Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm curious about how archive.org feel about this. Presumably they are paying for the outgoing bandwidth.


With respect to pure speed, at some point it would be cheaper/faster to show up at the facility with a trunk load of drives and plug right in. Superficially, this is a game for the locals, not someone like me. Then again, looking at how little it costs to ship me a couple terrabytes of drives, maybe this isn't as crazy as it sounds.

I can get a cheap empty prosumer NAS box for a couple hundred bucks and a couple TB of drives for a couple hundred more. If I gave someone (presumably the internet archive) a kilobuck and told them to keep the change (which would be a small but nice revenue stream for them), they could assemble a couple TB nas and ship it to me and I could plug it in and give it a public ipv6 address and aside from the kilobuck it would cost me a couple hundred bucks a year in electricity and presumably bandwidth, but at least it wouldn't have any labor cost. This might be going a bit far for individual citizens but you could probably guilt trip corporations into hosting your custom made NAS boxes for you. Just an idea to think about.

Another intriguing financial model might be to partner with public libraries and schools and double charge private companies in order to give public libraries and schools free backup NAS.

The world of non-profits opens up strange new ideas.


This is being done by Jason Scott and the rest of the team that runs archive.org. Presumably they're ok with using their own bandwidth!


It's being overseen by Jason Scott (me) but I am not utilizing other employees of Internet Archive - they're quite busy enough as it is.


Don't forget that the purpose of the Internet Archive is primarily to be an open archive. In that sense, I'd imagine that bandwidth use is not really a factor, and anything that contributes to the longevity of the archive is welcome.

"It's costing us bandwidth" is only a relevant factor if your goal is to turn a profit, and the bandwidth use isn't contributing to that. An archive doesn't have the same considerations.


I'm very aware that IA is an open, non-profit archive and I'm sure that they don't mind that other copies of the archive exist. My question was about the cost to IA of transferring 12 petabytes (according to Jason Scott [1]). Outbound bandwidth ultimately costs money. IA is a non-profit and "It's costing us bandwidth" is very relevant.

For instance, here [2] are the charges for outgoing bandwidth on Rackspace cloud servers. The lowest quoted rate if £0.06/GB for up to about half a petabyte. IA won't be paying that much as they have their own infrastructure. But even if bandwidth only costs the IA $0.01/GB, 12PB would cost around $125k.

12PB is a lot of data. I don't know what IA pay for outbound bandwidth but it won't be zero. Thats all I was asking about.

[1] https://news.ycombinator.com/item?id=9602868

[2] http://www.rackspace.com/cloud/public-pricing#bandwidth


Keep in mind, we're trying to make 3 copies of all this data - so multiply that bandwidth by 3.


The link into 300 Funston Ave is 10Gbit. There is no shortage of bandwidth.


Your intel is old, edward ;) they're past 40Gbit now.


Even saturating a 40Gbit link at 5GB/s would require nearly 50 days to transfer 20PB.

Realistically, I'd estimate mirroring the IA to take years... providing you can keep up with its own growth.


Don't forget that the purpose of the Internet Archive is primarily to be an open archive. In that sense, I'd imagine that bandwidth use is not really a factor, and anything that contributes to the longevity of the archive is welcome.

"It's costing us bandwidth" is only a relevant factor if your goal is to turn a profit, and the bandwidth use isn't contributing to that. An archive doesn't have the same considerations.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: