Given all the problems they've had with AWS, I wonder what the overall cost/benefit would be for someone like Reddit moving from the flexed-EC2-servers model to just one high-powered db server and some caching webservers in front of it.
Reddit's probably was using trying to use EBS as a backend for a transactional database. The issue is that EBS makes no guarantees on latency, only that they'll never lose your data.
Amazon is not saying it, but EBS is probably lying when doing an fsync().
In this post [1] explaining why Reddit was down, one problem was that the DB slave got ahead of the master, with the most probable explanation being that the master flagged the data as being safe for replication, before committing it to disk and I trust PostgreSQL more than I trust EBS.
In this forum answer Amazon is giving about this potential problem [2], they are dodging the question by saying that fsync() guarantees durability for instance failures, but not for volume failures, with the anual failure rate (AFR) being given as 0.1% - 0.5% for volumes (how accurate that is, it remains to be seen).
So EBS is probably lying about fsync's success, especially since the behavior of fsync in virtual environments is always a surprise. So you can definitely lose your data more frequently than if you had your own hardware.
Amazon is not saying it, but EBS is probably lying when doing an fsync().
Given that POSIX says "the nature of the transfer is implementation-defined" and they've defined what fsync does on their implementation: No, they're not lying.
I wouldn't say that they're being misleading, either. On a local physical disk, once fsync returns your data is safely stored unless/until the disk dies. According to the forum post you linked to, semantics on EBS are exactly the same.
fsync does not mean "has been written to spinning magnetic media".
Reddit could easily cut their spend on hosting by like 50-75% just by moving to a dedicated hosting provider. They'd also get bare metal speeds in the process.
Who knows why they've stuck with AWS, though. It's possible Amazon is giving them a discount so they can be used as an example.
