"Suppose you’re going from a single database to a service that hides the details of a new storage solution. Have the service wrap around the legacy storage, and ramp up writes to it slowly. With backfilling, comparison checks on read (another feature flag), and then slow ramp up of reads (yet another flag), you will have much more confidence and fewer disasters."
Does the author mean to say:
"Have the service wrap around the legacy storage, and ramp up writes to the new system slowly"?
It reads like he's saying ramping up writes to the legacy system.
Also I am confused by his use of the word "backfilling", what would being backfilled? IOPS to the new system after IOPs to the legacy system? Is that the idea.
This was a good read. Kudos to the author to putting it out there.
If you want to migrate from system A to system B you don't start writing all data to B immediately. You don't know if B can handle the load, if your code is right, if double writing will cause an unacceptable slowdown etc. Instead you mirror a % of all writes from A to B and increase it over time.
Now you have a problem because B only has a % of the data. To fix this you need a separate task that periodically identifies records in A missing from B and syncs them. This is the "backfill" - it isn't part of your running application and is only needed for the duration of the migration.
Once you think you have sufficient data in B you can start checking your work, run a % of your reads from both A AND B and make sure the results match.
Once that matches for a while you can start making a % of reads from B without ever contacting A. When this reaches 100% you stop writing to A.
"Suppose you’re going from a single database to a service that hides the details of a new storage solution. Have the service wrap around the legacy storage, and ramp up writes to it slowly. With backfilling, comparison checks on read (another feature flag), and then slow ramp up of reads (yet another flag), you will have much more confidence and fewer disasters."
Does the author mean to say:
"Have the service wrap around the legacy storage, and ramp up writes to the new system slowly"? It reads like he's saying ramping up writes to the legacy system.
Also I am confused by his use of the word "backfilling", what would being backfilled? IOPS to the new system after IOPs to the legacy system? Is that the idea.
This was a good read. Kudos to the author to putting it out there.