Data - Migration

Data System Architecture


Migrating data from an existing store to a new one.

See also: Database - Migration (Deployment|Versioning|Change Management)


The migration of data can be broken down as follows:

  1. Encapsulate access to the data in an appropriate data type.
  2. Modify the implementation to store data in both the old and the new stores.
  3. Bulk migrate existing data from the old store to the new store. This is done in the background in parallel to writing new data to both stores.
  4. Modify the implementation to read from both stores and compare the obtained data.
  5. When convinced that the new store is operating as intended, switch to using the new store exclusively (the old store may be maintained for some time to safeguard against unforeseen problems).

Facebook has used this process to transparently migrate database tables containing hundreds of billions of rows to new storage formats.

Discover More
Card Puncher Data Processing
Code Shipping - Feature Development / Planning

An application is composed logically from one or more features. jobYTQTgeUE?t=842Philip Fernbach - Why do we believe things that aren't true? Features can be in one of 4 states: Internal Incubating...
Data System Architecture
Database - Migration (Deployment|Versioning|Change Management)

with SQL database. Database migration is the process of changing the structure of the database in an other with mostly DDL statement. The flow sometimes will block any change that risks data-loss. See...

Share this page:
Follow us:
Task Runner