The first destination of the data that has been extracted from source is the staging area. Sometimes staging areas are also called landing zones for flat files, XML files, Cobol files and the like.
This logical layer:
- acts as a temporary storage area for data manipulation before it enters the data Warehouse
- and serves to isolate the rate at which data is received into the data warehouse from the frequency at which data is refreshed and made available to the end user.
It is also possible that in some implementations this layer is not necessary, as all data transformation processing will be done “on the fly” as data is extracted from the source system before it is inserted directly into the Operational data layer (Foundation).
The most basic approach for the staging layer is to have it be an identical schema (a replicate) of the data source:
- with the data that we need.
- and may be with some structural changes to the tables, such as range partitioning