Data Warehouse - Persistent layer (Fundamental Layer | Operational Data Store - ODS)

Data System Architecture

About

A persistent layer is a data warehouse layer where data is persisted (ie never deleted).

A persistent layer is also known as:

  • a fundamental layer
  • an operational data store (ODS)
  • an enterprise data warehouse (EDS)

To support operational reporting you need to store a lot of transitional information that changes frequently and has little historical or strategic value to the business. That type of stuff can significantly overburden the design and maintenance of a dimensional data warehouse, so, an ODS is used.

Implementing an ODS can significantly improve and simplify data capture for loading the data warehouse. Because you can control the design and implementation of the ODS, it can be closely integrated with the DW, such as sharing key dimensions and delta capture for fact loads.

Functionality

  • Actual Data: Mirror the source application structure
  • History Data: Save eventually the modification of the source data in a scd format

Characteristic

Advantages

  • Make the loading process of the Data Mart restartable
  • Enable cross source business logic
  • Can receive data directly in the good data type (Example from a web service)
  • The loading phase can be split in two: first, the loading of source data in the persistent layer then the design of the data.
  • Backup: As the whole system restartable is from this layer, you may backup only this layer.
  • Bad data file (ie that doesn't represent the reality such as test file in production) can be suppressed easily in this layer.

Documentation / Reference





Discover More
Dw Layers
Data Warehouse - Layer (Architecture)

Name Properties Stage Layer Real-Time, CDC, continuous refresh DataWarehouse Layer / ODS Normalized, Data History, refresh: 2-6 daily Data Mart Performance, Access layer, Star schema, refresh: 1-4...
Data System Architecture
Data Warehouse - Staging Area

The first destination of the data that has been extracted from source is the staging area. Sometimes staging areas are also called landing zones for flat files, XML files, Cobol files and the like. This...
Data Mart
Data Warehousing - Data Marts

A data mart is a subject-or application-specific multidimensional schema build on the top of an Enterprise Data Warehouse. Data marts are designed for a particular line of business and is an aggregation...
Datavault
Datawarehouse Data Modeling - Data Vault

The data vault (DV) is a relational data model technique used in the persistence layer of a datawarehouse The Data Vault consists of three primary entity types: ...



Share this page:
Follow us:
Task Runner