About
A pipeline is a finite or infinite automata known as a stream) where:
- the data
- transition (pipe)
- from one state to another
- via a series of transformations (work)
A pipeline creates a composition relationship.
A pipeline is also known as:
- Compose
- Chain (for instance Chain of command) - Daisy Chain ;)
Pipeline follows a compositional structure known as cascade of operations.
Flow vs Pipeline
A dataflow (data workflow) is:
- a directed graph structure (loop may be present)
- of pipeline (ie the nodes are pipelines)
Model
A pipeline is a combination of:
- an input source
- an output destination.
- and a sequence of data (that may be a byte of an object). In the general sense, it's called a message.
- that may be chained to form a pipeline
Type
Imperative
The pipeline is executed step by step
Declarative
The pipeline is executed only when the terminal operation is called.
All steps are building a composite type known as algebraic data type.
Example
Shell
In an OS Shell (Dos, bash), a serie of command connected by the pipe operators forms a pipeline. See Shell Data Processing - Pipeline
Code
By returning the calling object from a function, you can compose (or chain) functions. See Design Pattern - (Object) Builder. When we compose (chain) an operation, the output of one operation becomes the input for the next operation, and operations are applied from left to right.