MapReduce - Pipeline

About

A MapReduce app implements a pipeline

Articles Related

Syntax

(input) -> <k1, v1> 
    -> map -> <k2, v2> 
    -> combine -> <k2, v2> 
    -> sort -> <k3, v3> 
    -> reduce -> <k4, v4> 
    -> (output)

where:

• the input is transformed in key value pair stream/data
• the stream/data is process in paralleled via a map operations
• the result is then combined/shuffled
• sorted
• reduced
• and written to the disk