Spark Engine - Transformation Function

About

Transformations are functions that will not be completed at the time you write and execute the code.

They will only get executed once an action function is called.

Spark transformations create new data structure from an existing one (creating a chain).

Spark remembers the set of transformations that are applied to a base data structure

It can then optimize the required calculations and automatically recover from failures and slow workers.

Example

  • Converting an integer into a float or to filter a set of values.

Powered by ComboStrap