Spark - Engine
In Spark, the Spark engine is a SQL’s optimized execution engine and understand as input:
and is therefore sometimes known as the SQL Engine.
In Spark, functions are pipelined around their data structure.
A function will take a Spark data structure (data type) as input and output the same data structure (data type).
It results of a builder pattern where the last method performs an action whereas the previous one set parameters.
You are not manipulating the engine directly but via interfaces: