Spark Engine - Action Function

Spark Query Plan Generation

About

Actions are function that trigger the computation of all previous transformations in order to get back an actual result.

An action is composed of one or more jobs

Spark actions execute the specified set of transformations to the source data and return the results out of Spark at the driver.




Discover More
Spark Pipeline
Spark - Action

in RDD. Reduce aggregates a data set element using a function. Takeordered and take returns n elements ordered or not Collect returns all of the elements of the RDD as an array
Spark Jobs
Spark - Jobs

Job in Spark. A job is a unit of task for an application. A job consists of tasks that will be executed by the workers in parallel where possible. A job is triggered by an action function.
Spark Query Plan Generation
Spark Engine - (Operations | Functions )

Operations are divided into transformations and actions. Transformations are pipelined function (producing the same input type), and actions trigger computation and return results. Transformation functions...
Spark Query Plan Generation
Spark Engine - Logical Plan

Logical Plan in Spark. Each data structure represents a logical plan that describes the computation required to produce the data. When an action is invoked, Spark's query optimizer optimizes the...
Spark Query Plan Generation
Spark Engine - Physical Plan

in Spark When an action is invoked, Spark's query optimizer optimizes the logical plan and generates a physical plan for efficient execution. To explore the physical plan, use the explain...
Spark Query Plan Generation
Spark Engine - lazy

All spark data structure are lazy which means that computations are only triggered when an action is invoked



Share this page:
Follow us:
Task Runner