Spark - Log

Card Puncher Data Processing

Management

Directory

export SPARK_LOG_DIR=/var/log/spark2

Cluster

Standalone

Spark - Standalone installation (spark scheme) or Spark - Local Installation

Spark executor logs are located in the SPARK_HOME/work/app-

'' directory (where is the name of your application.
YARN

See Yarn - Log (Container, Application) - Tfile

Driver logs are by default printed to console





Discover More
Spark Program
Spark - Application

An application is an instance of a driver created via the initialization of a spark context (RDD) or a spark session (Data Set) This instance can be created via: a whole script (called batch mode)...



Share this page:
Follow us:
Task Runner