Spark - Spark-submit

About

The spark submit application to submit application.

The spark-submit script is used to launch applications on a cluster.

Spark jobs are generally submitted from an edge node

Syntax

./bin/spark-submit \
  --class <main-class> \
  --master <master-url> \
  --deploy-mode <deploy-mode> \
  --jars <additional1.jar,additional2.jar> \
  --conf <key>=<value> \
  ... # other options
  <application-jar> \
  [application-arguments]

where:

Example

Run application locally on 8 cores

./bin/spark-submit \
  --class org.apache.spark.examples.SparkPi \
  --master local[8] \
  /path/to/examples.jar \
  100

More ?. See Java submit

Documentation / Reference


Powered by ComboStrap