Spark - Spark-submit

About

The spark submit application to submit application.

The spark-submit script is used to launch applications on a cluster.

Spark jobs are generally submitted from an edge node

Articles Related

Syntax

./bin/spark-submit \
  --class <main-class> \
  --master <master-url> \
  --deploy-mode <deploy-mode> \
  --jars <additional1.jar,additional2.jar> \
  --conf <key>=<value> \
  ... # other options
  <application-jar> \
  [application-arguments]

where:

class is the main class of the jar
master is the connection URL. It can be:
- Spark - Yarn
deploy-mode can be client or cluster. See Yarn mode
jar is Spark - Jar
application-jar is the packaged application.

Example

Run application locally on 8 cores

./bin/spark-submit \
  --class org.apache.spark.examples.SparkPi \
  --master local[8] \
  /path/to/examples.jar \
  100

More ?. See Java submit

Documentation / Reference

https://spark.apache.org/docs/latest/submitting-applications.html
Class: org.apache.spark.deploy.SparkSubmit.main