Spark - Daemon

About

daemon in Spark

The daemon in Spark are the driver that starts the executors. See Spark - Cluster

The daemon in Spark are JVM running threads (known as core (or slot)

  • one driver = 1 JVM many core
  • one executor = 1 JVM many core

As the JVM has to startup and initialize certain data structures before it can begin running tasks, running more core is preferable as running more executors.


Powered by ComboStrap