Each driver program has a web UI, typically on port 4040, that displays information about:
The Spark UI will tell you which DataFrames and what percentages are in memory. Spark - (Executor) Cache
This UI is only available for the duration of the application. The cluster manager UI's (master en worker) are always available. Example with the built-in standalone cluster manager, the UI by default are available at the port 8080 for the master and 8081 for the workers.
http://<driver-node>:4040
where: