Yarn - Application Master Container (AM) - Job tracker

About

An Application Master (AM) is a per-application daemon to look after the lifecycle of the job.

For instance, in Spark, it's called the driver

The Application Master daemon is created when an application is started in the very first container.

The AM:

  • communicates with YARN cluster
  • handles application execution
  • negotiate resources from the ResourceManager
  • work with the NodeManager(s) to execute and monitor the tasks.

It has the responsibility to provide a web UI and to send that link to the RM.

During application launch time, the main tasks of the ApplicationMaster are:

Management

Rest Api

Memory

  • Required AM memory is (asked memory+384 MB)
  • Limit: max threshold yarn.scheduler.maximum-allocation-mb

Powered by ComboStrap