Yarn - Application Master Container (AM) - Job tracker

Yarn Hortonworks


An Application Master (AM) is a per-application daemon to look after the lifecycle of the job.

For instance, in Spark, it's called the driver

The Application Master daemon is created when an application is started in the very first container.

The AM:

  • communicates with YARN cluster
  • handles application execution
  • negotiate resources from the ResourceManager
  • work with the NodeManager(s) to execute and monitor the tasks.

It has the responsibility to provide a web UI and to send that link to the RM.

During application launch time, the main tasks of the ApplicationMaster are:


Rest Api

MapReduce Application Master REST API’s


  • Required AM memory is (asked memory+384 MB)
  • Limit: max threshold yarn.scheduler.maximum-allocation-mb

Discover More
Hive System Architecture
Hive - Architecture

Example of System architecture with Map/Reduce as Engine. where: (for UI) Job Tracker: ...
Spark Cluster
Spark - Driver

The driver is a (daemon|service) wrapper created when you get a spark context (connection) that look after the lifecycle of the Spark job. cluster managerapplication manager The driver: start as its...
Card Puncher Data Processing
Spark - Yarn

Yarn is a cluster manager supported by Spark. The deployment mode sets where the driver will run. The driver will run: In client mode, in the client process (ie in the current machine), and the...
Yarn Hortonworks
Yarn - App Dev

Yarn application development page The yarn application model development has thee main entries: the YarnClient to communicate between the Client to ResourceManager and submit an application....
Yarn Hortonworks
Yarn - Application (app)

An Hadoop application in the context of Yarn is either: a single job (ie a run of an application) or a DAG of jobs. On a application level (vs cluster level), Yarn consists of: a per-application...
Yarn Hortonworks
Yarn - Applications Manager (ASM)

The ApplicationsManager is responsible for: accepting job-submissions, negotiating the first container for executing the application specific ApplicationMaster restarting the ApplicationMaster...
Yarn Hortonworks
Yarn - Architecture / Topology / Components

The Yarn (ie MapReduce) framework consists of three topology level: federation of independent cluster cluster application (that the Yarn cluster runs) Federation allow multiple independent...
Yarn Hortonworks
Yarn - Job

A job is a running instance of an application and can be monitored via: the application master when running the History Server when finished See JobContext...

Share this page:
Follow us:
Task Runner