Yarn - Queue

Yarn Hortonworks


A queue is a scheduler data structure that allows scheduler implementation to categorize apps into queue.


Queue definitions and properties such as capacity, ACLs can be changed, at runtime.



The utilization of the queue may become full (100% utilization) because the rate at which new jobs get submitted is higher than the rate at which the old jobs are completed.

The following image shows a joblauncher queue at 714.4% overused.

Queue Full

This is acceptable so long as there is still free capacity in the default queue to borrow from. However, when the cluster is fully utilized and the YARN memory is at 100% capacity, new jobs must wait, which causes timeouts on the client side (submit)



Administrators can add additional queues at runtime.


Queues cannot be deleted at runtime.


Administrators can stop queues at runtime. Existing applications continue to completion

If a queue is in STOPPED state, new applications cannot be submitted.


A stopped queue can be started.


Yarn - Yarn CLI

yarn queue


yarn top


Yarn - Yarn CLI

yarn queue -status <QueueName>

State or status can be:

  • STOPPED - Existing applications continue to completion.

Move app to queue

See Yarn - Application (app)

Discover More
Yarn Hortonworks
Hadoop - Metrics

on Hadoop JvmMetrics RetryCache/NameNodeRetryCache: RetryCache metrics is useful to monitor NameNode fail-over. Each metrics record contains Hostname tag. rpc rpcdetailed namenode:...
Card Puncher Data Processing
Hive - WebHCat (Rest API for HCatalog)

WebHCat ((or Templeton) service is a REST operation based API for HCatalog. WebHCat provides a service that you can use to run Hadoop MapReduce (or YARN), Pig, Hive jobs or perform Hive metadata operations...
Card Puncher Data Processing
Spark - Yarn

Yarn is a cluster manager supported by Spark. The deployment mode sets where the driver will run. The driver will run: In client mode, in the client process (ie in the current machine), and the...
Yarn Hortonworks
Yarn - Application (app)

An Hadoop application in the context of Yarn is either: a single job (ie a run of an application) or a DAG of jobs. On a application level (vs cluster level), Yarn consists of: a per-application...
Yarn Hortonworks
Yarn - Fair Scheduler

The fair scheduler is a scheduler plugin that fairly distributes an equal share of resources for jobs in the YARN cluster. When there is a single app running, that app uses the entire cluster. When...
Yarn Hortonworks
Yarn - Preemption

If Preemption is enabled, higher-priority applications do not have to wait because lower priority applications have taken up the available capacity. Under-served queues can begin to claim their allocated...
Yarn Hortonworks
Yarn - Security

yarn.admin.acl: ACL of who can be admin of the YARN cluster. yarn.acl.enable (true) Queue accees control ...
Yarn Hortonworks
Yarn - Timeline Server (ie yarn app history server)

The storage and retrieval of application’s current and historic information Component: timeline store known as: ApplicationHistoryServer Map Reduce History Server resource manager Rest API...

Share this page:
Follow us:
Task Runner