Table of Contents

About

An Hadoop application in the context of Yarn is either:

  • a single job (ie a run of an application)
  • or a DAG of jobs.

On a application level (vs cluster level), Yarn consists of:

Yarn is able to run different types of applications.

Type

Type of application that Yarn can run.

Properties

Name

To create an application name: 

yarn app -save <Application Name> <File Name>

where:

  • -updateLifetime and -changeQueue alter the values provided in the file.
  • -appTypes specify which client implementation to use

Management

Via the application module of the yarn cli

See the source code entry at ApplicationCLI.java

Kill

Yarn - Yarn CLI 

yarn -kill application_id

Stop

yarn -stop <Application Name or ID>

Status

yarn app -status <ApplicationId or ApplicationName>

Launch

yarn app -launch <Application Name> -updateLifetime and -changeQueue -appTypes <Types> <File Name>

# or to start a saved application
yarn start <Application Name>

# or with YARN code bundled in a jar file 
yarn jar <jar> [mainClass] args...

where:

  • -updateLifetime and -changeQueue alter the values provided in the file.
  • -appTypes specify which client implementation to use

If a queue is in STOPPED state, new applications cannot be submitted.

If an app specifically lists a queue in a container resource request, the request is submitted to that queue.

A yarn app is submitted programmatically with a YarnClient interface. See Yarn - App Dev

Move to queue

Yarn - Yarn CLI move to queue 

yarn app -changeQueue <Queue Name>
# movetoqueue is Deprecated
#yarn app -movetoqueue <Application ID>

For the fairScheulder, an attempt to move an application to a queue will fail if the addition of the app’s resources to that queue would violate the its maxRunningApps or maxResources constraints.

Modify the number of container

change the number of components/containers running for an application 

yarn app -flex <Application Name or ID> -component <Component Name> <Count>

Develop

Yarn - App Dev

List

Yarn - Yarn CLI 

yarn app -list

Filter:

  • by type -appTypes <Types>
  • by tag -appTags <Tags>
  • by state -appStates <States>

See also top

Top

yarn top

YARN top - 14:00:34, up 11d, 2:41, 0 active users, queue(s): root
NodeManager(s): 3 total, 3 active, 0 unhealthy, 0 decommissioned, 0 lost, 0 reb
Queue(s) Applications: 20 running, 759 submitted, 0 pending, 739 completed, 0 k
Queue(s) Mem(GB): 153 available, 146 allocated, 0 pending, 0 reserved
Queue(s) VCores: 44 available, 50 allocated, 0 pending, 0 reserved
Queue(s) Containers: 50 allocated, 0 pending, 0 reserved

                  APPLICATIONID USER             TYPE      QUEUE   #CONT  #RCON
 application_1522322322495_0274 sshuser    informatica blaze    default      12
 application_1522230994452_0133 admin       mapreduce    default       5
 application_1522322322495_0687 admin       mapreduce    default       2
 application_1522322322495_0684 admin       mapreduce    default       2
 application_1522322322495_0682 admin       mapreduce    default       2
 application_1522307230623_0033 admin       mapreduce    default       2
 application_1522307230623_0060 admin       mapreduce    default       2
 application_1522307230623_0058 admin       mapreduce    default       2
 application_1522307230623_0020 admin       mapreduce    default       2
 application_1522307230623_0019 admin       mapreduce    default       2
 application_1522307230623_0018 admin       mapreduce    default       2
 application_1522307230623_0016 admin       mapreduce    default       2
 application_1522322322495_0233 admin       mapreduce    default       2
 application_1522322322495_0232 admin       mapreduce    default       2
 application_1522322322495_0231 admin       mapreduce    default       2
 application_1522322322495_0237 admin       mapreduce    default       2
 application_1522322322495_0235 admin       mapreduce    default       2
 application_1522322322495_0726 admin           spark    default       1
 application_1521626910869_0002 hive            spark  thriftsvr       1
 application_1521626910869_0001 hive            spark  thriftsvr       1

Log

yarn logs -applicationId application_1520929888341_0011

History

See: