About
An Hadoop application in the context of Yarn is either:
On a application level (vs cluster level), Yarn consists of:
- a per-application ApplicationMaster
- a per-application Container running on a NodeManager
Yarn is able to run different types of applications.
Type
Type of application that Yarn can run.
- …
Properties
Name
To create an application name:
yarn app -save <Application Name> <File Name>
where:
- -updateLifetime and -changeQueue alter the values provided in the file.
- -appTypes specify which client implementation to use
Management
Via the application module of the yarn cli
See the source code entry at ApplicationCLI.java
Kill
yarn -kill application_id
Stop
yarn -stop <Application Name or ID>
Status
yarn app -status <ApplicationId or ApplicationName>
Launch
yarn app -launch <Application Name> -updateLifetime and -changeQueue -appTypes <Types> <File Name>
# or to start a saved application
yarn start <Application Name>
# or with YARN code bundled in a jar file
yarn jar <jar> [mainClass] args...
where:
- -updateLifetime and -changeQueue alter the values provided in the file.
- -appTypes specify which client implementation to use
If a queue is in STOPPED state, new applications cannot be submitted.
If an app specifically lists a queue in a container resource request, the request is submitted to that queue.
A yarn app is submitted programmatically with a YarnClient interface. See Yarn - App Dev
Move to queue
Yarn - Yarn CLI move to queue
yarn app -changeQueue <Queue Name>
# movetoqueue is Deprecated
#yarn app -movetoqueue <Application ID>
For the fairScheulder, an attempt to move an application to a queue will fail if the addition of the app’s resources to that queue would violate the its maxRunningApps or maxResources constraints.
Modify the number of container
change the number of components/containers running for an application
yarn app -flex <Application Name or ID> -component <Component Name> <Count>
Develop
List
yarn app -list
Filter:
- by type -appTypes <Types>
- by tag -appTags <Tags>
- by state -appStates <States>
See also top
Top
yarn top
YARN top - 14:00:34, up 11d, 2:41, 0 active users, queue(s): root
NodeManager(s): 3 total, 3 active, 0 unhealthy, 0 decommissioned, 0 lost, 0 reb
Queue(s) Applications: 20 running, 759 submitted, 0 pending, 739 completed, 0 k
Queue(s) Mem(GB): 153 available, 146 allocated, 0 pending, 0 reserved
Queue(s) VCores: 44 available, 50 allocated, 0 pending, 0 reserved
Queue(s) Containers: 50 allocated, 0 pending, 0 reserved
APPLICATIONID USER TYPE QUEUE #CONT #RCON
application_1522322322495_0274 sshuser informatica blaze default 12
application_1522230994452_0133 admin mapreduce default 5
application_1522322322495_0687 admin mapreduce default 2
application_1522322322495_0684 admin mapreduce default 2
application_1522322322495_0682 admin mapreduce default 2
application_1522307230623_0033 admin mapreduce default 2
application_1522307230623_0060 admin mapreduce default 2
application_1522307230623_0058 admin mapreduce default 2
application_1522307230623_0020 admin mapreduce default 2
application_1522307230623_0019 admin mapreduce default 2
application_1522307230623_0018 admin mapreduce default 2
application_1522307230623_0016 admin mapreduce default 2
application_1522322322495_0233 admin mapreduce default 2
application_1522322322495_0232 admin mapreduce default 2
application_1522322322495_0231 admin mapreduce default 2
application_1522322322495_0237 admin mapreduce default 2
application_1522322322495_0235 admin mapreduce default 2
application_1522322322495_0726 admin spark default 1
application_1521626910869_0002 hive spark thriftsvr 1
application_1521626910869_0001 hive spark thriftsvr 1
Log
yarn logs -applicationId application_1520929888341_0011
- Set log level: see daemon log
- See also log location
History
See: