Yarn - App Dev

Yarn Hortonworks

About

Yarn application development page

Dev model

Low level

The yarn application model development has thee main entries:

  • the YarnClient to communicate between the Client to ResourceManager and submit an application.
  • AMRMClientAsync to communciatie between the ApplicationMaster and the ResourceManager (handling events asynchronously by AMRMClientAsync.CallbackHandler)
  • NMClientAsync: to communicate between the ApplicationMaster and the NodeManager - Launch containers. handling container events by NMClientAsync.CallbackHandler

This three entries wrap the yarn protocols.

High level library

YarnClient

Once a YarnClient is created:,

  • the client can then set up application context,
  • the client prepare the very first container of the application that contains the ApplicationMaster (AM). The client provide:
    • the dependency (local files,jars),
    • the command to be executed
    • the Unix process(es) that needs to be launched
  • the client submit the application.

Sample

Tuto based on hortonworks/simple-yarn-app




Discover More
Yarn Hortonworks
Yarn - Application (app)

An Hadoop application in the context of Yarn is either: a single job (ie a run of an application) or a DAG of jobs. On a application level (vs cluster level), Yarn consists of: a per-application...
Yarn Hortonworks
Yarn - Protocols

The three main protocols for YARN application are: ApplicationClientProtocol, ApplicationMasterProtocol and ContainerManagementProtocol They are wrapped by the 3 api for YARN applications development....
Yarn Hortonworks
Yarn - Slider (App shipping)

Slider is a collection of tools and technologies to simplify the packaging, deployment and management of long-running applications on YARN. Slider bypasses node installation for application and related...



Share this page:
Follow us:
Task Runner