Yarn - App Dev

About

Yarn application development page

Articles Related

Dev model

Low level

The yarn application model development has thee main entries:

• the YarnClient to communicate between the Client to ResourceManager and submit an application.
• AMRMClientAsync to communciatie between the ApplicationMaster and the ResourceManager (handling events asynchronously by AMRMClientAsync.CallbackHandler)
• NMClientAsync: to communicate between the ApplicationMaster and the NodeManager - Launch containers. handling container events by NMClientAsync.CallbackHandler

This three entries wrap the yarn protocols.

High level library

• Yarn - Slider (App shipping) - Slider is a framework for deployment and management of services applications in Hadoop.
• http://twill.apache.org/

YarnClient

Once a YarnClient is created:,

• the client can then set up application context,
• the client prepare the very first container of the application that contains the ApplicationMaster (AM). The client provide:
  • the dependency (local files,jars),
  • the command to be executed
  • the Unix process(es) that needs to be launched
• the client submit the application.

Sample

Tuto based on hortonworks/simple-yarn-app