Yarn - Resource Manager

Yarn Hortonworks

About

The ResourceManager is the central authority of the Yarn cluster. It has two main components:

Access

UI

See Yarn - Yarn UI (ResourceManager UI)

Conf: yarn.resourcemanager.webapp.address

Rest API

Yarn - ResourceManager Rest API

Application submissions

for application submissions. ie The address of the applications manager interface

IPC

Conf: yarn.resourcemanager.address

  • 8032 – Default port number
  • 8050 – Default port number when you create your cluster using Ambari.

HTTP

  • 8025 - conf yarn.resourcemanager.resource-tracker.address

Scheduler

See Scheduler port

Management

Resource Reservation

YARN supports the notion of resource reservation via the ReservationSystem, a component that allows users to specify a profile of resources over-time and temporal constraints (e.g., deadlines), and reserve resources to ensure the predictable execution of important jobs.

Restart

Restart is a feature that allow him to keep functioning across restarts and also makes the down-time invisible to end-users.

Documentation / Reference





Discover More
Yarn Hortonworks
Hadoop - Daemons / Process / Services

Hadoop process Common: Key management server Hdfs: Namenode – on the head node Datanode – on the worker nodes Rest API: Yarn: ResourceManager – on the head node Web Proxy...
Yarn Hortonworks
Hadoop - Head Node (Master node)

A head node is a node on a hadoop cluster that runs the (head|lead) services that manages the worker nodes. For example, the following process would be installed on a head node: For hdfs, the namenode...
Yarn Hortonworks
Hadoop - Metrics

on Hadoop JvmMetrics RetryCache/NameNodeRetryCache: RetryCache metrics is useful to monitor NameNode fail-over. Each metrics record contains Hostname tag. rpc rpcdetailed namenode:...
Yarn Hortonworks
Hue - Yarn

Yarn in Hue. It is a managed service of Hue. Job Browser Job...
Card Puncher Data Processing
Kubernete (kube)

Kubernete is an orchestrator for container. It permits to deploy and scale containers on a cluster of host.
Mapreduce Pipeline
MapReduce - Application

Applications: specify the input/output locations supply map and reduce functions via implementations of appropriate interfaces and/or abstract-classes. These, and other job parameters, comprise...
Card Puncher Data Processing
Spark - Yarn

Yarn is a cluster manager supported by Spark. The deployment mode sets where the driver will run. The driver will run: In client mode, in the client process (ie in the current machine), and the...
Card Puncher Data Processing
What is a Container/VM Orchestrator?

Container Orchestrator: manage, scale, start/stop and maintain container They are also known as scheduler. This is where you create and run container in production. ie a cluster resource...
Yarn Hortonworks
Yarn - App Dev

Yarn application development page The yarn application model development has thee main entries: the YarnClient to communicate between the Client to ResourceManager and submit an application....
Yarn Hortonworks
Yarn - Application (app)

An Hadoop application in the context of Yarn is either: a single job (ie a run of an application) or a DAG of jobs. On a application level (vs cluster level), Yarn consists of: a per-application...



Share this page:
Follow us:
Task Runner