Yarn - Scheduler (S)

About

The scheduler is a component of the Resource Manager.

The Scheduler is responsible for allocating resources to the applications.

It performs its scheduling function based on the resource requirements of the applications defined in resource Container (which incorporates elements such as memory, cpu, disk, network etc.)

The scheduler:

  • performs no monitoring or tracking of status for the application.
  • offers no guarantees about restarting failed tasks either due to application failure or hardware failures.

Management

List

Default

The default scheduler is the JobQueueTaskScheduler. See Yarn - Default Scheduler

Plugin

The Scheduler use plugin to implement schedulers functionality. They are responsible for partitioning the cluster resources among the various queues, applications etc.

Built-in Plugin:

UI

The ResourceManager web UI/scheduler web-page should show the resource usages of individual queues.

Port

  • HTTP: 8030 conf: yarn.resourcemanager.scheduler.address - The address of the scheduler interface
  • HTTP: 8033 conf: yarn.resourcemanager.admin.address - The address of the RM admin interface.

Load simulator

Documentation / Reference


Powered by ComboStrap