Hadoop - Metrics

About

What are Application Metrics? ie Perfcounter, Performance Metrics, Operational data, Monitoring, telemetry on Hadoop

Articles Related

Component

Jvm

• JvmMetrics

Rpc

• RetryCache/NameNodeRetryCache: RetryCache metrics is useful to monitor NameNode fail-over. Each metrics record contains Hostname tag.
• rpc
• rpcdetailed

Hdfs

• namenode: HDFS - NameNode
• FSNamesystem: hdfs global
• JournalNode: HDFS - JournalNode (JN)
• datanode: HDFS - DataNode
• FsVolume: Per-volume metrics contain Datanode Volume IO related statistics.

Yarn

• ClusterMetrics: shows the metrics of the YARN cluster from the .:yarn:rm|ResourceManager]]’s perspective.
• QueueMetrics: shows an application queue from the ResourceManager’s perspective.
• NodeManagerMetrics: shows the statistics of the containers in the node.
• ContainerMetrics: shows the resource utilization statistics of a container.

Ugi

• UgiMetrics

Metrics

• MetricsSystem

Default context

• StartupProgress

Documentation / Reference

• https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/Metrics.html
• https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/metrics2/package-summary.html