Yarn - Log (Container, Application) - Tfile

Yarn Hortonworks


The log of an application (ie from all the containers that the app use when running).


Application logs are not saved in text format. They are saved in a binary format called TFile.

A TFile format is a container of key value pairs, and it is not readable by humans.



Command Line


yarn logs -applicationId <applicationId>
yarn logs -applicationId <applicationId> -appOwner <user-who-started-the-application>
yarn logs -applicationId <applicationId> -appOwner <user-who-started-the-application> -containerId <containerId> -nodeAddress <worker-node-address>


Code: LogsCLI.java


With HortonWorks Ambari > Yarn > Quick Links:

Yarn Ui Log Ambari

Other tfile reader

tfile reader:



Logs for all the containers belonging to a single Application are aggregated and written out to a single (possibly compressed) log file at a configured location.

yarn.log-aggregation-enable to true

If disabled, node manager will keep the logs locally


The log should be under the yarn.nodemanager.log-dirs configuration properties

Screenshot from Ambari


Ambari Yarn.nodemanager.log Dirs



The application log will be at: yarn.nodemanager.log-dirs/application_appid


The Containers log will be at yarn.nodemanager.log-dirs/application_appid/container_{contid}


Application Framework Description
Map reduce job Each container directory will contain the files stderr, stdin, and syslog generated by that container.
Other frameworks They can choose to write more or less files not the location


For clusters with a lot of Yarn aggregated logs, it can be helpful to combine them into hadoop archives in order to reduce the number of small files, and hence the stress on the NameNode.

  • Usage
mapred archive-logs

More … Hadoop Archive Logs Guide

Discover More
Card Puncher Data Processing
Spark - Log

or Spark executor logs are located in the /work/app- name of your application Driver logs
Yarn Hortonworks
Yarn - Application (app)

An Hadoop application in the context of Yarn is either: a single job (ie a run of an application) or a DAG of jobs. On a application level (vs cluster level), Yarn consists of: a per-application...
Yarn Hortonworks
Yarn - Container (RmContainer|Resource Container)

A resource container incorporates resource elements such as memory, cpu, disk, network etc. The Docker...

Share this page:
Follow us:
Task Runner