Table of Contents

About

HDFS follows the same configuration scheme than the whole Hadoop platform. See Hadoop - Configuration (HADOOP_CONF_DIR)

The configuration are split between two files:

  • hdfs-site.xml, which provides default behaviors for the HDFS client.
  • core-site.xml, which sets the default filesystem name.

The hdfs configurations are stored in:

Management

Get a value

With the hdfs utility, to get the value of a key: 

hdfs getconf -confKey key

Example with the file system URI. You get the value here of a wasb azure file system 

hdfs getconf -confKey fs.defaultFS

wasb://[email protected]

Directory

hdfs envvars | grep -i HADOOP_CONF_DIR

HADOOP_CONF_DIR='/usr/hdp/2.6.2.25-1/hadoop/conf'

See HDFS - Environment Variable

Pass a value at runtime

  • Command line: You can pass a configuration at the command line with the D generic option.
hdfs dfs -D "fs.default.name=hdfs://mycluster/" -ls /
  • Code API (Java)
Configuration conf = new Configuration();
String hdfsUri = "hdfs://mycluster/";
conf.set("fs.defaultFS", hdfsUri);
FileSystem fileSystem = FileSystem.get(URI.create(hdfsUri), conf);