About
HDFS follows the same configuration scheme than the whole Hadoop platform. See Hadoop - Configuration (HADOOP_CONF_DIR)
The configuration are split between two files:
- hdfs-site.xml, which provides default behaviors for the HDFS client.
- core-site.xml, which sets the default filesystem name.
The hdfs configurations are stored in:
- and can be overwritten through a hdfs-site.xml file
Articles Related
Management
Get a value
With the hdfs utility, to get the value of a key:
hdfs getconf -confKey key
Example with the file system URI. You get the value here of a wasb azure file system
hdfs getconf -confKey fs.defaultFS
wasb://[email protected]
Directory
hdfs envvars | grep -i HADOOP_CONF_DIR
HADOOP_CONF_DIR='/usr/hdp/2.6.2.25-1/hadoop/conf'
Pass a value at runtime
- Command line: You can pass a configuration at the command line with the D generic option.
hdfs dfs -D "fs.default.name=hdfs://mycluster/" -ls /
- Code API (Java)
Configuration conf = new Configuration();
String hdfsUri = "hdfs://mycluster/";
conf.set("fs.defaultFS", hdfsUri);
FileSystem fileSystem = FileSystem.get(URI.create(hdfsUri), conf);