About
File System - Block in HDFS.
The block size can be changed by file.
Block are stored on a datanode and are grouped in block pool
Articles Related
Management
Info
Location
The location on where the blocks are stored is defined in hdfs-site.xml. Example:
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/hadoop/data/dfs/datanode</value>
</property>
Offset
For Hive, see Built-in BLOCKOFFSETINSIDE__FILE virtual column
Size
A typical block size used by HDFS is 128 MB. Thus, an HDFS file is chopped up into 128 MB chunks.
- From the config file (in bytes). ie 128 Mb
<property>
<name>dfs.blocksize</name>
<value>134217728</value>
</property>
- From the command line:
hdfs getconf -confKey dfs.blocksize
134217728
# of 128 Mb
Move
See the mover hdfs sub-command to move block replicas across storage types.
hdfs mover
Failure
Under-replicated
- web ui: The overview page gives you this information.
