Hadoop - Cluster Installation

About

Manual installation of an hadoop cluster. To start a Hadoop cluster you will need to start both the HDFS and YARN cluster. More …

For now, just download and unzip

Articles Related

Steps

• Download the binary archive at http://hadoop.apache.org/releases.html
• Untar it

tar -xvf hadoop-2.7.5.tar.gz

• Open the file environment file and set the JAVA_HOME

REM Windows
%HADOOP_HOME%\etc\hadoop\hadoop-env.cmd

set JAVA_HOME=C:\Java\jdk1.8.0_162

• For windows: download a compiled version of Winutils and put it in the bin. RStudio comes also with on at RStudio_home\bin\winutils

# Example for hadoop 2.7
https://github.com/steveloughran/winutils/blob/master/hadoop-2.7.1/bin/winutils.exe

Test

cd %HADOOP_HOME%%\bin
hadoop.cmd classpath
hdfs dfsadmin -help