Installing Hortonworks Hadoop for Windows

Installing Hortonworks for
Windows

Intro
• I installed Hortonworks for Windows on my local Hyper-V machine.
• The following Slides introduce you to the steps for installing on your
machine.
• The entire content can also be found on my blog:
• http://www.bloomconsultingbi.com/2013/10/installationhortonworks-hadoop-13-part.html
• http://www.bloomconsultingbi.com/2013/10/installationhortonworks-hadoop-13-part_22.html
• Enjoy~!

So today we are going to install Hadoop 1.3 single node cluster onto a
Hyper-V system.

Download the files from the Hortonworks website:
http://hortonworks.com/products/hdp-windows/
Version 1.3

Click the link to begin the download. Unzip
the file, creates a folder:

See the text file "clusterproperties.txt"

Install and load Hyper-V (Windows 8). Create
a new VM. Load Windows 2012 Server.

Start the Server: Be sure to Create a Network
Adapter, I created an "Internal" adapter:

Then set the network configuration (Version
4):

Next I copied the files up to the VM Server. Then
begin the install. Using the Hortonworks page as a
reference:

Pre-requisites
• Next open the Hortonworks page to view the pre-requisites for the install...
•
• http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win1.3.0/bk_installing_hdp_for_windows/content/win-chap2-singlenode.html
•
• http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win1.3.0/bk_installing_hdp_for_windows/content/win-getting-ready-2-3-1.html
•
• Download Python:
•
• http://www.python.org/download/

Create a folder on the VM, I named it
HWHadoop13:

Copy the Python install to the VM as well,
and update the Path variable...

Open PowerShell as Administrator. Rewrite
the line of code in PowerShell...Execute...
Python 2.7.5
*** MESSAGE TO READER ***
Be sure to add the Python executable path to the Environment Variable "PATH"...
Use the following instructions to manually install Python in your local environment:

1.Download Python from here to the workspace directory.
2.Update the PATH environment variable. Using Administrator privileges. From the Powershell window,
execute the following commands as Administrator user:
msiexec /qn /norestart /log %WORKSPACE%python-2.7.5.log /i %WORKSPACE%python-2.7.5.msi setx PATH "$env:path;C:Python27" /m

o%WORKSPACE%

o$env

is the full workspace directory path.

is the Environment setting for your cluster.

where

Note
Important

Ensure the downloaded Python MSI name
matches python-2.7.5.msi. If not, change the
above command to match the MSI file name.

Next download the C++ 2010 Redistributable
Package...

Copy the file to the HWHadoop (Your home
directory for Hadoop) folder...

Type this in the PowerShell command line...

Microsoft Visual C++ 2010 Redistributable
Package (64-bit)
1.Use the instructions provided here to download Microsoft Visual C++ 2010
Redistributable Package (64-bit) to the workspace directory.
2.Execute the following command from Powershell with Administrator privileges:
%WORKSPACE%vcredist_x64.exe /q /norestart

For example:
C:prereqsvcredist_x64.exe /q /norestart

Now, download the Microsoft Framework...
Microsoft.NET framework 4.0

*** MESSAGE TO READER ***
Be sure to connected to the internet, because it has to pull some files off the web,
if you're not connected, the install will fail...
1.Use the instructions provided here to download Microsoft.NET framework 4.0 to the workspace directory.
2.Execute the following command from Powershell with Administrator privileges:
%WORKSPACE%slavesetupdotNetFx40_Full_setup.exe /q /norestart /log %WORKSPACE%/dotNetFx40_Full_setup.exe

And now for the JDK:
• JDK 6.31 or higher

• *** MESSAGE TO READER ***
• During the installation process, it threw an error. Turns out you can
not have spaces in the path for JAVA_HOME. So uninstall and reinstall to new directory, i.e. C:Java instead of C:Program Files...

Use the instructions provided below to manually
install JDK to the workspace directory:
1.Check the version. From a command shell or Powershell window, type:
java -version

2.(Optional): Uninstall the Java package if the JDK version is less than v1.6 update 31.
3.Go to Oracle Java SE 6 Downloads page and accept the license.
Download the JDK installer to the workspace directory.

Note
Important
Ensure that no whitespace characters are present
in the installation directory's path.
For example, C:Program Files is not allowed.

Next
From Powershell with Administrator privileges, execute the following commands:

%WORKSPACE%jdk-6u31-windows-x64.exe /qn /norestart /log %WORKSPACE%jdk-6u31-windows-x64.log INSTALLDIR=C:javajdk1.6.0_31 setx JAVA_HOME
"C:javajdk1.6.0_31" /m where %WORKSPACE%

is the full workspace directory path.

Note
Important
Ensure the downloaded JDK .exe file's name matches with jdk-6u31-windows-x64.exe. If not, change the above
command to match the EXE file name.
For example:
C:prereqsjdk-6u31-windows-x64.exe /qn /norestart/log C:prereqsjdk-6u31-windows-x64.log
INSTALLDIR=C:javajdk1.6.0_31

Oracle
http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archive-downloads-javase6-419409.html#jdk-6u31-oth-JPR
Only problem is you have to have an Oracle account or you must create one.
execute the Power Shell command...

http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/bk_installing_hdp_for_windows/content/win-chap2-singlenode.html

Java_Home path
• After the pre-requisites are loaded, Python, DotNet, C++
Redistributables, Oracle JDK, you are now ready to proceed.
First, you'll want to set the JAVA_HOME path in the Environmental
Variables:

Bug
• Please keep in mind, there is a bug here, you may not have a "SPACE"
in your path, so you are advised to change the path to something like
this, after you re-install the Java JDK.

Next, set the PATH to include the Python
executable...

You will also want to set the HOSTS file to
translate the DNS from IP to Server name:

From the DOS prompt type hostname to
obtain your hostname:

Open the HOSTS file in Notepad and apply the
necessary change,

Now you'll want to Open all Ports:

Next
• Next you want to modify your ClientProperties.txt file, replace the
generic info with actual values, I believe it worked better with IP
Address rather than HostName... however, the screen capture had
the HostName...

And finally, begin the install of Hortonworks
Hadoop 1.3 for Windows:

Folders
• You will need to add some folders to you C: as you progress, I
experienced many errors and had to add the folders each time, here's
a view of some of the folder structure (not complete):

After some trial and error, we have
successfully loaded the application:

Workaround
• Mine failed here, and it turns out the HDFS was never formatted so to
help you out here's the article that explains how to format the HDFS
drive:
• http://hortonworks.com/community/forums/topic/namenodecannot-be-started-after-successful-hdp-1-3-installation/
• WORKAROUND:
1. Open the “Hadoop Command Line” Command Prompt shortcut.
2. Run the following command that sets up the NameNode
directories: “hadoop namenode -format”

As you can see here, the list of Services, you
may have to manually start the ones which did
not start automatically:

Here's another view of the C: folder structure:

And here's the Task/Job tracker web page:

And lastly, the working file system web page:

And here's the shortcuts on the desktop:

Finished
• And that concludes this presentation.
• Happy Hadooping~!

Jonathan Bloom
Current Position:
Senior BI Consultant
• Twitter:
• @SQLJon
• Linked-in:
• http://www.linkedin.com/BloomConsultintBI
• Email:
• JBloom@agilebay.com

Installing Hortonworks Hadoop for Windows

More Related Content

What's hot

Similar to Installing Hortonworks Hadoop for Windows

More from Jonathan Bloom

Recently uploaded

Installing Hortonworks Hadoop for Windows