Login to your Microsoft Azure account and got the Portal dashboard Click on MarketPlace
In the search field enter Hortonworks and let it reduce the option to show the HDP 2.5 option and then click that.
Ensure the 2.5 variant is selected on the left and click the Create button at the bottom right.
Enter the name of the host you want to create Choose VM Disk Type of SSD (default) Enter a username – I’ve chosen jeff Choose either SSH Key or Password – I’ve not shown mine but I use a public/private SSH Key Pair Leave Subscription set to Free Trial (default) Choose Create New Resource Group and give it a name Choose a Location
Choose a Machine type – I chose DS3_V2 Works out at about 26p per hour Click Select
Use Managed Disks (Non Default) for simplicity Leave the defaults for all other settings and click OK
Read the summary and click OK if you are happy
Accept the terms by checking the check box and click Purchase.
The VM will go off to create and you go back to the Portal page You can see that the VM is “deploying”, i.e. Azure is building it Eventually it finishes building – takes a couple of minutes
After deploying it shows as “Running”
Click on the VM and it takes you to the Summary page
If it’s refreshed, from the Portal page Click on the Public IP Address from the Resources box, otherwise click on the VM, then the IP address, then Configuration. Set a DNS Name Label (the name you can use to access your VM on the internet) The Fully Qualified Domain Name is the Domain Name Label you choose with the suffix shown on the right hand side, e.g. oramosshdp25sandbox.ukwest.cloudapp.azure.com
Navigate to Inbound Security Rules which is under the Network Security Group under the VM Notice the only rule that exists allows inbound SSH traffic. Click on Inbound security rule
Click +Add in the top left to add a new rule to allow inbound traffic on port 8888 (for accessing HDP) Enter a name (allow-8888) and Port Range (8888) Click on OK to create the rule
Now test you can access the site using your favourite terminal (I use MobaXterm) firstname.lastname@example.org In MobaXterm I set it up to use my private SSH key and the name of the user account (jeff) I specified earlier so it just logs in automatically without any further ado.
In your SSH session follow these instructions to firstly set a new password for root – the default password is hadoop. Next as root, run ambari-admin-password-reset to enter a password for the ambari admin user (admin).
Now navigate to the Fully Qualified Domain Name, port 8888. Fill out your details and hit Submit.
Once you’ve entered your details it presents you with this screen Click on the Launch Dashboard on the left hand side which brings up the Ambari login page Log in as raj_ops / raj_ops
Enter the admin user (admin) and the password you set a few moments ago with ambari-admin-password-reset. Alternatively use raj_ops / raj_ops which is the default Operator user already setup in the sandbox. Press Sign In
That’s it…we’re up and running with Hortonworks Data Platform 2.5 in the Microsoft Azure cloud.
This time, click on the Click Links on the right hand side which brings up the jump page for various tools
Hover over SSH Client. Navigate to port 4200 on the main Fully Qualified Domain Name: http://oramosshdp25sandbox.ukwest.cloudapp.azure.com:4200/
Ambari – Systems Management environment for HortonWorks Atlas – Metadata management Falcon – Data lifecycle management – replication and lineage Ranger – Security Zeppelin – Web based notebook for ingestion and exploration
Login using root and the password you have previously set.
Login to hdfs user with su – hdfs Put out the hdfs admin report: hdfs dfsadmin -report
Horton works hdp 2.5 sandpit on microsoft azure v0 5
HDP 2.5 SANDBOX
VM ON MICROSOFT
Installation Of Hortonworks HDP 2.5 on Microsoft Azure
Set Firewall Rules
Working with “big data” long before it became trendy!
Extensive Oracle background
25 years experience from Oracle 5 onwards
Former presenter UKOUG
Currently Supplying Services To E.On UK
Lead Technical Architect role
75TB Relational Enterprise Data Warehouse
Support for several other European warehouses
Microsoft Cloud platform
2nd largest by share, behind Amazon (AWS)
A collection of cloud services
Most trusted cloud (compliance certifications)
More geographical regions than Amazon
Microsoft, Oracle, SAP…
Open Source tools and technologies
Linux, Docker, Chef, Puppet, GitLab, WordPress…
Windows or Linux OS
Big Data, Machine Learning, Blockchain, IoT, Microservices
IAAS, PAAS, DBAAS, SAAS
Commercial Hadoop Distribution
One of the major distributions
Partnership with Microsoft to deliver on Azure (Cloud)
HORTONWORKS HDP SANDBOX
Useful for test and learn
A Microsoft Azure account
I used the free month trial which gives £125 credit
An SSH Client
A Web Browser
Optional (Not really)
A generated public/private key pair