Apache Whirr
●

What is it ?

●

How does it work?

●

The Cloud

●

Architecture

●

Use with Hadoop

www.semtech-solutio...
Apache Whirr – What is it ?
●

A library based cloud service system

●

API libraries for cloud providers

●

Choose a con...
Apache Whirr – How does it work ?
●

Libraries provided to offer high level API

●

Based on JClouds

●

Recipe based appr...
Apache Whirr – How does it work ?
●

Automatically start instances on the Cloud

●

Configure and start Hadoop

●

Add app...
Apache Whirr – Why go virtual ?
●

Whirr gives independence from Cloud vendor

●

Makes it easier to move vendors later

●...
Apache Whirr – Supported
●

What cloud suppliers are available
–
–

●

Amazon EC2
Rackspace Cloud Services

What services ...
Apache Whirr – Example config
Example Whirr configuration - .whirr/credentials
whirr.provider=aws-ec2
whirr.identity=your-...
Contact Us
●

Feel free to contact us at
–

www.semtech-solutions.co.nz

–

info@semtech-solutions.co.nz

●

We offer IT p...
Upcoming SlideShare
Loading in …5
×

An introduction to Apache Whirr

803 views

Published on

A short introduction to Apache Whirr. What is it and how does it relate to the cloud ?
How can it be used with Hadoop ?

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
803
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

An introduction to Apache Whirr

  1. 1. Apache Whirr ● What is it ? ● How does it work? ● The Cloud ● Architecture ● Use with Hadoop www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  2. 2. Apache Whirr – What is it ? ● A library based cloud service system ● API libraries for cloud providers ● Choose a configuration file to define a cluster ● High level interaction ● Service based on roles www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  3. 3. Apache Whirr – How does it work ? ● Libraries provided to offer high level API ● Based on JClouds ● Recipe based approach for cloud providers i.e. EC2 ● It is cloud neutral ● Create clusters as you need them for – Dev / test etc www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  4. 4. Apache Whirr – How does it work ? ● Automatically start instances on the Cloud ● Configure and start Hadoop ● Add applications like – Hive – Hbase – Yarn / MapReduce www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  5. 5. Apache Whirr – Why go virtual ? ● Whirr gives independence from Cloud vendor ● Makes it easier to move vendors later ● Save money by only using what you need ● Expand the cluster as demand requires ● Reduce the cluster when possible ● Compress data as much as possible to reduce costs ● Virtual cost < physical cost ( should be ) until – Data sizes in high Tbyte – low Pbyte range www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  6. 6. Apache Whirr – Supported ● What cloud suppliers are available – – ● Amazon EC2 Rackspace Cloud Services What services do they support ? – Cassandra – Hadoop – Zoo Keeper – Hbase – Elastic Search – Voldemort – Hama www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  7. 7. Apache Whirr – Example config Example Whirr configuration - .whirr/credentials whirr.provider=aws-ec2 whirr.identity=your-aws-key whirr.credential=your-aws-secret Whirr Hadoop configuration - hadoop.properties whirr.cluster-name=myhadoopcluster whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker whirr.provider=aws-ec2 Start Whirr whirr launch-cluster --config hadoop.properties www.semtech-solutions.co.nz info@semtech-solutions.co.nz
  8. 8. Contact Us ● Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz ● We offer IT project consultancy ● We are happy to hear about your problems ● You can just pay for those hours that you need ● To solve your problems

×