Hadoop in the Cloud


Published on

Overview of Public Clou

Published in: Technology, Business

Hadoop in the Cloud

  1. 1. The Elephant in the Room Jim O’Neil Developer Evangelist, Microsoft jim.oneil@microsoft.com  @jimoneil
  2. 2. DIYRoll your own Hadoop cluster…. welcome to DevOps “Isotope” Pallet
  3. 3. AppliancesOracle Big Data Appliance – 18 server / 12 core each / 40Gb Infiniband – Partnering with Cloudera on the distributionGreenplum HD Data Computing Appliance – 18 nodes, 12 core each – Straight up Apache HadoopNetApp Open Solution for Hadoop – Storage arrays only (E2660 and FAS2040) – Partnership with Cloudera
  4. 4. The Elephant in the CloudJim O’NeilDeveloper Evangelist, Microsoftjim.oneil@microsoft.com  @jimoneil
  5. 5. Cloud: a Notional Definition Private Cloud Deployment Models Hybrid Cloud Community Cloud Public Cloud Essential Characteristics Infrastructure as a Service Platform as a Service Broad network Rapid Elasticity accessSoftware as Resource Pooling a Service On-demand self-service Measured service
  6. 6. Hadoop in the CloudGoogle App Engine appengine-mapreduce API (not really Hadoop)Amazon Web Services 66 Public AMIs (including Cloudera) Elastic Map ReduceWindows Azure Hadoop on AzureIBM SmartCloud Infosphere BigInsights
  7. 7. Google App Engine MapreducePipeline ClassExperimental!Mapreduce is an experimental, innovative, and rapidlychanging new feature for App Engine.Unfortunately, being on the bleeding edge means thatwe may make backwards-incompatible changes toMapreduce. We will inform the community when thisfeature is no longer experimental.
  8. 8. Amazon EMRu
  9. 9. Windows Azure http://HadoopOnAzure.comCurrently in Customer Technology PreviewPartnership with Hortonworks Windows updates to Apache JavaScript framework Hive ODBC connector
  10. 10. IBM SmartCloudInfoSphere BigInsights IBM distribution of Hadoop (0.20.2) Jaql query language BigSheets BigInsight Scheduler “Hadoop ecosystem” Hive, Avro, Hbase, Pig, Oozie, Flume
  11. 11. I meant what I said, and I said what I meant.An elephants faithful, one hundred percent. Jim O’Neil Developer Evangelist, Microsoft jim.oneil@microsoft.com  @jimoneil