Hadoop for the disillusioned

1,190 views

Published on

Published in: Technology, News & Politics
  • Be the first to comment

Hadoop for the disillusioned

  1. 1. Hadoop for the disillusioned Steve Watt, Red Hat CC flickr rubenswieringa @wattsteve
  2. 2. @wattsteve
  3. 3. Wired Magazine - July 2008 @wattsteve
  4. 4. Hadoop in 2013 Platform Layers Technologies Computational Runtimes YARN, GiRAPH, MapReduce, HBase, Phoenix, Spark/BDAS, Drill, Impala, Stinger & more FileSystems Azure, CassandraFS, CephFS, CleverSafe, GlusterFS, GridGain, HDFS, Lustre MapR FS, S3, SWIFT, Quantcast FS, Symantec VCFS & more Infrastructures System on a Chip, x86, Virtualization and Cloud Distributions Cloudera, Hortonworks, IBM, Intel, MapR, WanDisco CC flickr lowfatbrains @wattsteve
  5. 5. Source: Gartner Hype Cycle @wattsteve
  6. 6. Your data is growing beyond your ability to manage & query it CC flickr kakadu @wattsteve
  7. 7. Save money when asking the same questions of your data CC flickr martijnsnels @wattsteve
  8. 8. Hadoop Customer, “Great, but now what?” Innovators Early Adopters Early Majority Late Majority Laggards CHASM Geoffrey Moore’s Technology Adoption Lifecycle @wattsteve
  9. 9. new and build data products CC flickr cbcastro @wattsteve
  10. 10.      Ask your domain experts and LOB folks what unanswered questions they have Where can you get the data you need to answer that question? (domain experts should know where to get it) Some of this data may be outside your organization (Social Media, Sensor Data, Data brokerages/Marketplaces, Web Pages) and some of it may be inside. If the data for the query doesn’t exist, figure out how to instrument or gather it. Pair your domain experts with your data engineers so they can work out how to obtain and massage the data given the types of queries desired CC flickr birdwatcher63 @wattsteve
  11. 11. • Building data products is a similar exercise except that it involves typical product planning, such as identifying a market. • This is also a great way for an organization to explore what assets they have within their data CC flickr syume @wattsteve
  12. 12. Mapping the night sky CC flickr bobfamiliar @wattsteve
  13. 13. Analyzing farm soil content to predict human conflict CC flickr oxfam @wattsteve
  14. 14. Crisis Management for the Chilean Earthquake CC flickr flodigrip @wattsteve
  15. 15. Thanks for listening Steve Watt swatt@redhat.com @wattsteve

×