Hadoop in a Box

3,805 views
3,650 views

Published on

You can run Hadoop on pretty much any hardware out there, as it makes very few assumptions about its environment. In this talk I present the modular and highly energy-efficient storage server which is currently being developed by the OpenStoragePod project.

I explain the motivations for basing our design on large numbers of 2.5" laptop disks, and discuss why you might want to consider using OpenStoragePod as the basis for building your (next) Hadoop cluster.

Hadoop in a Box

  1. 1. Hadoop in a Box
  2. 2. @tlossen
  3. 3. flickr.com/photos/brent_nashville/2228396914
  4. 4. flickr.com/photos/bizkit/2698782365
  5. 5. data CPU
  6. 6. data S.E.T.I. CPU
  7. 7. data Avatar S.E.T.I. CPU
  8. 8. data most others Avatar S.E.T.I. CPU
  9. 9. History
  10. 10. 1956 ed-thelen.org
  11. 11. 1980 en.wikipedia.org
  12. 12. 2010
  13. 13. Open StoragePod
  14. 14. Mission
  15. 15. Return on Byte (ROB) = value to be extracted from byte / cost of storing that byte (Amr Awadallah, Cloudera)
  16. 16. Total Cost of Ownership Cap Ex OpEx
  17. 17. Total Cost of Ownership Cap Ex OpEx
  18. 18. Total Cost of Ownership Cap Ex OpEx
  19. 19. Total Cost of Ownership Cap Ex OpEx
  20. 20. Total Cost of Ownership Cap Ex OpenStoragePod OpEx
  21. 21. “Petascale storage for the rest of us”
  22. 22. Design
  23. 23. 3.5” 2.5” price €80 €180 power 3.4 W idle power 2.8 W weight 330 g volume 195 cm³
  24. 24. 3.5” 2.5” price €80 €180 power 3.4 W 1.7 W idle power 2.8 W 0.75 W weight 330 g 153 g volume 195 cm³ 88 cm³
  25. 25. Node 170 x 170 x 170 mm 100 W (12V) 20 TB (= 20 disks)
  26. 26. Pod 6 nodes, 4U 600 W (220V) 120 TB (= 120 disks)
  27. 27. Rack 10 pods 6 KW 1.2 PB (= 1200 disks)
  28. 28. Container 8 racks 48 KW 9.6 PB (= 9600 disks)
  29. 29. Status
  30. 30. Cube 140 x 140 x 140 mm 15 W (5V) 5 TB (= 5 disks)
  31. 31. Q&A
  32. 32. openstoragepod.org speakerrate.com/talks/3465

×