Ironfan: Build a Hadoop                 Cluster in 20 Minutes         Dhruv Bansal            Nathaniel Eliot         @dhr...
Saturday, May 19, 2012
OutlineSaturday, May 19, 2012
Why Ironfan?                                             Outline                    • Why we love Chef                    ...
Why Ironfan?                                                Outline                    • Why we love Chef                 ...
Why Ironfan?                                                Outline                    • Why we love Chef                 ...
Why we love Chef                                Infrastructure as code:                    Version control               T...
Why we love Chef                         Modular, composable units:              MySQL                Nginx              S...
Why we love ChefSaturday, May 19, 2012
Why we don’t love Chef                           Anything is possible                           Nothing is simple         ...
Why we don’t love Chef       Write               Upload       Compile         Converge             Development Machine    ...
Why we don’t love Chef           How do we make @server_ips dynamic?Saturday, May 19, 2012
Enter Ironfan                         We wrote Ironfan toSaturday, May 19, 2012
Enter Ironfan                         We wrote Ironfan to           • Simplify, unify, and standardize our usage of       ...
Enter Ironfan                         We wrote Ironfan to           • Simplify, unify, and standardize our usage of       ...
Enter Ironfan                         We wrote Ironfan to           • Simplify, unify, and standardize our usage of       ...
Enter Ironfan                         We wrote Ironfan to           • Simplify, unify, and standardize our usage of       ...
What does Ironfan do?Saturday, May 19, 2012
What does Ironfan do?                         ChefSaturday, May 19, 2012
What does Ironfan do?                                                        Ironfan                                 Simpl...
What does Ironfan do?Saturday, May 19, 2012
What does Ironfan do?   Dynamic service discovery:Saturday, May 19, 2012
What does Ironfan do?   Dynamic service discovery:Saturday, May 19, 2012
What does Ironfan do?Saturday, May 19, 2012
What does Ironfan do?                                A simple DSL for                                defining clusters    ...
DEMO TIME!Saturday, May 19, 2012
Thanks for coming!          Learn more about what          Ironfan and Infochimps can                  sales@infochimps.co...
Upcoming SlideShare
Loading in...5
×

Ironfan: Build a Hadoop Cluster in 20 minutes

2,479
-1

Published on

The slides from our talk at ChefConf 2012

Published in: Business, Technology

Ironfan: Build a Hadoop Cluster in 20 minutes

  1. 1. Ironfan: Build a Hadoop Cluster in 20 Minutes Dhruv Bansal Nathaniel Eliot @dhruvbansal @temujin9Saturday, May 19, 2012
  2. 2. Saturday, May 19, 2012
  3. 3. OutlineSaturday, May 19, 2012
  4. 4. Why Ironfan? Outline • Why we love Chef • Why we don’t • Why we wrote IronfanSaturday, May 19, 2012
  5. 5. Why Ironfan? Outline • Why we love Chef • Why we don’t • Why we wrote Ironfan Ironfan in Action • Build a Hadoop cluster • Run a Hadoop job • Put that cluster awaySaturday, May 19, 2012
  6. 6. Why Ironfan? Outline • Why we love Chef • Why we don’t • Why we wrote Ironfan Ironfan in Action • Build a Hadoop cluster • Run a Hadoop job • Put that cluster away Ironfan roadmap • Better environments • Cloud agnosticism • Continuous integration • StacksSaturday, May 19, 2012
  7. 7. Why we love Chef Infrastructure as code: Version control Testable Shareable RecapitulableSaturday, May 19, 2012
  8. 8. Why we love Chef Modular, composable units: MySQL Nginx SOLR My ApplicationSaturday, May 19, 2012
  9. 9. Why we love ChefSaturday, May 19, 2012
  10. 10. Why we don’t love Chef Anything is possible Nothing is simple And there’s much repetition (not DRY)Saturday, May 19, 2012
  11. 11. Why we don’t love Chef Write Upload Compile Converge Development Machine Managed Server Hard Coded Dynamic Too much is hard-coded at development/upload time!Saturday, May 19, 2012
  12. 12. Why we don’t love Chef How do we make @server_ips dynamic?Saturday, May 19, 2012
  13. 13. Enter Ironfan We wrote Ironfan toSaturday, May 19, 2012
  14. 14. Enter Ironfan We wrote Ironfan to • Simplify, unify, and standardize our usage of the Chef toolsetSaturday, May 19, 2012
  15. 15. Enter Ironfan We wrote Ironfan to • Simplify, unify, and standardize our usage of the Chef toolset • Build further abstractions on top of ChefSaturday, May 19, 2012
  16. 16. Enter Ironfan We wrote Ironfan to • Simplify, unify, and standardize our usage of the Chef toolset • Build further abstractions on top of Chef • Give us superpowers that Chef doesn’t have yetSaturday, May 19, 2012
  17. 17. Enter Ironfan We wrote Ironfan to • Simplify, unify, and standardize our usage of the Chef toolset • Build further abstractions on top of Chef • Give us superpowers that Chef doesn’t have yet http://github.com/infochimps-labs/ironfanSaturday, May 19, 2012
  18. 18. What does Ironfan do?Saturday, May 19, 2012
  19. 19. What does Ironfan do? ChefSaturday, May 19, 2012
  20. 20. What does Ironfan do? Ironfan Simple helpers in the silverware cookbook abstract common Chef patterns and keep things DRY ChefSaturday, May 19, 2012
  21. 21. What does Ironfan do?Saturday, May 19, 2012
  22. 22. What does Ironfan do? Dynamic service discovery:Saturday, May 19, 2012
  23. 23. What does Ironfan do? Dynamic service discovery:Saturday, May 19, 2012
  24. 24. What does Ironfan do?Saturday, May 19, 2012
  25. 25. What does Ironfan do? A simple DSL for defining clusters of machinesSaturday, May 19, 2012
  26. 26. DEMO TIME!Saturday, May 19, 2012
  27. 27. Thanks for coming! Learn more about what Ironfan and Infochimps can sales@infochimps.com do for you. Come work at a fast- growing, cutting-edge careers@infochimps.com technology company based in Austin, TX. http://github.com/infochimps-labs/ironfan http://github.com/infochimps-labs/wukongSaturday, May 19, 2012
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×