Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intro to Apache Storm

252 views

Published on

A quick tour of the basic concepts in Apache Storm. The talk is meant to be accessible to beginners who have no prior experience with distributed computing, Clojure, or Apache Storm.

This talk was given at the September 28, 2015 meeting of Vancouver Functional Programmers, hosted at Unbounce.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Intro to Apache Storm

  1. 1. Intro to Storm David Y. Kay
  2. 2. Who am I? Med Device Skate Instructor“Big names”
  3. 3. Nathan Marz
  4. 4. Why Storm? • Real-time, not batch • Scalable • Fault-tolerant & no data loss • Language-agnostic*
  5. 5. Storm Alternatives
  6. 6. What does it look like?
  7. 7. Example System HDFS Storm Hadoop Twitter
  8. 8. Topology
  9. 9. Example Twitter Topology
  10. 10. Re-used New & Reusable One-off Spout
  11. 11. Terminology • Tuple - An immutable, list-like data structure • Spout - A starting point. Emits tuples. • Bolt - Consumes and emits tuples. • Topology - A deployable DAG of bolts & spouts
  12. 12. Hello World Topology
  13. 13. Hello, Bolt!
  14. 14. Bolt name Output Elements Input Parameters {function body emit tuple to the next bolt “tuple succeeded!”
  15. 15. Hello, Spout!
  16. 16. Spout name Output Elements Input Parameters Called when “tuple succeeded!” {{ Spout object { Storm: “make a new tuple!” emit tuple
  17. 17. Hello, Topology!
  18. 18. Hello World Topology
  19. 19. from (defbolt …) Spout Identifier Bolt Identifier {Spouts {Bolts from (defspout …) from (defspout …) - parameterized params bolt instantiation params Inputs NAMED Inputs
  20. 20. Live Demo
  21. 21. Recap • Spouts are where streams begin • Bolts consume and emit tuples - “links in a chain” • “Lego block” re-use model • Topologies • Wire the bolts and spouts together, then deploy
  22. 22. Storm Cluster
  23. 23. Resources • Udacity: “Real-Time Analytics with Apache Storm” • storm.apache.org • “Apache Storm vs. Spark Streaming” - P. Goetz
  24. 24. Fin @DavidYKay DavidYKay@gmail.com DavidYKay.com

×