Recommended
Recommended
More Related Content
Similar to Storm - Altamira University Presentation
Similar to Storm - Altamira University Presentation (20)
Recently uploaded
Recently uploaded (20)
Storm - Altamira University Presentation
- 1. Apache Storm A distributed, real-time computation system Ryan Lanman Some content borrowed from Nathan Marz’ Presentation of a similar name
- 2. Objectives 1.Their Motivation 2.Our Motivation 3.Storm Basics 4.Demo
- 3. Their Motivation How Storm Came To Be
- 10. What They Wanted • • • • • • Guaranteed data processing Horizontal scalability Fault-tolerance No intermediate message brokers! Higher level abstraction than message passing “Just works”
- 11. Our Motivation Why We Chose Storm ^
- 12. Lumify Ingest Raw Data Text Extraction Entity Extraction Text Highlighting Location Extraction Full Text Indexing
- 13. Issues • • • • • No Reducers High DB Read/Writes Batch-style processing M/R Overhead Zero Fault Tolerance
- 14. What We Really Wanted • Distributed, Stream-type Processing • Simple Logical DAG • Better Fault Tolerance
- 15. Storm Ingest Workflow Documents Raw Data Content Sorter Video Images Text Extraction Video Frame Splitting Image Text Extraction Video Frame Text Extraction Text …
- 16. Storm Basics What the heck’s a Topology?
- 22. Tuples • Single unit of data in Storm • Examples – Tweet – User Activity Log Entry – File Info
- 23. Streams Tuple Tuple Tuple Tuple Tuple An unbound sequence of Tuples Tuple Tuple
- 25. Bolts Tuple Process input streams to create new streams Tuple
- 26. Examples Spout Examples • HDFS Filesystem Spout • Kafka Queue Spout Bolt Examples • Filtering • Aggregation • DB Operations
- 28. Demo