Storm
Upcoming SlideShare
Loading in...5
×
 

Storm

on

  • 504 views

 

Statistics

Views

Total Views
504
Views on SlideShare
504
Embed Views
0

Actions

Likes
0
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Storm Storm Presentation Transcript

  • Distributed Stream Processing With Storm Petar Kostov March 2013
  • Contents• Concepts• Cluster structure• Parallelism• Reliability• Abstractions – DRCP, Trident• Utils Storm March 2013 #2
  • Concepts• Bolt• Spout• Tuple• Stream Storm March 2013 #3 View slide
  • Cluster structure Storm March 2013 #4 View slide
  • Parallelism Storm March 2013 #5
  • Taming Parallelism• Stream groupings Consumer component Emitter component Task 1 Source Task Task 2 … Task N Storm March 2013 #6
  • Reliability• Tuple ACK-ing – @10K acks/sec the system will fail in 50 000 000 years• Reliable/Unreliable spout Storm March 2013 #7
  • Abstractions: DRPC Storm March 2013 #8
  • Abstractions: Trident• Stateful stream processing• Exactly once semantics• The new way to do DRPC Storm March 2013 #9
  • Abstractions: TridentTridentTopology topology = new TridentTopology();TridentState wordCounts = topology.newStream("spout1", spout).each(new Fields("sentence"), new Split(), new Fields("word")).groupBy(new Fields("word")).persistentAggregate(new MemoryMapState.Factory(), new Count(),new Fields("count")).parallelismHint(6); Storm March 2013 #10
  • Utils• Local mode• storm-deploy• Storm UI Storm March 2013 #11
  • THANK YOU! Storm March 2012 #12