실시간 인벤트 처리

  • 598 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
598
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
6
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. 실시간 이벤트 처리 문병원byeongweon.moon@redduck.com 레드덕
  • 2. 실시간 이벤트 처리
  • 3. 솔루션EsperApache S4 비슷함Twitter StormHStreaming
  • 4. Event StreamProcessingComplex EventProcessing
  • 5. Event StreamProcessing 은 이벤트 스트림의 빠른 처리와 단순한 산술계산에 초첨을 두고 있다.
  • 6. Complex Event Processing은 패턴 분석에 주안점을 둔데이터 분석을 포함하고 있다
  • 7. Esper• ESP/CEP with expressive Event Processing Language: continuous queries, aggregation, joins, causality and missing events, joins to historical data, output flow control...• High throughput, low latency• Standalone or embedded, open API• Multi platform support
  • 8. Computing Rates Per Feedinsert into TicksPerSecondselect feed, rate(10) as cnt from MarketDataEvent group by feed
  • 9. Detecting a Fall-offselect feed, avg(cnt) as avgCnt, cnt asfeedCnt from TicksPerSecond.win:time(10 seconds) group by feedhaving cnt < avg(cnt) * 0.75
  • 10. StormStream processingContinuous computationDistributed RPC
  • 11. Storm do• Guaranteed message processing• Robust process management• Fault detection and automatic reassignment• Efficient message passing• Local mode and distributed mode
  • 12. TopologyBuilder builder = new TopologyBuilder();builder.setSpout(1, new KestrelSpout("kestrel.backtype.com", 22133, "sentence_queue", new StringScheme()));builder.setBolt(2, new SplitSentence(), 10) .shuffleGrouping(1);builder.setBolt(3, new WordCount(), 20) .fieldsGrouping(2, new Fields("word"));
  • 13. public class SplitSentence implements IBasicBolt { public void execute(Tuple tuple, BasicOutputCollector collector) { String sentence = tuple.getString(0); for(String word: sentence.split(" ")) { collector.emit(new Values(word)); } } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("word")); }}
  • 14. public class WordCount implements IBasicBolt { private Map<String, Integer> _counts = new HashMap<String,Integer>(); public void execute(Tuple tuple, BasicOutputCollector collector) { String word = tuple.getString(0); int count; if(_counts.containsKey(word)) { count = _counts.get(word); } else { count = 0; } count++; _counts.put(word, count); collector.emit(new Values(word, count)); } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("word", "count")); }}