실시간 인벤트 처리
Upcoming SlideShare
Loading in...5
×
 

실시간 인벤트 처리

on

  • 861 views

 

Statistics

Views

Total Views
861
Views on SlideShare
861
Embed Views
0

Actions

Likes
1
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

실시간 인벤트 처리 실시간 인벤트 처리 Presentation Transcript

  • 실시간 이벤트 처리 문병원byeongweon.moon@redduck.com 레드덕
  • 실시간 이벤트 처리
  • 솔루션EsperApache S4 비슷함Twitter StormHStreaming
  • Event StreamProcessingComplex EventProcessing
  • Event StreamProcessing 은 이벤트 스트림의 빠른 처리와 단순한 산술계산에 초첨을 두고 있다.
  • Complex Event Processing은 패턴 분석에 주안점을 둔데이터 분석을 포함하고 있다
  • Esper• ESP/CEP with expressive Event Processing Language: continuous queries, aggregation, joins, causality and missing events, joins to historical data, output flow control...• High throughput, low latency• Standalone or embedded, open API• Multi platform support
  • Computing Rates Per Feedinsert into TicksPerSecondselect feed, rate(10) as cnt from MarketDataEvent group by feed
  • Detecting a Fall-offselect feed, avg(cnt) as avgCnt, cnt asfeedCnt from TicksPerSecond.win:time(10 seconds) group by feedhaving cnt < avg(cnt) * 0.75
  • StormStream processingContinuous computationDistributed RPC
  • Storm do• Guaranteed message processing• Robust process management• Fault detection and automatic reassignment• Efficient message passing• Local mode and distributed mode
  • TopologyBuilder builder = new TopologyBuilder();builder.setSpout(1, new KestrelSpout("kestrel.backtype.com", 22133, "sentence_queue", new StringScheme()));builder.setBolt(2, new SplitSentence(), 10) .shuffleGrouping(1);builder.setBolt(3, new WordCount(), 20) .fieldsGrouping(2, new Fields("word"));
  • public class SplitSentence implements IBasicBolt { public void execute(Tuple tuple, BasicOutputCollector collector) { String sentence = tuple.getString(0); for(String word: sentence.split(" ")) { collector.emit(new Values(word)); } } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("word")); }}
  • public class WordCount implements IBasicBolt { private Map<String, Integer> _counts = new HashMap<String,Integer>(); public void execute(Tuple tuple, BasicOutputCollector collector) { String word = tuple.getString(0); int count; if(_counts.containsKey(word)) { count = _counts.get(word); } else { count = 0; } count++; _counts.put(word, count); collector.emit(new Values(word, count)); } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("word", "count")); }}