Your SlideShare is downloading. ×
Real time analytics with Netty, Storm, Kafka
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Real time analytics with Netty, Storm, Kafka

4,448
views

Published on

Published in: Technology

1 Comment
24 Likes
Statistics
Notes
No Downloads
Views
Total Views
4,448
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
95
Comments
1
Likes
24
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Real-time Analytics with Netty, Apache Kafka and Storm Case study with “lambda architecture” http://nguyentantrieu.info Update: 07/06/2013
  • 2. Agenda 1. Overview Architecture 2. Log HTTP-Handler and producer: Netty 4 3. Kafka 0.8 (Stream Data Log Storage) 4. Storm Analytics Cluster
  • 3. Overview System Architecture
  • 4. Concept Flow
  • 5. Concept Flow JavaScript Tracking Mobile SDK Http Log Server Kafka
  • 6. S2 HTTP Log Server netty framework 4
  • 7. Netty.io Netty is a non-blocking I/O (NIO) client-server framework for the development of Java network applications such as protocol servers and clients. The asynchronous event-driven network application framework and tools are used to simplify network programming such as TCP and UDP socket servers.[2] Netty includes an implementation of the reactor pattern of programming. http://en.wikipedia.org/wiki/Netty_(software) http://nguyentantrieu.info/blog/backend-system-with-netty-io
  • 8. Apache Kafka (version 0.8) https://cwiki.apache.org/confluence/display/KAFKA/Index
  • 9. In Production ● Clustering ( 4 nodes) ● Partitions ○ user-activity: 24 partitions ● Producer and consumer are replication aware
  • 10. Core Producer Class (S2 HTTP Log Server)
  • 11. Storm Analytics Cluster
  • 12. The Storm Topology Kafka Cluster topic: user-activity Tokenizer Bolt Parser Bolt Aggregate Bolt Redis Statistics Bolt Raw Data Save DWH Bolt Kafka Consumer Spout
  • 13. Useful Links ● https://storm.incubator.apache.org ● https://www.youtube.com/watch? v=DRgs8L4AOgc ● https://cwiki.apache. org/confluence/display/KAFKA/Index ● http://nguyentantrieu.info/blog/backend-system- with-netty-io ●

×