The document discusses real-time stream processing. It covers topics like Storm, NoSQL, and Hadoop. It also outlines an agenda that includes a real-time analysis of tweets use case, an Ambari demo, and a real-time stream processing demo. The document discusses how stream processing solutions are designed to handle high volumes of data in real-time in a scalable, fault-tolerant way to enable analysis of data in motion.

1. ì
Real-time stream processing
15/11/15 Alain Douangpraseuth

2. $(whoami)
ì Alain Douangpraseuth
ì So$ware Consultant – France
fr.linkedin.com/in/alain-douangpraseuth-b5577972
@douangpraseuth
alain.douangpraseuth@gmail.com
15/11/15 Alain Douangpraseuth

3. Agenda
ì Real-Dme stream processing
ì Storm, NoSQL and Hadoop
ì Hortonworks Data PlaLorm
ì Use case « Real-Dme analysis of Tweets »
ì Ambari Demo
ì Real-Dme stream processing Demo
ì Q&A
15/11/15 Alain Douangpraseuth

4. Real-time stream processing
ì Big Data
ì Tons of informaDon arriving every second
ì Processing big volumes of data is not enough
ì Fast Data
ì Data has to be processed fast
ì React to changing business condiDons
15/11/15 Alain Douangpraseuth

5. Real-time stream processing
ì Basics concepts
ì Stream Processing
ì Streaming AnalyDcs
ì Stream processing soluDons
ì Designed to handle high volume in real Dme
ì Scalable, highly available and fault tolerant
architecture
ì Analysis of data in moDon
15/11/15 Alain Douangpraseuth

6. Real-time stream processing
ì Domains
ì Trading
ì Fraud detecDon
ì System monitoring
ì Many other examples
15/11/15 Alain Douangpraseuth

7. Storm, NoSQL and Hadoop
ì Storm
ì Is a highly distributed realDme computaDon system
ì NoSQL
ì ElasDcsearch
ì VisualizaDon tool, Kibana
ì Hadoop
ì HDFS
ì YARN+MapReduce2
15/11/15 Alain Douangpraseuth

8. Hortonworks Data Platform
15/11/15 Alain Douangpraseuth

9. Use case « Real-time analysis of Tweets »
5-node Hadoop Cluster
Twi_er Data
YARN + MapReduce2
Storm
Zookeeper
HDFS
Log Data …
Ambari
SOURCES REAL-TIME
VISUALIZATION
NoSQL
REAL-TIME
INDEX
REAL-TIME
PROCESS
Hive
SQL
VISUALIZATION
15/11/15 Alain Douangpraseuth

10. Ambari Demo
ì 5-node Hadoop cluster
ì HDP 2.3 with Ambari 2.1
ì Cloud - EC2 with RHEL 7.1
15/11/15 Alain Douangpraseuth

11. Real-time stream processing Demo
15/11/15 Alain Douangpraseuth

12. Q&A
15/11/15 Alain Douangpraseuth