Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
L’analyse en temps réel
de Big Data
 Le monitoring de flux
par l’exemple
#DevoxxMA #BigData @a_bouchama
Abdellatif BOUCHA...
Abdellatif BOUCHAMA
@a_bouchama
• Middleware architect, and
passionate about the new
technology: Big Data,IoT and
contribu...
Agenda
Big Data in 2015:Trends & Statistics
The emergence of real time
Use Case: Flow Activity Monitoring
Demo
@a_bouchama...
What is Big Data?
Big Data
Velocity
30KB/s --> 30GB/s
· Batch
· Real-Time
· Streaming
Volume
1-2 Terabytes  ∞
Variety
· S...
Big Data Expectations:
Things That Excite Executives About Big Data
It’s the “next oil.” from the Ginni Rometty, CEO of IB...
Big Data view by Gartner:
@a_bouchama#DevoxxMA #BigData
Big Data in 2015:
75% of Companies Are Investing or Planning to Invest in Big Data in the NextTwoYears
Goals for Big Data ...
The emergence of Real-Time
Low
Pure Batch
Operationalizing
Near RealTime
Or
Interactive
Analytics
High
RealTime
Analytics
...
Use case: Flow Activity
Monitoring
@a_bouchama#DevoxxMA #BigData
ESB
ESB
ESB
Objectives
Monitor all data flows which goes through the
different layers (transport, mediation, ..etc).
Analyze and centr...
Constraints & Requirements
Constraints
• Non intrusive system
• No modification on business flows.
• We can plug it and un...
Architecture
CH
ESB
US
ESB
FR
ESB
ElasticSearch
Why Elasticsearch stack?
Open source
Easy to deploy
Distributed
Linear scalability
REST Interface
Kibana & Logstash to com...
Data flow and constraints
Collect
JMX
Store & transport
Transform & Access
Modelize & Analyze
Visualize & Predict
JMS
DemoTime!
Be prepared for it to fail, because demos always do
@a_bouchama#DevoxxMA #BigData
What’s Next ?
• Enrichment analysis, after adding business
information in the logs
• Integration of backend and frontend
a...
THANKS FOR LISTENING
Any Questions?
#DevoxxMA #BigData @a_bouchama
Upcoming SlideShare
Loading in …5
×

Analyse en temps réel de BigData par l'exemple

1,606 views

Published on

Tous les projets stratégiques ont besoin de gestion et de surveillance, et les questions qui se posent souvent:

1. Comment puis-je visualiser la santé de ma plate-forme?
2. Comment puis-je analyser les données de mes applications? 3. Comment puis-je visualiser la performance de l'entreprise et la part des objectifs atteints?

Dans cette présentation nous allons voir comment nous pouvons répondre à ces questions et donner des tableaux de bord spécifiques fondés sur un ensemble de produits open source qui permettent de centraliser et d'analyser des données de votre SI en temps réel.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Analyse en temps réel de BigData par l'exemple

  1. 1. L’analyse en temps réel de Big Data  Le monitoring de flux par l’exemple #DevoxxMA #BigData @a_bouchama Abdellatif BOUCHAMA
  2. 2. Abdellatif BOUCHAMA @a_bouchama • Middleware architect, and passionate about the new technology: Big Data,IoT and contributor in the Open source • Co-founder of BusHorn.com @a_bouchama#DevoxxMA #BigData
  3. 3. Agenda Big Data in 2015:Trends & Statistics The emergence of real time Use Case: Flow Activity Monitoring Demo @a_bouchama#DevoxxMA #BigData
  4. 4. What is Big Data? Big Data Velocity 30KB/s --> 30GB/s · Batch · Real-Time · Streaming Volume 1-2 Terabytes  ∞ Variety · Structured · Semi-Structured · UnStructured
  5. 5. Big Data Expectations: Things That Excite Executives About Big Data It’s the “next oil.” from the Ginni Rometty, CEO of IBM The potential to revolutionize industries, and change business models (e.g., Uber and Airbnb), with what we learn from the data. The ability to provide real-time graphing solutions for data relationships. Real-time operational and business data allows people to make decisions faster thereby saving significant money. The opportunities it gives us to help clients solve real business problems. https://dzone.com/articles/14-things-that-excite-executives-about-big-data?oid=big_data
  6. 6. Big Data view by Gartner: @a_bouchama#DevoxxMA #BigData
  7. 7. Big Data in 2015: 75% of Companies Are Investing or Planning to Invest in Big Data in the NextTwoYears Goals for Big Data initiatives: 1. Enhancing the customer experience 2.Streamlining existing processes 3.Achieving more targeted marketing and reducing costs Last year, 37% of big data projects were initiated by the CIO, while 25% were initiated by business unit heads. In 2015, the roles are nearly tied, at 32% and 31 %, respectively. http://www.gartner.com/newsroom/id/3130817@a_bouchama#DevoxxMA #BigData
  8. 8. The emergence of Real-Time Low Pure Batch Operationalizing Near RealTime Or Interactive Analytics High RealTime Analytics • Right timeReal time • Smart dataBig Data « It is the right time for Real Time to use Big Data as a Smart Data »
  9. 9. Use case: Flow Activity Monitoring @a_bouchama#DevoxxMA #BigData
  10. 10. ESB ESB ESB
  11. 11. Objectives Monitor all data flows which goes through the different layers (transport, mediation, ..etc). Analyze and centralize data Perform the correlation between the functional and technical data Provide a centralized view in real time. Deals Integrate all data flows, analyze them and correlate them quickly. Manage a very high data rate, near millions of events Manage a number of messages per second @a_bouchama#DevoxxMA #BigData
  12. 12. Constraints & Requirements Constraints • Non intrusive system • No modification on business flows. • We can plug it and unplug it easily. Requirements • System cost should be mastered and adaptable • System automatized • Measurement @a_bouchama#DevoxxMA #BigData
  13. 13. Architecture CH ESB US ESB FR ESB ElasticSearch
  14. 14. Why Elasticsearch stack? Open source Easy to deploy Distributed Linear scalability REST Interface Kibana & Logstash to complete the Landscape @a_bouchama#DevoxxMA #BigData
  15. 15. Data flow and constraints Collect JMX Store & transport Transform & Access Modelize & Analyze Visualize & Predict JMS
  16. 16. DemoTime! Be prepared for it to fail, because demos always do @a_bouchama#DevoxxMA #BigData
  17. 17. What’s Next ? • Enrichment analysis, after adding business information in the logs • Integration of backend and frontend applications • Audit @a_bouchama#DevoxxMA #BigData
  18. 18. THANKS FOR LISTENING Any Questions? #DevoxxMA #BigData @a_bouchama

×