Real Time Big Data Framework - Sharath Hosagrahar Lead Architect, Red Hat Global Partners & Alliances Organisation

1. Real Time Big Data Framework
Sharath Hosagrahar
Lead Architect, Red Hat Global Partners & Alliances Organisation
8 October 2015

4. Source: http://blogs.ifsworld.com/2013/02/how-will-big-data-influence-your-finance-team/
Simple Definition
Big data is a collection of data
from traditional and digital
sources inside and outside your
company that represents a
source for ongoing discovery and
analysis

5. TURN BIG DATA
INTO
ACTIONABLE
BUSINESS
INSIGHTS

6. • Collected from various sources
• Analyze and act on data separately
• Using past data to make strategic
decisions
• Batch processing driven
BIG DATA AT REST

7. • Similar collection process as big data
at rest
• Analytics occur in real time as the
event happens
• Common in Telco e.g. analyzing mobile
phone subscriber usage patterns
• Complex Event Processing with low
latency
BIG DATA IN MOTION

8. • Real time data analysis and as a result,
real time action
• Captured data can be stored in a big-data
store for batch analysis
• Batch analysis to identify and model
customer segments
• Real time analysis to act on an identified
customer segment
• Combining batch and real time analysis
provides maximum actionable business
insights
COMBINING THE TWO

9. RED HAT REAL TIME BIG DATA
FRAMEWORK

10. • Analyze large quantities of stream data in real time
• Scale out architecture
• A flexible analytical approach using JBoss BRMS
• Virtual integration with structured and unstructured
data and connects to Internet of Things (IoT) devices
• Open Source & Standards Based
• Proven Success
HIGHLIGHTS
Agile business insights and actions through real time analysis and integration of
data in motion and data at rest

11. • Real Time Marketing / Promotions
– Upsell/cross-sell
• Social media analysis
– Trends and News
• Location Based Services (LBS)
• Network Channel Analysis
ACTUAL CUSTOMER USE CASES

12. HIGH LEVEL SOLUTION ARCHITECTURE

13. • Distributed execution of Map-Reduce workloads based on data
from various sources performed within JBoss Data Grid
– Faster distributed processing in memory and within network
– More complex processing scenarios can be achieved by combining
this workload with the decision management capabilities within JBoss
BRMS
FEATURE 1: REAL TIME MAP REDUCE
Real-Time Processing Short Cycle Execution
Keyword
extraction
Keyword
extraction
Keyword
extraction
Sum
Sum
Sum
Relay
receive
receive
receive morphological
analysis
morphological
analysis
morphological
analysis
Watch the
word
ranking
Rule
matching

14. • Modify decision rules in real time
• Accumulate decision knowledge as rules and evolve
into an intelligent operational decision engine
• Short Cycle Execution
FEATURE 2: REAL TIME DECISIONING
Real-Time Processing Short Cycle Execution
Keyword
extraction
Keyword
extraction
Keyword
extraction
Sum
Sum
Sum
Relay
receive
receive
receive morphological
analysis
morphological
analysis
morphological
analysis
Watch the
word
ranking
Rule
matching

15. • Build virtual data views using JBoss Data
Virtualization
– Integrate real time analysis of JBoss Data Grid with
master information from RDBMS via JBoss Data
Virtualization
– Analyst can view information via dashboard or existing
business intelligence or custom applications
– Developers/Administrators can build virtual tables
with an IDE-based tool or standard SQL notations
FEATURE 3: REAL TIME BUSINESS INSIGHTS

16. FEATURE 3: REAL TIME BUSINESS INSIGHTS
Data
Virtualization
RDBMS
JBoss Data Virtualization
RDBMS
JBoss Data Grid
JBoss Data Grid
Deploy
Data Integration/
Virtualization

17. RED HAT JBOSS MIDDLEWARE COMPONENTS

18. RED HAT
CONSULTING APPROACH

19. RED HAT METHODOLOGY

20. RED HAT CONSULTING SERVICES

21. CALL CENTER SYSTEM (AS-IS)
IVR
call
call
call
call
call
call
Tele Operator
End User
Call Log
Call LogCall Log
Call Log
Call Log
Call LogCall Log
Call Log
Call Log
Call Log
Call Log
Call Log
Call Log
Call Log
Automatic
input
input
input
Write
Program Hadoop
Batch Platform
Report
Manager
Analyze Report
per 1 day
Call Center System

22. CALL CENTER SYSTEM (TO-BE)
IVR
call
call
call
call
call
call
Tele Operator
End User
Call Log
Call LogCall Log
Call Log
Call Log
Call LogCall Log
Call Log
Call Log
Call Log
Call Log
Call Log
Call Log
Call Log
Automatic
input
input
input
Write
Program Hadoop
Batch Platform
Report
Manager
Analyze Report
per 1 day
Call Center System
JBoss Data Grid
JBoss BRMS
JBoss EAP
JBoss
DV
Alert over threshold
Hive
Real Time Big Data Framework

23. Thank you

24. Red Hat Forum 2015
Energize Your Enterprise