This document summarizes a webinar on real-world applications of streaming analytics. It discusses case studies of companies in various industries using the StreamAnalytix platform for real-time analytics on large data streams. Examples include classifying 250 million messages per day for an intelligence company and monitoring response times for a healthcare application. The webinar focuses on business problems solved through streaming analytics and the StreamAnalytix product capabilities.

1. WEBINAR
Real World Applications
of Streaming Analytics
Recorded version available at http://bit.ly/1AeGdxM
1 © 2014 Impetus Technologies

2. Recent Webcast Recap– Archived on the
Website
Recorded version available at http://bit.ly/1AeGdxM
2 © 2014 Impetus Technologies
Real-time Streaming
Analytics for
Enterprises based on
Apache Storm
Real-time Streaming
Analytics: Business Value,
Use Cases, and Architectural
Considerations

3. Why rapid growth and
demand for real-time
analytics
Q&A
Recorded version available at http://bit.ly/1AeGdxM
Agenda
3 © 2014 Impetus Technologies
StreamAnalytix –
Product Overview
Real World Case Studies
Business Problem, Solution
Architecture and Outcomes

4. • Big Data Solutions & Services company
• Unique in depth, expertise – started implementing in 2008
• Proven with customer success
• IP and Products
• We deliver - Business Impact from Big Data Solutions
• Technology expertise
• Data Science
• Business Analytics
• Serving Fortune 1000 companies since 1996
• Large-scale and mission critical software platforms
• HQ: Los Gatos, CA; 1500 people
• Offshore operations in 3 cities in India
Recorded version available at http://bit.ly/1AeGdxM
Brief Intro
4 © 2014 Impetus Technologies

5. Drivers for Real-time Streaming Analytics
Fleet Operations & Logistics Security
Mobile Devices and Apps Energy Industry IT Operations
Recorded version available at http://bit.ly/1AeGdxM
5 © 2014 Impetus Technologies

6. Drivers for Real-time Streaming Analytics
You and I :
The ‘CUSTOMER’
Recorded version available at http://bit.ly/1AeGdxM
6 © 2014 Impetus Technologies

7. Drivers for Real-time Streaming Analytics
Recorded version available at http://bit.ly/1AeGdxM
7 © 2014 Impetus Technologies

8. Drivers for Real-time Streaming Analytics
Context
Sensitive service
Recorded version available at http://bit.ly/1AeGdxM
Multi-channel
engagement in
real-time
8 © 2014 Impetus Technologies
Happy
customers,
Loyalty,
Revenue,
Profits, Growth

9. Drivers for Real-time Streaming Analytics
Recorded version available at http://bit.ly/1AeGdxM
Business
Operations
9 © 2014 Impetus Technologies
Business
Analytics
Real-time
Streaming
Analytics

10. Real-time Business Analytics – The “Batch Gap”
The batch workflow is too slow
Views are out of date
Not yet
absorbed. Data absorbed into Batch
Views
Recorded version available at http://bit.ly/1AeGdxM
10 © 2014 Impetus Technologies
Now
Time
Just a few hours of data.

11. Blended View – Historical and NOW
Recorded version available at http://bit.ly/1AeGdxM
t
11 © 2014 Impetus Technologies
now
Hadoop works great back
here
Storm works
here
Blleennddeedd Vviieew

12. Big Data and Fast Data Combined
Batch Layer
Pre-computed
information
Batch re-compute
Speed Layer
Pre-computed
information
Real time
increment
Recorded version available at http://bit.ly/1AeGdxM
All data
All data
12 © 2014 Impetus Technologies
Serving Layer
Batch view
Batch view
Merge
Real time view
Real time view
Incoming
Data Query

13. Poll
Where are you in the process of
implementing real-time streaming analytics?
Recorded version available at http://bit.ly/1AeGdxM
13 © 2014 Impetus Technologies

14. Enterprise Class Real time Streaming Analytics Platform
A Product developed and offered by
Recorded version available at http://bit.ly/1AeGdxM
14 © 2014 Impetus Technologies

15. StreamAnalytix is a software platform that enables enterprises to analyze and
respond to events in real-time at Big Data scale. It is designed to rapidly build
and deploy streaming analytics applications for any industry vertical, any data
format, and any use-case
Recorded version available at http://bit.ly/1AeGdxM
At a Glance
15 © 2014 Impetus Technologies

16. StreamAnalytix Block Diagram
Recorded version available at http://bit.ly/1AeGdxM
16 © 2014 Impetus Technologies

17. Case Studies - Real World Applications
Recorded version available at http://bit.ly/1AeGdxM
17 © 2014 Impetus Technologies

18. Case Study 1 – Intelligence Solutions Company
Basic Schematic Architecture
Recorded version available at http://bit.ly/1AeGdxM
Problem:
18 © 2014 Impetus Technologies
Numerous " non-voice "
communications

19. Case Study 1 – Intelligence Solutions Company
• Classify streaming text in real-time based on topic
• Sentiment Analysis on the stream in real-time
• 250 million messages a day
• Variety: weblogs, chats, emails, tweets etc.
Recorded version available at http://bit.ly/1AeGdxM
• Accuracy
Classification - 99.99%
Sentiment analysis - 80%
19 © 2014 Impetus Technologies
20 Predefined Categories
"Arts_culture_entertainment
" "law_crime_justice"
"disaster_accident"
"economy_finance"
"education"
"environment_weather"
"health" "lifestyle" "politics"
"religion" "science" "society"
"sports" "conflict_war"
"literature" "computing"
"labor" "travel"
"governance_government"
"human_interest"
Problem statement

20. Case Study 1 – Intelligence Solutions Company
Problem statement
• English and Arabic content
• Other languages = “other” (no metadata)
Had CSS and JavaScript files
To be categorized as “scripts”
• Ingest, Store, Index, Query
Metadata and Raw binary data
Petabytes
• Query SLA – On any 4 hour window "cold data"
4 to 5 seconds
ETSI compliant encryption
Recorded version available at http://bit.ly/1AeGdxM
• Data very raw
20 © 2014 Impetus Technologies

21. Case Study 1 – Intelligence Solutions Company
Classification
Recorded version available at http://bit.ly/1AeGdxM
Content
Extraction and
Preprocessing
22 © 2014 Impetus Technologies
Sentiment
Analysis
Tokenization of words based on delimiters (space)
Elimination of all “Stop Words”, non-contributory words
Removal of non-ASCII and Non UTF-8
Models built offline
and scoring online

22. Case Study 1 – Intelligence Solutions Company
Real-time Classification
• 20 categories; Multiple labels if applicable
• Semantic similarity approach based on matrix
• Language independent (with caveats)
• Low Latency achieved by two step process
Recorded version available at http://bit.ly/1AeGdxM
decomposition
-Pre-processing
-Numerical computation
23 © 2014 Impetus Technologies

23. Case Study 1 – Intelligence Solutions Company
Sentiment Analysis
• Dictionary or Lexicon approach; Unsupervised model
• Prepared offline with matrix decomposition
• Polarities assigned to adjectives (+ - 0 )
-Surrounding words could negate, amplify etc.
-Clusters of words treated separately
-Feature extraction possible for distinct sentiment
Recorded version available at http://bit.ly/1AeGdxM
24 © 2014 Impetus Technologies

24. Case Study 1 – Intelligence Solutions Company
Learnings - Analytics
• Language independent technique worked well
• 50-60 documents per topic was sufficient
-Is not 100% tokenizable – no spaces
-Did not hamper accuracy significantly
-Needed language expert to test model (for any foreign language)
Recorded version available at http://bit.ly/1AeGdxM
• Arabic content
25 © 2014 Impetus Technologies

25. Case Study 1 – Intelligence Solutions Company
Learnings - Architecture
• Task lends well to parallelization and scale out
• StreamAnalytix is a good fit – linear scale out
• Event size/ throughput – trade off
• Unique sharding and indexing for query optimization
• Many more types of use-cases possible
Recorded version available at http://bit.ly/1AeGdxM
• Flexible topology
26 © 2014 Impetus Technologies

26. Case Study 2 – Hosted Contact Center
Solution
Recorded version available at http://bit.ly/1AeGdxM
27 © 2014 Impetus Technologies

27. Case Study 2 – Hosted Contact Center
Solution
IVR
Agent Queue
Recorded version available at http://bit.ly/1AeGdxM
28 © 2014 Impetus Technologies

28. Case Study 2 – Hosted Contact Center
Solution
• Proactive –
Business teams
want to understand
dominant call paths
• Lower “Queue” time
Recorded version available at http://bit.ly/1AeGdxM
Problem Statement
• Reactive –
Customer service
complaints on “What
happened to my call ?”
Diagnostics
- Easier
- Faster
29 © 2014 Impetus Technologies
• Proactive -
Abandoned call
analysis
Hang up on
IVR/hold

29. Case Study 2 – Hosted Contact Center
Solution
Technical Requirements
Log Aggregation
• Stream raw log events from
multiple remote servers
• Filter incoming log events – before
storage
• Index/search of log events
Real-Time Dashboard and Alerts
Auto Correlate Logs in
Real-time
• Correlate log events arriving at
different time intervals based on
System ID, Channel ID, Call ID
• Visualize the complete call path
for a particular id
• Ability to show counters on the existing Log Monitoring dashboard. For eg. #of inbound calls per tenant
• SLA based alarms – ability to generate alarms based on SLA threshold values over a moving time window per
Recorded version available at http://bit.ly/1AeGdxM
tenant.
30 © 2014 Impetus Technologies
IVR Dominant
Path

30. Case Study 2 – Hosted Contact Center
Solution
Recorded version available at http://bit.ly/1AeGdxM
31 © 2014 Impetus Technologies

31. Case Study 2 – Hosted Contact Center
Solution
Recorded version available at http://bit.ly/1AeGdxM
32 © 2014 Impetus Technologies

32. Case Study 2 – Hosted Contact Center
Solution
Recorded version available at http://bit.ly/1AeGdxM
33 © 2014 Impetus Technologies

33. Case Study 2 – Hosted Contact Center
Solution
Outcome, Next steps
• Next steps
Recorded version available at http://bit.ly/1AeGdxM
34 © 2014 Impetus Technologies
– Sentiment analysis in real-time
(chat)
– Audio to text: Sentiment
analytics on transcript
– Rich real-time dash-boarding
and live counters
• Successfully solved key problems
– Call log aggregation, indexing and
search
– Real-time call path picture
– Dominant path analytics

34. Case Study 3 – Digital Content Provider
• Scholarly journals, educational, research
content
• Institutional Subscribers – 1000s of users
each
• Business wants real-time visibility and
analytics of customer behavior patterns
Recorded version available at http://bit.ly/1AeGdxM
35 © 2014 Impetus Technologies

35. Case Study 3 – Digital Content Provider
• 10s of millions of events per day
– Clickstream data – complex XML events
– Complex XML events parsed, filtered in real-time
• Recommendation engine
Recorded version available at http://bit.ly/1AeGdxM
Problem Statement
• Real-time ETL
• Clickstream-Analytics:
– Double click detection
– BOT detection
– Upsell/ cross-sell
36 © 2014 Impetus Technologies

36. Case Study 3 – Digital Content Provider
Data Flow and Real-time Pipeline Design
Recorded version available at http://bit.ly/1AeGdxM
37 © 2014 Impetus Technologies

37. Case Study 4 – Web Application SLA Monitoring
• Healthcare insurance exchange software platform
• Server response time to front end application is a key
• Complaints from key customers (potential revenue impact)
• Triggered need for aggressive monitoring and alerting
Recorded version available at http://bit.ly/1AeGdxM
metric
system
38 © 2014 Impetus Technologies

38. Case Study 4 – Web Application SLA Monitoring
Recorded version available at http://bit.ly/1AeGdxM
39 © 2014 Impetus Technologies
• Alert if response
breaches 4
second threshold
• Real-time
counters/
dashboard for a
variety of metrics
• Monthly report
Problem Statement

39. Case Study 4 – Web Application SLA Monitoring
Syslog
Server
Kafka
Server
StreamAnalytix Agent Features
• The agent can publish to multiple destinations
• The agent can send encrypted data (optional)
StreamAnalytix
Real-Time Pipeline
Recorded version available at http://bit.ly/1AeGdxM
Remote
Node
StreamAnalyti
x
Agent
Syslog
Kafka
via
TCP
40 © 2014 Impetus Technologies
Index
Store
Down
Stream
System
SLA
Events
Report
generation
SLA
Alerts
Real-Time
Counters
Data Flow and Real-time Pipeline Design

40. Successful outcomes with all early customers
A few others in process
• Tier1 Healthcare Insurance Carrier – variety of use-cases
• Major Credit Card Brand and Bank – variety of use-cases
• End-point Security Application – On-prem and SaaS
• Mobile Field Devices – Real-time monitoring, predictive analytics
Recorded version available at http://bit.ly/1AeGdxM
41 © 2014 Impetus Technologies

41. Q&A
?
Email us at inquiry@streamanalytix.com
www.StreamAnalytix.com
Request: On-premise and Cloud based trial and/or Proof of concept
Recorded version available at http://bit.ly/1AeGdxM
42 © 2014 Impetus Technologies

42. StreamAnalytix Product Highlights
An “App Server” for real-time apps – on-premise and
cloud
Focus on your business logic - leave infra to us
Handle all the 3V’s of Big Data on one platform
Recorded version available at http://bit.ly/1AeGdxM
43 © 2014 Impetus Technologies
Significant time to market acceleration
Seamless integration with Hadoop and NoSQL

43. Data Parsing
- Variety
Recorded version available at http://bit.ly/1AeGdxM
Key Features
High Speed
Data
Ingestion
Elastic
Scaling –
Volume,
Velocity
44 © 2014 Impetus Technologies
Pluggable
Persistence
Real-time
Index and
Search
Dynamic
Message
Routing
Rule Based
Alert
Pluggable
Workflow
Management
Fault
Tolerance
and Data
Integrity
Optimized for
High
Performance