2. Introduction
2
Enterprises are moving away from simple monitoring and search-based tools
and towards trying to understand the meaning and causes of business
problems. From large organizations to young enthusiastic entrepreneurs and
startups, everyone is able to see the role that Big Data and analytics play in
decision making across the various domains of business.
The high velocity flow of data from real-time data sources such as market
data, the Internet of Things, mobile, sensors, clickstream, and even
customer transactions remain largely unutilized by most firms. Forrester
calls this data perishable because they occur at a moment’s notice and you
must act on the insights they reveal within a narrow window of opportunity
before they quickly lose their value.
Businesses are looking at a solution to collect, integrate, analyze, and
visualize data, as it is generated. Real-time streaming analytics processes
all the activities at the very moment the data is being generated, without
disrupting the activity of existing sources, storage, and enterprise systems.
This whitepaper explores several of real-time streaming analytics industry
examples and use cases across various sectors.
“A streaming analytics platform is
a software that can filter, aggre-
gate, enrich, and analyze a high
throughput of data from multiple
disparate live data sources and in
any data format to identify simple
and complex patterns to visualize
business in real-time, detect
urgent situations, and automate
immediate actions.” 1
1
Mike Gualtieri and Rowan Curran, The Forrester Wave™: Big Data Streaming Analytics Platforms, Q3 2014
3. The Need for RTSA
3
Over time, data loses its relevance to unfolding events. With each
increment of delay, data value decreases, becoming less a force for
rapid response than a resource for diagnostics and descriptive reports
of the past.
The value of predictive analytics rests on the accuracy of the data under
analysis, which in turn depends on its currency. When data are sensed and
analyzed in real-time, businesses can act with certainty, confident that
subsequent actions are rooted in a relevant, timely understanding of
unfolding events. It makes complex decision making easier.
A real-time streaming analytics addresses all these needs. With zero data
waiting time, nothing gets lost, overseen or outdated, because the variety of
data is not an issue. The results of the analytics are translated and fed back
into the local systems in real-time, which means the distance between the
incoming data and the outgoing data is as low as a few milliseconds.
$$$$$
Before
$$$
Now
$$
Later
Predictive analytics based on current events
Value depends on accuracy
Real-time
Certainty is high
Value depends on immediate use
Descriptive
Diagnostic
Least value
4. Real-time Use Cases of RTSA
Some of the key use cases for streaming analytics applications include
the following major areas:
4
Enterprises are realizing that the ability to tap into unstructured, real-time
data sources creates a potential goldmine of information.
New
Opportunities
MissedOpportunities
Routine Operations
Cutting Preventable losses
New Products and
Service Offerings
New Revenues from
Current Offerings
More Profits from Current
Operations (Efficiency)
Revenue and Asset
Protection
...........................
...............
......
......
• Predictive Maintenance – Manufacturing, Oil & Gas
• Clinical Care and Patient Management – Healthcare Clinical
• Sensor Analytics – IOT, Manufacturing,
• Fleet Operations – Transportation,
• Fraud and Anomaly Detection – IT Security, Financial Services
• Gaming Analytics – Entertainment, Gaming
• Churn Analytics – Telecom, Banking, Retail
• Network Traffic Analysis and Optimization – Telco
• Internet Advertising – Retail, e-commerce
Verticals
• Customer Experience
• Clickstream Analytics
• Context -sensitive Offers And Recommendations
• IT Log Analytics
• Security
Horizontals
• Internet of Things
• Mobile App Analytics
• Call Center Monitoring and Analytics
Combo
5. Customer Behavior Analytics
Industry Examples
5
Because profitability for most businesses depends on the acquisition of new
customers and the ongoing management of existing customer relationships,
enterprises need to understand their customers and work to build
relationships with them.
Customer behavior analytics help businesses adjust their plans and strategies
to become more competitive, minimize potential risk and make the best
decisions in real-time.
With the use of predictive analytics that use a combination of real-time and
historical data, businesses can create a series of targeted offers and make
them available in real-time at the next point of customer interaction. These
targeted offers help enhance customer satisfaction and improve ROI.
Our client, one of the largest wireless telecommunications providers in the
United States, wanted to capture, process, and stream their customers’ journey
in real-time. These would help them to predict customer behavior and enhance
the customer experience.
Impetus deployed StreamAnalytix to develop a pipeline within 3 business days
that captures, processes, and analyzes the data to provide the insights they
were seeking.
Customer Behavior Analytics can do
the following:
Provide a 360° customer view
Correlate information to present the right
offer at the right time based on personal
profiles and preferences
Monitor and optimize the performance
of your digital marketing efforts
Uncover opportunities for growth and
areas of improvement
Kafla
DVS Mapper Streaming
Custom DVS
Parser
HBase MTN
Lookup HBase
HBase
Customer Journery
Dashboard
................................................
............
Primary POC
6. Industry Examples
Business Activity Monitoring
6
Business Activity Monitoring can do
the following:
Audit business processes
Send event-driven alerts that trigger
process adjustments
Alert individuals to changes in the
business that may require action
Provide aggregated insight to
executives involved in strategic
planning
Enterprises can achieve significant return on investment by using Business
Activity Monitoring (BAM) as a real-time, intervention-focused tool for
measuring and managing business processes.
Business Activity Monitoring enables enterprises to monitor their business
processes, identify failures or exceptions, and address them in real-time.
Additionally, because BAM tracks processes and knows when they succeed
or fail, it builds valuable records of behavior that can lead to overall process
improvement. It also provides a useful tool to manage compliance, assure
The client, a leading source of residential mortgage credit in the US
secondary market, has a complex pipeline for processing loans and
needed a system for monitoring payload and business
For example, they needed to be able to do the following:
Monitor business activity to discover how many loans of each
loan type were processed every day by each processing stage.
Monitor payloads to find out how many files of each loan type
were processed every day by each processing stage
Reconciliation BAM
ESB
Data
Upstream Systems Loan Sourcing EDW
Compare, Audit, CEP, Alerting
RT
Dashboard
s
Impetus deployed StreamAnalytix to build probes/interceptors to get
information about payloads and business activity as it flows through the
processing pipeline.
7. Internet of Things (IoT) Analytics
7
Internet of Things Analytics can do
the following:
Enhance situational awareness
Optimize processes and resource
consumption
Control and respond to complex autono-
mous systems instantly
Track behavior for real-time marketing
Industry Examples
Our client, one of the world's largest airlines based on number of
destinations served, wanted us to build applications that would analyze
their data to do the following:
Extract and analyze data from customer surveys, email data and twitter
feeds to improve customer service and experiences.
Correlate these insights with other data sources such as maintenance
logs, crew logs, etc. for future use.
Allow customers to book a seat of their choice and provide their
analysts insights into the filling pattern of their flights to enhance
pricing.
Update travelers on TSA wait time.
Create a seat recommendation engine for frequent flyers
The Internet of Things (IoT) and the data it is producing are becoming main
catalysts of transformation for businesses and consumers alike.
For example, businesses can analyze sensor data in real-time to make key
decisions impacting patient healthcare or resource management, and
consumers can receive more personalized, timely and connected products
and experiences such as tailored deals from retailers.
Enabling the Mobile app
With RT notifications of
estimated queue time at TSA
Sensor 1
Sensor 2
TSA – Security Line
CEP Engine - TSA
Wait time estimation
Delay Compute
Proactive Notification / Updates
Beacon / Proximity Sensor
Ingest, Pre-process, Filter
Impetus successfully deployed StreamAnalytix to solve all the above
challenges. The graphic shown here illustrates how we used StreamAnalytix to
create a predictive model that provides mobile notifications to travelers about
estimated TSA wait times. The solution calculates the wait time using real time
Beacon data.
8. Clickstream Analytics
8
Clickstream Analytics can do the
following:
Optimize click paths
Analyze basket data
Recommend the next best product
Allocate website resources
Analyze customer segmentation on a
granular level
Industry Examples
Our client, one of the largest wireless telecommunications providers in the
United States, wanted to search their clickstream data. Using customer
search data, they wanted to track customer activity and provide related offers.
Clickstreams are the virtual trails that users create while surfing the Internet.
Aclickstream is a record of a user's activity, including every page of every web
site the user visits and how long the user was on a page, the order the pages
were visited. This data is becoming increasingly valuable to Internet marketers
and advertisers.
Clickstream analytics give businesses a predictive edge to see what products
customers tend to buy together. As these purchasing correlations are
recognized, businesses can look at what a customer purchases and send them
real-time offers for the products that they will most likely buy next, thus
increasing the chances of making another sale either during the same online
visit or in the future.
Analysis of clickstream also provides a granular look at how individual customer
segments are using a given website. As a result, enterprises can gain actionable
insights to help personalize the user experience and convert more web visitors
from browsers to buyers.
Video Playback Statistics Coding
Web Performance Management
Pricing & Revenue Management
Real-time Dashboards
Web Page
Clickstream Analytics
Impetus used StreamAnalytix to develop a pipeline to parse the unstructured
data, perform the ETL as per business rules, and store data in HBase.
