Evaluation guide to Streaming Analytics

Evaluation Guide to Streaming Analytics
by Can Alhas and Ersegun Kocoglu
Evaluation Guide to
StreamingAnalytics
January 2017
The insights to choose
the best streaming data analy3cs tool for your business

How to Evaluate and
Select Streaming Data
Analytics Solutions
The term “Big Data” originated in the 1990’s as data became larger and more
complex than data warehousing systems were capable of analyzing. Today, Big
Data continues to grow in complexity, with a mix of structured and unstructured
data, and increasing velocity and volume.
Big Data strategies have focused on capture and analysis of data, beginning with
Data Warehousing, and more recently involving Hadoop, Map Reduce and other
techniques. Surveys by the Gartner Group indicate that Big Data strategies have
followed earlier data warehousing patterns with mixed outcomes. Gartner
reports that 49 percent of organizations are “. . . struggling to get value from
Hadoop” (Gartner 2015 Hadoop Adoption Survey).
1
Evaluation Guide to Streaming Analytics January 2017

Streaming Analytics solutions respond
to events within 100 milliseconds.
Nancy can receive a custom offer
while browsing a web page, or receive
a push message while in the store.
Financial service providers respond
immediately to suspected fraud.
Streaming Analytics are also used to
manage Internet of Things (IOT)
devices.
Streaming Data Analytics has developed in recent years to focus on high
frequency data, with real time processing and action. Streaming Data Analytics
shifts focus from systems of record (“what were last quarter’s sales of product
X?”), to real time insight and action (“what is Nancy most likely to buy, and what
form of engagement will best influence her behavior?”). Streaming Analytics
shifts our focus from being reactive to proactive.
It’s Time for
Streaming Data Analytics
Streaming Analytics has emerged as
among the most practical Big Data
strategies. Focusing on a subset of
the data stream, systems are simpler
to implement and deliver immediate
results. Streaming Analytics systems
are known for increased revenues,
customer retention (and reduced
churn), higher system uptime for
managed devices, and lower customer
support costs.
Business
Value
$$$$
$
$$
$$$
Time
OPTIMUM BUSINESSVALUE
Business Event
Data Preparation for Analysis
Analysis Complete
Business Decision
Business Action
Streaming Analytics Solutions Deliver Business Value in Real Time
2

The Gartner Group
recognizes the following
vendors are leaders in
Streaming Analytics*:
Apache Foundation EVAM
OracleMicrosoft
Software AG SAP
Tibco SoftwareSAS
* Gartner Hype Cycle for Data Science, 2016 Published: 25 July 2016
3

Overview of this
Evaluation Guide
While Streaming Analytics solutions can be implemented in a matter of days, they
involve a range of capabilities and technologies. This Evaluation Guide is
organized into seven categories that align with organizational interests of
Marketing, Finance, Operations, IT, and other groups. Each category includes a
number of detailed topics.
4
1 Business considerations: Time to Solution, Cost, and vendor value-add
2 Architecture
3
Event Collection, Ingestion, Enhancement Data, Stream Processing,
and Action
4 Security, Audit Support, Logging and Monitoring
5 Operations, and Testing
6 Analytics
7 Business Process Modeling, Scenario Design and Management

Streaming Data
and Real Time
Event Processing
in a Nutshell
Real time event processing begins with events such as a social
media post, a web browsing session, a wireless call or text, or
updates to a database. Events are recognized and formatted
by “event listeners,” that filter and format the events for efficient
processing. Formatted message are routed to an ingestion
service such as Amazon’s AWS Kinesis, or directly into the
event processing system.
Customer profile information is required for events to be acted
upon. For example, a wireless customer that experiences four
dropped calls may fulfill an “scenario” for the wireless operator.
To respond to the customer in real-time, the service provider
needs the customer name and contact methods. This
“customer profile” needs to be available in-memory to enable
response within the 100 milliseconds time window. In large
scale systems the event listeners can query and incorporate
specific data into the formatted event, enhancing scalability and
performance of the event processing engine. Customer profile
data is referred to as “enrichment data” throughout this
evaluation guide.

5

Collecting events in real time is challenging, but the larger
challenge is combining events into scenarios that drive real time
actions. The creation of scenarios, the ability to adapt or
update scenarios, and supporting hundreds of scenarios, is
determined by the technical frameworks for Scenarios and
event engine design. The technical aspects of scenario design,
and management of scenarios, is a crucial focus in
understanding and evaluating Streaming Analytics solutions.
6
This diagram illustrates the functional elements involved in real time event processing.

1
7
Evaluating Streaming Analytics Systems
Business considerations: Time to
Solution,Cost,and vendor value-add
With any strategic technology it’s important to consider the time and effort
involved in getting started, as well as cost of purchase, maintenance and
support, and the value a vendor brings to your business. The goal is to achieve
the highest business benefits (ROI), with lowest possible risk.
(1)  What is the cost and effort to implement a Pilot to support 5 scenarios for 1M users?
(2)  What is the cost to implement a large scale solution that includes 100 scenarios and
5M users?
(3)  What drives the cost over time as the system scales? Is the driver associated with
real -time event processing, or is the complexity and cost associated with Scenario
definition and support?
(4)  What resources are required to define and implement new Scenarios? Does the
system support scenario definition and implementation by business users, or require
programmers?
(5)  How quickly can new scenarios be developed and implemented?
(6)  Rate the vendor reputation for support and customer success by speaking with
references
(7)  Does the vendor have expertise in delivering solutions for your industry or vertical?
(8)  Does the solution support both public cloud or on premise use?
(9)  Does the solution allow for future flexibility to Integrate with Open Source projects?


2
8
Architecture
The structure and design of a Streaming Analytics system must meet the
technical and operational requirements, while optimizing the performance,
security, and management. An important aspect of the architecture is
whether it’s modern, and open to plug-and-play integration of open source
libraries. Openness can be evaluated by the support for on-premise and public
cloud use (AWS or Azure), and plug-and-play support for open source libraries such
as Kafka, R, and others.
The system requires a functionally partitioned design similar to the architecture illustrated
above. The architecture includes event listeners, streaming event ingestion, event
processing, scenario design, and enrichment data, persistent data stores for logging and
analytics, and support for actions. The capabilities should include:
(1)  The system should be capable of scaling to support departmental or enterprise-wide
use. The system should be economical to support a single node, or support tens of
millions of users across multiple nodes.
(2)  The system should deliver event-to-action response in 100 milliseconds or less.
(3)  The system should have documented support for on-premise or public cloud use.
(4)  The system should include a modular architecture, with event listeners, event ingestion,
and event processing capable of running on separate compute nodes.
(5)  For businesses that may involve support of remote and IOT devices, the system should
be supportable on distributed devices, such as the Rasberry Pi.
(6)  The system should be able to scale-out linearly, by adding processing nodes
(7)  The solution should support a clustered, resilient architecture, capable of non-stop
operation when a node fails.
(8)  The solution should be able to redistribute events in the cluster upon the failure or
abnormal termination of one node.
(9)  The solution must provide automatic load-balancing capabilities.
(10)  The system should be recoverable from node failure, restoring operation from an
asynchronous replication of system data to a persistent data store
(11)  The system should include an open architecture, capable of integrating with cloud
services such as RedShift, Kinesis, as well as open source projects such as Flume,
Kafka, etc.


3
9
Event Collection,Ingestion,Stream
Processing,Enrichment data,and
Actions
Real time event processing is a sophisticated undertaking due to the varied
sources of event data, including social media sites, database operations, legacy
systems, payment systems, and web browsing sessions. The core of a real time event
processing system is the ability to recognize events, format and deliver those events for
efficient ingestion, and ultimately to the “event processor.” Scenarios also require use of
“enrichment” data, such as customer name, contact methods, and other data not included in
the real time event.
a. Event Collection:
Where many Big Data strategies begin by aggregating data from many sources, a Streaming
Analytics strategy focuses on capturing a selected (limited) set of high frequency data
events. “Event Listeners” integrate with source systems to capture the event in real time,
format the event for the event processing engine. Event listeners should be available to
integrate with all major database systems (where events are the creation of a new database
record), social media systems, logging systems, legacy applications, and others.
(1)  Event listeners should support parsing, filtering, and reformatting capabilities.
(2)  Each processed event should be tagged with a unique event ID, and time-stamped.
(3)  Event listeners should also be capable of incorporating enrichment data into the
formatted event as needed.
(4)  Event listeners should support events in Oracle, SQL Server, and other database
logging systems
(5)  There should be event listener for FILES, WEB LOGS, XML, JSON, KAFKA,
RabbitMQ, Kinesis
(6)  The system should include a documented SDK to support development of new event
listeners
(7)  Event listeners should support open source data feeding mechanisms (flume, log stash
etc.)
(8)  There should be ready web clickstream detection listeners
(9)  Event ingestion should support both synchronous and asynchronous event models,
and support Scenario return actions within a synchronous event session.
(10)  The solution should be able to distribute events to processing engines using pre-
defined algorithms/formulas.
(11)  The solution should support out of the box social media events listeners


3
10
Actions
b. Enrichment Data:
Real time events are generated by either a person or a device, and taking action on
events requires data about that person or device. This “enrichment” data can include
customer name, customer segment, contact information, and selected history.
(1)  Enrichment data should be accessible through a web service, or SQL query, and
incorporated into an in-memory distributed cache
(2)  Enrichment data should be updated asynchronously, based on a schedule, and be
include support for expiration.
(3)  Enrichment data should be able to cached if required
(4)  There should be configuration support for canceling long running enrichment queries
(5)  Enrichment query times should be logged and average times for each enrichment
should be seen in dashboards
(6)  Enrichment data should be accessible by the stream processing engine, as well as by
Event Listeners, to incorporate selected enrichment data into specific event streams.
c. Event Processing Engine:
The event processing engine processes formatted events, combined with enrichment data,
to register against defined Scenarios (discussed further below).
(1)  The event processing engine should support persistent queues to ensure all events are
processed
(2)  The event processing engine should employ in-memory cache to ensure real time
performance and economical system design
(3)  The in-memory cache should be accessible by a restful API as well as SQL query to
third party applications
(4)  The event processing engine should support asynchronous replication of event data to
a persistent data store, to support recovery of the system in the event of a failure
(5)  The event processing engine should support dynamic logging, with the logging being
capable of being updated during operation from normal to “debug” mode, without
interrupting system operation.
(6)  The event processing engine should log each event by processing time, enrichment
time, and action.


3
11
Actions
(7)  The event processing engine should be capable of responding to a scenario
that consumes excessive resources. The scenario can be suspended, or alert can be
generated.
(8) The event processing engine should support varied time windows sizes (seconds or
weeks) without requiring more memory or system resources.
(9) The system should be able to run in test mode for selected scenarios, where actions are
suspending when in test mode.
(10) The event processer should be capable of dynamically adding nodes to respond to
event processing needs.
(11) Engine should support Windows and Linux OS
(12)  The solution should support impact analysis easily. For example, if any resource
systems API change is needed, it should be quickly reported in impact analysis
including all the scenarios used.
(13) The solution should support a variety of time window capabilities:
•  Time-based windows (e.g., keep 10 seconds of data)
•  Count-based windows (e.g., hold 100 events)
•  Count-based windows partitioned by an attribute (e.g., the last 100 trades for
each stock)
•  Sliding windows (e.g., a window that continually keeps the last 10 seconds of
data)
•  Jumping (or tumbling) windows (e.g., a window that resets every 10 seconds)
•  Top-k / Bottom-k windows (e.g., the top 10 stocks by trade volume)
•  Complex windows (e.g. the last 10 readings for each reader that are less than 1
minute old)
•  Explicitly-controlled windows (e.g., allows rows to be arbitrarily inserted or
deleted)


3
12
Actions
d. Actions:
Actions are the goal for Streaming Analytics systems. A system should have a library
of supported actions, each implemented within 100 milliseconds of a triggering event. A
documented SDK should support the development of new actions as needed. Basic
actions should include SMS, email, calling a third party rules engine, invoking a web service,
calling a restful API, or requesting a business process or database operation.
(1)  The system should be able to invoke any web service to effect an action
(2)  The system should be able to invoke any restful API to effect an action
(3)  A documented SDK should support the development of new actions
(4)  Synchronous and Asynchronous actions should be supported. A suspected fraudulent
transaction should be responded to immediately, during the (synchronous) transaction.
(5)  New actions can be added to system without interruption of system operation
(6)  Action durations should be logged and average times for each action should be seen in
dashboards


4
13
Security, Audit Support,Logging,and
Monitoring
A system designed to drive automated actions to end-user and customers,
requires rigorous security and safeguards. Security is enhanced by audit
support, which emphasizes the importance of logging and analytical reporting on
system performance and changes.
a. Security and Audit
Security for Streaming Analytics systems is based on role-based usage and authentication,
and managerial review and controlled releases of scenarios and consequent actions.
(1)  The system should support role-based users, and require authenticated user login.
(2)  Scenario Design Tool should have the identification and authentication feature
(3)  The Scenario Design Tool will have user identification and authentication, and
authorization for use of certain features.
(4)  In the software configuration, it will not keep the unencrypted password in connection
with other external systems.
(5)  The Scenario Design Tool will have the ability to create new users and manage the role
based features of users.
(6)  Scenarios should have the ability to store relevant actions and events to provide a full
audit trail


4
14
Security, Audit Support,Logging,and
Monitoring
b. Logging and Monitoring
Logging and monitoring are critical, both to enhance the secure use of the
system, and support efficient operations.
(1)  The event processing engine should support dynamic logging, so logging levels can
be changed during operation (from normal, to debug, and back).
(2)  The event processing system should log all events, with unique event ID and
timestamp
(3)  Scenario creation and updates should be logged by author and time.
(4)  Enhancement data updates, and processing times should be logged
(5)  Action processing times should be logged and summarized in the dashboard
(6)  Monitoring data should be available for query and reporting by 3rd party applications
(7)  The system should provide an open data repository for the enterprise's enterprise
reporting systems and other business systems to read scenario data and the status of
individual users in scenarios.
(8)  The monitoring and reporting tool should support selection of actions, events or states
in which scenarios should be monitored.
(9)  The monitoring and reporting tool should provide the capacity to build customized
dashboards
(10)  The monitoring and reporting tool should have the ability to make time-based
aggregations such as counting, summing, averaging parameters over time windows
(i.e. average credit card spending in the last hour) at a per scenario level
(11)  The solution should be able to have location base reporting (Geo-mapping).
(12)  In monitoring screen, it should be possible to change scenario variables on the fly
without going to scenario designer
(13)  It should be possible to add new chart types by plugin development


5
15
6
Operations andTesting
Efficient management of the system is supported by a full functioned central
console, and with strong support for testing of scenarios prior to deployment.
(1)  The system console should include a full range of functions, enabling the
system to be started, stopped, or queried to see queue length, and other
operating parameters.
(2)  Scenarios should be self-documenting, portable, easily reviewable and
understandable to operators and managers.
(3)  The system should include support for change management, where scenarios are
reviewed and approved prior to being released into production use
(4)  The system should support arithmetic operations for calculations of rewards or bonus.
(5)  The system should support event simulation, to test scenarios.
(6)  The system should support scenarios in test mode, where events are processed, but
actions are simply logged and not implemented.
(7)  The system should support the ability to create control or test groups, based on
quantities or percentages within a campaign, for all levels of campaign hierarchy
(campaign, segments, and sub-segments).
(8)  The system should support up to 20,000 TPS per core under simple stress tests.

Analytics
In addition to basic logging and monitoring capabilities, which keep users
informed on system and scenario performance, the Analytical capabilities
should allow for insightful analysis of the system data.
(1)  The system shouldn’t be limited to real time or batch reporting, but also
support alerts to system users
(2)  The system should support integration and use of analytic tools like R, MOA, H2O.
(3)  The platform should support extensibility in Analytical capabilities, by adding optional
Analytical modules, such as Frequent Pattern Analysis, Enhanced Real-Time
clustering, and other analytical methods.


7
16
Business Process Modeling,
Scenarios, Scenario Design and
Management
The many capabilities listed above are challenging, but are building blocks
required for a flexible and cost effective Streaming Analytics solution. The great
divide among Streaming Analytics solutions is in how events are defined and combined in
scenarios that drive actions.
Scenarios combine one or more real time events with enhancement data, and criteria such
as time and counts, to trigger action. System designs take one of two approaches, with
scenarios based either on technical events or a business process model. Business
process modeling requires the creation of a catalog of business objects, including business
events, enhancement data, and actions. This business process model supports the creation
of Scenarios that are reviewable by managers, and can be defined and implemented by non-
programmers.
Larger scale systems support hundreds of scenarios, and events often combine to fulfill
scores of scenarios in rapid succession. Systems must be designed to support prioritization
of scenarios, and support scenarios constraints. Without these capabilities customers can
be subjected to a literal “flood” of actions as scenarios are fulfilled.


7
17
Business Process Modeling,
Scenarios, Scenario Design and
Management
(1)  Scenarios combine one or more events with enrichment data and actions.
(2)  Scenarios are designed from a catalog of business events, enrichment data, and
actions.
(3)  A visual designer provides a drag and drop visual presentation for the steps involved in
completing a scenario.
(4)  A visual designer depicts the experience of an individual user or device (event actor).
The scenario should be easily reviewed by a manager or super-user.
(5)  A scenario designer should provide users access to parameters and time or count
based aggregations, such as money transferred out of accounts in the past 24 hours.
(6)  A scenario design should support the creation of a scenario template, with a catalog of
available parameters, to support application of the scenario to varied customer
segments and actions.
(7)  The system should support scenario prioritization, to ensure the most important
scenario actions are prioritized.
(8)  The system should support the ability to constrain or limit the number of actions any
actor is provided within a specified timeframe.
(9)  Scenarios can act as synthetic events to provide input for other scenarios, using a
global variable or generating a synthetic event.
(10)  A scenario designer should support analysis of scenarios. It should be straightforward
to understand, for example, how many scenarios for a customer segment are based on
a shared real time event.
(11)  Scenarios should support the definition of non-events. For example, a customer who
purchases a new service or device, but fails to register the device or use the service,
could be defined as a non-event.
(12)  A scenario designer should validate scenario errors, and propose fixes.
(13)  The scenario designer should include support for a control group feature, with a period
function for measuring the success of a scenario.


This Evaluation Guide to Streaming Analytics is based on the collective experience at EVAM.
Over forty enterprises globally rely on EVAM systems to manage interactions with over 200
million people daily.
This article provides a basic introduction to Streaming Analytics systems and their capabilities.
A comprehensive Evaluation Guide, suitable for adaptation as an RFP is available as a free
download at https://www.evam.com/evaluation-guide
EVAM welcomes your questions, inquiries, and feedback. If you would like to receive more
information about streaming data analytics solutions and for all other inquiries, please contact
us at ersegun@evam.com. We will get in touch with you shortly.
Industry Based Use Cases
EVAM has many uses cases across almost all industries for streaming data analytic solutions.
Banking, Telco, Retail, Insurance, Travel, Energy and more.
EVAM makes these and more use cases possible in any business where needed. Contact us
now to understand more about how EVAM can help your business.
About EVAM
EVAM is a leader in Streaming Analytics solutions, with over forty enterprises relying on EVAM
systems to support over 200 million end users daily. EVAM delivers real-time customer
engagement solutions for Retail, Financial Services, Manufacturing, and Wireless Services.
EVAM supports public cloud on AWS and Azure, and on-premise solutions, with plug and play
integration with open source frameworks such as Apache Spark, Kafka, and others.
For further information, please visit www.evam.com
About Authors
Can Alhas is the founder and chief architect at EVAM, Ersegun Kocoglu is the founder of
MarketMe Marketing Consultancy and CMO & Business Partner at EVAM.
About This Evaluation Guide
18

© 2017 EVAM. All rights reserved. EVAM Streaming Analytics Platform products and solutions are either
trademarks or registered trademarks of EVAM. Other product and company names mentioned herein may
be the trademarks of their respective owners.

Evaluation guide to Streaming Analytics

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Evaluation guide to Streaming Analytics

Similar to Evaluation guide to Streaming Analytics (20)

More from Evam Streaming Analytics

More from Evam Streaming Analytics (7)

Recently uploaded

Recently uploaded (20)

Evaluation guide to Streaming Analytics