Real World Use Cases and Success Stories for In-Memory Data Grids (TIBCO ActiveSpaces, Oracle Coherence, Infinispan, IBM WebSphere eXtreme Scale, Hazelcast, Gigaspaces, GridGain, Pivotal Gemfire, not: SAP HANA)

In-Memory Computing
“Real
World
Use
Cases”
Kai Wähner
Technical Lead
kwaehner@tibco.com
@KaiWaehner
www.kai-waehner.de
LinkedIn / Xing à Please connect!

Kai Wähner
Consulting
Developing
Coaching
Speaking
Writing
Selling
Main Tasks
Requirements Engineering
Enterprise Architecture Management
Business Process Management
Architecture and Development of Applications
Service-oriented Architecture
Integration of Legacy Applications
Cloud Computing
Big Data
Contact
Email: kontakt@kai-waehner.de
Blog: www.kai-waehner.de/blog
Twitter: @KaiWaehner
Social Networks: LinkedIn, Xing

Disclaimer
!
These
opinions
are
my
own
and
do
not
necessarily
represent
my
employer

Key Messages
In-Memory Computing is used for Acting in Real-Time!
In-Memory is NOT just for Caching and Storing – A Data Grid offers much more!
Eventing and Fault-Tolerance move In-Memory Computing to another Level!

© Copyright 2000-2014 TIBCO Software Inc. 5
Agenda
• Introduction to In-Memory Computing
• Use Cases / Customer Success Stories

Agenda

Time
Business
Value
Business Event
Data Ready for Analysis
Analysis Completed
Decision Made
$$$$
$$$
$$
$
In-Memory Computing
and Event Processing
speed action and
increase business value
by seizing
opportunities while
they matter
Action Taken
Business Value of Events over Time

Drivers for In-Memory Computing
• Hardware costs declining
• Data Processing Requirements
exploding
• Traditional Approaches not
scaling
– Relational Databases
– Clustered Databases
– In-Memory Caches
– Messaging Systems

New Categories of Technology
• Two parallel responses to the 21st century data
processing needs
• NoSQL Databases
– Disk based with some in-memory caching
– Horizontal Scalability on Commodity
Hardware
– Alternatives to Relational Databases and SQL
– Basically Available Soft-state Eventually
(BASE)
– No ACID (transactions / concurrency control)
• In-Memory Data Grid Technology
– Memory for data storage
– Pooling Memory from multiple machines
– Use database for persistence
– ACID Properties
– Eventing – Notifications, Continuous Queries

SAP
HANA
is
not
an
In-‐Memory
Data
Grid!
Database Landscape in 2014
h;p://blogs.the451group.com/
informaDon_management/2014/03/18/
updated-‐data-‐plaIorms-‐landscape-‐
map-‐february-‐2014/

Product Example: TIBCO ActiveSpaces
Best of both Worlds (NoSQL + InMemory)!
Distributed In-memory System of Record
Stores platform / language independent key-value data structures in memory with the option to persist
data in parallel on local disks on a cluster of elastic horizontally scalable commodity hardware
High Performance ACID compliant NoSQL Data Grid
Offers all benefits of NoSQL databases and immediate consistency with full ACID compliance for
transactions and concurrency control
Minimal configuration and easy-to-use APIs (Java, C, .NET, “TIBCO Products”)
Uses proprietary consistent hashing algorithm that that ensures a single network hop for fetching
data. No need for partitioning, no complex XML configuration files
Querying
Data can be queried using an SQL-like language and queries can be accelerated through full
indexing capabilities such as composite indexes and tree or hash index types.

Agenda

Caching for Fast Data Access
LOADER
• Cache
to
slower
systems
• Read-‐only
• Not
the
system
of
record
• No
persistence
required
• Side
benefit:
Backend
load
is
reduced

Caching + Dynamic Load
SERVICE
• Dynamically
loaded
into
Memory
when
the
data
is
first
accessed
by
a
client
applicaDon
• Service
can
present
a
standard
interface
• Client
applicaDons
are
not
required
to
implement
any
In-‐Memory
specific
code
(1)
Check
Cache
(2)
Load
from
DB
if
not
in
Cache

Routing Messages to Back-Office Applications
• Receive
a
common
data
feed
that
needs
to
be
parsed
and
routed
to
several
back-‐office
applicaDons
can
use
• In-‐Memory
holding
reference
informaDon
for
the
rouDng
applicaDon.
The
router
can
quickly
determine
where
to
send
the
data.
• Examples:
Bank
payments,
insurance
claims
processing

Off-loading expensive systems
Expensive
in
terms
of
response
Dme
and
/
or
transacDon
costs!

Personalized Customer Experience
“With
38
million
fans,
MGM
knows
how
to
put
its
customers
first,
it
takes
more
than
a
smile
too.
Customers
want
a
personalized,
tailored
experience,
one
that
knows
their
name
and
can
anDcipate
their
needs.
With
the
help
of
TIBCO
technologies
that
leverage
big
data
and
give
customers
a
digital
idenDty,
MGM
can
send
personalized
offers
directly
to
customers,
save
them
a
seat,
and
have
their
favorite
drink
on
the
way.
With
mulDple
customer
touch
points
and
channels,
MGM
can
reach
customers
in
more
ways,
and
in
more
places,
than
ever
before.”
h;ps://www.youtube.com/watch?v=X-‐7S3kCOx9k
Latency
Problems:
• Several
Legacy
Systems
• Processing
via
ERP,
CRM,
Host,
etc.
In-‐Memory:
• Events
and
CorrelaDons
• Enable
Real
Time
• Only
customers
that
have
checked
in

Fault Tolerance and Disaster Recovery
Enabling Active-Active Fault Tolerance in Applications:
In-‐Memory
CompuDng
is
reliable,
scalable
and
fault-‐tolerant!

Fault Tolerance and Disaster Recovery
Multisite Data Replication:
In-‐Memory
CompuDng
is
reliable,
scalable
and
fault-‐tolerant!

Handling temporary spikes on a slow ‘system of record’
• An
In-‐Memory
event
listener
gets
noDfied
whenever
a
data
value
is
changed
and
sends
updates
through
a
message
queue
for
updaDng
the
master
system
of
record.
• The
back
office
system
can
also
be
updated
through
other
channels.
• Examples:
Christmas
Shopping
in
E-‐Commerce,
Ticket
Sales,
Online
Bekng

à
Operational Data Store (Local File System)
In-‐Memory
as
“system
of
record”

Operational Data Store (Local File System)
• Low-‐latency,
high-‐throughput
operaDonal
data
– Customer
data:
e.g.
account
status
and
balance,
purchase
history:
real-‐Dme
loyalty
(promoDons,
cross-‐selling),
fraud
detecDon,
...
– Market
data:
e.g.
risk
assessment,
porIolio
mgmt,
producDon
output
opDmizaDon,
buyer-‐seller
matching
– Sensor
data:
e.g.
smart
metering
/
grid,
public
transport
safety
– Track
and
trace:
e.g.
barcode
scans,
RFID:
logisDcs,
airlines
• Why
In-‐Memory?
– Much
faster
than
tradiDonal
DB,
especially
many
small
transacDons
(XTP)
– State
/
data
management
not
addressed
by
messaging
soluDons
– EvenDng
is
a
first
class
feature,
changes
can
be
‘pushed’
in
real-‐Dme
to
interested
parDes
(subscribe
to
changes,
conDnuous
queries)
– Provides
for
distributed
process
synchronizaDon
– Integrated
with
CEP
engine
(TIBCO
BusinessEvents)

Situation
Retailer: Inventory Management
• Master data management system stores over 800 million customer records across more than 30 enterprise apps.
• Stores real-time inventory data to enable ‘Buy online and pick-up at store’ and ‘Smart fulfillment’ features
Problem
• Due to lack of correlation between Point of Sale data and inventory, the website contained outdated inventory data.
Products were listed as out of stock when there was actually inventory.
• Need to leverage store inventory as well as inventory located fulfillment centers
Solution
• In-Memory stores real-time inventory data for the website, the fulfillment application, and other applications that need
access to inventory data
Business Impact
• Reduction in customer churn
• Intelligent fulfillments leading to greater customer satisfaction
• Improved overall efficiency of fulfillment centers and store inventory

Distribution of Rapidly Changing Data
à
Examples
are
monitoring
data
for
a
power
plant,
stock
market
data,
telemetry
data
for
a
complex
system
(example,
a
satellite),
or
the
status
and
locaDon
of
packages
for
a
major
logisDcs
or
shipping
company.

Telco: Real-Time Offer Generation and Fulfillment by Different Subcontractors
Purchase 3G Package
Cross-sell Voice/SMS package to subscriber
who purchases 3G Mobile Package
Total: 3 mio / day
Peak: 50 events per sec
Reload
Give 100 free SMS to subscriber who tops-up
Total: 12 mio top-up / day
Peak: 300 top-up per sec
Voice Call
Give discount VOIP package to subscriber who
makes a IDD call
Total: 200 mio / day
Peak: 12,000 events per sec
SMS Usage
Give discounted SMS package to subscriber
who sends SMS more than 10 times a day
Total: 750 mio / day
Peak: 27,000 events per sec
Purchase BB Package
Event Cloud
Reload
Voice Call
IDD Call
OnNet Call
SMS Usage
Event Handling and
Processing
Touchpoint Integration
Fulfill SMS
Package
Fulfill 3G Package
Fulfill Voice
Package
Fulfill SMS
Package
Billing, Offer
Fulfilled
46.7 million subscribers
2,000 SMS
notifications per
seconds
500 offer
fulfillments per
second
Offer
Message
Reminder
Message
Fulfillment
Message

Storing State-full Data for Enterprise Applications
State-‐full
Data

Super Fast Compute Grid for Intermediary Calculations for Analytics

Super Fast Compute Grid for Intermediary Calculations for Analytics
• Technical
issues
in
distributed
grid
compuDng
with
large
scale
data
– Work
load
distribuDon
– Process
synchronizaDon
– Data
transfer
• Examples
– Risk
assessment
and
management
– OpDmizaDon
problems:
scheduling,
cargo
assignment,
load
distribuDon
in
power
network
/
grid
• Why
In-‐Memory?
– Many
useful
synchronizaDon
features
(e.g.
atomic
“take”)
– LocaDon
transparency
and
fault-‐tolerance
– Real-‐Dme
instead
of
nightly
/
weekly
/
...
Data-‐Warehousing
approach

Questions?
Kai Wähner
kwaehner@tibco.com
@KaiWaehner
www.kai-waehner.de
LinkedIn / Xing à Please connect!

Real World Use Cases and Success Stories for In-Memory Data Grids (TIBCO ActiveSpaces, Oracle Coherence, Infinispan, IBM WebSphere eXtreme Scale, Hazelcast, Gigaspaces, GridGain, Pivotal Gemfire, not: SAP HANA)

More Related Content

What's hot

Similar to Real World Use Cases and Success Stories for In-Memory Data Grids (TIBCO ActiveSpaces, Oracle Coherence, Infinispan, IBM WebSphere eXtreme Scale, Hazelcast, Gigaspaces, GridGain, Pivotal Gemfire, not: SAP HANA)

More from Kai Wähner

Recently uploaded

Real World Use Cases and Success Stories for In-Memory Data Grids (TIBCO ActiveSpaces, Oracle Coherence, Infinispan, IBM WebSphere eXtreme Scale, Hazelcast, Gigaspaces, GridGain, Pivotal Gemfire, not: SAP HANA)