Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
In-Memory Computing 
“Real 
World 
Use 
Cases” 
Kai Wähner 
Technical Lead 
kwaehner@tibco.com 
@KaiWaehner 
www.kai-waehn...
Kai Wähner 
Consulting 
Developing 
Coaching 
Speaking 
Writing 
Selling 
Main Tasks 
Requirements Engineering 
Enterprise...
Disclaimer 
! 
These 
opinions 
are 
my 
own 
and 
do 
not 
necessarily 
represent 
my 
employer
Key Messages 
In-Memory Computing is used for Acting in Real-Time! 
In-Memory is NOT just for Caching and Storing – A Data...
© Copyright 2000-2014 TIBCO Software Inc. 5 
Agenda 
• Introduction to In-Memory Computing 
• Use Cases / Customer Success...
© Copyright 2000-2014 TIBCO Software Inc. 6 
Agenda 
• Introduction to In-Memory Computing 
• Use Cases / Customer Success...
Time 
Business 
Value 
Business Event 
Data Ready for Analysis 
Analysis Completed 
Decision Made 
$$$$ 
$$$ 
$$ 
$ 
In-Me...
Drivers for In-Memory Computing 
• Hardware costs declining 
• Data Processing Requirements 
exploding 
• Traditional Appr...
New Categories of Technology 
• Two parallel responses to the 21st century data 
© Copyright 2000-2014 TIBCO Software Inc....
SAP 
HANA 
is 
not 
an 
In-­‐Memory 
Data 
Grid! 
© Copyright 2000-2014 TIBCO Software Inc. 10 
Database Landscape in 2014...
Product Example: TIBCO ActiveSpaces 
Best of both Worlds (NoSQL + InMemory)! 
Distributed In-memory System of Record 
Stor...
© Copyright 2000-2014 TIBCO Software Inc. 12 
Agenda 
• Introduction to In-Memory Computing 
• Use Cases / Customer Succes...
Caching for Fast Data Access 
LOADER 
• Cache 
to 
slower 
systems 
• Read-­‐only 
• Not 
the 
system 
of 
record 
• No 
p...
Caching + Dynamic Load 
SERVICE 
• Dynamically 
loaded 
into 
Memory 
when 
the 
data 
is 
first 
accessed 
by 
a 
client ...
Routing Messages to Back-Office Applications 
• Receive 
a 
common 
data 
feed 
that 
needs 
to 
be 
parsed 
and 
routed 
...
Off-loading expensive systems 
Expensive 
in 
terms 
of 
response 
Dme 
and 
/ 
or 
transacDon 
costs!
Personalized Customer Experience 
“With 
38 
million 
fans, 
MGM 
knows 
how 
to 
put 
its 
customers 
first, 
it 
takes 
...
Fault Tolerance and Disaster Recovery 
Enabling Active-Active Fault Tolerance in Applications: 
In-­‐Memory 
CompuDng 
is ...
Fault Tolerance and Disaster Recovery 
Multisite Data Replication: 
In-­‐Memory 
CompuDng 
is 
reliable, 
scalable 
and 
f...
Handling temporary spikes on a slow ‘system of record’ 
• An 
In-­‐Memory 
event 
listener 
gets 
noDfied 
whenever 
a 
da...
à 
Operational Data Store (Local File System) 
In-­‐Memory 
as 
“system 
of 
record”
Operational Data Store (Local File System) 
• Low-­‐latency, 
high-­‐throughput 
operaDonal 
data 
– Customer 
data: 
e.g....
Situation 
Retailer: Inventory Management 
• Master data management system stores over 800 million customer records across...
Distribution of Rapidly Changing Data 
à 
Examples 
are 
monitoring 
data 
for 
a 
power 
plant, 
stock 
market 
data, 
t...
Telco: Real-Time Offer Generation and Fulfillment by Different Subcontractors 
Purchase 3G Package 
Cross-sell Voice/SMS p...
Storing State-full Data for Enterprise Applications 
State-­‐full 
Data
Super Fast Compute Grid for Intermediary Calculations for Analytics
Super Fast Compute Grid for Intermediary Calculations for Analytics 
• Technical 
issues 
in 
distributed 
grid 
compuDng ...
Key Messages 
In-Memory Computing is used for Acting in Real-Time! 
In-Memory is NOT just for Caching and Storing – A Data...
Questions? 
Kai Wähner 
kwaehner@tibco.com 
@KaiWaehner 
www.kai-waehner.de 
LinkedIn / Xing à Please connect!
Upcoming SlideShare
Loading in …5
×

Real World Use Cases and Success Stories for In-Memory Data Grids (TIBCO ActiveSpaces, Oracle Coherence, Infinispan, IBM WebSphere eXtreme Scale, Hazelcast, Gigaspaces, GridGain, Pivotal Gemfire, not: SAP HANA)

10,211 views

Published on

A lot of data grid products are available. TIBCO ActiveSpaces, Oracle Coherence, Infinispan, IBM WebSphere eXtreme Scale, Hazelcast, Gigaspaces, GridGain, Pivotal Gemfire to name most of the important ones. Not SAP HANA!

The goal of my talk was not very technical. Instead, I discussed several different real world use cases and success stories for using in-memory data grids. Here is the abstract for my talk:

NoSQL is not just about different storage alternatives such as document store, key value store, graphs or column-based databases. The hardware is also getting much more important. Besides common disks and SSDs, enterprises begin to use in-memory storages more and more because a distributed in-memory data grid provides very fast data access and update. While its performance will vary depending on multiple factors, it is not uncommon to be 100 times faster than corresponding database implementations. For this reason and others described in this session, in-memory computing is a great solution for lifting the burden of big data, reducing reliance on costly transactional systems, and building highly scalable, fault-tolerant applications.The session begins with a short introduction to in-memory computing. Afterwards, different frameworks and product alternatives are discussed for implementing in-memory solutions. Finally, the main part of this session shows several different real world uses cases where in-memory computing delivers business value by supercharging the infrastructure.

Published in: Technology

Real World Use Cases and Success Stories for In-Memory Data Grids (TIBCO ActiveSpaces, Oracle Coherence, Infinispan, IBM WebSphere eXtreme Scale, Hazelcast, Gigaspaces, GridGain, Pivotal Gemfire, not: SAP HANA)

  1. 1. In-Memory Computing “Real World Use Cases” Kai Wähner Technical Lead kwaehner@tibco.com @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!
  2. 2. Kai Wähner Consulting Developing Coaching Speaking Writing Selling Main Tasks Requirements Engineering Enterprise Architecture Management Business Process Management Architecture and Development of Applications Service-oriented Architecture Integration of Legacy Applications Cloud Computing Big Data Contact Email: kontakt@kai-waehner.de Blog: www.kai-waehner.de/blog Twitter: @KaiWaehner Social Networks: LinkedIn, Xing
  3. 3. Disclaimer ! These opinions are my own and do not necessarily represent my employer
  4. 4. Key Messages In-Memory Computing is used for Acting in Real-Time! In-Memory is NOT just for Caching and Storing – A Data Grid offers much more! Eventing and Fault-Tolerance move In-Memory Computing to another Level!
  5. 5. © Copyright 2000-2014 TIBCO Software Inc. 5 Agenda • Introduction to In-Memory Computing • Use Cases / Customer Success Stories
  6. 6. © Copyright 2000-2014 TIBCO Software Inc. 6 Agenda • Introduction to In-Memory Computing • Use Cases / Customer Success Stories
  7. 7. Time Business Value Business Event Data Ready for Analysis Analysis Completed Decision Made $$$$ $$$ $$ $ In-Memory Computing and Event Processing speed action and increase business value by seizing opportunities while they matter Action Taken Business Value of Events over Time
  8. 8. Drivers for In-Memory Computing • Hardware costs declining • Data Processing Requirements exploding • Traditional Approaches not scaling © Copyright 2000-2014 TIBCO Software Inc. 8 – Relational Databases – Clustered Databases – In-Memory Caches – Messaging Systems
  9. 9. New Categories of Technology • Two parallel responses to the 21st century data © Copyright 2000-2014 TIBCO Software Inc. 9 processing needs • NoSQL Databases – Disk based with some in-memory caching – Horizontal Scalability on Commodity Hardware – Alternatives to Relational Databases and SQL – Basically Available Soft-state Eventually (BASE) – No ACID (transactions / concurrency control) • In-Memory Data Grid Technology – Memory for data storage – Pooling Memory from multiple machines – Use database for persistence – ACID Properties – Eventing – Notifications, Continuous Queries
  10. 10. SAP HANA is not an In-­‐Memory Data Grid! © Copyright 2000-2014 TIBCO Software Inc. 10 Database Landscape in 2014 h;p://blogs.the451group.com/ informaDon_management/2014/03/18/ updated-­‐data-­‐plaIorms-­‐landscape-­‐ map-­‐february-­‐2014/
  11. 11. Product Example: TIBCO ActiveSpaces Best of both Worlds (NoSQL + InMemory)! Distributed In-memory System of Record Stores platform / language independent key-value data structures in memory with the option to persist data in parallel on local disks on a cluster of elastic horizontally scalable commodity hardware High Performance ACID compliant NoSQL Data Grid Offers all benefits of NoSQL databases and immediate consistency with full ACID compliance for transactions and concurrency control Minimal configuration and easy-to-use APIs (Java, C, .NET, “TIBCO Products”) Uses proprietary consistent hashing algorithm that that ensures a single network hop for fetching data. No need for partitioning, no complex XML configuration files Querying Data can be queried using an SQL-like language and queries can be accelerated through full indexing capabilities such as composite indexes and tree or hash index types.
  12. 12. © Copyright 2000-2014 TIBCO Software Inc. 12 Agenda • Introduction to In-Memory Computing • Use Cases / Customer Success Stories
  13. 13. Caching for Fast Data Access LOADER • Cache to slower systems • Read-­‐only • Not the system of record • No persistence required • Side benefit: Backend load is reduced
  14. 14. Caching + Dynamic Load SERVICE • Dynamically loaded into Memory when the data is first accessed by a client applicaDon • Service can present a standard interface • Client applicaDons are not required to implement any In-­‐Memory specific code (1) Check Cache (2) Load from DB if not in Cache
  15. 15. Routing Messages to Back-Office Applications • Receive a common data feed that needs to be parsed and routed to several back-­‐office applicaDons can use • In-­‐Memory holding reference informaDon for the rouDng applicaDon. The router can quickly determine where to send the data. • Examples: Bank payments, insurance claims processing
  16. 16. Off-loading expensive systems Expensive in terms of response Dme and / or transacDon costs!
  17. 17. Personalized Customer Experience “With 38 million fans, MGM knows how to put its customers first, it takes more than a smile too. Customers want a personalized, tailored experience, one that knows their name and can anDcipate their needs. With the help of TIBCO technologies that leverage big data and give customers a digital idenDty, MGM can send personalized offers directly to customers, save them a seat, and have their favorite drink on the way. With mulDple customer touch points and channels, MGM can reach customers in more ways, and in more places, than ever before.” h;ps://www.youtube.com/watch?v=X-­‐7S3kCOx9k Latency Problems: • Several Legacy Systems • Processing via ERP, CRM, Host, etc. In-­‐Memory: • Events and CorrelaDons • Enable Real Time • Only customers that have checked in
  18. 18. Fault Tolerance and Disaster Recovery Enabling Active-Active Fault Tolerance in Applications: In-­‐Memory CompuDng is reliable, scalable and fault-­‐tolerant!
  19. 19. Fault Tolerance and Disaster Recovery Multisite Data Replication: In-­‐Memory CompuDng is reliable, scalable and fault-­‐tolerant!
  20. 20. Handling temporary spikes on a slow ‘system of record’ • An In-­‐Memory event listener gets noDfied whenever a data value is changed and sends updates through a message queue for updaDng the master system of record. • The back office system can also be updated through other channels. • Examples: Christmas Shopping in E-­‐Commerce, Ticket Sales, Online Bekng
  21. 21. à Operational Data Store (Local File System) In-­‐Memory as “system of record”
  22. 22. Operational Data Store (Local File System) • Low-­‐latency, high-­‐throughput operaDonal data – Customer data: e.g. account status and balance, purchase history: real-­‐Dme loyalty (promoDons, cross-­‐selling), fraud detecDon, ... – Market data: e.g. risk assessment, porIolio mgmt, producDon output opDmizaDon, buyer-­‐seller matching – Sensor data: e.g. smart metering / grid, public transport safety – Track and trace: e.g. barcode scans, RFID: logisDcs, airlines • Why In-­‐Memory? – Much faster than tradiDonal DB, especially many small transacDons (XTP) – State / data management not addressed by messaging soluDons – EvenDng is a first class feature, changes can be ‘pushed’ in real-­‐Dme to interested parDes (subscribe to changes, conDnuous queries) – Provides for distributed process synchronizaDon – Integrated with CEP engine (TIBCO BusinessEvents)
  23. 23. Situation Retailer: Inventory Management • Master data management system stores over 800 million customer records across more than 30 enterprise apps. • Stores real-time inventory data to enable ‘Buy online and pick-up at store’ and ‘Smart fulfillment’ features Problem • Due to lack of correlation between Point of Sale data and inventory, the website contained outdated inventory data. Products were listed as out of stock when there was actually inventory. • Need to leverage store inventory as well as inventory located fulfillment centers Solution • In-Memory stores real-time inventory data for the website, the fulfillment application, and other applications that need access to inventory data Business Impact • Reduction in customer churn • Intelligent fulfillments leading to greater customer satisfaction • Improved overall efficiency of fulfillment centers and store inventory
  24. 24. Distribution of Rapidly Changing Data à Examples are monitoring data for a power plant, stock market data, telemetry data for a complex system (example, a satellite), or the status and locaDon of packages for a major logisDcs or shipping company.
  25. 25. Telco: Real-Time Offer Generation and Fulfillment by Different Subcontractors Purchase 3G Package Cross-sell Voice/SMS package to subscriber who purchases 3G Mobile Package Total: 3 mio / day Peak: 50 events per sec Reload Give 100 free SMS to subscriber who tops-up Total: 12 mio top-up / day Peak: 300 top-up per sec Voice Call Give discount VOIP package to subscriber who makes a IDD call Total: 200 mio / day Peak: 12,000 events per sec SMS Usage Give discounted SMS package to subscriber who sends SMS more than 10 times a day Total: 750 mio / day Peak: 27,000 events per sec Purchase BB Package Event Cloud Reload Voice Call IDD Call OnNet Call SMS Usage Event Handling and Processing Touchpoint Integration Fulfill SMS Package Fulfill 3G Package Fulfill Voice Package Fulfill SMS Package Billing, Offer Fulfilled 46.7 million subscribers 2,000 SMS notifications per seconds 500 offer fulfillments per second Offer Message Reminder Message Fulfillment Message
  26. 26. Storing State-full Data for Enterprise Applications State-­‐full Data
  27. 27. Super Fast Compute Grid for Intermediary Calculations for Analytics
  28. 28. Super Fast Compute Grid for Intermediary Calculations for Analytics • Technical issues in distributed grid compuDng with large scale data – Work load distribuDon – Process synchronizaDon – Data transfer • Examples – Risk assessment and management – OpDmizaDon problems: scheduling, cargo assignment, load distribuDon in power network / grid • Why In-­‐Memory? – Many useful synchronizaDon features (e.g. atomic “take”) – LocaDon transparency and fault-­‐tolerance – Real-­‐Dme instead of nightly / weekly / ... Data-­‐Warehousing approach
  29. 29. Key Messages In-Memory Computing is used for Acting in Real-Time! In-Memory is NOT just for Caching and Storing – A Data Grid offers much more! Eventing and Fault-Tolerance move In-Memory Computing to another Level!
  30. 30. Questions? Kai Wähner kwaehner@tibco.com @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!

×