Db2 Event Store
A complete solution for Fast Data
Data must be stored
and shared efficiently
Large data sets require
cost effective storage
Data must be available
to entire organization
without requiring
replication
What is Fast Data?
Platform must be
always available
If ingest stops, data
could be lost forever
Interruption in fast
insights results in sub-
optimal business
decisions
Data is useless
without fast insights
Data value decays
rapidly over time
Insights must be
derived quickly
Data is arriving faster
than ever before
Billions of data events
are generated every
day
Data must be landed
quickly, or thrown
away
Fast Data is the application of big data analytics to large data sets, in
real-time or near real-time, to solve a problem or create business value.
Fast Data poses several challenges:
Fast Data in Practice
IBM CONFIDENTIAL
Financial fraud detection
• Fraud detection in Millions of
financial transactions every
hour
• Fraud must be detected
quickly, accurately, and
intelligently
Capacity management
• Monitor and track network
usage to optimally allocate
capacity
• Network data must be tracked
at several levels to be useful
Predictive Maintenance
• Monitor and track thousands to
millions of devices to predict
failures
• Early maintenance avoids
catastrophic failures & outages
… with storage cost
efficiency
Writes to shared
storage in compressed
Apache Parquet
format
Able to leverage low-
cost object storage
Single copy of the data
Db2 Event Store
… and is built for high
availability
Remains available to
ingest and analytics
in the presence of
failures
Architected to scale
to very large clusters
… and real-time
analytics
Real-time analytics
over ALL ingested data
Super-fast lookups
and intelligent scans
Integrated machine
learning capabilities
via Data Science
Experience
Delivers lightning fast
ingest
250 billion events/day
Ingest scales linearly
with additional nodes
Data ingested quickly,
then refined and
enriched
A unified offering, built from the ground up to support the specific
demands of Fast Data
IBM CONFIDENTIAL
Db2 Event Store
Lightning fast
ingest
Deep & Real-time
Analytics
Shared storage
Db2 Event Store Ecosystem
IBM
Streams
demo
IBM CONFIDENTIAL
Traditional Big Data
query processing
Raw Parquet files
Not typically cached
Synopsis requires loading full file
No indexing
Raw Parquet files + external synopsis (cacheable)
Immutable index (object storage + cache)
Faster analytics = Faster insights + answers to
more questions
synopsis immutable index
Typical IoT Customer Case
Customer
Needs
• Complete stack to monitor expensive or plentiful “devices”
• Must be able to manage data at high volume, and store it cost
effectively
• Required ability to incorporate predictive modeling
• Existing architectures not fast enough to handle ingest and
analytical needs
Benefits • Integrated stack - IBM Db2 Event Store + DSX Local
• High speed ingest into Db2 Event Store - 3 million data points
per second using a 3 node Event Store cluster
• Analytical queries against large data sets blows away the
competition
Complete modernization of entire data
stack provides improved stability, ability
to scale as their workload demands
increase, and a modern analytics
environment for their data scientists.
Results:
Developer Edition
Free Download and Go edition
Great for getting started, writing your first application
Packaged with Desktop version of DSX
Runs on MacOS, Linux, Windows via Docker
Download from www.ibm.biz/eventstore
Enterprise Edition
Production level offering
Includes high availability, monitoring UI, REST API
Packaged with DSX Local
Comes in 1 and 3 node installers
Docker + Kubernetes for orchestration
Db2 Event Store Packaging
Backup
Who is Lightbend?
Lightbend is the company behind open source technologies Scala, Akka, Play &
Lagom
Lightbend’s Reactive Platform is an application development platform and runtime
for building modern, microservices-based distributed applications in Scala or Java
and running on the JVM
Lightbend’s Fast Data Platform adds a curated set of technologies designed to
make it faster and easier to build large-scale, real-time streaming applications
Lightbend’s customers are Global 2000 enterprises developing custom
applications, especially those investing in digital transformation, real-time business
processes, IoT, mobile business and legacy modernization
• Enables the
developer and data
scientists to
efficiently bring to
market fast data
solutions
• Incorporates
microservices,
streaming data
analytics, and
machine learning
IBM Db2 Event Store
• For fast, scalable persistence
IBM DSX Local
• For Data Scientists to collaborate, build and
export machine learning models
Lightbend Fast Data Platform
• Build streaming analytic solutions and
microservices
IBM Fast Data
Platform
IBM Fast Data Platform - Architecture

Db2 event store

  • 1.
    Db2 Event Store Acomplete solution for Fast Data
  • 2.
    Data must bestored and shared efficiently Large data sets require cost effective storage Data must be available to entire organization without requiring replication What is Fast Data? Platform must be always available If ingest stops, data could be lost forever Interruption in fast insights results in sub- optimal business decisions Data is useless without fast insights Data value decays rapidly over time Insights must be derived quickly Data is arriving faster than ever before Billions of data events are generated every day Data must be landed quickly, or thrown away Fast Data is the application of big data analytics to large data sets, in real-time or near real-time, to solve a problem or create business value. Fast Data poses several challenges:
  • 3.
    Fast Data inPractice IBM CONFIDENTIAL Financial fraud detection • Fraud detection in Millions of financial transactions every hour • Fraud must be detected quickly, accurately, and intelligently Capacity management • Monitor and track network usage to optimally allocate capacity • Network data must be tracked at several levels to be useful Predictive Maintenance • Monitor and track thousands to millions of devices to predict failures • Early maintenance avoids catastrophic failures & outages
  • 4.
    … with storagecost efficiency Writes to shared storage in compressed Apache Parquet format Able to leverage low- cost object storage Single copy of the data Db2 Event Store … and is built for high availability Remains available to ingest and analytics in the presence of failures Architected to scale to very large clusters … and real-time analytics Real-time analytics over ALL ingested data Super-fast lookups and intelligent scans Integrated machine learning capabilities via Data Science Experience Delivers lightning fast ingest 250 billion events/day Ingest scales linearly with additional nodes Data ingested quickly, then refined and enriched A unified offering, built from the ground up to support the specific demands of Fast Data
  • 5.
    IBM CONFIDENTIAL Db2 EventStore Lightning fast ingest Deep & Real-time Analytics Shared storage Db2 Event Store Ecosystem IBM Streams
  • 6.
  • 9.
    Traditional Big Data queryprocessing Raw Parquet files Not typically cached Synopsis requires loading full file No indexing Raw Parquet files + external synopsis (cacheable) Immutable index (object storage + cache) Faster analytics = Faster insights + answers to more questions synopsis immutable index
  • 10.
    Typical IoT CustomerCase Customer Needs • Complete stack to monitor expensive or plentiful “devices” • Must be able to manage data at high volume, and store it cost effectively • Required ability to incorporate predictive modeling • Existing architectures not fast enough to handle ingest and analytical needs Benefits • Integrated stack - IBM Db2 Event Store + DSX Local • High speed ingest into Db2 Event Store - 3 million data points per second using a 3 node Event Store cluster • Analytical queries against large data sets blows away the competition Complete modernization of entire data stack provides improved stability, ability to scale as their workload demands increase, and a modern analytics environment for their data scientists. Results:
  • 11.
    Developer Edition Free Downloadand Go edition Great for getting started, writing your first application Packaged with Desktop version of DSX Runs on MacOS, Linux, Windows via Docker Download from www.ibm.biz/eventstore Enterprise Edition Production level offering Includes high availability, monitoring UI, REST API Packaged with DSX Local Comes in 1 and 3 node installers Docker + Kubernetes for orchestration Db2 Event Store Packaging
  • 12.
  • 13.
    Who is Lightbend? Lightbendis the company behind open source technologies Scala, Akka, Play & Lagom Lightbend’s Reactive Platform is an application development platform and runtime for building modern, microservices-based distributed applications in Scala or Java and running on the JVM Lightbend’s Fast Data Platform adds a curated set of technologies designed to make it faster and easier to build large-scale, real-time streaming applications Lightbend’s customers are Global 2000 enterprises developing custom applications, especially those investing in digital transformation, real-time business processes, IoT, mobile business and legacy modernization
  • 14.
    • Enables the developerand data scientists to efficiently bring to market fast data solutions • Incorporates microservices, streaming data analytics, and machine learning IBM Db2 Event Store • For fast, scalable persistence IBM DSX Local • For Data Scientists to collaborate, build and export machine learning models Lightbend Fast Data Platform • Build streaming analytic solutions and microservices IBM Fast Data Platform
  • 15.
    IBM Fast DataPlatform - Architecture