+
Jason Stamper and Gary Orenstein
IN-MEMORY COMPUTING WEBCAST
MARKET PREDICTIONS 2017
1
Market Predictions 2017:
In-Memory Computing
Jason Stamper, Analyst, 451 Research
@jasonstamper
451 Research is an information
technology research & advisory company
Founded in 2000
210+ employees, including over 100 analysts
1,000+ clients: Technology & Service providers, corporate
advisory, finance, professional services, and IT decision makers
10,000+ senior IT professionals in our research community
Over 52 million data points each quarter
4,500+ reports published each year covering 2,000+
innovative technology & service providers
Headquartered in New York City with offices in London,
Boston, San Francisco, and Washington D.C.
451 Research and its sister company Uptime Institute
comprise the two divisions of The 451 Group
Research & Data
Advisory Services
Events
3
4
451 Research’s view of the ‘Total Data’ Model
5
Growth?
2015 2016 2017 2018 2019 2020
$69,612
$79,612
$90,777
$103,126
$116,796
$132,049
14%
2015-20 CAGR
Source: 451 Research, Forecast: Total Data 2016 - Data
Platforms and Analytics Market Sizing and Forecasts.
284 Vendors included in this
analysis
9 Market segments included
Defining the in-memory database
“An in-memory database (IMDB; also main memory database system or MMDB or memory
resident database) is a database management system that primarily relies on main memory for
computer data storage. It is contrasted with database management systems that employ a disk
storage mechanism.
Main memory databases are faster than disk-optimized databases because the disk access is
slower than memory access, the internal optimization algorithms are simpler and execute fewer
CPU instructions. Accessing data in memory eliminates seek time when querying the data,
which provides faster and more predictable performance than disk.
Applications where response time is critical, such as those running telecommunications
network equipment and mobile advertising networks, often use main-memory databases.
IMDBs have gained a lot of traction, especially in the data analytics space, starting in the mid-
2000s - mainly due to multi-core processors that can address large memory and due to less
expensive RAM.”
- Wikipedia
6
7
In-memory systems come in several guises
 Pure in-memory database
 Disk-based/persistent database with an
in-memory ‘option’ or column store
 In-memory data grid or fabric
What’s driving in-memory adoption?
• Increasing # of users & transactions
• More data!
• Growing # of writes
• Insufficient capacity
• Declining throughput
• Performance inconsistencies
• Cost of ETL processes
8
How are different data platform and analytic
technologies shaping up?
$120,000
$100,000
$80,000
$60,000
$40,000
$20,000
$0
2015 2016 2017 2018 2019 2020
Event/Stream Processing
Data Management
Distributed Data Grid/Cache
Analytic Database
Search
Performance Management
Reporting and Analytics
Hadoop
Operational Databases
$140,000
9
Some questions to ask
 Will my data fit in memory?!
 Is the platform optimized for analytics, transactions or both?
 Is the platform durable (ACID compliant)? What about
restores?
 What is the programming model – does it support SQL?
 Is there an open source or community edition?
 Does it support production requirements such as high
availability, cross data center replication, granular user
permissions, and SSL?
 Can I run it in the cloud, on-premises or both?
10
Some potential solutions, and their pros and cons
• In-memory ‘options’ added to existing relational databases
• Pure in-memory databases
• Data streaming offerings
• Analytics as a service or database as a service – on-
prem/hybrid/cloud
• In memory data grid/cache
11
Some predictions for 2016/7
• Multi-modal databases – that can handle both transactions (OLTP) and
analytics (OLAP) become the norm (with in-memory being a key enabler)
• In-memory databases continue to grow – both pure and ‘hybrid’
• E-commerce, ad-tech, gaming, financial services, high tech grow in-memory
use particularly fast, but all businesses waking up to ‘latency sensitivity’
• From a slow start, the Internet of Things (IoT) gathers pace, thanks to both
demand, and the ability to do something about it
• A new President of the US, and Andy Murray to become world #1
12
Twitter.com/jasonstamper
13
Architecting with In-Memory
Gary Orenstein
The nature of
transactions
has changed.
1515
16
Traditional Transactions Modern Transactions
BATCH
• Exactly-Once
• Governed
• Structured
• ERP/CRM Applications
REAL TIME
• Duplicates
• Optional auditing
• Unstructured
• Social and Sensor
feeds
Modern Transactions
REAL TIME
• Duplicates
• Optional auditing
• Unstructured
• Social and Sensor
feeds
17
Traditional Transactions
BATCH
• Exactly-Once
• Governed
• Structured
• ERP/CRM Applications
Converged Transactions
REAL TIME
• Exactly-Once
• Governed
• Any Structure
• All Sources
A Real-Time Data Platform
Processing real-time and batch data
to maximize traditional
and modern transactions
18
Architecting A Real-Time Data Platform
Database Workloads
Data Warehouse
Workloads
Real-Time
Streaming
19
20
Architecting A Real-Time Data Platform
Orchestration / Containers
Cloud / On-Premises Platform
MessagingInputs Real-Time Applications
Business Intelligence
Dashboards
Relational Key-Value Document Geospatial
Existing Data Stores
Database Workloads
Data Warehouse
Workloads
Real-Time
Streaming
Hadoop Amazon S3MySQL
Transformation
21
A Deeper Look at Two Industry Sectors
Customer 360
Supply Chain Logistics
22
23
DellEMC Adopts MemSQL for Customer 360 Application
+
24
The Need for “CALM”
Customer Asset Lifecycle Management
For
enterprise sales
Who need
accurate and timely customer information
CALM is a
real-time application
Providing
up to the moment customer 360 dashboards
For enterprise sales
Who need accurate and timely customer information
CALM is a real-time application
Providing up to the moment customer 360
o
dashboards
Install Base
Pricing
Device Config
Contacts
Contracts
Analytics Contracts
Component
Data
Offers
Scorecard
Source: DellEMC
25
Data Lake Architecture
D A T A P L A T F O R M
V M W A R E V C L O U D S U I T E
E X E C U T I O N
P R O C E S S GREENPLUM DBSPRING XD PIVOTAL HD
Gemfire
H A D O O P
INGESTION
DATAGOVERNANCE
Cassandra PostgreSQL MemSQL
HDFS ON ISILON
HADOOP ON SCALEIO
VCE VBLOCK/VxRACK | XTREMIO | DATA DOMAIN
A N A L Y T I C S
T O O L B O X
Network WebSensor SupplierSocial Media Market
S T R U C T U R E DU N S T R U C T U R E D
CRM PLMERP
APPLICATIONS
ApacheRangerAttivioCollibra
Real-TimeMicro-BatchBatch
Source: DellEMC
BUSINESS BENEFITS
 Deliver real-time customer updates to EMC Sales Department
 Scale customer analytics to increase sales productivity
 Drive real-time personalization for operational efficiency
TECHNICAL ACHIEVEMENT
 Execute joins across multiple distributed data sets
2626
The Pace of Supply Chain Success
27
Amazon
30 minute post-order
drone deliveries
FedEx
317 million packages
shipped over Xmas
with real-time tracking
Uber Freight
Matching shippers
with trucks in real time
28
| MemEx
An IoT showcase application that powers supply chain
monitoring and management through predictive analytics.
30
TECHNICAL BENEFITS
 Processes 2 million data points, based on 2,000 sensors across
1,000 warehouses
 Two million reads and writes per second
 Combines MemSQL, Apache Kafka, and Streamliner with
machine learning, IoT sensors, and predictive analytics
 Enables enterprises to predict throughput of supply warehouses
| MemEx
30
Data Producers
(simulating
sensor activity)
Raw Sensor 1 + Predictive Score 1
S1 P1
1
| MemEx
Real-Time
Scoring
MemEx UI
32
33
THANK YOU!
Follow our speakers on Twitter
@jasonstamper @garyorenstein
Try MemSQL today at memsql.com/download

In-Memory Computing Webcast. Market Predictions 2017

  • 1.
    + Jason Stamper andGary Orenstein IN-MEMORY COMPUTING WEBCAST MARKET PREDICTIONS 2017 1
  • 2.
    Market Predictions 2017: In-MemoryComputing Jason Stamper, Analyst, 451 Research @jasonstamper
  • 3.
    451 Research isan information technology research & advisory company Founded in 2000 210+ employees, including over 100 analysts 1,000+ clients: Technology & Service providers, corporate advisory, finance, professional services, and IT decision makers 10,000+ senior IT professionals in our research community Over 52 million data points each quarter 4,500+ reports published each year covering 2,000+ innovative technology & service providers Headquartered in New York City with offices in London, Boston, San Francisco, and Washington D.C. 451 Research and its sister company Uptime Institute comprise the two divisions of The 451 Group Research & Data Advisory Services Events 3
  • 4.
    4 451 Research’s viewof the ‘Total Data’ Model
  • 5.
    5 Growth? 2015 2016 20172018 2019 2020 $69,612 $79,612 $90,777 $103,126 $116,796 $132,049 14% 2015-20 CAGR Source: 451 Research, Forecast: Total Data 2016 - Data Platforms and Analytics Market Sizing and Forecasts. 284 Vendors included in this analysis 9 Market segments included
  • 6.
    Defining the in-memorydatabase “An in-memory database (IMDB; also main memory database system or MMDB or memory resident database) is a database management system that primarily relies on main memory for computer data storage. It is contrasted with database management systems that employ a disk storage mechanism. Main memory databases are faster than disk-optimized databases because the disk access is slower than memory access, the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory eliminates seek time when querying the data, which provides faster and more predictable performance than disk. Applications where response time is critical, such as those running telecommunications network equipment and mobile advertising networks, often use main-memory databases. IMDBs have gained a lot of traction, especially in the data analytics space, starting in the mid- 2000s - mainly due to multi-core processors that can address large memory and due to less expensive RAM.” - Wikipedia 6
  • 7.
    7 In-memory systems comein several guises  Pure in-memory database  Disk-based/persistent database with an in-memory ‘option’ or column store  In-memory data grid or fabric
  • 8.
    What’s driving in-memoryadoption? • Increasing # of users & transactions • More data! • Growing # of writes • Insufficient capacity • Declining throughput • Performance inconsistencies • Cost of ETL processes 8
  • 9.
    How are differentdata platform and analytic technologies shaping up? $120,000 $100,000 $80,000 $60,000 $40,000 $20,000 $0 2015 2016 2017 2018 2019 2020 Event/Stream Processing Data Management Distributed Data Grid/Cache Analytic Database Search Performance Management Reporting and Analytics Hadoop Operational Databases $140,000 9
  • 10.
    Some questions toask  Will my data fit in memory?!  Is the platform optimized for analytics, transactions or both?  Is the platform durable (ACID compliant)? What about restores?  What is the programming model – does it support SQL?  Is there an open source or community edition?  Does it support production requirements such as high availability, cross data center replication, granular user permissions, and SSL?  Can I run it in the cloud, on-premises or both? 10
  • 11.
    Some potential solutions,and their pros and cons • In-memory ‘options’ added to existing relational databases • Pure in-memory databases • Data streaming offerings • Analytics as a service or database as a service – on- prem/hybrid/cloud • In memory data grid/cache 11
  • 12.
    Some predictions for2016/7 • Multi-modal databases – that can handle both transactions (OLTP) and analytics (OLAP) become the norm (with in-memory being a key enabler) • In-memory databases continue to grow – both pure and ‘hybrid’ • E-commerce, ad-tech, gaming, financial services, high tech grow in-memory use particularly fast, but all businesses waking up to ‘latency sensitivity’ • From a slow start, the Internet of Things (IoT) gathers pace, thanks to both demand, and the ability to do something about it • A new President of the US, and Andy Murray to become world #1 12
  • 13.
  • 14.
  • 15.
  • 16.
    16 Traditional Transactions ModernTransactions BATCH • Exactly-Once • Governed • Structured • ERP/CRM Applications REAL TIME • Duplicates • Optional auditing • Unstructured • Social and Sensor feeds
  • 17.
    Modern Transactions REAL TIME •Duplicates • Optional auditing • Unstructured • Social and Sensor feeds 17 Traditional Transactions BATCH • Exactly-Once • Governed • Structured • ERP/CRM Applications Converged Transactions REAL TIME • Exactly-Once • Governed • Any Structure • All Sources
  • 18.
    A Real-Time DataPlatform Processing real-time and batch data to maximize traditional and modern transactions 18
  • 19.
    Architecting A Real-TimeData Platform Database Workloads Data Warehouse Workloads Real-Time Streaming 19
  • 20.
    20 Architecting A Real-TimeData Platform Orchestration / Containers Cloud / On-Premises Platform MessagingInputs Real-Time Applications Business Intelligence Dashboards Relational Key-Value Document Geospatial Existing Data Stores Database Workloads Data Warehouse Workloads Real-Time Streaming Hadoop Amazon S3MySQL Transformation
  • 21.
    21 A Deeper Lookat Two Industry Sectors Customer 360 Supply Chain Logistics
  • 22.
  • 23.
    23 DellEMC Adopts MemSQLfor Customer 360 Application +
  • 24.
    24 The Need for“CALM” Customer Asset Lifecycle Management For enterprise sales Who need accurate and timely customer information CALM is a real-time application Providing up to the moment customer 360 dashboards For enterprise sales Who need accurate and timely customer information CALM is a real-time application Providing up to the moment customer 360 o dashboards Install Base Pricing Device Config Contacts Contracts Analytics Contracts Component Data Offers Scorecard Source: DellEMC
  • 25.
    25 Data Lake Architecture DA T A P L A T F O R M V M W A R E V C L O U D S U I T E E X E C U T I O N P R O C E S S GREENPLUM DBSPRING XD PIVOTAL HD Gemfire H A D O O P INGESTION DATAGOVERNANCE Cassandra PostgreSQL MemSQL HDFS ON ISILON HADOOP ON SCALEIO VCE VBLOCK/VxRACK | XTREMIO | DATA DOMAIN A N A L Y T I C S T O O L B O X Network WebSensor SupplierSocial Media Market S T R U C T U R E DU N S T R U C T U R E D CRM PLMERP APPLICATIONS ApacheRangerAttivioCollibra Real-TimeMicro-BatchBatch Source: DellEMC
  • 26.
    BUSINESS BENEFITS  Deliverreal-time customer updates to EMC Sales Department  Scale customer analytics to increase sales productivity  Drive real-time personalization for operational efficiency TECHNICAL ACHIEVEMENT  Execute joins across multiple distributed data sets 2626
  • 27.
    The Pace ofSupply Chain Success 27 Amazon 30 minute post-order drone deliveries FedEx 317 million packages shipped over Xmas with real-time tracking Uber Freight Matching shippers with trucks in real time
  • 28.
  • 29.
    | MemEx An IoTshowcase application that powers supply chain monitoring and management through predictive analytics.
  • 30.
    30 TECHNICAL BENEFITS  Processes2 million data points, based on 2,000 sensors across 1,000 warehouses  Two million reads and writes per second  Combines MemSQL, Apache Kafka, and Streamliner with machine learning, IoT sensors, and predictive analytics  Enables enterprises to predict throughput of supply warehouses | MemEx 30
  • 32.
    Data Producers (simulating sensor activity) RawSensor 1 + Predictive Score 1 S1 P1 1 | MemEx Real-Time Scoring MemEx UI 32
  • 33.
    33 THANK YOU! Follow ourspeakers on Twitter @jasonstamper @garyorenstein Try MemSQL today at memsql.com/download

Editor's Notes

  • #9 Because of these! Today modern area, web-scale. Social. Online. Right here, right now. Virtualization, and cloud… commodity hardware You customers will have some combination of those