More Related Content Similar to 3 Benefits of Multi-Temperature Data Management for Data Analytics (20) More from MapR Technologies (15) 3 Benefits of Multi-Temperature Data Management for Data Analytics1. © 2017 MapR Technologies 1
3 Benefits of Multi-Temperature Data
Management for Data Analytics
Steven Garcia & Joe Love, SAP Mobile Services
Sameer Nori, MapR
April 25th, 2017
2. © 2017 MapR TechnologiesMapR Confidential 2
Agenda
• Trends in Storage
• SAP Consumer Insight 365 Use Case
• MapR & SAP Integration
• Q&A
3. © 2017 MapR TechnologiesMapR Confidential 3
Trends in Storage
4. © 2017 MapR TechnologiesMapR Confidential 4
Changing Enterprise IT Mindset
Pet Cattle
Storage
Intelligence
in Hardware
Storage
Intelligence
in Software
5. © 2017 MapR TechnologiesMapR Confidential 5
Next Gen Apps/Device are driving File & Object Storage Demand
6. © 2017 MapR TechnologiesMapR Confidential 6
New Use Cases & Workloads Are Changing “Storage”
Scientific
Apps
Machine
Data
Analytics +
Log
Image/video
Processing Build
Test
Containers
Global
Repository
ERP
CRM
Cold Data Backup
Home
Dir
OLTP
VMs
Perf
Capacity
Need Storage platform that delivers
• High Capacity
• High Concurrency
• High Throughput
• and …Cost Efficiency
ALL DIGITAL
SCIENCE & MEDIA
ANALYTICS
OF EVERYTHING
CLOUD NATIVE
APPLICATIONS
1
2
3
Creativity & Discovery Are
Now Born Digital
Real Time Responses,
Billions of Connected devices,
A New Framework to Develop
Continuously , Scale on Demand
MapR
Focus
7. © 2017 MapR TechnologiesMapR Confidential 7 7
What customers are telling us …
HDFSS3POSIX Geo-Replication
Self Healing
Assume Failures are Common
Many Small Sites
A Few Big Sites
Leverages Cloud
Globally Protected
Globally Accessible
Globally Managed
NFS
Need Scalable Storage Platform Around the globe, Around the clock
Data Protection
8. © 2017 MapR TechnologiesMapR Confidential 8
Customer Benefits
Global Namespace
Eliminate
Silos
Hot
Multi-Temp
Data Mgmt
Web
Apps
File
Apps
Bring Apps
Closer to Data
Container
Apps Storage Software
Deploy
Anywhere
Public
Cloud
Warm
Cold
Commodity
Servers
S3NFS HDFS POSIX
CoreEdge Cloud
9. © 2017 MapR TechnologiesMapR Confidential 9
Current Landscape
Customer Spend
Hardware Software
Customer Spend
Hardware Software
New Usage
•Longer data history (cost effective
warm tier for Hana)
•Better performance for existing IQ
apps
Current Scenario
SAN/NAS
MapR
+
Commodity
H/W
SAP IQ
SAP IQ
10. © 2017 MapR TechnologiesMapR Confidential 10
Database
MapR-DB
Event Streaming
MapR Streams
High Availability
Web-Scale Storage
MapR-FS
Real Time Unified Security Multi-tenancy Disaster Recovery Global Namespace
MapR-A Different Approach: Converged Data Platform
Files, Tables, Streams
together on same platform
Shared Services
Supports Open-Source APIs
On-Premise, In the Cloud, Hybrid
Patented Architecture
11. © 2017 MapR TechnologiesMapR Confidential 11
SAP Consumer Insight 365
13. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 13Public
SAP Mobile Services Overview
500+
enterprises
and their customers are
engaged each year
1,095
operators
interconnected
worldwide
1+ billion
messages
processed daily
331
billion
total messages
processed in 2016
10.8
billion A2P
messages
processed in 2016
320
billion P2P
messages
processed in 2016
220
countries
and territories
reached*
7.38 billion
or 99%
of subscribers
reached worldwide
500+
operators
supported on
IPX
Enterprise services
SAP
Mobile
Services
Consumer data
and analytics SAP add-ons
Easily execute and manage
campaigns, authenticate users,
and apply rules-based delivery to
customers around the globe
Aggregated, anonymized, and
enhanced mobile ambient data
creates new revenue streams and
insight services
Operator services
Simplify interconnectivity and
IoT management, and optimize
SMS traffic through a single
contract and connection
Enable messaging extensions
for SAP Hybris, SAP
SuccessFactors, and other
SAP solutions
Our Network and Capabilities Our Portfolio and Services
14. Enabling digital business and conversations
require global reach and connectivity
Connectivity and reach
SAP
Mobile
Services
Consumer data and
analytics Internet of Things
SAP Enterprise SMS 365 provides SMS-
based connectivity to consumers around
the globe
SAP Intelligent Notifications 365 brings
rules-based in-app push,
e-mails, T2V, and SMS delivery
SAP Digital Consumer Insight 365
aggregated, anonymized, and enhanced
consumer mobile data
Engagement services
SAP Engagement 365 easily configures
and manages campaigns and promotions
SAP Authentication 365 offers
SMS-based 2FA capabilities
SAP Add-on enables messaging
extensions for SAP Hybris, SAP
SuccessFactors, and other SAP solutions
SAP IoT Connect 365 simplifies global
connected-object management through a
single contract and connection
15. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 15Public
321
SAP Consumer Insight 365 unlocks and enhances mobile data
Anonymized near real-time
mobile subscriber data available
by different countries/region
from Mobile Network Operators
Data enriched with other
sources and enhanced with
predictive algorithms, offered
as a cloud-based service
Detailed reports available for pre-defined
use cases, ready for consumption based
on industry workflows - delivered through
a web based portal UI and or API’s
Portal UI
APIAPI
Retail
CP
Media &
Entertainment
Agencies
SAP Mobile
Services Cloud
Analytics
Data Enrichment & Predictive Enhancement
Maps Points of Interest Demographics
SAP Consumer Insight 365
16. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 16Public
SAP Consumer Insight 365
Catchment
Where are they coming from?
Clickstream
What are they doing?
Footfall
Who are my customers?
Determine home location of
consumers, to drive circular
ad campaigns
Display origin of home
location for a given retail
store or location
List top 10 locations of origin
Optimize digital advertising
ROI by understanding
consumer digital click
through behavior / where did
it break in the conversion
chain?
What are people searching
for in a particular location
Determine demographics of
people are coming through a
location
Compute dwell time, input
rate, exit rate over time
across multiple locations
Compare competitors and
location over time
Enables brands /ad agencies
to understand where they
should spend their ad dollars
Drives brand / ad agency
digital ad spend
Enables brands / ad
agencies to understand ad
effectiveness/demographics
SAP Consumer Insight 365 is a cloud service powered by HANA that
enables insights on Trillions of anonymized mobile operator records
17. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 17Public
SAP is uniquely positioned to bring to market these mobile focused
consumer insights
Suppliers (Mobile Operators)
Trust: SAP Mobile Services serves 1000
network operators worldwide as part of
its SMS Intelligent Hubbing business and
programmable communication services.
Big Data Expertise: Extract up to 40-50
TB/day/operator worldwide
Technology Levers: Predictive Analytics, Real Time HANA / IQ Based Big Data Platform
Business Levers: Mobile Operator Relationships as part of SAP Mobile Services
Buyers (Brand & Agencies, others..)
Ease of Consumption: Cloud based
offering
Completeness of Solution: Effective
integration and delivery with existing
LoB and Industry Solutions
18. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 18Public
SAP Consumer Insight 365 is already winning industry acclaim
Finalist - Design for Experience Award,
“Bringing Order to Big Data”, UX Magazine
Winner - Emerging Technologies Award (E-
TECH), September 2014, Las Vegas
Gold Medal, 2014 UX Design Awards, San
Francisco, September 2014
Winner - Communication Design.
Red Dot Awards 2016
Winner - Product Excellence.
Spotlight Big Data As A Service
Award, 2016
19. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 19Public
Challenges
Volumes based on Single Supplier with 25 million Subscribers
20. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 20Public
Benefits of Deploying the MapR Platform
Architectural
• MapR platform is
modular &
componentized via
Hadoop design
• Easy implementation
• Work on few nodes at
a time and maintain
cluster integrity &
performance
Operational
• Scaling up or down is
very easy
• Many upgrade
options
• Retrofit and/or
improvement of
components can be
done without bringing
the system down
Economical
• Between a 2.5 to 1 in
lower tiered cost
saving and a 10 to 1
for the high end
commodity storage.
21. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 21Public
POC Performance Benchmarks
8 Node MapR cluster with 256GB RAM each, 4x10GB links
1.5 GB/s sustained write speeds per multiplex node (tested up to 4 IQ nodes), near linear
scale per node
No observed performance impact during forced MapR node failures.
3 nodes were failed within a few minutes of each other and created NO errors.
Once 4th node was failed, IQ disk errors began to occur. A simple shutdown of IQ and restart of the MapR
cluster nodes resulted in complete recovery.
22. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 22Public
What our Processing Landscape Looks Like
20 Volumes dedicated to IQ DBSpaces – Extremely high activity
Compression off (IQ compression is used)
Striped across multiple files (currently groups of 8 at a time), enforce 8 files to 8 distinct nodes.
Many dbspaces go static once fully loaded
Mirror volumes in use for changing data db spaces.
18 Volumes for raw data archives - low activity
4-6 Volumes for other systems backups – burst activity
8 volumes per carrier for file processing and data loading – Extremely high activity
Several volumes are only duplicate because of high transient nature
Hundreds of thousands of files in some directories, with tens of thousands open at any time.
23. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 23Public
SAP Consumer Insight 365
Architectural Overview
SAP Consumer
Insight 365
SAP Consumer
Insight 365
dashboard
SAP Consumer Insight
365 subscriber
SAP Consumer Insight
365 admin
MapR Converged Data Platform
24. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 24Public
Our Real World Experience
We got ahead of ourselves because of ease of implementation
Data growth rates which significantly exceeded our projections slowed our implementation of system,
installation automation as well as service provisioning
We have been learning as we go (We intend to publish much of this)
MapR (the company) is VERY helpful when we have a problem and been a true partner
Getting above 90% space usage across the cluster seems to be a pain point.
Things change around this number (give or take)
Our latest version of 5.2 is working better around this
25. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 25Public
Our Real World Experience
Large numbers of files in a single volume can affect processing (not unique to MapR-FS)
For most things it just works and is highly durable.
• We have had to learn a few tricks to force container/file distribution with respect to ultra-
high performance requirements for IQ
• This was fairly easy to learn how to do and was done in conjunction with MapR and Mobile Services during
the POC on IQ.
26. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 26Public
Mobile Services MapR Roadmap
Continued work with various tuning options to achieve best performance per volume profile
Disk pool sizes
Compression changes
Replication management
Tiered storage and use of higher speed drive groups (ssd, flash, etc.)
“Non-Stop IQ” or NSIQ complete implementation
Using mirrors, snapshots and cooperative technology for point in time recovery and offline database
maintenance procedures
Converting other IQ implementations to MapR based storage for data servers as well as
application related file systems
27. © 2017 SAP SE or an SAP affiliate company. All rights reserved. 27Public
Mobile Services MapR Roadmap
Enhanced monitoring and benchmarking
POC and implementation for other database engines(ASE, HANA, etc.)
POSIX/FUSE client use
Evaluate, design, and implement where appropriate other MapR ecosystem components
User tools for provisioning and volume management
28. © 2017 MapR TechnologiesMapR Confidential 28
MapR and SAP Integration
29. © 2017 MapR TechnologiesMapR Confidential 29
Open Source Engines & Tools Commercial Engines & Applications
Enterprise-Grade Platform Services
DataProcessing
Web-Scale Storage
MapR-FS MapR-DB
Search
and
Others
Real Time Unified Security Multi-tenancy Disaster Recovery Global NamespaceHigh Availability
MapR Streams
Cloud
and
Managed
Services
Search and
Others
UnifiedManagementandMonitoring
Search
and
Others
Event StreamingDatabase
Custom
Apps
MapR Converged Data Platform
HDFS API POSIX, NFS Kafka APIHBase API OJAI API
30. © 2017 MapR TechnologiesMapR Confidential 30
Commercial Engines & Applications
DataProcessing
UnifiedManagementandMonitoring
MapR Converged Data Platform
SAP
HANA
Vora™
Data Temperature
Enterprise-Grade Platform Services
Web-Scale Storage
MapR-FS
MapR-DB
Real Time Unified Security Multi-tenancy Disaster Recovery Global NamespaceHigh Availability
MapR Streams
Event StreamingDatabase
HDFS APIPOSIX, NFS Kafka APIHBase API OJAI API
31. © 2017 MapR TechnologiesMapR Confidential 31
Data Placement Control for Multi-Temperature Data
Data from most
recent 6 months
(volumes on most powerful nodes)
Data beyond
6 months
(volumes on typical nodes)
Archive data
(volumes on dense disk nodes)
Hot data Cold data
Faster nodes,
faster disks
Lower cost nodes,
higher disk densities
Volumes tied to specific grouping
of nodes (topologies) in cluster
via data placement control
32. © 2017 MapR TechnologiesMapR Confidential 32
MapR/ SAP IQ Integration**
Supercharging SAP IQ Multiplex with MapR
Better Performance
Increased system performance
Linear I/O Scalability
Multi-temperature Petascale analytics
Reduced Infrastructure Cost
Standard Switching
Leverage commodity hardware
Big data scale-out capacity
Built-in high availability, data
protection, and disaster recovery• **SAP/MapR Integration allows clients to leverage MapR Data Platform to
support SAP applications such as Dynamic Tiering, SAP IQ, and NLS (Near-
Line Storage) for BW.
Standard Networking
33. © 2017 MapR TechnologiesMapR Confidential 33
Summary
• Unified platform for multi-temperature data management
• Reduce storage silos
• Minimize data movement
34. © 2017 MapR TechnologiesMapR Confidential 34
ResourcesEngage with us!
1. SAP IQ & MapR Technical Brief
2. MapR File System selected by SAP for cloud storage layer
3. Try out the MapR sandbox