From Data to Services at the Speed of Business
Ali Hodroj
Vice President, Products and Strategy
Overview
• Analytics value chain
• Challenges of analytics-driven
transformation
• Microservices approach to
analytics with In-memory
computing
• Q&A
Direct customers
300+
Fortune / Organizations
50+ / 500+
Large installations in
production (OEM)
5,000+
ISVs
25+
GigaSpaces Technologies
In-memory computing insight platform for mission critical applications
4
Direct customers
300+
Fortune / Organizations
50+ / 500+
Large installations in
production (OEM)
5,000+
ISVs
25+
From Data to Services at the Speed of Business
Analytics
Machine Learning
Agility
Autonomy
Low total cost of ownership
Real-time
Just-in-time decisions
Time-sensitive data
New business models
Fast innovation
Value-driven alignment
Ingest
Data
The analytics and data-driven “value chain”
The key to becoming insight-driven is to optimize the analytics value chain
Analytics Action Performance
Perform Trigger Improve
r
Strategic thinking & High Value Questions
Design and Implementation
$13.01 forevery$1
a company spends on analytics, it
gets back spend on data
management and analytics
Source: MIT Sloan, NucleusResearch
Value of leveraging analytics and data-driven decisions
74%of firms say they want to be data-
driven, but only 23%are successful
Source: Forbes: Actionable Insight: Missing Link between Data and Value
2x [companies are twice] likely to
outperform their peers if they use
advanced analytics
Source: MIT Sloan
From Data to Services at the Speed of Business
Slow innovation path Lack of
realtime
view
Misalignment
between
analytics and
business
decisions
Top 3 Challenges
Businesses want real-time
view into critical data
Analytics infrastructures are
focused on data accumulation and
retrospective analysis
Paradox#1:Rear-viewmirrorarchitectures
Analytics-driven means:
experiment, fail fast, recover
fast and learn rapidly
Data science is 80% data
preparation and 20% analytics
Paradox#2:80/20data-to-analyticsratio
Analytics need tight
alignment with business
decisions
Most efforts are focused on
managing technology platforms
and data lake governance
Paradox#3:SOAdéjàvu
From Data to Services at the Speed of Business
Distributed Analytics
(Apache Spark)
Microservices-driven
Architecture
In-Memory Data and
Business Logic
Processing
Cloud-Native
and Agile
Infrasructure
Microservice-driven and In-Memory Computing
Approach?
What is In-Memory Computing?
Spark + In-Memory Data Grid
Large-scale distributed
analytics framework
Unified, scale-out, low-latency data store
Transactional capabilities:
ACID, Event-Driven, Rich Data
modeling
Microservices
16
Elastic Scale-out In-Memory Storage
(Shared-nothing, Linear scalability, Elastic capacity)
Low latency and high throughput
(co-located ops, event-driven, fast indexing)
High availability and Resiliency
(auto-healing, multi-data center replication, fault tolerance)
Rich API and Query Language
(SQL, Spring, Java, .NET, C++)
GigaSpaces XAP In-Memory Data Grid
17
In-Memory Data Grid: How it works
18
Offload data tiers to partitioned in-memory
The database goes
to the background
Partition your data
and store it in memory
19
Horizontally scale to converge additional data sources
Partitioned, co-located
in-memory data storage
20
Incrementally migrate new workloads and applications
Business logic, data &
messaging co-located
& partitioned into
processing units
21
GigaSpaces automatically ensures High Availability
Hot backup for each
partition for high
availability
22
On-board new applications and micro-services
Host your web
application on the
XAP infrastructure
23
Harness commodity/cloud infrastructure and multi-tier storage
Auto-scale out & in
based on real-time
performance & load
24
Result:
Real-time distributed
compute and storage for
transactions and analytics
25
• Unified & Concise API
• Highly Flexible Data Store Integration
• Massive Community and Adoption
Why Spark?
Geo-Spatial Full Text
In-Memory Data Grid + Spark Convergence
Unified Hybrid Transactional/Analytics Architecture
node 1
Spark master
Grid
master
node 2
Spark worker
Grid
Partition
node 3
Spark worker
Grid
Partition
Lightweight
workers,
small JVMs
Large JVMs,
Fast
indexing
• Push-down predicates (ultra-low latency processing,
30x performance improvement)
• Stateful data-360 sharing across analytics jobs
• Data-locality for high throughput
• Five 9s High Availability
Decoupled Hybrid Architecture
In-Memory Data Grid
Realtime Replication
• Scoring models
• Trigger actions
• Events
Transactions Analytics
• Useful when analytics are
mostly batch or long-
running queries.
• Analytics grid can be used
for frequent model training
(CPU intensive), without
impacting transactional
apps
• Flexibility in write-heavy
(transactions) and read-
heavy (analytics)
independent scaling Application
developers
Data Scientists &
Analysts
From Data to Services at the Speed of Business
Putting it all together
(GigaSpaces InsightEdge Platform)
GigaSpaces In-Memory Computing Platform
Core In-Memory Data Grid
Event
Processing
Data Models
(Spatial, POJO, JSON)
RPC &
Map/Reduce
Web
Containers
Storage-Class Memory
(3DXPoint)
Cluster Management and Service Discovery
Microservices (REST)
Event
Processing
Streaming
Machine
Learning
Spark SQL
Analytics & Big Data
SearchSQL/JDBC
Search and Query
RESTOrchestration
ManagementandMonitoring
SecurityandAuditing
WebMobile Devices
On-Premises Cloud Hybrid
Transactional Apps and
Customer Experience
Message Brokers and Data
Science Workbench
Business Intelligence and
Text Mining
Why In-Memory Data Grid?
SQL-99, Polyglot
Data & Search
Multi-Tiered Data
Storage
Cloud-Nativeand Horizontally
Scalable
• RAM
• SSD/Flash
• Storage-Class Memory
(3DXPoint)
• SQL ‘99
• Graph
• JSON
• POJO
• GeoSpatial
• Full Text
Distributed In-Grid Analytics
• SQL
• Streaming
• Machine Learning
• Graph Processing
• Deep Learning
• Textmining
• Geospatial
• In-Memory Event-Driven
Processing
• Distributed Tasks and Compute
Grid
• Real-time Web Services
• In-Memory Aggregations
Advanced In-Grid Transactions and Analytics
Processing Across Center, Edge and Cloud
GigaSpaces
Hadoop
Embracing an open source analytics ecosystem
Pick your own fast data architecture (lambda, kappa) and co-locate transaction processing
Kafka
Spark
Simplified Lambda Architecture
(Realtime + Historical)
Case Study: Magic Software
IoT Hub + Predictive Analytics (Automotive Telematics)
Challenge:
• Implement predictive analytics and anomaly detection
• Expand insight context through customer/data-360
integration
• Trigger transactional workflows based on prediction criteria
Solution:
• Simplified HTAP with Streaming data pipeline (3 tiers)
• IoT streaming analytics with 9s high availability
“GigaSpaces enables our
customers to simplify and
accelerate telemetry
ingestion, to gain full
business value from IoT
adoption.”
Yuval Lavi, VP of Innovation
Magic Software
http://www.magicsoftware.com
Key Takeaways
By the end of this presentation, you hopefully understood that:
➔ Microservices architectures provides data science agility!
Capturing business value from real-time insight can be accomplished by
embracing an agile open source tool chain coupled with in-memory data
grids.
➔ Insight platforms scale at the speed of business
Embracing insight platforms enables an agile path towards real-time digital
transformation and the data-driven enterprise.
➔ Try it all out – It’s open source!
http://insightedge.io / http://gigaspaces.com
http://github.com/InsightEdge
http://insightedge.slack.com
hello@insightedge.io
Book a demo:
Webinars
UPCOMING WEBINARS
Wed, Aug 23 | 2:00 PM EST
Machine Learning and Insight Platforms
RECORDED WEBINAR
From Data to Services at the Speed of
Business
Powering IoT Integration Solutions:
Turning Data from Sensors into Real-time
Actionable Insights
Real-time Microservices: From Zero to
Production in Under 3 Months
Visit our site to sign up:
www.gigaspaces.com/webinars
Q&A
www.gigaspaces.com
Thank you for attending!

From Data to Services at the Speed of Business

  • 1.
    From Data toServices at the Speed of Business Ali Hodroj Vice President, Products and Strategy
  • 2.
    Overview • Analytics valuechain • Challenges of analytics-driven transformation • Microservices approach to analytics with In-memory computing • Q&A
  • 3.
    Direct customers 300+ Fortune /Organizations 50+ / 500+ Large installations in production (OEM) 5,000+ ISVs 25+ GigaSpaces Technologies In-memory computing insight platform for mission critical applications
  • 4.
  • 5.
    Direct customers 300+ Fortune /Organizations 50+ / 500+ Large installations in production (OEM) 5,000+ ISVs 25+
  • 6.
    From Data toServices at the Speed of Business Analytics Machine Learning Agility Autonomy Low total cost of ownership Real-time Just-in-time decisions Time-sensitive data New business models Fast innovation Value-driven alignment
  • 7.
    Ingest Data The analytics anddata-driven “value chain” The key to becoming insight-driven is to optimize the analytics value chain Analytics Action Performance Perform Trigger Improve r Strategic thinking & High Value Questions Design and Implementation
  • 8.
    $13.01 forevery$1 a companyspends on analytics, it gets back spend on data management and analytics Source: MIT Sloan, NucleusResearch Value of leveraging analytics and data-driven decisions 74%of firms say they want to be data- driven, but only 23%are successful Source: Forbes: Actionable Insight: Missing Link between Data and Value 2x [companies are twice] likely to outperform their peers if they use advanced analytics Source: MIT Sloan
  • 9.
    From Data toServices at the Speed of Business Slow innovation path Lack of realtime view Misalignment between analytics and business decisions Top 3 Challenges
  • 10.
    Businesses want real-time viewinto critical data Analytics infrastructures are focused on data accumulation and retrospective analysis Paradox#1:Rear-viewmirrorarchitectures
  • 11.
    Analytics-driven means: experiment, failfast, recover fast and learn rapidly Data science is 80% data preparation and 20% analytics Paradox#2:80/20data-to-analyticsratio
  • 12.
    Analytics need tight alignmentwith business decisions Most efforts are focused on managing technology platforms and data lake governance Paradox#3:SOAdéjàvu
  • 13.
    From Data toServices at the Speed of Business Distributed Analytics (Apache Spark) Microservices-driven Architecture In-Memory Data and Business Logic Processing Cloud-Native and Agile Infrasructure Microservice-driven and In-Memory Computing Approach?
  • 14.
  • 15.
    Spark + In-MemoryData Grid Large-scale distributed analytics framework Unified, scale-out, low-latency data store Transactional capabilities: ACID, Event-Driven, Rich Data modeling Microservices
  • 16.
    16 Elastic Scale-out In-MemoryStorage (Shared-nothing, Linear scalability, Elastic capacity) Low latency and high throughput (co-located ops, event-driven, fast indexing) High availability and Resiliency (auto-healing, multi-data center replication, fault tolerance) Rich API and Query Language (SQL, Spring, Java, .NET, C++) GigaSpaces XAP In-Memory Data Grid
  • 17.
  • 18.
    18 Offload data tiersto partitioned in-memory The database goes to the background Partition your data and store it in memory
  • 19.
    19 Horizontally scale toconverge additional data sources Partitioned, co-located in-memory data storage
  • 20.
    20 Incrementally migrate newworkloads and applications Business logic, data & messaging co-located & partitioned into processing units
  • 21.
    21 GigaSpaces automatically ensuresHigh Availability Hot backup for each partition for high availability
  • 22.
    22 On-board new applicationsand micro-services Host your web application on the XAP infrastructure
  • 23.
    23 Harness commodity/cloud infrastructureand multi-tier storage Auto-scale out & in based on real-time performance & load
  • 24.
    24 Result: Real-time distributed compute andstorage for transactions and analytics
  • 25.
    25 • Unified &Concise API • Highly Flexible Data Store Integration • Massive Community and Adoption Why Spark?
  • 26.
    Geo-Spatial Full Text In-MemoryData Grid + Spark Convergence
  • 27.
    Unified Hybrid Transactional/AnalyticsArchitecture node 1 Spark master Grid master node 2 Spark worker Grid Partition node 3 Spark worker Grid Partition Lightweight workers, small JVMs Large JVMs, Fast indexing • Push-down predicates (ultra-low latency processing, 30x performance improvement) • Stateful data-360 sharing across analytics jobs • Data-locality for high throughput • Five 9s High Availability
  • 28.
    Decoupled Hybrid Architecture In-MemoryData Grid Realtime Replication • Scoring models • Trigger actions • Events Transactions Analytics • Useful when analytics are mostly batch or long- running queries. • Analytics grid can be used for frequent model training (CPU intensive), without impacting transactional apps • Flexibility in write-heavy (transactions) and read- heavy (analytics) independent scaling Application developers Data Scientists & Analysts
  • 29.
    From Data toServices at the Speed of Business Putting it all together (GigaSpaces InsightEdge Platform)
  • 30.
    GigaSpaces In-Memory ComputingPlatform Core In-Memory Data Grid Event Processing Data Models (Spatial, POJO, JSON) RPC & Map/Reduce Web Containers Storage-Class Memory (3DXPoint) Cluster Management and Service Discovery Microservices (REST) Event Processing Streaming Machine Learning Spark SQL Analytics & Big Data SearchSQL/JDBC Search and Query RESTOrchestration ManagementandMonitoring SecurityandAuditing WebMobile Devices On-Premises Cloud Hybrid Transactional Apps and Customer Experience Message Brokers and Data Science Workbench Business Intelligence and Text Mining
  • 31.
    Why In-Memory DataGrid? SQL-99, Polyglot Data & Search Multi-Tiered Data Storage Cloud-Nativeand Horizontally Scalable • RAM • SSD/Flash • Storage-Class Memory (3DXPoint) • SQL ‘99 • Graph • JSON • POJO • GeoSpatial • Full Text Distributed In-Grid Analytics • SQL • Streaming • Machine Learning • Graph Processing • Deep Learning • Textmining • Geospatial • In-Memory Event-Driven Processing • Distributed Tasks and Compute Grid • Real-time Web Services • In-Memory Aggregations Advanced In-Grid Transactions and Analytics Processing Across Center, Edge and Cloud
  • 32.
    GigaSpaces Hadoop Embracing an opensource analytics ecosystem Pick your own fast data architecture (lambda, kappa) and co-locate transaction processing Kafka Spark Simplified Lambda Architecture (Realtime + Historical)
  • 33.
    Case Study: MagicSoftware IoT Hub + Predictive Analytics (Automotive Telematics) Challenge: • Implement predictive analytics and anomaly detection • Expand insight context through customer/data-360 integration • Trigger transactional workflows based on prediction criteria Solution: • Simplified HTAP with Streaming data pipeline (3 tiers) • IoT streaming analytics with 9s high availability “GigaSpaces enables our customers to simplify and accelerate telemetry ingestion, to gain full business value from IoT adoption.” Yuval Lavi, VP of Innovation Magic Software http://www.magicsoftware.com
  • 34.
    Key Takeaways By theend of this presentation, you hopefully understood that: ➔ Microservices architectures provides data science agility! Capturing business value from real-time insight can be accomplished by embracing an agile open source tool chain coupled with in-memory data grids. ➔ Insight platforms scale at the speed of business Embracing insight platforms enables an agile path towards real-time digital transformation and the data-driven enterprise. ➔ Try it all out – It’s open source! http://insightedge.io / http://gigaspaces.com http://github.com/InsightEdge http://insightedge.slack.com hello@insightedge.io Book a demo:
  • 35.
    Webinars UPCOMING WEBINARS Wed, Aug23 | 2:00 PM EST Machine Learning and Insight Platforms RECORDED WEBINAR From Data to Services at the Speed of Business Powering IoT Integration Solutions: Turning Data from Sensors into Real-time Actionable Insights Real-time Microservices: From Zero to Production in Under 3 Months Visit our site to sign up: www.gigaspaces.com/webinars
  • 36.
  • 37.

Editor's Notes

  • #4 Gigaspaces is a software company focused on in-memory computing, fast data analytics, and insight platforms. We were established in the early 2000s. We were one of the first to digitize wall street with real-time trade reconciliation, market data systems...etc. We serve a portfolio of about 300 customers, 50 of which are fortune 500, and we're also an enabling technology to many OEM installations (mostly in Telco) with a deployment footprint in the thousands.
  • #5  Our flagship product is XAP, and in-memory data grid. But our insight platform, InsightEdge, extends quite beyond that. We have fused Spark with In-Memory Data Grid
  • #6 In terms of companies that do business with GigaSpaces, you'll notice the who's who of the global 2000. And while they are diverse in their verticals, they are trying to......
  • #7 They are trying to get from data to services.... So what does that mean?
  • #8 So what does that typically look like? To set the context...
  • #9 ......because the economic value of insight-driven transformation is undeniable Recent research shows really interesting numbers for what you might call insight-driven businesses From an ROI perspective, firms are seeing a %1300 ROI The majority of those who haven’t become fully insight-driven, about 74%, already have plans for introducing analytics at every corner for their business This is mainly due to the recognition that, having analytics, not only as means of differentiation, but as a fast innovation engine, to be twice as innovate and ourperform their peers.
  • #10 So how come only 23% are successful?? We can look at some challenges.... So we see generally three paradoxes...
  • #11 The first one is what I would call the "rear-view mirror architecture paradox"
  • #12 The second is the 80/20 paradox. Or we can call it the "lean engineering" paradox.
  • #13 Finally, what we would call the "SOA" deja vu paradox For those of us who can remember. If we take a trip down the memory lane to 2008-2009, remembering the days of when SOA had big promises of transforming the enterprise by ALIGNING BUSINESS WITH ITA THROUGH SERVICES ORIENTATION. History is repeating itself.
  • #14 So can we take different approaches?
  • #15  For those not familiar, in-memory computing means using RAM as the primary storage medium for business and analytics. There by eliminating any form of Disk I/O or network I/O latency, therefore operating at millisecond latencies at very high throughput. So while traditionally, we were
  • #16 This approach consists of two components
  • #26 all common API tap into other data stores on demand Data science is in high demand, but short supply – so the ability to leverage the know how, capability, and production readiness eliminates a lot of pain points.
  • #29 Goal is to provide a unified environment where application developers and data scientists can collaborate. Data science by itself is an iterative activity which requires a lot of trial and error