From Data to Services at the Speed of Business
•
1 like•272 views
From Data to Services at the Speed of Business: Applying cloud-native paradigm to combine fast data analytics with microservices architecture for hybrid workloads.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to From Data to Services at the Speed of Business
Similar to From Data to Services at the Speed of Business (20)
Recently uploaded
Recently uploaded (20)
From Data to Services at the Speed of Business
- 1. From Data to Services at the Speed of Business Ali Hodroj Vice President, Products and Strategy
- 2. Overview • Analytics value chain • Challenges of analytics-driven transformation • Microservices approach to analytics with In-memory computing • Q&A
- 3. Direct customers 300+ Fortune / Organizations 50+ / 500+ Large installations in production (OEM) 5,000+ ISVs 25+ GigaSpaces Technologies In-memory computing insight platform for mission critical applications
- 4. 4
- 5. Direct customers 300+ Fortune / Organizations 50+ / 500+ Large installations in production (OEM) 5,000+ ISVs 25+
- 6. From Data to Services at the Speed of Business Analytics Machine Learning Agility Autonomy Low total cost of ownership Real-time Just-in-time decisions Time-sensitive data New business models Fast innovation Value-driven alignment
- 7. Ingest Data The analytics and data-driven “value chain” The key to becoming insight-driven is to optimize the analytics value chain Analytics Action Performance Perform Trigger Improve r Strategic thinking & High Value Questions Design and Implementation
- 8. $13.01 forevery$1 a company spends on analytics, it gets back spend on data management and analytics Source: MIT Sloan, NucleusResearch Value of leveraging analytics and data-driven decisions 74%of firms say they want to be data- driven, but only 23%are successful Source: Forbes: Actionable Insight: Missing Link between Data and Value 2x [companies are twice] likely to outperform their peers if they use advanced analytics Source: MIT Sloan
- 9. From Data to Services at the Speed of Business Slow innovation path Lack of realtime view Misalignment between analytics and business decisions Top 3 Challenges
- 10. Businesses want real-time view into critical data Analytics infrastructures are focused on data accumulation and retrospective analysis Paradox#1:Rear-viewmirrorarchitectures
- 11. Analytics-driven means: experiment, fail fast, recover fast and learn rapidly Data science is 80% data preparation and 20% analytics Paradox#2:80/20data-to-analyticsratio
- 12. Analytics need tight alignment with business decisions Most efforts are focused on managing technology platforms and data lake governance Paradox#3:SOAdéjàvu
- 13. From Data to Services at the Speed of Business Distributed Analytics (Apache Spark) Microservices-driven Architecture In-Memory Data and Business Logic Processing Cloud-Native and Agile Infrasructure Microservice-driven and In-Memory Computing Approach?
- 14. What is In-Memory Computing?
- 15. Spark + In-Memory Data Grid Large-scale distributed analytics framework Unified, scale-out, low-latency data store Transactional capabilities: ACID, Event-Driven, Rich Data modeling Microservices
- 16. 16 Elastic Scale-out In-Memory Storage (Shared-nothing, Linear scalability, Elastic capacity) Low latency and high throughput (co-located ops, event-driven, fast indexing) High availability and Resiliency (auto-healing, multi-data center replication, fault tolerance) Rich API and Query Language (SQL, Spring, Java, .NET, C++) GigaSpaces XAP In-Memory Data Grid
- 17. 17 In-Memory Data Grid: How it works
- 18. 18 Offload data tiers to partitioned in-memory The database goes to the background Partition your data and store it in memory
- 19. 19 Horizontally scale to converge additional data sources Partitioned, co-located in-memory data storage
- 20. 20 Incrementally migrate new workloads and applications Business logic, data & messaging co-located & partitioned into processing units
- 21. 21 GigaSpaces automatically ensures High Availability Hot backup for each partition for high availability
- 22. 22 On-board new applications and micro-services Host your web application on the XAP infrastructure
- 23. 23 Harness commodity/cloud infrastructure and multi-tier storage Auto-scale out & in based on real-time performance & load
- 24. 24 Result: Real-time distributed compute and storage for transactions and analytics
- 25. 25 • Unified & Concise API • Highly Flexible Data Store Integration • Massive Community and Adoption Why Spark?
- 26. Geo-Spatial Full Text In-Memory Data Grid + Spark Convergence
- 27. Unified Hybrid Transactional/Analytics Architecture node 1 Spark master Grid master node 2 Spark worker Grid Partition node 3 Spark worker Grid Partition Lightweight workers, small JVMs Large JVMs, Fast indexing • Push-down predicates (ultra-low latency processing, 30x performance improvement) • Stateful data-360 sharing across analytics jobs • Data-locality for high throughput • Five 9s High Availability
- 28. Decoupled Hybrid Architecture In-Memory Data Grid Realtime Replication • Scoring models • Trigger actions • Events Transactions Analytics • Useful when analytics are mostly batch or long- running queries. • Analytics grid can be used for frequent model training (CPU intensive), without impacting transactional apps • Flexibility in write-heavy (transactions) and read- heavy (analytics) independent scaling Application developers Data Scientists & Analysts
- 29. From Data to Services at the Speed of Business Putting it all together (GigaSpaces InsightEdge Platform)
- 30. GigaSpaces In-Memory Computing Platform Core In-Memory Data Grid Event Processing Data Models (Spatial, POJO, JSON) RPC & Map/Reduce Web Containers Storage-Class Memory (3DXPoint) Cluster Management and Service Discovery Microservices (REST) Event Processing Streaming Machine Learning Spark SQL Analytics & Big Data SearchSQL/JDBC Search and Query RESTOrchestration ManagementandMonitoring SecurityandAuditing WebMobile Devices On-Premises Cloud Hybrid Transactional Apps and Customer Experience Message Brokers and Data Science Workbench Business Intelligence and Text Mining
- 31. Why In-Memory Data Grid? SQL-99, Polyglot Data & Search Multi-Tiered Data Storage Cloud-Nativeand Horizontally Scalable • RAM • SSD/Flash • Storage-Class Memory (3DXPoint) • SQL ‘99 • Graph • JSON • POJO • GeoSpatial • Full Text Distributed In-Grid Analytics • SQL • Streaming • Machine Learning • Graph Processing • Deep Learning • Textmining • Geospatial • In-Memory Event-Driven Processing • Distributed Tasks and Compute Grid • Real-time Web Services • In-Memory Aggregations Advanced In-Grid Transactions and Analytics Processing Across Center, Edge and Cloud
- 32. GigaSpaces Hadoop Embracing an open source analytics ecosystem Pick your own fast data architecture (lambda, kappa) and co-locate transaction processing Kafka Spark Simplified Lambda Architecture (Realtime + Historical)
- 33. Case Study: Magic Software IoT Hub + Predictive Analytics (Automotive Telematics) Challenge: • Implement predictive analytics and anomaly detection • Expand insight context through customer/data-360 integration • Trigger transactional workflows based on prediction criteria Solution: • Simplified HTAP with Streaming data pipeline (3 tiers) • IoT streaming analytics with 9s high availability “GigaSpaces enables our customers to simplify and accelerate telemetry ingestion, to gain full business value from IoT adoption.” Yuval Lavi, VP of Innovation Magic Software http://www.magicsoftware.com
- 34. Key Takeaways By the end of this presentation, you hopefully understood that: ➔ Microservices architectures provides data science agility! Capturing business value from real-time insight can be accomplished by embracing an agile open source tool chain coupled with in-memory data grids. ➔ Insight platforms scale at the speed of business Embracing insight platforms enables an agile path towards real-time digital transformation and the data-driven enterprise. ➔ Try it all out – It’s open source! http://insightedge.io / http://gigaspaces.com http://github.com/InsightEdge http://insightedge.slack.com hello@insightedge.io Book a demo:
- 35. Webinars UPCOMING WEBINARS Wed, Aug 23 | 2:00 PM EST Machine Learning and Insight Platforms RECORDED WEBINAR From Data to Services at the Speed of Business Powering IoT Integration Solutions: Turning Data from Sensors into Real-time Actionable Insights Real-time Microservices: From Zero to Production in Under 3 Months Visit our site to sign up: www.gigaspaces.com/webinars
- 36. Q&A
- 37. www.gigaspaces.com Thank you for attending!
Editor's Notes
- Gigaspaces is a software company focused on in-memory computing, fast data analytics, and insight platforms. We were established in the early 2000s. We were one of the first to digitize wall street with real-time trade reconciliation, market data systems...etc. We serve a portfolio of about 300 customers, 50 of which are fortune 500, and we're also an enabling technology to many OEM installations (mostly in Telco) with a deployment footprint in the thousands.
- Our flagship product is XAP, and in-memory data grid. But our insight platform, InsightEdge, extends quite beyond that. We have fused Spark with In-Memory Data Grid
- In terms of companies that do business with GigaSpaces, you'll notice the who's who of the global 2000. And while they are diverse in their verticals, they are trying to......
- They are trying to get from data to services.... So what does that mean?
- So what does that typically look like? To set the context...
- ......because the economic value of insight-driven transformation is undeniable Recent research shows really interesting numbers for what you might call insight-driven businesses From an ROI perspective, firms are seeing a %1300 ROI The majority of those who haven’t become fully insight-driven, about 74%, already have plans for introducing analytics at every corner for their business This is mainly due to the recognition that, having analytics, not only as means of differentiation, but as a fast innovation engine, to be twice as innovate and ourperform their peers.
- So how come only 23% are successful?? We can look at some challenges.... So we see generally three paradoxes...
- The first one is what I would call the "rear-view mirror architecture paradox"
- The second is the 80/20 paradox. Or we can call it the "lean engineering" paradox.
- Finally, what we would call the "SOA" deja vu paradox For those of us who can remember. If we take a trip down the memory lane to 2008-2009, remembering the days of when SOA had big promises of transforming the enterprise by ALIGNING BUSINESS WITH ITA THROUGH SERVICES ORIENTATION. History is repeating itself.
- So can we take different approaches?
- For those not familiar, in-memory computing means using RAM as the primary storage medium for business and analytics. There by eliminating any form of Disk I/O or network I/O latency, therefore operating at millisecond latencies at very high throughput. So while traditionally, we were
- This approach consists of two components
- all common API tap into other data stores on demand Data science is in high demand, but short supply – so the ability to leverage the know how, capability, and production readiness eliminates a lot of pain points.
- Goal is to provide a unified environment where application developers and data scientists can collaborate. Data science by itself is an iterative activity which requires a lot of trial and error