Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MapR Streams and MapR Converged Data Platform

2,643 views

Published on

We're introducing MapR Streams, a reliable, global event streaming system that connects data producers and data consumers across shared topics of information. With the integration of MapR Streams, comes the industry’s first and only converged data platform that integrates file, database, event streaming, and analytics to accelerate data-driven applications and address emerging IoT needs.

Are you ready to accelerate your business with the power of a truly global platform for integrating data-in-motion with data-at-rest?

Published in: Technology
  • Be the first to comment

MapR Streams and MapR Converged Data Platform

  1. 1. © 2015 MapR Technologies 1© 2015 MapR Technologies
  2. 2. © 2015 MapR Technologies 2 Topics • Business imperative: Converging data-in-motion & data-at-rest • Introducing MapR Streams • Live Demo: MapR Streams in action • Use cases: putting event streaming to work for your business
  3. 3. © 2015 MapR Technologies 3 Speakers Steve Wooledge VP, Product Marketing Anil Gadre SVP, Product Management Will Ochandarena Director, Product Management Nick Amato Director, Technical Marketing
  4. 4. © 2015 MapR Technologies 4 Empowering the “as-it-happens” business by speeding up the data-to-action cycle
  5. 5. © 2015 MapR Technologies 5 Top Brands Across All Major Industries FINANCIAL SERVICES RETAIL & CPG SECURITY ONLINE SERVICES & SOFTWARE MEDIA & ENTERTAINMENT MANUFACTURING, UTILITIES, OIL & GAS ADVERTISING HEALTH COMMUNICATIONS GOVERNMENT Fortune 10 Retailer
  6. 6. © 2015 MapR Technologies 6© 2015 MapR Technologies Converging data-in-motion & data-at-rest
  7. 7. © 2015 MapR Technologies 7 The Rise of IoT and Data in Motion 1 Billion 6 Billion 50 Billion 2000s: Mobile Internet 2020: Internet of People and Things 1990s: Fixed Internet Connected Devices Worldwide By 2020, 21% of all “high value” data will come from IoT - IDC
  8. 8. © 2015 MapR Technologies 8 Introducing MapR Streams Global Publish/Subscribe Event Streaming Producers Publish billions of messages/sec to a topic. Consumers Reliable delivery to all consumers. Immediately. Global Tie together geo-dispersed clusters. Worldwide.
  9. 9. © 2015 MapR Technologies 9 The Struggle to Deliver New Apps as Big Data Grows More kinds of data More data sources Batch to more real-time analytics More consuming apps • Multiple technologies • Harder to develop the apps • Latencies everywhere • Silos coming back • Data copies proliferating • Even harder to maintain apps • More management issues • Multiple cluster expense How to create breakthrough value for the business rapidly
  10. 10. © 2015 MapR Technologies 10 A Once-in-30-Year Shift is UnderwayLegacy Data-to-action Applications Middleware Expensive Specialized Compute / Storage Enterprise Applications Commodity Hardware RDBMs Batch Analytics Message Bus Global Event Streaming BigDataAge Single Namespace Operational Analytics Structured data Semi- structured Data Unstructured Data
  11. 11. © 2015 MapR Technologies 11 ProcessingData Batch Streaming SQL MapR Converged Data Platform Problem: A Patchwork of Data Silos is EmergingApps Customer ExperienceData Architecture Optimization Security Investigation & Event Management Operational Intelligence Managed Services & Custom Apps
  12. 12. © 2015 MapR Technologies 12 ProcessingData MapR Converged Data Platform Files Tables Documents Streams Design Goal: Common Data Services for All ApplicationsApps Customer ExperienceData Architecture Optimization Security Investigation & Event Management Operational Intelligence Managed Services & Custom Apps Batch Streaming SQL MapR Converged Data Platform MapR Converged Data Platform Converged platform with the power of Hadoop, Spark, NoSQL database, SQL, event streaming, and Web-scale storage
  13. 13. © 2015 MapR Technologies 13 MapR Converged Data Platform Open Source Engines & Tools Commercial Engines & Applications Utility-Grade Platform Services DataProcessing Enterprise Storage MapR-FS MapR-DB MapR Streams Database Event Streaming Global Namespace High Availability Data Protection Self-healing Unified Security Real-time Multi-tenancy Search & Others Cloud & Managed Services Custom Apps UnifiedManagementandMonitoring
  14. 14. © 2015 MapR Technologies 14 Hadoop + Enterprise Storage Hadoop + Enterprise Storage + NoSQL Hadoop + Enterprise Storage + NoSQL + Interactive SQL Hadoop + Enterprise Storage + NoSQL + Interactive SQL + Event Streaming 2011 2013 2014 2015 Top Ranked Hadoop Top Ranked NoSQL Top Ranked SQL Consistently Executing On Our Converged Vision
  15. 15. © 2015 MapR Technologies 15 MapR Platform Services: Open API Architecture Assures Interoperability, Avoids Lock-in MapR-FS Enterprise Storage MapR-DB NoSQL Database MapR Streams Global Event Streaming HDFS API POSIX NFS SQL, Hbase API JSON API Kafka API
  16. 16. © 2015 MapR Technologies 16 MapR Streams Creates the Industry’s First and Only Converged Data Platform High-throughput streaming converges data-in-motion with data-at-rest 1 2 3 Converged : Single cluster for event streams, database, and file-based apps with unified security Continuous: Continuous analytics with always-on reliability enables “streams of record” Global: Optimized for global real-time applications including “intermittent” IoT connections
  17. 17. © 2015 MapR Technologies 17 Big Data is Generated One Event at a Time “time” : “6:01.103”, “event” : “RETWEET”, “location” : “lat” : 40.712784, “lon” : -74.005941 “time: “5:04.120”, “severity” : “CRITICAL”, “msg” : “Service down” “card_num” : 1234, “merchant” : ”Apple”, “amount” : 50
  18. 18. © 2015 MapR Technologies 18 Batch Processing Has Many Use Cases ● Clickstream analysis ● Predictive maintenance ● Fraud detection ● Coupon offers ● Risk models ● Customer 360 ● Sentiment analysis
  19. 19. © 2015 MapR Technologies 19 Real-time Processing is Complementary ● Ops dashboards ● Failure alerts ● Breach detection ● Real-time fraud detection ● Real-time offers ● Push notifications ● Trending now ● News feed
  20. 20. © 2015 MapR Technologies 20 The Challenge with Data Pipelines Filtering & Aggregation Alerting Processing
  21. 21. © 2015 MapR Technologies 21 Streams Simplify Data Movement Filtering & Aggregation Alerting Processing Streams Reliable publish/subscribe transport between sources and destinations.
  22. 22. © 2015 MapR Technologies 22 Legacy Systems: Message Queues IBM MQ, TIBCO, RabbitMQ OrdersFront End Order Processing Order Processing Usage/Requirements ● Tight, transactional conversations between systems ● 1:1 or Few:Few ● Low data rates ● Mission-critical delivery Approach ● Queue-oriented design ● Each message replicated to N output queues ● Messages popped when read ● Scale-up, master/slave Doesn’t Do ● High message rates (>100K/s) ● Slow consumers ● Queue replay/rewind
  23. 23. © 2015 MapR Technologies 23 Evolving “Big Data” Event Streams: Distributed Logs Kafka, Hydra, DistributedLog Usage/Requirements ● High throughput data transferred from decoupled systems ● Many->1 ● 1->Many ● Different speeds Approach ● Log-oriented design ● Write messages to log files ● Consumers pull messages at their own pace ● Scale-out Doesn’t Do ● Global applications ● Message persistence ● Integrated analytics (data movement required) DB_Changes Stream Processing Search/ EDW DB
  24. 24. © 2015 MapR Technologies 24 MapR: Rethinking a Platform for Event Streams ● “Big data” scale and performance ● Global applications and data collection ● Multi-tenant and multi-application ● Secure ● Analytics-ready (no movement) ● Converged: no cluster sprawl Stream Processing Analytics Ad Impressions App Logs Sensor Data
  25. 25. © 2015 MapR Technologies 25© 2015 MapR Technologies MapR Streams Converged, Continuous, Global
  26. 26. © 2015 MapR Technologies 26 MapR Streams: Global Pub-sub Event Streaming System for Big Data Producers publish billions of messages/sec to a topic in a stream. Guaranteed, immediate delivery to all consumers. Tie together geo-dispersed clusters. Worldwide. Standard real-time API (Kafka). Integrates with Spark Streaming, Storm, Apex, and Flink Direct data access (OJAI API) from analytics frameworks. To pi c Stream TopicProducers Consumers Remote sites and consumers Streaming Batch analytics
  27. 27. © 2015 MapR Technologies 27 Global Provides ● Arbitrary topology of thousands of clusters ● Automatic loop prevention ● DNS-based discovery ● Globally synchronized message offsets and consumer cursors Enables ● Global applications & data collection ● Producer & consumer failover ● Analysis/filtering/aggregation at the edge ● “Occasional” connections Producers Consumers
  28. 28. © 2015 MapR Technologies 28 Top Differentiators MapR Streams Converged Global Secure & Multi-Tenant Single cluster for files, tables, and streams. Global, IoT-scale “fabrics” with failover. Tenant-owned streams, logical grouping of topics and messages. Authentication, authorization, encryption. Unified policy with all other platform services. Data persistence and direct data access to batch processing frameworks.
  29. 29. © 2015 MapR Technologies 29© 2015 MapR Technologies See MapR Streams in Action – Demo Nick Amato
  30. 30. © 2015 MapR Technologies 30© 2015 MapR Technologies Event Streaming & Processing Use Cases Will Ochandarena
  31. 31. © 2015 MapR Technologies 31 All Industries Web 2.0 Healthcare Telecom • ETL / DW optimization • Mainframe optimization • Application & network monitoring • Security information & event management • Recommendation engines & targeting • Customer 360 • Click-stream analysis • Social media analysis • Ad optimization • Smart hospitals • Biometrics • Patient vital monitoring • Fraud detection • Antenna optimization • Charging & billing • Equipment monitoring & preventative maintenance • Smart meter analysis Sample Verticals & Use Cases Oil & Gas Financial Services Retail Ad Tech • Pump monitoring & alerting • Seismic trace identification • Equipment maintenance • Safety & environment • Security • Real-time fraud/risk monitoring • Mobile notifications of transactions • Real-time supply chain optimization • Inventory management • Real-time coupons • Ad targeting & optimization • Global campaign dashboards
  32. 32. © 2015 MapR Technologies 32 USE CASE Stream Application/Infrastructure Monitoring Logs Metrics Business Results ● Real-time detection and alerting on failures, security breaches ● Global dashboards on utilization, availability, performance Why Streaming ● Real-time delivery from apps/infra to ETL & processing systems. ● Reliable buffering of data in case of slow/failing systems Why MapR ● Converged platform brings all components together. ● Global event replication enables centralized monitoring. ● Secure multi-tenancy allows cluster sharing.
  33. 33. © 2015 MapR Technologies 33 Database Change Capture ForSmart Credit Card Processing Business Results ● Improved user satisfaction with real-time mobile notifications of purchases. ● More fraud detected in real-time. ● More productive staff with data exploration. Why Streams ● Seamless, real-time connection between mainframe RDBMS and ETL/processing. Why MapR ● Utility-grade reliability means no transactions are lost. ● Converged platform security gives unified authentication, authorization, encryption. USE CASE Transactions Fraud Detection Streaming 1
  34. 34. © 2015 MapR Technologies 34 Stream System of Record for Healthcare, Finance Business Results ● Data agility - up-to-date views in JSON, Graph, Search formats. ● HIPAA, PCI Compliance. ● Compliance with in-country data regulations. Records JSON DB (MapR-DB) Graph DB (Titan on MapR-DB) Search Engine (Elastic-Search) Write APIs Read APIs Why Streams ● Streams are immutable, re-windable, and auditable data structures. ● Pub-sub allows real-time replication to JSON-DB, Graph DB, ElasticSearch. Why MapR ● Converged platform gives single cluster, single security model for data in motion and at rest. ● Selective, reliable global replication for DR. EU USE CASE
  35. 35. © 2015 MapR Technologies 35 Global, Consolidated Analytics in Ad Tech … + APJ, EMEA Business Results ● New, real-time customer dashboards for ad spend and performance. ● Reduced global time-to-insight - hours to minutes. ● Addition of disaster recovery capability. Why Streams ● Simpler, more reliable data/ETL pipelines than legacy log-shipping method. Why MapR ● Converged platform gives single cluster, single security model for data in motion and at rest. ● Reliable global replication for distributed collection, analysis, and DR. US1 Ad Application HQ US2 USE CASE
  36. 36. © 2015 MapR Technologies 36 IoT Data Transport & Processing USE CASE Business Results ● New revenue streams from collecting and processing data from “things”. ● Low response times by placing collection and processing near users. Why Streams ● IoT is event-based, and needs an event streaming architecture. Why MapR ● Converged platform gives single cluster, single security model for data in motion and at rest. ● Reliable global replication for distributed collection, analysis, and DR. Global Dashboards, Alerts, Processing Local Collection, Filtering, Aggregation
  37. 37. © 2015 MapR Technologies 37 Q&A MapR Streams Creates Industry’s First and Only Converged Data Platform 1 2 3 Converged : Single cluster for event streams, database, and file-based apps with unified security Continuous: Continuous analytics with always-on reliability enables “streams of record” Global: Optimized for global real-time applications including “intermittent” IoT connections Learn more at www.mapr.com/streams

×