Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Journey to the Modern App with Containers, Microservices and Big Data

351 views

Published on

Journey to the Modern App with Containers, Microservices and Big Data

Published in: Technology
  • Be the first to comment

Journey to the Modern App with Containers, Microservices and Big Data

  1. 1. © 2017 Mesosphere, Inc. All Rights Reserved. Eryn Muetzel, Product Marketing for Big Data @errbear3 JOURNEY TO THE MODERN APP WITH CONTAINERS, MICROSERVICES AND BIG DATA Datacenter Operating System (DC/OS) The simplest way to build, run, scale and manage modern enterprise apps
  2. 2. © 2017 Mesosphere, Inc. All Rights Reserved.
  3. 3. © 2017 Mesosphere, Inc. All Rights Reserved. INDUSTRY TRANSITIONS Customer analytics 2000s Electronic customer records 1980s Online customer engagement 1990s Real-time & predictive customer engagement 2013+
  4. 4. © 2017 Mesosphere, Inc. All Rights Reserved.
  5. 5. © 2017 Mesosphere, Inc. All Rights Reserved.
  6. 6. WE LIVE IN THE ALWAYS CONNECTED ECONOMY. 3.7BMobile Broadband Users 80%New Data is Unstructured 13Connected Devices per Household (N. Amer) 40%Annual Enterprise Data Growth
  7. 7. IN THE ALWAYS CONNECTED ECONOMY SOFTWARE IS YOUR BUSINESS. Build more apps. Personal and meaningful customer experiences Innovate faster. Capture value from new value streams and actionable data insights Modernize IT Always-available services that are secure and cost effective
  8. 8. 1M trips daily across 311 cities, disrupting multiple industries. Personal behavior-based insurance using IoT. Real-time, personalized clinical care. MODERN ENTERPRISE APPS IN ACTION.
  9. 9. © 2017 Mesosphere, Inc. All Rights Reserved. ARCHITECTURAL SHIFT TRADITIONAL ENTERPRISE APPLICATIONS MODERN ENTERPRISE APPLICATIONS Data Code Latency Users 2+ Billion smartphone users 3+ Billion internet users Enterprise data growth (CAGR): 40%+ Source: KPCB Internet Trends 2017, EMC Digital Universe 2014
  10. 10. © 2017 Mesosphere, Inc. All Rights Reserved. ARCHITECTING FOR SCALE AT TWITTER BEFORE AFTER
  11. 11. © 2017 Mesosphere, Inc. All Rights Reserved. Streaming 11 THE MODERN ENTERPRISE APP PLATFORM SERVICES MICROSERVICES STATELESS CONTAINERS ANALYTICS Batch Machine Learning Search DATABASES Time Series SQL / NoSQL
  12. 12. © 2017 Mesosphere, Inc. All Rights Reserved. Web Apps and Microservices Connected Devices Machine Learning and Simulation Industrial IoT Connected CarFraud DetectionSocial Media DIGITAL TRANSFORMATION REQUIRES MODERN APPS Modern Apps Operational expertise Continuous integration and delivery Big Data Analytics Message Queue Distributed Database Search Future services... Platform Services Container Orchestration How do you develop these skills in your organization?
  13. 13. MODERN ENTERPRISE APPS REQUIRE NEW CAPABILITIES. Distributed computing expertise. Engineering and operations of secure & highly reliable services at scale DevOps process & culture. Frequent & reliable releases supported by an automated CI/CD toolchain Cloud-native technologies. Containers, microservices, data servies (e.g., Spark, Kafka, Cassandra)
  14. 14. © 2017 Mesosphere, Inc. All Rights Reserved. MODERN ENTERPRISE APPS Containers Data Services MicroservicesOpen source & Social coding1 2 3 4
  15. 15. © 2017 Mesosphere, Inc. All Rights Reserved. OPEN SOURCE SOFTWARE PROVIDES LEVERAGE1
  16. 16. © 2017 Mesosphere, Inc. All Rights Reserved. CONTAINERS SIMPLIFY PACKAGING AND DEPLOYMENT Private Copy Shared App 3 Libraries Guest Operating System Infrastructure App 1 Libs Host Operating System Start / stop time One minute to few seconds Milliseconds Workload density 10 - 100x1x App 2 Libraries Guest Operating System App 1 Libraries Guest Operating System App 2 Libs App 3 Libs Virtual Machines Containers Host Operating System Hypervisor Infrastructure App 4 Libs App 5 Libs 2
  17. 17. © 2017 Mesosphere, Inc. All Rights Reserved. JUST HOW FAST ARE CONTAINERS? Larry Rau from @Verizon with @flo Launching 50,000 containers in seconds with @mesosphere #DC/OS
  18. 18. © 2017 Mesosphere, Inc. All Rights Reserved. MICROSERVICES ENABLE FAST AND FREQUENT RELEASES Traditional Architecture Many functions in a single process Siloed teams RESTAPIs Microservices Architecture Cross-functional teams organized around capabilities Scales individually Each element of functionality defined as “microservices” Scales monolithically 3
  19. 19. © 2017 Mesosphere, Inc. All Rights Reserved. DATA SERVICES PROVIDE CONNECTION AND PERSISTENCE Data Ingestion Response Devices ClientSensors Event Bus Reactive App Database Analytics Use Case Examples ● Anomaly detection ● Personalization ● IoT Applications ● Predictive Analytics ● Machine Learning 4
  20. 20. © 2017 Mesosphere, Inc. All Rights Reserved. 20 BUT WHAT ABOUT THE INFRASTRUCTURE? CaaSPaaS Container AppContainer App Stateful Service #1 V1 Big Data Analytics #1 V1 Stateful Service #2 V1 Stateful Service #2 V2 PaaS Container App Team A Big Data Analytics #1 V2 Big Data Analytics #2 V1 Team B
  21. 21. © 2017 Mesosphere, Inc. All Rights Reserved. 21 BUT WHAT ABOUT THE INFRASTRUCTURE? CaaSPaaS Container AppContainer App Stateful Service #1 V1 Big Data Analytics #1 V1 Stateful Service #2 V1 Stateful Service #2 V2 PaaS Container App Team A Big Data Analytics #1 V2 Big Data Analytics #2 V1 Team B ● Extremely low utilization - less than 10%, due to static partitioning ● Manual operations - Weeks to provision, wasted time managing individual machines ● Difficult to experiment with new tech - months to provision environments to evaluate new technologies ● High risk - Failure from manual operations and difficulties maintaining high availability
  22. 22. © 2015 Mesosphere, Inc. All Rights Reserved. WHERE HAVE WE SEEN THIS PROBLEM BEFORE? You are launching Google Chrome. Which core would you like to use? Core 1 Core 2 Core 3 Core 4 You are launching a distributed app requiring 500 cores. Which of your 50,000 cores would you like to use? Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 8 Core 9 Core 10Core 7 Core 11 Core 13 Core 14 Core 15Core 12 Core 16 Core 18 Core 19 Core 20Core 17 Core 21 Core 23 Core 24 Core 25Core 22 Core 26 Core 28 Core 29 Core 30Core 27 Core 31 Core 33 Core 34 Core 35Core 32 Core 36 Core 38 Core 39 Core 40Core 37 Core 41 Core 43 Core 44 Core 45Core 42
  23. 23. © 2015 Mesosphere, Inc. All Rights Reserved. MODERN APPS REQUIRE OPERATIONS AT THE DC LEVEL Traditional App Modern Enterprise App
  24. 24. © 2015 Mesosphere, Inc. All Rights Reserved. YES, THIS TOO CAN BE SOLVED WITH ABSTRACTION Scheduler Executor Task Launch Task Launch Task Status Task Status Resource Offer Executor Executor Executor Scheduler Executor Task Launch Executor Executor Task Status Scheduler Executor Task Launch Executor Executor Task Status Distributed System A Distributed System B Datacenter Operating System Approach Distributed Systems A+B+C+... Apache Mesos Two-level Scheduling
  25. 25. © 2017 Mesosphere, Inc. All Rights Reserved. 25 MODERN APPS WITH THE DC/OS MODEL Traditional Approach Big Data Analytics Stateful Service DC/OS Model Container App Container App CaaS PaaS ● Turnkey install of datacenter-wide services ● Simplified operations & efficiency ● Lowers barriers to experiment with new tech (e.g., Spark) Container App Datacenter-scale operating system
  26. 26. © 2017 Mesosphere, Inc. All Rights Reserved. BRINGING MODERN APPS TO EVERYONE ● 100% Open source ● No limits
  27. 27. 27 THE DC/OS COMMUNITY
  28. 28. YOUR DATACENTER OR CLOUD IS NOW ONE GIANT COMPUTER
  29. 29. © 2017 Mesosphere, Inc. All Rights Reserved. DC/OS SERVICE INSTALL
  30. 30. © 2017 Mesosphere, Inc. All Rights Reserved. DC/OS SERVICE INSTALL
  31. 31. © 2017 Mesosphere, Inc. All Rights Reserved. 31 DC/OS SERVICE OPS
  32. 32. PLATFORM SERVICES INSTALLED ACROSS DATACENTER OR CLOUD IN A SINGLE COMMAND
  33. 33. © 2017 Mesosphere, Inc. All Rights Reserved. TRY DC/OS IN UNDER 20 MINUTES dcos.io
  34. 34. BUSINESS APPS MESOSPHERE ENTERPRISE DC/OS CONTAINER ORCHESTRATION CI/CD BIG DATA ANALYTICS MESSAGE QUEUE DISTRIBUTED DATABASE SEARCH PLATFORM SERVICES RESOURCE MANAGEMENT OPERATIONS SECURITY MULTI-TENANCY DATACENTER-CLOUD OPERATING SYSTEM APACHE MESOS PRIVATE CLOUD AWS INFRASTRUCTURE (EC2) GOOGLE CLOUD MICROSOFT AZURE MESOSPHERE ENTERPRISE DC/OS
  35. 35. Container Orchestration PaaS Stateful Data Services Analytics Hybrid Cloud Platform Services Ecosystem Production Proven at Scale DATACENTER
  36. 36. MESOSPHERE DC/OS CUSTOMER IMPACT Accelerate Time-to-Market. Capture value from data insights and quickly roll out new services to stay competitive. Ensure Resilience and Automation. Provide always available and secure business services on hybrid clouds. Save Money. Get the most out of infrastructure with high utilization and cost predictability.
  37. 37. © 2017 Mesosphere, Inc. All Rights Reserved. “We’re running real time analytics and visualization of data from connected sensors Before DC/OS we had no where near the scale we had today, from thousands of to millions of events per second. Next, we’re doing predictive GIS” Adam Mollenkopf, Real Time Geospatial Information Systems Capability Lead
  38. 38. © 2017 Mesosphere, Inc. All Rights Reserved. “We are building a PaaS and services on an existing OpenStack cloud to enable developers to build microservices on Docker containers and use big data technologies... DC/OS was a force multiplier for us to get these new services out the door fast... We transitioned the full enterprise infrastructure at 2X the speed of previous efforts.” Tim Pletcher Engineering Director, Application Services
  39. 39. © 2017 Mesosphere, Inc. All Rights Reserved. “We came across Mesosphere while planning for the coming IoT market.. If I build everything in a silo, I have no chance... With DC/OS, I have one single contiguous cluster and I can bring my data and store it and I can run all my applications as well.. and I think that's a huge advantage” Larry Rau, Director Architecture & Infrastructure, Verizon Labs
  40. 40. © 2017 Mesosphere, Inc. All Rights Reserved. McKinsey Quarterly 2015: “The perils of ignoring software development” Based on assessment of software development projects at 1,300 companies Top companies bring new software to market 3x faster
  41. 41. © 2017 Mesosphere, Inc. All Rights Reserved. Thank You Questions? @errbear3 Learn more at mesosphere.com, including our whitepaper: Modern Enterprise App Operations with DC/OS Lessons from Running Containers, Microservices, and Stateful Big Data Services in Production

×