Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

EMC World 2016 - code.16 Running Stateful Services on Cloud Native Platforms with Mesos

Many of today's PaaS systems are focused on stateless applications, scaling them from 1 to infinity and automatically rescheduling them when something goes wrong. But what about the data they create? How can we create scalable data persistence backends for our services to make sure our stored data is highly available? In this session we will demonstrate stateless applications running on PaaS systems, connecting to data persistence layers like relational and NoSQL databases, all running on Mesos and all stored on highly available distributed storage platforms.

  • Login to see the comments

EMC World 2016 - code.16 Running Stateful Services on Cloud Native Platforms with Mesos

  1. 1. 1© Copyright 2016 EMC Corporation. All rights reserved. 1© Copyright 2016 EMC Corporation. All rights reserved. RUNNING STATEFUL SERVICES ON CLOUD NATIVE PLATFORMS WITH MESOS JONAS ROSLAND, EMC {code} DEVELOPER ADVOCATE & COMMUNITY MANAGER
  2. 2. 4© Copyright 2016 EMC Corporation. All rights reserved. • How are Cloud Native Platforms managed? • Handling stateless apps and stateful services • Everything running in containers • Making the platforms understand data persistence AGENDA
  3. 3. 5© Copyright 2016 EMC Corporation. All rights reserved. • Clustered servers • Manual deployments • Redundancy, not necessarily high availability BACK IN THE OLD DAYS
  4. 4. 6© Copyright 2016 EMC Corporation. All rights reserved. • Infrastructure as a Service • Deploy virtual machines • Manual/half-automated way of deploying apps • Almost efficient THEN SOME TIME AGO
  5. 5. 7© Copyright 2016 EMC Corporation. All rights reserved. • Collection of compute resources • Spread out horizontally • Brings scale to your deployments NOW – CLOUD NATIVE PLATFORMS
  6. 6. 8© Copyright 2016 EMC Corporation. All rights reserved. Standard definitions
  7. 7. 9© Copyright 2016 EMC Corporation. All rights reserved. PAAS SITS ON TOP OF SCHEDULERS
  8. 8. 10© Copyright 2016 EMC Corporation. All rights reserved. • Manages all physical/virtual resources • Deploys and schedules apps • Handles hardware failures RESOURCE MANAGERS AND SCHEDULERS
  9. 9. 11© Copyright 2016 EMC Corporation. All rights reserved. SCHEDULING
  10. 10. 12© Copyright 2016 EMC Corporation. All rights reserved. • Enables apps and infra to automatically find parts of each other • Keeps track of how to connect and use the application services SERVICE DISCOVERY
  11. 11. 13© Copyright 2016 EMC Corporation. All rights reserved. SERVICE DISCOVERY
  12. 12. 14© Copyright 2016 EMC Corporation. All rights reserved. APPLICATIONS RUN IN CONTAINERS Code Code and requirements Container
  13. 13. 15© Copyright 2016 EMC Corporation. All rights reserved. You don’t have to rewrite your apps!
  14. 14. 16© Copyright 2016 EMC Corporation. All rights reserved. • Storing data (state) is a critical component of any application • Where they store this state defines their class – Connected database – In memory – Local disk APPLICATION STATE DIFFERENCES
  15. 15. 17© Copyright 2016 EMC Corporation. All rights reserved. • Store no critical data locally • Can be scaled as needed • Recovers quickly STATELESS APPLICATIONS
  16. 16. 18© Copyright 2016 EMC Corporation. All rights reserved. • Services that store state • Have usually been treated as HA apps • Can be scale-out or scale-up STATEFUL SERVICES
  17. 17. 19© Copyright 2016 EMC Corporation. All rights reserved. FROM oraclelinux:latest ENV PACKAGE_URL https://repo.mysql.com/yum/mysql-5.7-community/docker/x86_64/mysql-community-server-minimal-5.7.12- 1.el7.x86_64.rpm RUN rpmkeys --import http://repo.mysql.com/RPM-GPG-KEY-mysql && yum install -y $PACKAGE_URL && yum install -y libpwquality && rm -rf /var/cache/yum/* RUN mkdir /docker-entrypoint-initdb.d VOLUME /var/lib/mysql COPY docker-entrypoint.sh /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] EXPOSE 3306 33060 CMD ["mysqld"] MYSQL IN A CONTAINER
  18. 18. 20© Copyright 2016 EMC Corporation. All rights reserved. 20© Copyright 2016 EMC Corporation. All rights reserved. APPLICATIONS BUILT FROM SERVICES Front End or Non-Persistent Data Services (Persistent) Scale-Up Scale-Out Cassandra Postgres Redis CouchDB Elastic Search Hadoop MariaDB Mysql Nginx HAProxy MongoDB RabbitMQ Tomcat Memcached ScaleIO ECS ApacheRails Language Specific HTTP
  19. 19. 21© Copyright 2016 EMC Corporation. All rights reserved. • No! – Without proper storage mapped, the service will lose data – Data doesn’t travel with the service if it’s stored on local disk • Yes! – Automatically mapping modern storage backends such as ScaleIO, VMAX, XtremIO, Isilon and others enables your data to be stored safely STATEFUL SERVICES IN CONTAINERS?
  20. 20. 22© Copyright 2016 EMC Corporation. All rights reserved. • Have been mostly focused on stateless apps • Working hard on making stateful services part of their inventory • So how do you handle both types? CLOUD NATIVE PLATFORMS
  21. 21. 23© Copyright 2016 EMC Corporation. All rights reserved. • Data is stored outside the platform EXAMPLE 1
  22. 22. 24© Copyright 2016 EMC Corporation. All rights reserved. • Data is stored on the same platform EXAMPLE 2
  23. 23. 25© Copyright 2016 EMC Corporation. All rights reserved. 25© Copyright 2016 EMC Corporation. All rights reserved. Virtual Machine PERSISTENCE AND P3 Platform 2 Platform 3 High Availability Live Migration Highly Available Container Platform Lifecycle Management Container Platform Lifecycle Management
  24. 24. 26© Copyright 2016 EMC Corporation. All rights reserved. • Cloud native applications have no concept of local storage • They require state being stored somewhere else • So we need our platform to understand storage CLOUD NATIVE APPS AND STORAGE
  25. 25. 27© Copyright 2016 EMC Corporation. All rights reserved. Making the platform understand storage
  26. 26. 28© Copyright 2016 EMC Corporation. All rights reserved. LET’S START WITH THE BASICS GCE ? AWSEMC OpenStack
  27. 27. 29© Copyright 2016 EMC Corporation. All rights reserved. SAY HELLO TO MY LITTLE FRIEND – REX-RAY
  28. 28. 30© Copyright 2016 EMC Corporation. All rights reserved. NOW WE’RE GETTING SOMEWHERE GCEAWSEMC OpenStack
  29. 29. 31© Copyright 2016 EMC Corporation. All rights reserved. AND NOW IT’S GETTING INTERESTING GCEAWSEMC OpenStack Mesos Kubernetes
  30. 30. 32© Copyright 2016 EMC Corporation. All rights reserved. • Gives you the possibility to use any storage platform, and even multiple platforms at the same time • Always in sync with Docker’s development cycle • Integrated with Docker, Mesos and Kubernetes REX-RAY BENEFITS
  31. 31. 33© Copyright 2016 EMC Corporation. All rights reserved. How does Cloud Foundry and Mesos work together?
  32. 32. 35© Copyright 2016 EMC Corporation. All rights reserved. • You already have your apps divided into stateless and stateful • Focus on your stateless apps first • Make real decisions to move from archaic master- slave clusters • Move to new platforms to handle your old, new and expanding apps SUMMARY
  33. 33. 37© Copyright 2016 EMC Corporation. All rights reserved. @EMCcode @jonasrosland emccode.com community.emccode.com Questions?
  34. 34. 40© Copyright 2016 EMC Corporation. All rights reserved. 40© Copyright 2016 EMC Corporation. All rights reserved.

×