Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Microservices, Data Services and Containers for Cloud Native Architectures (Devops st.05)

2,184 views

Published on

EMC DevOps presentation - containers and more

Published in: Technology
  • Be the first to comment

Microservices, Data Services and Containers for Cloud Native Architectures (Devops st.05)

  1. 1. 1© Copyright 2015 EMC Corporation. All rights reserved. 1© Copyright 2015 EMC Corporation. All rights reserved.
  2. 2. 2© Copyright 2015 EMC Corporation. All rights reserved. MICROSERVICES, DATA SERVICES AND CONTAINERS FOR CLOUD NATIVE ARCHITECTURES JOSH LONG AND RAGHAVAN “RAGS” SRINIVAS 2© Copyright 2015 EMC Corporation. All rights reserved.
  3. 3. 3© Copyright 2015 EMC Corporation. All rights reserved. •  Spring Developer Advocate at Pivotal (a company that empowers enterprise transformation) •  Open Source Contributor – Spring Boot, Spring Integration, Vaadin and Activiti •  JavaOne Rock Star •  Book author on Spring JOSH LONG
  4. 4. 4© Copyright 2015 EMC Corporation. All rights reserved. •  Rags - work for EMC CODE (emccode.github.io) •  Middleware and Application programmer •  Architect and Evangelist •  Part time teaching at UML and writing at InfoQ •  Philosophy: It’s better to have an unanswered question than a unquestioned answer RAGS
  5. 5. 5© Copyright 2015 EMC Corporation. All rights reserved. •  Case Study •  Introduction, Motivation and 12-factor App •  Data Services ecosystem •  Containers and Platforms •  Spring Boot – A lightweight container •  Non-functional Requirements with Cloud Foundry •  Conclusions, Call for Action, Q & A AGENDA
  6. 6. 6© Copyright 2015 EMC Corporation. All rights reserved. CASE STUDY
  7. 7. 7© Copyright 2015 EMC Corporation. All rights reserved. EXAMPLE CASE-STUDY SCALING FROM STARTUP TO WEBSCALE •  1 Engineer •  1 Web Server •  1 MYSQL DB •  Rackspace Cloud DAY = 0 •  2 Engineers •  1 Load-Balancer •  4 Web Servers •  1 MySQL DB M/S •  1 Task Queue •  1 NoSQL DB •  AWS EC2+S3+CDN YEAR = 1 •  3 Engineers •  2 Load-Balancer •  16 Web Servers •  2 API Engines •  5 MySQL DB •  4 Cassandra Nodes •  15 Membase Nodes •  8 Memcache Nodes •  10 Redis Nodes •  7 Task Routers •  4 ElasticSearch Nodes •  3 MongoDB Clusters •  AWS EC2+S3+CDN YEAR = 2 Source: High Scalability Blog – Pinterest Architecture
  8. 8. 8© Copyright 2015 EMC Corporation. All rights reserved. EXAMPLE CASE-STUDY SCALING FROM STARTUP TO WEBSCALE BILLIONS OF PAGEVIEWS BILLIONS OF PICTURES BILLIONS OF ACTIONS BILLIONS OF API REQUESTS SCALE ELB TIER SCALE CACHING TIER SCALE WEB TIER SCALE DATA TIER 3 ENGINEERS
  9. 9. 9© Copyright 2015 EMC Corporation. All rights reserved. INTRODUCTION, MOTIVATION AND 12-FACTOR APP.
  10. 10. 10© Copyright 2015 EMC Corporation. All rights reserved. Relational Database Data Access Service HTML JavaScript MVC Service Monolithic ApplicationBrowser MONOLITHS •  Twitter moved from a Rails monolith •  Facebook moved from PHP monolith •  Netflix moved from a Java monolith
  11. 11. 11© Copyright 2015 EMC Corporation. All rights reserved. •  Not monolithic J •  Smaller units of a larger system – independent deployability •  Single Responsibility Principle •  Runs in its own process – Unix pipes •  Lightweight communication protocols •  Polyglot persistence - multiple persistence techniques (not just SQL) WHAT ARE MICROSERVICES?
  12. 12. 12© Copyright 2015 EMC Corporation. All rights reserved. •  REST is the underlying Protocol POLYGLOT PERSISTENCE (ONE SIZE DOES NOT FIT ALL) REST Redis Hadoop
  13. 13. 13© Copyright 2015 EMC Corporation. All rights reserved. DATA SERVICES ECOSYSTEM
  14. 14. 14© Copyright 2015 EMC Corporation. All rights reserved. •  Sharding –  Distributing Data over multiple nodes –  In conjunction with a Hashing/Partitioning algorithm •  Replication –  Protect against Node(s) failure –  Provide Data resiliency to node failure(s) via replication (1 – many replicas) TECHNIQUES 1
  15. 15. 15© Copyright 2015 EMC Corporation. All rights reserved. CONSTRAINTS: CAP THEOREM/BREWER’S CONJECTURE The CAP theorem states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: •  Consistency means that each node/client always has the same view of the data, •  Availability means that all clients can always read and write, •  Partition tolerance means that the system works well across physical network partitions.
  16. 16. 16© Copyright 2015 EMC Corporation. All rights reserved. 16 DATA SERVICES ECOSYSTEM Database Description Strengths Weakness/Limitations MySQL Scalability with MySQL replication Relational SQL – Developer friendly, familiar tools Vendor specific Hadoop/HDFS Generalized parallel algorithm based on Map/Reduce Data Locality, Rich ecoystem Batch oriented (newer initiatives for real time queries) Cassandra and HBase Fast key/value Database (good for time series data) Works well with HDFS (no Data movement required) No joins, transactions, etc. MongoDB Simple Document- oriented Database Denormalized extensible schema As schema gets more complex harder to maintain
  17. 17. 17© Copyright 2015 EMC Corporation. All rights reserved. THE ECOSYSTEM 17
  18. 18. 18© Copyright 2015 EMC Corporation. All rights reserved. PLATFORMS AND CONTAINERS
  19. 19. 19© Copyright 2015 EMC Corporation. All rights reserved. •  LXC containers - default VM for the cloud –  Lifecycle management of VMs –  Abstracts the OS infrastructure (ports, etc.) –  It’s virtualization (VM-like agility) –  Lightweight •  Just enough Operating System (JeOS) •  Servlet containers – default container for web –  Lifecycle management of web applications –  Abstracts the application infrastructure (transaction, etc.) CONTAINERS
  20. 20. 20© Copyright 2015 EMC Corporation. All rights reserved. Cloud Foundry and LatticeSpring Boot PLATFORMS FOR MICROSERVICES
  21. 21. 21© Copyright 2015 EMC Corporation. All rights reserved. SPRING BOOT
  22. 22. Josh Long - @starbuxman title
  23. 23. 23© Copyright 2015 EMC Corporation. All rights reserved. NON-FUNCTIONAL REQUIREMENTS WITH CLOUD FOUNDRY AND LATTICE
  24. 24. 24© Copyright 2015 EMC Corporation. All rights reserved. CLOUD FOUNDRY ARCHITECTURE Ops  Manager  UI   Ops  Manager  Director   Operations Manager Service Service  Broker   Service  Nodes   Service  Broker   Service  Nodes   Service App  Log  Aggregator   Login  Server   Dynamic  Router   Cloud  Controller   UAA   Health  Manager   DEA  Pool   Messaging  (NATS)   Apps   Metrics  CollecGon   Apps   HA  Proxy  LB   Elastic Runtime
  25. 25. 25© Copyright 2015 EMC Corporation. All rights reserved. BOSH DB Blob Store Worker NATS Health Monitor CPI DirectorCLI Inner shell Outer shell Agents 1.  Target a BOSH director using CLI 2.  Upload a Stemcell 3.  Get a Release from a repo 4.  Create a deployment manifest 5.  BOSH Deploy Cloud Foundry
  26. 26. 26© Copyright 2015 EMC Corporation. All rights reserved. Provision  VM   Install  ApplicaGon  RunGme   Deploy  ApplicaGon   Configure  Load  Balancer   Configure  Service  ConnecGvity   Configure  Firewall   cf  push   Configure  SSL  TerminaGon   cf  bind-­‐service   A PLATFORM THAT HANDLES THE NON-FUNCTIONAL REQS. cf  scale  
  27. 27. 27© Copyright 2015 EMC Corporation. All rights reserved. Blobstore BOSH Health Monitor DB Deploy my Services IaaS Worker VMsBOSH Director NATS Cassandra Node Target VMCassandra Node Target VMCassandra Node Target VM http://bosh.cloudfoundry.org/ DATA SERVICES
  28. 28. 28© Copyright 2015 EMC Corporation. All rights reserved. DB Router DATA SERVICES BINDING Service credentials reserve resources obtain connection data CLI Cloud Controller Service Broker Data Service Runtime create service (HTTP) bind service (HTTP) create service (HTTP) bind service (HTTP) http://docs.cloudfoundry.org/services/api.html
  29. 29. 29© Copyright 2015 EMC Corporation. All rights reserved. •  A lightweight container rewritten in Go •  Ability to run Dockerized applications WHY LATTICE? App  Log  Aggregator   Dynamic  Router   DEA  Pool   Apps   Metrics  CollecGon   Apps   Elastic Runtime 1.0 Diego   Apps   Apps   Lattice
  30. 30. 30© Copyright 2015 EMC Corporation. All rights reserved. SUMMARY, CALL FOR ACTION, Q&A
  31. 31. 31© Copyright 2015 EMC Corporation. All rights reserved. •  Cloud Native Architectures – a new paradigm –  Scale and Speed is not just for Netflix/Twitter/Baidu/etc. –  Microservices not Monoliths –  12-factor app. and DevOps –  Polyglot persistence/Multiple Data services –  Containers •  Containers come in multiple shapes, types and sizes •  Choose a Platform that enables microservices, containers, different services and enables DevOps SUMMARY
  32. 32. 32© Copyright 2015 EMC Corporation. All rights reserved. •  Former Netflix DevOps Guru Adrian Cockroft on DevOps + MS http://www.infoq.com/interviews/adrian-cockcroft-microservices-devops •  Bootiful Applications with Spring Boot http://www.youtube.com/watch?v=eCos5VTtZoI •  Chris Richardson’s http://microservices.io site and his Decomposing Applications for Scalability talks •  The Netflix Techblog http://techblog.netflix.com MICROSERVICES
  33. 33. 33© Copyright 2015 EMC Corporation. All rights reserved. •  Sign up for the Cloud Foundry and BOSH mailing lists. •  Learn how to write 12-factor applications. •  Try out Lattice. •  Free workshop and roadshows. CFAD! •  Register for the next Cloud Foundry Summit (May next week) •  Try out Pivotal Web Services (http://run.pivotal.io) CLOUD FOUNDRY
  34. 34. 34© Copyright 2015 EMC Corporation. All rights reserved. BOOK Available to you compliments of Pivotal! Get the FREE e-book at http://bit.ly/cloud-native-book!

×