Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings


Published on

Tutorial for IEEE CloudCom 2018

Published in: Technology
  • Be the first to comment

Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings

  1. 1. 12/12/2018 1Demetris Trihinas 1Tutorial | CloudCom 2018 Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings Demetris Trihinas+*, Athanasios Tryfonos*, Marios D. Dikaiakos* * Department of Computer Science University of Cyprus + Department of Computer Science University of Nicosia IEEE International Conference on Cloud Computing Technology and Science (IEEE CloudCom 2018)
  2. 2. 12/12/2018 2Demetris Trihinas 2Tutorial | CloudCom 2018 Tutorial Overview • Software architecture paradigms • Containerization and Microservices • Microservices design and development • Microservices challenges at scale • Unicorn DevOps-as-a-Service Platform Monolithic SOA Microservices
  3. 3. 12/12/2018 3Demetris Trihinas 3Tutorial | CloudCom 2018 The Good Old Days… • You had an idea for an application… Books UI Ratings Shopping Cart Reviews Book Store (WAR/EAR) Application Server
  4. 4. 12/12/2018 4Demetris Trihinas 4Tutorial | CloudCom 2018 Hello… Monolith • A monolith is a software application whose architectural modules cannot be executed independently. • The term monolith first appeared in “The Art of Unix Programming” to describe systems that get too big.
  5. 5. 12/12/2018 5Demetris Trihinas 5Tutorial | CloudCom 2018 Time Passes By... • As you add more features it becomes difficult to track bugs and maintain a healthy code base. • You enter “dependency hell” because adding or updating libraries in shared projects results in inconsistencies. Monoliths are difficult to maintain and evolve due to their complexity Michael Jang, “Linux Annoyances for Geeks”, 2006
  6. 6. 12/12/2018 6Demetris Trihinas 6Tutorial | CloudCom 2018 Your User-Base Grows… • You hire more developers to cope with adding new features, improvements and bugs. • Embrace cloud computing to handle increased demand with pay-as-you-use model.usage
  7. 7. 12/12/2018 7Demetris Trihinas 7Tutorial | CloudCom 2018 Cloud Computing Books UI Ratings Shopping Cart Reviews Book Store (WAR/EAR) Application Server Pay as you use model
  8. 8. 12/12/2018 8Demetris Trihinas 8Tutorial | CloudCom 2018 Cloud Computing - Vertical Scaling Books UI Ratings Shopping Cart Reviews Book Store (WAR/EAR) Application Server
  9. 9. 12/12/2018 9Demetris Trihinas 9Tutorial | CloudCom 2018 Cloud Computing - Vertical Scaling Books UI Ratings Shopping Cart Reviews Book Store (WAR/EAR) Application Server Pay as you use
  10. 10. 12/12/2018 10Demetris Trihinas 10Tutorial | CloudCom 2018 Challenges of a Monolith • To cope with increasing load, a larger server/VM is allocated. • But… VM pricing does not scale linearly. • Any change in one module of a monolith requires rebooting the whole application. • For large-sized monoliths rebooting means downtime.
  11. 11. 12/12/2018 11Demetris Trihinas 11Tutorial | CloudCom 2018 Cloud Computing - Horizontal Scaling Books UI Ratings Shopping Cart Reviews Book Store (WAR/EAR) Application Server
  12. 12. 12/12/2018 12Demetris Trihinas 12Tutorial | CloudCom 2018 Challenges of a Monolith • But… sub-optimal deployment because of conflicting requirements. • For example one module is memory-bound, others cpu- bound, while another may be sparsely accessed. • To satisfy every module’s requirements, you end up with LARGE MONOLITHS.
  13. 13. 12/12/2018 13Demetris Trihinas 13Tutorial | CloudCom 2018 Lack of Innovation Books UI Ratings Shopping Cart Reviews Book Store (WAR/EAR) Application Server Build/Test/Release Long build cycles and releases take months. who broke the built?
  14. 14. 12/12/2018 14Demetris Trihinas 14Tutorial | CloudCom 2018 Software Architecture Paradigms time
  15. 15. 12/12/2018 15Demetris Trihinas 15Tutorial | CloudCom 2018 Hello… Service-Oriented Architecture • The monolith is decomposed into services. • Services are based on separation of concerns. • Services expose their offerings to other services over a communication mechanism –universally agreed among all services– part of the application. • Enterprise Service Bus (EBS) Georgakopoulos, D., Papazoglou, M.P.: Service-Oriented Computing. The MIT Press, Cambridge (2008)
  16. 16. 12/12/2018 16Demetris Trihinas 16Tutorial | CloudCom 2018 Benefits of SOA • Dynamism – launch only instances of the services in demand and use only the needed resources. • Services can be reused in other systems. Book Service Rating Service Cart Service • New services easily integrated by embracing comm. protocol*. • Services can be independently developed*. N. Dragoni “Microservices: yesterday, today, and tomorrow”, Software Engineering, Springer, 2017
  17. 17. 12/12/2018 17Demetris Trihinas 17Tutorial | CloudCom 2018 The Enterprise Service Bus Book Service Rating Service Cart Service A. Barker, C. D. Walton, D. Robertson, “Choreographing Web Services”. IEEE Transactions on Services Computing, 2009. • Sophisticated communication facilities via choreography: • Message-passing and business rules application to establish service collaboration.
  18. 18. 12/12/2018 18Demetris Trihinas 18Tutorial | CloudCom 2018 Drawbacks of SOA • The ESB does not scale and is ungovernable as you add more services. Book Service Rating Service Cart Service • To change anything in the ESB or the communication protocol ALL services must be changed as well • So… builds still take forever. Jim Webber and Martin Fowler, “Does My Bus Look Big In This”, 2008
  19. 19. 12/12/2018 19Demetris Trihinas 19Tutorial | CloudCom 2018 Software Architecture Paradigms time
  20. 20. 12/12/2018 20Demetris Trihinas 20Tutorial | CloudCom 2018 Hello… Microservices • The Monolith is decomposed into even smaller services. • Each discrete business functionality is a service. • Microservices run independently and inter- communicate using lightweight communication mechanisms. • Platform-agnostic API’s over the internet (bye-bye EBS).
  21. 21. 12/12/2018 21Demetris Trihinas 21Tutorial | CloudCom 2018 Decentralized Governance and Data • Microservices are to be as decoupled and as cohesive as possible. • Each service can be independently developed in different prog. Languages and frameworks, and even have its own database. • Each (micro-)service owns its own domain logic and acts more as a filter in the classical Unix sense. • A service receives a request, it applies logic as appropriate and produces a response that is served back to the client. J. Thones. Microservices. IEEE Software, Jan 2015.
  22. 22. 12/12/2018 22Demetris Trihinas 22Tutorial | CloudCom 2018 Microservices Architecture Book Service Rating Service Cart Service Review Service RESTTCP SOAP REST
  23. 23. 12/12/2018 23Demetris Trihinas 23Tutorial | CloudCom 2018 Microservices == SOA Revamped? • The term “microservices” was first introduced in 2011 at a SOA workshop (Venice, Italy). • Netflix initially used the name “Fine-grained SOA” to describe its microservices architecture. • Martin Fowler - “In contrast to SOA, microservices are about smart endpoints and dumb pipes” James Lewis, “Micro services - Java, the Unix Way ”, 2011 Netflix, “Announcing Ribbon: : Tying the Netflix Mid-Tier Services Together”, 2013
  24. 24. 12/12/2018 24Demetris Trihinas 24Tutorial | CloudCom 2018 Microservices and DevOps Embrace agility and autonomicity to increase innovation! Build/Test/Release UI Build/Test/Release Books Build/Test/Release Ratings Build/Test/Release Reviews Build/Test/Release Carts For each service one can understand, alter and write new code without knowing anything about the internals of its peers.
  25. 25. 12/12/2018 25Demetris Trihinas 25Tutorial | CloudCom 2018 Amazon e-Commerce Platform Peter Dalbhanjan, Solutions Architect AWS, 2016
  26. 26. 12/12/2018 26Demetris Trihinas 26Tutorial | CloudCom 2018 Does Size Matter? • How Big Should a “Micro” Service Be? • It’s about the team… Amazon’s “Two pizza rule” • J. Bezos: If you cant feed a team with two pizzas its too large. • It’s about discretizing functionality. • “Micro” has nothing to do with size – 5MB or 5GB is irrelevant.
  27. 27. 12/12/2018 27Demetris Trihinas 27Tutorial | CloudCom 2018 Conway’s Law - DevOps in 1967 …any organization that designs a system will inevitably produce a design whose structure is a copy of the organization’s communication structure… Melvin E. Conway, 1967
  28. 28. 12/12/2018 28Demetris Trihinas 28Tutorial | CloudCom 2018 Service Deployments • Decoupling services means builds and releases need less coordination (from 1 app -> N services). • You build it, you run it. • Deployments are also more often. • This means… deployments must be: • Fast, Reliable and Automated. Jim Gray, “A Conversation with Werner Vogels: Learning from the Amazon technology platform”, ACM Queue, 2006.
  29. 29. 12/12/2018 29Demetris Trihinas 29Tutorial | CloudCom 2018 Containerization • Containerization allows applications to share a single host OS in a portable manner. • Perfect for microservices as complex applications are split into discrete and modular units. • Reduced management complexity because problem or change related to one service does not require an overhaul of the overall application. R. Morabito, V. Cozzolino, A. Y. Ding, N. Beijar and J. Ott, "Consolidate IoT Edge Computing with Lightweight Virtualization,” IEEE Networks, 2018.
  30. 30. 12/12/2018 30Demetris Trihinas 30Tutorial | CloudCom 2018 Containers are an OS Technology GENERAL DISTRIBUTION CONTAINERS ARE AN OS TECHNOLOGY TRADITIONAL OS CONTAINERS Hardware LIB A Host OS LIB B LIB C Hardware LIB A OS Deps Container Host OS Container Container App A App B App C App A LIB B OS Deps App B LIB C OS Deps App C Remember dependency hell? Container Isolation
  31. 31. 12/12/2018 31Demetris Trihinas 31Tutorial | CloudCom 2018 But… Containers are NOT a New Technology • Part of the Linux ecosystem for more than a decade. • Resource isolation at the OS level through kernel namespaces and control groups. • Namespaces deal with resource isolation for a single process. • cgroups manage access for groups of processes. S. Hogg, "Software Containers: Used More Frequently than Most Realize”, Network Magazine, 2015.
  32. 32. 12/12/2018 32Demetris Trihinas 32Tutorial | CloudCom 2018 VMs vs Containers GENERAL DISTRIBUTION VIRTUAL MACHINES AND CONTAINERS VIRTUAL MACHINES CONTAINERS virtual machines are isolated apps are not containers are isolated so are the apps VM OSDependencies Kernel Hypervisor Hardware App App App App Hardware Container Host (Kernel) Container App OSdeps Container App OSdeps Container App OSdeps Container App OSdeps
  33. 33. 12/12/2018 33Demetris Trihinas 33Tutorial | CloudCom 2018 At the end… What are Containers? • It depends who you ask… GENERAL DISTRIBUTION WHAT ARE CONTAINERS? It Depends Who You Ask ● Sandboxed application processes on a shared Linux OS kernel ● Simpler, lighter, and denser than virtual machines ● Portable across different environments ● Package my application and all of its dependencies ● Deploy to any environment in seconds and enable CI/CD ● Easily access and share containerized components INFRASTRUCTURE APPLICATIONS Amir, Zipory (Red Hat Openshift), “DevOps with Containers: From Hype to Reality”, 2018.
  34. 34. 12/12/2018 34Demetris Trihinas 34Tutorial | CloudCom 2018 Docker – Container Engine • To run containers, specialized software must streamline and coordinate the process on top of the OS. • Golden standard for containerized technologies.
  35. 35. 12/12/2018 35Demetris Trihinas 35Tutorial | CloudCom 2018 Docker Mantra • Build, ship, run any application, anywhere. • Build – package app in a container. • Ship – move container from one machine to another. • Run – execute the container runtime. • Any application – literally anything. • Anywhere – Cloud, laptop, bare metal, microcontroller, sensor (?). Dirk Merkel, “Docker: lightweight Linux containers for consistent development and deployment”, Linux J., 2014(239),March 2014.
  36. 36. 12/12/2018 36Demetris Trihinas 36Tutorial | CloudCom 2018 Designing and Developing Microservices
  37. 37. 12/12/2018 37Demetris Trihinas 37Tutorial | CloudCom 2018 Tutorial Code is Publically Available
  38. 38. 12/12/2018 38Demetris Trihinas 38Tutorial | CloudCom 2018 Microservices Development “Decomposing” the monolith
  39. 39. 12/12/2018 39Demetris Trihinas 39Tutorial | CloudCom 2018 Decomposing the Monolith into Services • Books – product information (e.g. title, author, summary, ISBN) • Ratings – star-based score • Reviews – user feedback • Cart – items to be purchased • Each Service has its own data store Books Ratings Carts Reviews Database UI
  40. 40. 12/12/2018 40Demetris Trihinas 40Tutorial | CloudCom 2018 The Book Store
  41. 41. 12/12/2018 41Demetris Trihinas 41Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Reviews Cart Separation of Logic!
  42. 42. 12/12/2018 42Demetris Trihinas 42Tutorial | CloudCom 2018 Microservices Development • Java framework providing complimentary to the business logic features for app development: • Embedded servers to deploy within JVM containers. • Application components are auto-configured externally upon instantiation and at runtime. • Inherently distributed with tools for service discovery, security, API gateways, circuit breaking, message-passing, database integration. • Open-source ecosystem with tons of smaller frameworks created (e.g., major contributors include Netflix, AWS, Spotify).
  43. 43. 12/12/2018 43Demetris Trihinas 43Tutorial | CloudCom 2018 Microservices Development • Spring Boot is based on Spring Framework: • Aspect-Oriented Programming (AOP). • Model-View-Controller (MVC) approach for web service development. • Annotations for including Spring –complimentary– features. • Authentication, Authorization, Logging, Testing… • Limited to running on a single JVM container.
  44. 44. 12/12/2018 44Demetris Trihinas 44Tutorial | CloudCom 2018 Annotation-Based Programming • Annotations were initailly a form of metadata providing instructions that are not part of the actual application. • So they do not “directly” affect the program semantics. • They provide compilers and build engines with: • Useful information and hints
  45. 45. 12/12/2018 45Demetris Trihinas 45Tutorial | CloudCom 2018 Annotation-Based Programming • They provide compilers and build engines with: • Avoids bloating of business logic! Code injection and interpretation at compile or deployment time.
  46. 46. 12/12/2018 46Demetris Trihinas 46Tutorial | CloudCom 2018 Other Microservices Frameworks • Eclipse Vert.X • Lightweight framework (use only the libs you need). • Polyglot (Java, JavaScript, Groovy, Ruby, Scala and Kotlin). • Reactive programming (event-driven and non-blocking). • Framework functionality mixes with business logic. • Relatively new with need of more documentation and guides.
  47. 47. 12/12/2018 47Demetris Trihinas 47Tutorial | CloudCom 2018 Other Microservices Frameworks • Thorntail (previously Wildfly Swarm) • RedHat’s open-source Java application server micro-framework. • Lightweight framework (use only the libs you need). • Framework functionality mixes with business logic. • New with need of more documentation and guides. • Oracle Helidon (another Java framework, introduced 2018). • Nameko (python, RPC over AMQP) • Java seems the language of trade.
  48. 48. 12/12/2018 48Demetris Trihinas 48Tutorial | CloudCom 2018 The “Books” (Micro-) Service The Books microservice entry point
  49. 49. 12/12/2018 49Demetris Trihinas 49Tutorial | CloudCom 2018 The “Books” (Micro-) Service • Spring Boot does all the configuration (e.g., dependency management, parameter passing). • Bundles app with web server (Apache Tomcat by default). All is done with the addition of a single annotation
  50. 50. 12/12/2018 50Demetris Trihinas 50Tutorial | CloudCom 2018 The “Book” Service • To use Spring Boot you need to include the respected dependency and plugin to your dependency management tool (e.g., mvn).
  51. 51. 12/12/2018 51Demetris Trihinas 51Tutorial | CloudCom 2018 The “Book” Model
  52. 52. 12/12/2018 52Demetris Trihinas 52Tutorial | CloudCom 2018 The “Book” Model No mention of underlying data store (e.g., MySQL, MariaDB)! Create relational entity (a.k.a table) Auto-generated primary key! Define properties that cannot be left blank!
  53. 53. 12/12/2018 53Demetris Trihinas 53Tutorial | CloudCom 2018 The “Book” Repository
  54. 54. 12/12/2018 54Demetris Trihinas 54Tutorial | CloudCom 2018 The “Book” Repository • All CRUD operations are auto-generated and made available by Spring Boot. • Create new record. • Update existing record. • Delete record. • Find one record. • Find All records.
  55. 55. 12/12/2018 55Demetris Trihinas 55Tutorial | CloudCom 2018 The “Book” Repository • Repository functionality can be extended. Extend lookup queries to use book title --secondary index-- not just book id.
  56. 56. 12/12/2018 56Demetris Trihinas 56Tutorial | CloudCom 2018 Configuring Data Store via Spring Through your microservice config file -> NO code changes!
  57. 57. 12/12/2018 57Demetris Trihinas 57Tutorial | CloudCom 2018 Configuring Data Store via Spring Through your microservice config file -> NO code changes!
  58. 58. 12/12/2018 58Demetris Trihinas 58Tutorial | CloudCom 2018 Spring Data for Entity Model and Repository • In JPA are the modeling handlers for persistent storage (entity, etc.). • Connector to use MySQL (not part of Spring Boot!).
  59. 59. 12/12/2018 59Demetris Trihinas 59Tutorial | CloudCom 2018 The “Books” Controller
  60. 60. 12/12/2018 60Demetris Trihinas 60Tutorial | CloudCom 2018 The “Books” Controller • Handles all “book” requests. • Provides REST API for accessing book repository. • Maps requests to database/repository queries.
  61. 61. 12/12/2018 61Demetris Trihinas 61Tutorial | CloudCom 2018 The “Books” Controller REST API landing URI Automatic dependency injection Microservices in microservices! Book Controller returns a JSON array of ALL Book entities in DB Change from JSON to XML, plain text or HTML by only changing parameter!
  62. 62. 12/12/2018 62Demetris Trihinas 62Tutorial | CloudCom 2018 Microservices only Rely on Each Others Public API • Database technology or schema can change but the client doesn’t care as access to data is provided via the API. Microservice Y Public API Microservice X Public API Nope!
  63. 63. 12/12/2018 63Demetris Trihinas 63Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Reviews Cart
  64. 64. 12/12/2018 64Demetris Trihinas 64Tutorial | CloudCom 2018 Microservices Development • The Ratings, Reviews and Cart Microservices developed following similar design pattern. • Omitting due to time. • But… code publically available:
  65. 65. 12/12/2018 65Demetris Trihinas 65Tutorial | CloudCom 2018 Service Discovery and Load Balancing How do services find each other
  66. 66. 12/12/2018 66Demetris Trihinas 66Tutorial | CloudCom 2018 Service Discovery • Service discovery uses a registry to keep a real-time list of services, their location, and their health. • Services query the registry to discover the location of other services and then connect directly. • This allows services to scale up/down, load balancing and gracefully handle failure in dynamic infrastructures.
  67. 67. 12/12/2018 67Demetris Trihinas 67Tutorial | CloudCom 2018 The Self-Registration Pattern • A service instance is responsible for (de-)registering itself with the service registry. • The service instance sends heartbeat requests to prevent its registration from expiring.
  68. 68. 12/12/2018 68Demetris Trihinas 68Tutorial | CloudCom 2018 The Client-Side Discovery Pattern • The client is responsible for determining the network locations of available services and load balancing requests across them. • The client queries a service registry, which is essentially a database with the available services locations. • The client then selects one of the available services (e.g. load-balancing algorithm) to make its request.
  69. 69. 12/12/2018 69Demetris Trihinas 69Tutorial | CloudCom 2018 The Client-Side Discovery Pattern Drawback #1 -> The client is highly coupled with the service registry. Drawback #2 -> Service discovery logic MUST be implemented for each programming language and frameworks used by service clients. • Client knows about all available services -> intelligent and application-specific load-balancing decisions.
  70. 70. 12/12/2018 70Demetris Trihinas 70Tutorial | CloudCom 2018 • The client makes a request to a service via a load balancer. • The load balancer queries the service registry and routes each request to an available service instance. The Server-Side Discovery Pattern Service Registry + Load Balancer two for the price of one!
  71. 71. 12/12/2018 71Demetris Trihinas 71Tutorial | CloudCom 2018 The Server-Side Discovery Pattern • etcd - Highly available key-value store used for shared configuration and service discovery (CloudFoundry, Kubernetes). • Consul - provides API that allows clients to register and discover services. Performs health checks to determine service availability.
  72. 72. 12/12/2018 72Demetris Trihinas 72Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Cart Service Registry
  73. 73. 12/12/2018 73Demetris Trihinas 73Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Reviews Cart Service Registry Register “Reviews” as a service
  74. 74. 12/12/2018 74Demetris Trihinas 74Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Reviews Cart Service Registry
  75. 75. 12/12/2018 75Demetris Trihinas 75Tutorial | CloudCom 2018 Service Discovery with Consul • Integrated well with Spring Boot with code annotations and auto-configuration enabled. All is done with the addition of a single annotation
  76. 76. 12/12/2018 76Demetris Trihinas 76Tutorial | CloudCom 2018 Service Discovery with Consul • To use Consul annotations simply add the respected dependency to mvn.
  77. 77. 12/12/2018 77Demetris Trihinas 77Tutorial | CloudCom 2018 Configuring Service Discovery • Through your microservice config file -> NO code changes! Auto-generated Instance IDs for dynamic resource provisioning! Tags to categorize your services!
  78. 78. 12/12/2018 78Demetris Trihinas 78Tutorial | CloudCom 2018 Service Discovery with Consul
  79. 79. 12/12/2018 79Demetris Trihinas 79Tutorial | CloudCom 2018 Service Discovery with Consul The tags declared in microservice configuration!
  80. 80. 12/12/2018 80Demetris Trihinas 80Tutorial | CloudCom 2018 Service Discovery with Consul Same Service ID But… unique InstanceID
  81. 81. 12/12/2018 81Demetris Trihinas 81Tutorial | CloudCom 2018 Service Discovery with Consul Heath Check Report
  82. 82. 12/12/2018 82Demetris Trihinas 82Tutorial | CloudCom 2018 • Map requests to services and retrieve resources. • Distribute load among services. Reverse Proxy and Load Balancing Horizontal Scalability
  83. 83. 12/12/2018 83Demetris Trihinas 83Tutorial | CloudCom 2018 Reverse Proxy and Load Balancing • Traditional reverse-proxies (e.g., HAproxy) require that you statically configure each route that will connect paths and sub-domains to each microservice. • In a dynamic environment you add, remove, kill, upgrade, or scale; your services many times a day. • keeping routes up-to-date becomes tedious.
  84. 84. 12/12/2018 84Demetris Trihinas 84Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Reviews Carts Load Balancer Service Registry Request Route Books#3 Ratings#1 Reviews#1 Carts#2 Services Books{1,2,3} Ratings{1} Reviews{1,2} Carts{1,2,3}
  85. 85. 12/12/2018 85Demetris Trihinas 85Tutorial | CloudCom 2018 Traefik • Reverse proxy and load balancer. • Auto-configurable and supports dynamic discovery. • Traefik listens to your service registry (e.g., Consul) or orchestrator API (e.g., Kubernetes). • Instantly generates the routes so your microservices are connected to the outside world -- without manual intervention.
  86. 86. 12/12/2018 86Demetris Trihinas 86Tutorial | CloudCom 2018 Traefik – Proxy Service Catalog Micro-services categorized by declared tags
  87. 87. 12/12/2018 87Demetris Trihinas 87Tutorial | CloudCom 2018 Traefik – Load Balancing Define Load Balancing Strategies Adjust weighting to skew the load A/B testing 10% of traffic to test beta version There is even option for dynamic weighting which increases weights on servers that perform better than others.
  88. 88. 12/12/2018 88Demetris Trihinas 88Tutorial | CloudCom 2018 Reverse Proxy and Load Balancing • Load balancer can change anytime without any code changes in to the microservices! • Other Reverse Proxies and Load Balancers • Google’s Envoy: • Adopts side-car paradigm for containers • Netflix OSS: • Eureka (SD) + Ribbon (LB) + Hystrix (CB) • Must be configured through app.
  89. 89. 12/12/2018 89Demetris Trihinas 89Tutorial | CloudCom 2018 API Gateway “Homogenizing” resource access.
  90. 90. 12/12/2018 90Demetris Trihinas 90Tutorial | CloudCom 2018 The Book Store
  91. 91. 12/12/2018 91Demetris Trihinas 91Tutorial | CloudCom 2018 API Gateway and Identity Service • Coordinates how all client requests are processed and also performs request validation. • Allows for clean clients - no need to implement different request technologies (e.g., REST, SOAP, etc). • Gateway authenticates user via identity service which provides access token. • All requests going to microservices without a valid token are rejected.
  92. 92. 12/12/2018 92Demetris Trihinas 92Tutorial | CloudCom 2018 API Gateway • Developed also by using microservices design pattern.
  93. 93. 12/12/2018 93Demetris Trihinas 93Tutorial | CloudCom 2018 API Gateway • Endpoints to services are NOT hardcoded, instead they are “autowired” -> microservices in microservices. API version request mapping Autowired endpoints to book store services
  94. 94. 12/12/2018 94Demetris Trihinas 94Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Reviews Cart Load Balancer API Gateway Service Registry Identity Service
  95. 95. 12/12/2018 95Demetris Trihinas 95Tutorial | CloudCom 2018 “Behind” the Book Store Books Ratings Reviews Cart Load Balancer API Gateway Service Registry Identity Service The “front” door Behind private network
  96. 96. 12/12/2018 96Demetris Trihinas 96Tutorial | CloudCom 2018 Circuit Breaking • Prevents high loads on failing servers based on various strategies: • One service cannot bring down the entire application. Books Ratings CartCB Bug in latest version of ratings service.
  97. 97. 12/12/2018 97Demetris Trihinas 97Tutorial | CloudCom 2018 The Book Store Frontend One service cannot bring down the entire application.
  98. 98. 12/12/2018 98Demetris Trihinas 98Tutorial | CloudCom 2018 Circuit Breaking ASYNChronous resource fetching with 3 second timeout.
  99. 99. 12/12/2018 99Demetris Trihinas 99Tutorial | CloudCom 2018 Other Circuit Breaking Strategies • Error-ratio over 10s sliding window: • Latency at quantile in milliseconds: • Ratio of response code range: NetworkErrorRatio() > 0.5 LatencyAtQuantileMS(50.0) > 50 ResponseCodeRatio(500, 600) > 0.5
  100. 100. 12/12/2018 100Demetris Trihinas 100Tutorial | CloudCom 2018 Circuit Breaking Without taking steps to ensure fault tolerance, 30 dependencies each with 99.99% uptime would result in 2+ hours downtime/month (99.99%^30 = 99.7% uptime = 2+ hours in a month) Ben Christensen, Netflix Engineer
  101. 101. 12/12/2018 101Demetris Trihinas 101Tutorial | CloudCom 2018 API Gateway - Versioning • A key factor of resilience is that when you make upgrades to your services, your clients don’t suffer from downtime. • In the "old" days, upgrades were notorious for breaking clients. • An upgrade to the server required an update to the client. • Nowadays, minutes of downtime for doing an upgrade can cost millions in revenue.
  102. 102. 12/12/2018 102Demetris Trihinas 102Tutorial | CloudCom 2018 API Gateway - Versioning • For example: • Instead of author, management decides to change field to author_firstname, author_lastname. • Easy update for db engineer but what about your clients? • Remember: downtime -> lose of money • Naïve solution… wake up at 2am to do upgrades. • But… it‘s not 2am everywhere when offering a service over the internet.
  103. 103. 12/12/2018 103Demetris Trihinas 103Tutorial | CloudCom 2018 API Versioning – Backwards Compatible • Old strategy that precedes microservices by many years. • Never delete a column in a database (M. Stonebraker) • You can add columns to a DB but never take them away. • Not every change is possible with duplication. In such case v2 is introduced and clients not upgraded are served from v1. “Database Decay and How to Avoid It”. M. Stonebraker, D. Deng, M. L. Brodie, IEEE Big Data, 2016 Duplicate info but no client downtime
  104. 104. 12/12/2018 104Demetris Trihinas 104Tutorial | CloudCom 2018 Token-Based Authentication • Spring Security extends Spring Boot functionality to support authentication. • Each request includes token (ideally in request header), allowing access to routes, services, and resources. • Token based authentication: JWT, OAuth2
  105. 105. 12/12/2018 105Demetris Trihinas 105Tutorial | CloudCom 2018 JSON Web Token Authentication • JWT is open standard (RFC 7519) for secure data transmission between parties as a JSON object. • Data can be trusted because it is digitally signed. • Secret key (HMAC algorithm) or a private/public RSA key pair. • Token Signature is calculated using the header and the token payload -> verify the integrity of the content.
  106. 106. 12/12/2018 106Demetris Trihinas 106Tutorial | CloudCom 2018 JSON Web Token Authentication Books Microservices Ratings Microservices Reviews Microservices Cart Microservices Token validated at the microservice level so that auth server is never stressed. Only creates new tokens and refreshes expired ones.
  107. 107. 12/12/2018 107Demetris Trihinas 107Tutorial | CloudCom 2018 JSON Web Token Authentication JWT validation in Spring is just a WebFilter that checks token signature.
  108. 108. 12/12/2018 108Demetris Trihinas 108Tutorial | CloudCom 2018 API Documentation • Documentation is a headache to maintain. • A single API change and the documentation is out-of- date. • API documentation should be updated and visualized automatically. • If the API is public then the “public” must be able to access it.
  109. 109. 12/12/2018 109Demetris Trihinas 109Tutorial | CloudCom 2018 Swagger API Documentation
  110. 110. 12/12/2018 110Demetris Trihinas 110Tutorial | CloudCom 2018 Swagger API and UI dependency for mvn. Swagger API Documentation REST controller and methods decorated with API annotations.
  111. 111. 12/12/2018 111Demetris Trihinas 111Tutorial | CloudCom 2018 Containerization Dockerizing the microservices
  112. 112. 12/12/2018 112Demetris Trihinas 112Tutorial | CloudCom 2018 Dockerizing the Microservices • Package and deploy microservices in container runtimes for portability across private and public clouds. • Prepare Dockerfile for each service to configure the bundling of the service with the container. • Prepare Docker Compose description to automate the configuration and deployment of the entire platform.
  113. 113. 12/12/2018 113Demetris Trihinas 113Tutorial | CloudCom 2018 The Dockerfile • A text file that contains all commands needed to prepare a Docker image = container + service config. • A Docker image is comprised of read-only layers with each layer representing a Dockerfile instruction. • The layers are stacked and each one is a delta of the changes from the previous layer.
  114. 114. 12/12/2018 114Demetris Trihinas 114Tutorial | CloudCom 2018 Book Service Dockerfile Use as a base for this image another image or start FROM scratch Add files to image from your local machine What commands to run within the container Instructions to build the container (e.g., install java 8, set environmental variables)
  115. 115. 12/12/2018 115Demetris Trihinas 115Tutorial | CloudCom 2018 Book Service Dockerfile These commands are used in ALL our services (e.g., rating, review, cart) Hello… BookBase Docker image
  116. 116. 12/12/2018 116Demetris Trihinas 116Tutorial | CloudCom 2018 Ratings Service Dockerfile A container built from other containers!
  117. 117. 12/12/2018 117Demetris Trihinas 117Tutorial | CloudCom 2018 Docker Compose • Putting everything together. • Multi-container/Multi-service declaration and configuration. • Common subnet network. • Persistent volume for consul and databases. • CI workflow integration.
  118. 118. 12/12/2018 118Demetris Trihinas 118Tutorial | CloudCom 2018 Docker Compose • Books, Ratings, Reviews and Cart Services Put Books service behind cloudcom private network!
  119. 119. 12/12/2018 119Demetris Trihinas 119Tutorial | CloudCom 2018 Docker Compose • Consul Service Registry No dockerfile… consul image available in docker hub Also part of the private cloudcom subnet
  120. 120. 12/12/2018 120Demetris Trihinas 120Tutorial | CloudCom 2018 Microservices Challenges
  121. 121. 12/12/2018 121Demetris Trihinas 121Tutorial | CloudCom 2018 Microservices at Scale • In the cloud era, as applications grow by adding more services, dynamic resource allocation and security enforcement become significant challenges. • These challenges must be fostered through autonomicity.
  122. 122. 12/12/2018 122Demetris Trihinas 122Tutorial | CloudCom 2018 Monitoring and Diagnostics • Containers ease deployment for users – no need to configure (virtual) infrastructure, network(s), storage. • Nightmare for monitoring tools -> no guest OS to deploy agents side-by-side with application. Monitoring must be deployed and run through container engine or be part of the application Monitoring is integral part of app design and cannot be decided after deployment.
  123. 123. 12/12/2018 123Demetris Trihinas 123Tutorial | CloudCom 2018 Monitoring and Diagnostics • Granularly slicing an application into services inherently introduces heterogeneity. • Book service is latency sensitive while books accessed are logged for analytics (e.g., popular titles). • Ratings service is cpu-intensive. • Reviews service is memory-bound while reviews are monitored for helpful/deceitful reviews. • Requires full customization of the monitoring process to perform diagnostics and receive helpful insights.
  124. 124. 12/12/2018 124Demetris Trihinas 124Tutorial | CloudCom 2018 Monitoring and Diagnostics • Monitoring ephemeral, decomposed and highly dynamic applications -> Non-neglectable monitoring cost! • Overhead on monitoring source itself. • Overhead (network, storage) and actual cost on monitoring topology. • Overhead to process real-time data and trawl historic monitoring data. D. Trihinas and G. Pallis and M. D. Dikaiakos, "Low-Cost Adaptive Monitoring Techniques for the Internet of Things", IEEE Transactions on Services Computing, 2018.
  125. 125. 12/12/2018 125Demetris Trihinas 125Tutorial | CloudCom 2018 Auto-Scaling and Optimization • Scaling to meet demand is a challenge for most applications and microservices are no exception. • But I thought… microservices and containers are inherently easier to scale: • Simply create more copies of the services overwhelmed by demand (horizontal scaling) and fastly boot your containers. • So which one is it? Lopez et al. “Towards quantifiable boundaries for elastic horizontal scaling of microservices”, IEEE/ACM UCC, 2017.
  126. 126. 12/12/2018 126Demetris Trihinas 126Tutorial | CloudCom 2018 Auto-Scaling and Optimization • At scale, significant profiling is required to optimize performance, cost and quality: • Identify what should be monitored • When and How to scale • Investigate, in a distributed and granular deployment, if one service is currently affecting the performance of another service? • Anticipate demand? “[Best Paper] ADVISE – a Framework for Evaluating Cloud Service Elasticity Behavior.” Copil, G. and Trihinas, D. et al., ICSOC 2014.
  127. 127. 12/12/2018 127Demetris Trihinas 127Tutorial | CloudCom 2018 Orchestration in Hybrid Cloud Deployments • Software teams are increasingly choosing to work with multiple cloud offerings and/or cloud providers. • Although containers –ideally can run anywhere- this does not mean that a deployment can span across: • Geographic regions • Cloud availability zones • Different cloud sites
  128. 128. 12/12/2018 128Demetris Trihinas 128Tutorial | CloudCom 2018 Orchestration in Hybrid Cloud Deployments • Constructing and managing a cross site overlay network? • Network performance issues (e.g., propagation delays) for service communication especially when network traffic must pass across datacenter boundaries. • Service placement strategies: • Suitability, cost, availability, latency?
  129. 129. 12/12/2018 129Demetris Trihinas 129Tutorial | CloudCom 2018 Security Enforcement and Privacy Protection • For a monolith there is only one thing to protect… and its still a challenge. • Decomposing an app into services with each service inter-communicating over the network raises significant security risks.
  130. 130. 12/12/2018 130Demetris Trihinas 130Tutorial | CloudCom 2018 Security Enforcement and Privacy Protection • Maintaining identity and access management across the entire deployment. • Mitigating new sets of security rules at runtime without service disruption? • We cant just shut down thousands service instances for security rule alteration. Sun et al, Security-as-a-Service for Microservices-Based Cloud Applications, IEEE CloudCom 2015.
  131. 131. 12/12/2018 131Demetris Trihinas 131Tutorial | CloudCom 2018 Security Enforcement and Privacy Protection • Privacy leaks due to code vulnerabilities in diversified service stack. • In-house developed code. • Third-party libraries, frameworks, dependencies. • Runtime overhead for security enforcement and privacy preserving mechanisms? Shu et al., “A Study of Security Vulnerabilities on Docker Hub”, ACM CODASPY, 2017. Docker Security Scanning,
  132. 132. 12/12/2018 132Demetris Trihinas 132Tutorial | CloudCom 2018 DevOps as a Service Platform
  133. 133. 12/12/2018 133Demetris Trihinas 133Tutorial | CloudCom 2018 Unicorn Vision Increase SME and Startup productivity by offering a DevOps-as-a- Service platform to ease the design and deployment of secure and elastic microservices.
  134. 134. 12/12/2018 134Demetris Trihinas 134Tutorial | CloudCom 2018 Unicorn Dashboard • One collaborative and unified environment: • Develop apps • Share workspaces • Ship coded artefacts to the cloud • Manage the entire deployment lifespan
  135. 135. 12/12/2018 135Demetris Trihinas 135Tutorial | CloudCom 2018 Unicorn Dashboard: Developer Perspective • Design libraries for policy and constraint definition through code annotations for minimum code intrusion. • Eclipse Che IDE plugin for collaborative development.
  136. 136. 12/12/2018 136Demetris Trihinas 136Tutorial | CloudCom 2018 Unicorn Dashboard: Manager Perspective • Annotations mapped to Service Graph description. • Administrators can enrich graph with additional policies without having to use code annotations.
  137. 137. 12/12/2018 137Demetris Trihinas 137Tutorial | CloudCom 2018 Unicorn Dashboard: App Description • Service graph description extends Docker Compose model. • No need to learn another proprietary technology. • Description can still be used in any other Docker runtime.
  138. 138. 12/12/2018 138Demetris Trihinas 138Tutorial | CloudCom 2018 Unicorn Platform • Service graph validation (e.g. antagonizing policies). • Smart and interoperable multi-cloud orchestration. • Runtime policy and constrain enforcement.
  139. 139. 12/12/2018 139Demetris Trihinas 139Tutorial | CloudCom 2018 Unicorn Technology Stack Developed on top of popular and open-source frameworks including Kubernetes, Docker, CoreOS to support multi-cloud application runtime management.
  140. 140. 12/12/2018 140Demetris Trihinas 140Tutorial | CloudCom 2018 Unicorn Technology Stack Kubernetes to support the orchestration of large-scale distributed containerized deployments spanning across multiple hosts. Kubernetes Limitations - (De-)provisioning infrastructure resources. - Auto-scaling. - Cross-cloud deployments.
  141. 141. 12/12/2018 141Demetris Trihinas 141Tutorial | CloudCom 2018 Unicorn Technology Stack Unicorn Smart Orchestrator extends open-source Arcadia framework to enable Kubernetes across multiple cloud sites. - Cloud adaptors to probe and program underlying infrastructure. - Taps into auto-scaling offered by cloud offerings to estimate and assess app elasticity behavior and scaling effects. - Cross-cloud network overlay management to reliably handle SDN accessibility.
  142. 142. 12/12/2018 142Demetris Trihinas 142Tutorial | CloudCom 2018 Unicorn Technology Stack Underlying kernel for the containerized environment is CoreOS which enables fast boot times and secure-out-of-the Docker runtime. Unicorn “side-car” services - Orchestrator service for HA host management. - Low-cost and self- adaptive monitoring to reduce network traffic propagation. - Security service to filter network traffic and apply privacy preserving ruling.
  143. 143. 12/12/2018 143Demetris Trihinas 143Tutorial | CloudCom 2018 Sign-in to Unicorn
  144. 144. 12/12/2018 144Demetris Trihinas 144Tutorial | CloudCom 2018 Unicorn Dashboard
  145. 145. 12/12/2018 145Demetris Trihinas 145Tutorial | CloudCom 2018 Unicorn Design Libraries • Available for downloading through Unicorn Dashboard. • Design libraries for monitoring, elastic scaling, security enforcement and vulnerability assessment. • Code annotations -> no business-logic bloating. • Currently available for java apps.
  146. 146. 12/12/2018 146Demetris Trihinas 146Tutorial | CloudCom 2018 Monitoring Design Library • Maven Dependency also available to ease design library usage:
  147. 147. 12/12/2018 147Demetris Trihinas 147Tutorial | CloudCom 2018 Monitoring Design Library • Simply add the Unicorn Monitoring annotation and monitoring is enabled automatically with default configuration!
  148. 148. 12/12/2018 148Demetris Trihinas 148Tutorial | CloudCom 2018 Customize Monitoring – Add Probes Create your own Monitoring Probes
  149. 149. 12/12/2018 149Demetris Trihinas 149Tutorial | CloudCom 2018 Customize Monitoring – Add Probes • Spring Boot Probe (avg response time, avg throughput, requests since last update): Customize and extend monitoring
  150. 150. 12/12/2018 151Demetris Trihinas 151Tutorial | CloudCom 2018 Customize Monitoring – Create Probe
  151. 151. 12/12/2018 152Demetris Trihinas 152Tutorial | CloudCom 2018 Register Cloud Providers • Register and manage cloud enpoints and access tokens.
  152. 152. 12/12/2018 153Demetris Trihinas 153Tutorial | CloudCom 2018 Manage Cloud Availability Regions Enable and disable cloud availability regions for app deployments.
  153. 153. 12/12/2018 154Demetris Trihinas 154Tutorial | CloudCom 2018 Container Registry • Users are allowed to push their docker images to the Unicorn private registry instead of public docker hub.
  154. 154. 12/12/2018 155Demetris Trihinas 155Tutorial | CloudCom 2018 Add Microservices
  155. 155. 12/12/2018 156Demetris Trihinas 156Tutorial | CloudCom 2018 Creating an Application Description
  156. 156. 12/12/2018 157Demetris Trihinas 157Tutorial | CloudCom 2018 Service Graph Model Unicorn handles service registry, load balancing, elastic scaling and cluster management! Only configure and drag n’ drop your services.
  157. 157. 12/12/2018 158Demetris Trihinas 158Tutorial | CloudCom 2018 Application Deployment
  158. 158. 12/12/2018 159Demetris Trihinas 159Tutorial | CloudCom 2018 Runtime Policy and Service Graph Config
  159. 159. 12/12/2018 160Demetris Trihinas 160Tutorial | CloudCom 2018 Elastic Scaling
  160. 160. 12/12/2018 161Demetris Trihinas 161Tutorial | CloudCom 2018 Service Graph Description Auto-Updated
  161. 161. 12/12/2018 162Demetris Trihinas 162Tutorial | CloudCom 2018 Runtime Policy and Service Graph Config (2)
  162. 162. 12/12/2018 163Demetris Trihinas 163Tutorial | CloudCom 2018 Security Rules and Alerts
  163. 163. 12/12/2018 164Demetris Trihinas 164Tutorial | CloudCom 2018 Unicorn Validation Contest • Startups with an idea for an app or have already developed a cloud app… and are interested in: • Converting their app to a microservices architecture by using the Unicorn design libraries • Deploying their app through the Unicorn platform • Startups can apply for a Unicorn Validation Contract. • 12 contracts -> 10.000 euros
  164. 164. 12/12/2018 165Demetris Trihinas 165Tutorial | CloudCom 2018 Unicorn Validation Contest
  165. 165. 12/12/2018 166Demetris Trihinas 166Tutorial | CloudCom 2018 Summary • Microservices are applications decomposed into single- function services that inter-communicate over lightweight communication protocols. • Designed and developed independently. • Decentralized governance. • Communication only relies on API’s. • Continuous delivery. • Failure isolation. • Portability and flexibility via containerization.
  166. 166. 12/12/2018 167Demetris Trihinas 167Tutorial | CloudCom 2018 Summary • Autonomicity can foster microservices scalability but it bears challenges: • Monitoring: part of app development process, service heterogeneity, significant overheads. • Auto-Scaling: determine optimization strategies, detect bottlenecks. • Cross-cloud Orchestration: service placement strategies. • Security: must secure multiple services instead of singleton.
  167. 167. 12/12/2018 168Demetris Trihinas 168Tutorial | CloudCom 2018 Summary • Unicorn DevOps-as-a-Service Platform • Design libraries for microservices development • Cloud IDE plugin for collaborative development • Policy and constraint definition via service graph model • Deployment via containerized runtimes (enhanced Docker Compose) • Multi-cloud orchestration • Real-time monitoring, auto-scaling and security enforcement
  168. 168. 12/12/2018 169Demetris Trihinas 169Tutorial | CloudCom 2018 Unicorn Team @unicorn_H2020 Dr. Marios D. Dikaiakos Dr. George Pallis Dr. Demetris Trihinas Athanasios Tryfonos Zacharias Georgiou Moysis Simeonidis Maria Poveda
  169. 169. 12/12/2018 170Demetris Trihinas 170Tutorial | CloudCom 2018 Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings Thank you! @unicorn_H2020