Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deploying (micro)services with Disnix

3,420 views

Published on

Talk given about deployment, Disnix, and microservices including a few simple examples and a real-life usage scenario @ NixCon2015

Published in: Software
  • Be the first to comment

Deploying (micro)services with Disnix

  1. 1. Deploying (micro)services with Disnix Sander van der Burg Nov 14, 2015 Sander van der Burg Deploying (micro)services with Disnix
  2. 2. Microservice architectures Microservice architectures have become quite popular: Sander van der Burg Deploying (micro)services with Disnix
  3. 3. Microservice architectures There are even books available: Sander van der Burg Deploying (micro)services with Disnix
  4. 4. Microservice architectures: What are they? In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. – Martin Fowler Sander van der Burg Deploying (micro)services with Disnix
  5. 5. Microservice architectures: What are they? In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. – Martin Fowler Sander van der Burg Deploying (micro)services with Disnix
  6. 6. Microservice architectures: are they a revolution? Sander van der Burg Deploying (micro)services with Disnix
  7. 7. Microservice architectures: are they a revolution? Sander van der Burg Deploying (micro)services with Disnix Service oriented architectures Services are autonomous, platform-independent entities that can be described, published, discov- ered, and loosely coupled in novel ways. They per- form functions that range from answering simple requests to executing sophisticated business pro- cesses requiring peer-to-peer relationships among multiple layers of service consumers and providers. Any piece of code and any application component deployed on a system can be reused and trans- formed into a network-available service.
  8. 8. Microservice architectures: are they a revolution? “A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third parties.” Sander van der Burg Deploying (micro)services with Disnix
  9. 9. Let’s implement an example system! Sander van der Burg Deploying (micro)services with Disnix
  10. 10. Let’s implement an example system! Sander van der Burg Deploying (micro)services with Disnix Stafftracker Let’s maintain a collection of staff members and their rooms From the room number I can determine the zipcode From the zipcode I can determine the street and city Divide the system into processes communicating through “light weight” network protocols (HTTP, TCP, ...)
  11. 11. Designing an architecture Sander van der Burg Deploying (micro)services with Disnix
  12. 12. Designing an architecture Sander van der Burg Deploying (micro)services with Disnix Pros Strong Module Boundaries: You can build teams around each service Independent Deployment: Less likely that a failure crashes the whole system A small component is typically easier to deploy (e.g. fewer dependencies) Technology Diversity: One service can be implemented in JavaScript, another in Python etc. Any storage means possible (e.g. databases) We can use many operating systems, hardware architectures etc. (source: http://martinfowler.com/microservices)
  13. 13. Designing an architecture Sander van der Burg Deploying (micro)services with Disnix Cons Distribution: Network links are slow and subject to failure Eventual Consistency: Difficult to guarantee that data remains consistent Operational Complexity: ... (source: http://martinfowler.com/microservices)
  14. 14. Operational Complexity All services can be deployed to one machine, but each service can also run on a dedicated machine: Worst case: 8 machines need to be managed Sander van der Burg Deploying (micro)services with Disnix
  15. 15. Operational Complexity Diverse technology imposes many kinds of deployment procedures: Deploying a Node.js package is different than deploying a Python package Different operating systems, different dependencies, many variants Sander van der Burg Deploying (micro)services with Disnix
  16. 16. Operational Complexity How to update the deployment frequently? How not to break the system while upgrading? How to minimize downtimes? How to roll back in case of a failure? Sander van der Burg Deploying (micro)services with Disnix
  17. 17. The Nix project Sander van der Burg Deploying (micro)services with Disnix
  18. 18. The Nix project Automated deployment using declarative specifications with the following properties: Generic. Can be used with many programming languages, component technologies, and operating systems. Reproducible. (Almost) no impurities – if inputs are the same, result should be the same regardless of its location Reliable. Dependency completeness, (almost) atomic upgrades and rollbacks. Efficient. Only the required deployment activities are executed. Sander van der Burg Deploying (micro)services with Disnix
  19. 19. The Nix project: Tools Nix – declarative specification of package build procedures (including their dependencies) NixOS – declarative specification of system aspects Hydra – declarative specification of jobs NixOps – declarative specification of a network of machines Deployment activities are implicit – a user writes or adapts a deploy- ment specification, and the tools will carry out the required activities, such as building, transfering, activating, and deactivating. Sander van der Burg Deploying (micro)services with Disnix
  20. 20. Deploying (micro)services How to deploy (micro)service(-oriented) systems the “Nix-way”? Sander van der Burg Deploying (micro)services with Disnix
  21. 21. Disnix $ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix Sander van der Burg Deploying (micro)services with Disnix
  22. 22. Disnix: Modeling a service build {stdenv, inetutils}: let makeFlags = "PREFIX=$out "+ "inetutils=${inetutils}"; in stdenv.mkDerivation { name = "hello-world-client"; src = ../../../services/hello-world-client; buildPhase = "make ${makeFlags}"; installPhase = "make ${makeFlags} install"; } Services are specified in a similar way as “ordinary” Nix packages: Define a function taking local (intra-dependencies) as a parameters Body describes how to build package from source and its dependencies Sander van der Burg Deploying (micro)services with Disnix
  23. 23. Disnix: Modeling a service build Sander van der Burg Deploying (micro)services with Disnix
  24. 24. Disnix: Modeling a service build {stdenv, inetutils}: {hello world server}: let makeFlags = "PREFIX=$out "+ "helloWorldHostname=${hello world server.target.hostname} "+ "helloWorldPort=${toString (hello world server.port)} "+ "inetutils=${inetutils}"; in stdenv.mkDerivation { name = "hello-world-client"; src = ../../../services/hello-world-client; buildPhase = "make ${makeFlags}"; installPhase = "make ${makeFlags} install"; } Disnix expressions also take inter-dependencies into account Sander van der Burg Deploying (micro)services with Disnix
  25. 25. Composing services locally {system, pkgs}: let callPackage = pkgs.lib.callPackageWith (pkgs // self); self = { hello_world_server = callPackage ../pkgs/hello-world-server { }; hello_world_client = callPackage ../pkgs/hello-world-client { }; }; in self As with ordinary Nix packages, we need to compose their intra- dependencies by providing them as function parameters. Sander van der Burg Deploying (micro)services with Disnix
  26. 26. Services model {system, distribution, invDistribution, pkgs}: let # Imports the intra-dependency composition shown previously customPkgs = import ../top-level/all-packages.nix { inherit system pkgs; }; in rec { hello_world_server = rec { name = "hello_world_server"; pkg = customPkgs.hello_world_server { inherit port; }; port = 3000; type = "wrapper"; # Specifies how to activate and deactivate a service }; hello_world_client = { name = "hello_world_client"; # Refers to the intra-dependency closure of the client pkg = customPkgs.hello_world_client; dependsOn = { # Inter-dependency composition inherit hello_world_server; }; type = "echo"; }; } Sander van der Burg Deploying (micro)services with Disnix
  27. 27. Infrastructure model { test1 = { # x86 Linux machine (Kubuntu) reachable with SSH hostname = "192.168.56.101"; system = "i686-linux"; targetProperty = "hostname"; clientInterface = "disnix-ssh-client"; }; test2 = { # x86-64 Linux machine (NixOS) reachable with SOAP/HTTP hostname = "192.168.56.102"; system = "x86_64-linux"; targetEPR = http://192.168.56.102:8080/DisnixWebService/...; targetProperty = "targetEPR"; clientInterface = "disnix-soap-client"; }; test3 = { # x86-64 Windows machine (Windows 7) reachable with SSH hostname = "192.168.56.103"; system = "x86_64-cygwin"; targetProperty = "hostname"; clientInterface = "disnix-ssh-client"; }; } Sander van der Burg Deploying (micro)services with Disnix
  28. 28. Distribution model {infrastructure}: { hello_world_server = [ infrastructure.test2 ]; hello_world_client = [ infrastructure.test1 ]; } Maps services in the services model to target machines in the infras- tructure model Sander van der Burg Deploying (micro)services with Disnix
  29. 29. Demo Sander van der Burg Deploying (micro)services with Disnix
  30. 30. Demo Initial deployment: $ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix Adding a second client (by adding test2): $ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix Moving the server from test2 to test3 and building on targets: $ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix --build-on-targets Sander van der Burg Deploying (micro)services with Disnix
  31. 31. Disnix: Communication flow disnix-env communicates with target machines through a custom connection client to remotely execute deployment activities: SSH (the default) SOAP/HTTP through DisnixWebService (http://github.com/svanderburg/DisnixWebService) NixOps through DisnixOS (http://github.com/svanderburg/disnixos) Sander van der Burg Deploying (micro)services with Disnix
  32. 32. Disnix: Communication flow Two deployment tools are invoked to carry out remote deployment activities: Nix (http://nixos.org/nix): package manager responsible for building, distributing Dysnomia (http://github.com/svanderburg/dysnomia): plugin system responsible for activating, deactivating, locking, snapshotting, restoring Sander van der Burg Deploying (micro)services with Disnix
  33. 33. Disnix: Process decomposition of disnix-env Each deployment activity is implemented by a command-line tool. Sander van der Burg Deploying (micro)services with Disnix
  34. 34. Disnix: Process decomposition of disnix-env Two additional tools are used when building on targets has been enabled. Sander van der Burg Deploying (micro)services with Disnix
  35. 35. How can we deploy our example system? Sander van der Burg Deploying (micro)services with Disnix
  36. 36. How can we deploy our example system? Sander van der Burg Deploying (micro)services with Disnix Implementation details Each database: a MongoDB database instance Each service: a process with REST API implemented in Node.js Web application front-end: A web application implemented in Node.js nginx proxy
  37. 37. Infrastructure deployment Before we can use Disnix, we need machines first! With the following characteristics: A Disnix service instance for executing deployment steps remotely MongoDB DBMS Machines can be deployed with various kinds of (external) solutions (including: Disnix itself). Sander van der Burg Deploying (micro)services with Disnix
  38. 38. Infrastructure deployment NixOps can be used to automatically instantiate and deploy a network of NixOS machines having the required characteristics: { test1 = { { pkgs, ...}: { networking.firewall.enable = false; # Enable the Disnix service services.disnix.enable = true; # Enabling MongoDB also configures Disnix # to use the Dysnomia plugin that can activate it services.mongodb.enable = true; services.mongodb.bind_ip = "0.0.0.0"; } test2 = ... test3 = ... } Sander van der Burg Deploying (micro)services with Disnix
  39. 39. Integrating service and infrastructure deployment Disnix’s service deployment and NixOps’ infrastructure deployment facilities can be combined with an extension toolset called: DisnixOS. Major difference with the basic toolset: DisnixOS uses networked NixOS configuration files instead of an infrastructure model Sander van der Burg Deploying (micro)services with Disnix
  40. 40. Demo Sander van der Burg Deploying (micro)services with Disnix
  41. 41. Demo Deploying a network of NixOS machines with NixOps: $ nixops create ./network.nix ./network-virtualbox.nix -d test $ nixops deploy -d test Deploying services to machines with DisnixOS: $ export NIXOPS DEPLOYMENT=test $ disnixos-env -s services.nix -n network.nix -n network-virtualbox.nix -d distribution.nix --use-nixops Sander van der Burg Deploying (micro)services with Disnix
  42. 42. A real world example: Conference Compass Sander van der Burg Deploying (micro)services with Disnix
  43. 43. A real world example: Conference Compass Conference Compass provides a service to improve the way people experience events Most visible part of the service: apps for conference attendees Each customer basically gets “their own” app. We have a product-line using a Nix-based build infrastructure, including Hydra Sander van der Burg Deploying (micro)services with Disnix
  44. 44. A real world example: Conference Compass The app’s contents is customizable with a configurator service allowing organizers to create and update their content Apps connect to a configurator to retrieve the data to be displayed and other configuration settings Integration with third party information systems is also possible Sander van der Burg Deploying (micro)services with Disnix
  45. 45. Architectural decisions Each app connects to its own dedicated configurator service: No single point of failure More scalable – slow configurator of one app does not affect another More flexible – we can easily move configurators around Multiple version support – we can add new features for a new app, without breaking old apps Implemented in Node.js Each third party integration (called: channel) is a seperate program: They may consume a lot of system resources Implemented in Python, because we find that more appropriate for data conversion Processes communicate through REST/HTTP APIs Sander van der Burg Deploying (micro)services with Disnix
  46. 46. Deploying services We deploy four kinds of services with Disnix: Configurators (Node.js programs) Channels (Python programs) Mongo databases (one for each configurator) nginx proxies (one for each machine) Responsible for forwarding requests to configurators and caching expensive requests Sander van der Burg Deploying (micro)services with Disnix
  47. 47. Deploying services A simple deployment scenario: (Diagram generated with disnix-visualize) Sander van der Burg Deploying (micro)services with Disnix
  48. 48. Deploying services We might want to move services to more powerful machines when an event goes live (because more users of an app imply that more system resources are needed, e.g. network bandwidth): Sander van der Burg Deploying (micro)services with Disnix
  49. 49. Deploying services For really big events, we may want to deploy redundant instances configurators and channels to handle all the load: Sander van der Burg Deploying (micro)services with Disnix
  50. 50. Some statistics We have been using Disnix + NixOps for deploying our production environment since January 2015 Between 6-11 Amazon EC2 machines are managed by NixOps ±80 configurators have been deployed with Disnix including their inter-dependencies Translates to ±200 services managed by Disnix Production environment is frequently updated. Sometimes three times a day. Transition phase duration is ±5 minutes in worst case. Most simple upgrades only bring a few seconds of downtime. Sander van der Burg Deploying (micro)services with Disnix
  51. 51. Conclusions I have explained the relevance of componentized distributed system architectures (e.g. microservices) I have explained how to do service deployment with Disnix I have explained how to integrate infrastructure deployment and service deployment I have shown a real-life usage scenario Sander van der Burg Deploying (micro)services with Disnix
  52. 52. References There is much more you can do with Disnix! This presentation only covers a few basic aspects! Disnix homepage: http://nixos.org/disnix TCP proxy example: https: //github.com/svanderburg/disnix-proxy-example StaffTracker examples: Java/WebServices/MySQL version: https://github.com/ svanderburg/disnix-stafftracker-java-example PHP/MySQL version: https://github.com/svanderburg/ disnix-stafftracker-php-example Node.js/MongoDB version: https://github.com/ svanderburg/disnix-stafftracker-nodejs-example Disnix should be considered an advanced prototype tool! Sander van der Burg Deploying (micro)services with Disnix
  53. 53. References Sander van der Burg Deploying (micro)services with Disnix
  54. 54. Questions Sander van der Burg Deploying (micro)services with Disnix

×