Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Zenoss Control Center Introduction


Published on

Control Center is an open source application management and orchestration system. Control Center was developed by Zenoss; however, apart from working with the Zenoss platform Control Center can be used to manage other Docker applications, from a simple web application to a multi-tiered stateful application stack.

Control Center is based on a service oriented architecture which enables applications to run as a set of distributed services that can span hosts, datacenters, and geographic regions. A simple declarative application template file is used to describe how an application is configured, deployed, managed, and monitored.

Published in: Technology
  • Be the first to comment

Zenoss Control Center Introduction

  1. 1. Zenoss Control Center Open Source Application Management & Orchestration System for Docker
  2. 2. Overview • History • Brief introduction to Docker • Why Docker? • What is Control Center? • Demo • Questions
  3. 3. History • Zenoss founded in 2005 as Open Source Enterprise IT Management Solution • Docker was chosen to support the scale requirements for monitoring the modern datacenter • Need was created for managing Zenoss within Docker • Birth of Control Center • Zenoss Control Center Open Sourced in August 2014
  4. 4. What is Docker? Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application.
  5. 5. ● Based on Linux containers ● Minimal overhead (cpu/io/network) ● Can run on any Linux system that supports LXC, nsinit ● Docker container contains everything it needs to run ● A docker container should be able to run anywhere that Docker can run. What is Docker?
  6. 6. Why Docker? • Scalability – Ability to spin up new daemons on the fly • Easy Installs – Just pull image and deploy – Dependencies exist in the container • Reliable History Tracking – Improves upgrades – Easy to roll back
  7. 7. What Docker doesn’t provide (yet) ● service discovery ● a good log interface ● cross host volumes ● cross host scheduling … and anything else you may find in a cluster manager
  8. 8. Where Docker Ends, Control Center Begins App Containers Control Center Container Manager Host OS (UOS) Physical/Virtual Server CC Control Center (CC) is an application written in Go! (aka golang). CC takes care of all interactions with Docker and the host system. You install CC on every computer that you intend to run Zenoss on. CC is open source software (Apache 2 license).
  9. 9. Terms ● serviced: Control Center command-line client ● Service: An entity that manages a container and/or other services. ● Service Definition: Describes how a service is run and configured ● Service Template: A document of service definitions relating to a platform. We tell Control Center how to execute the application inside of a container image via a Service Template. The same container may act completely differently depending on the service definition used to start it.
  10. 10. CONTROL CENTER - SYSTEMS The Control Center has 3 important systems that divide the work of orchestrating and monitoring of services. Master - This subsystem houses several subservices and is generally responsible of the overall orchestration of the system. There will only be one Control Center Master. Node Controller - This controller is responsible for all docker interaction, starting/stopping services, mounting of service resources and metric collection. There is one of these per host. Container Controller - This program is the first program that gets launched inside a container. It is responsible for starting the actual service, providing network proxies, forwarding logs/events, and forwarding service specific stats.
  11. 11. CONTROL CENTER - SYSTEM OVERVIEW Master Logstash Zookeeper Metric Svc Docker Registry UI/Rest API RPC API Node Controller RPC API Docker Container Container controller Container controller controller logstash-forwarder logstash-forwarder Your Service Your Service Your Service logstash-forwarder TCP Mux Shared FS Host Boundary Node Controller RPC TCP Mux
  12. 12. CONTROL CENTER MASTER Elasticsearch Database for control center Zookeeper Coordination service Logstash Log consolidation Uses its own copy of Elasticsearch Metrics -- CC centric metric system OpenTSDB Standalone Hbase Consumer Query service Celery Schedule tasks Registry docker-registry Shared Storage (NFS) Web server for CC UI Rest API / RPC API
  13. 13. CC NODE CONTROLLER ● 1 Node Controller per host ● Configured how to talk to master ● Watches the master’s zookeeper for services that has been scheduled to start on its hosts ● Pulls down images from Master’s Docker Registry ● Starts/Stops/Manages services as needed ● Mux distributes all network traffic to this box to the appropriate containers
  14. 14. CONTAINER CONTROLLER ● Injected into each container ● Serviced binary is bind mounted into every container that it runs automatically ● The Serviced container controller is what runs and then executes the actual service command ● Imports endpoints from other services ● transparently proxies connections to remote endpoints and makes them appear local ● Bind mounts Logstash forwarder
  15. 15. CC - CORE CONCEPTS Pool - a collection of compute, network and storage resources. ● IP Addresses ● Hosts ● Storage Systems (future)
  16. 16. CC - CORE CONCEPTS Host - a compute node (physical or virtual) that is available to the control center. ○ Hostname ○ Pool ID ○ Kernel Version ○ OS Distribution ○ Docker Version ○ ServiceD Version ○ Memory Size ○ CPU Count ○ Interface (NICs) Names & Addresses ○ Monitoring Profile omain/host/host.go#L19
  17. 17. CC - CORE CONCEPTS Service Template - A hierarchical description of a services that deliver an Application Resource Manager Zproxy (Nginx) Zope Query OpenTSDB HBase MariaDB ZEP RabbitMQ MariaDB 2 Zookeeper Not Limited to Tree Dependencies
  18. 18. CC - CORE CONCEPTS Service - Is a description of an individual process. ● ImageID - what container image does this service use ● Startup - what command must run to start the service ● Config Files - what configuration files to inject into the container before startup ● Log Files - what log files the service writes to inside the container and what is the format of those log files ● Memory Limit - what is the maximum allowable amount of memory the service can use ● Volumes - what persistent disk storage is needed by the service and where that storage should be mounted ● Endpoints - what network endpoints does this service depend on or does this service expose ● Health Checks - what scripts can be run to determine the health of a service ● Metrics - what datapoints the service exposes ● Thresholds - what thresholds can be applied to the exposed metrics ● Graphs - what graphs are relevant to the service
  19. 19. CC - CORE CONCEPTS Schema - the relationship between the various domain objects in the control center Service Control Center Template Pool [default] Host [] Host [] Host [] Service A Service A Service A ServiceA @T0
  20. 20. CC - NETWORK CONNECTIONS Imagine your service needs to talk to Redis. It thinks it can access redis via The container controller actually accepts the connection Your Service Container Controller localhost interface (lo) Zookeeper The controller finds where redis is actually running via ZK. Mux The controller makes an encrypted connection to the mux. Redis Container The mux contacts the exposed port of the redis container.
  21. 21. MariaDB Sample Service Template { "Command": "/usr/bin/mysqld_safe", "Endpoints": [ { "Application": "mysql", "PortNumber": 3306, "Protocol": "tcp", "Purpose": "export" } ], "ImageId": "zenoss/zenoss5x", "Instances": { "min": 1 }, "Launch": "auto", "LogConfigs": [ { "path": "/var/log/mysqld.log", "type": "mysql" } ], "Name": "MySQL", "Snapshot": { "Pause": "PAUSE_CHECK_TIMEOUT=60 LOCK_HOLD_DURATION=600 $ZENHOME/bin/ pause", "Resume": "$ZENHOME/bin/ resume" }, "Tags": [ "daemon" ] }
  22. 22. Persistent Volumes For those services that have to maintain state between execution, the Control Center will mount a folder from the Shared File System into the container. The SFS is used so that even if the service starts up on a different machine it can still access the same files. Service 1 Service 2 Control Center Docker SFS Operating System Service 3 Service 4 Service 6 Control Center Service 5 Docker SFS Operating System
  23. 23. Install Control Center on Ubuntu 14.04 # install docker repo wget -O - | sh # install zenoss repo sudo apt-key adv --keyserver --recv-keys AA5A1AD7 sudo sh -c 'echo "deb [ arch=amd64 ] trusty universe" > /etc/apt/sources.list.d/zenoss.list’ # update repo database sudo apt-get update # install Control Center sudo apt-get -y install serviced # then start services sudo start serviced
  24. 24. Demo Time
  25. 25. Questions?