Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How OpenStack is Built - Anton Weiss - OpenStack Day Israel 2016

476 views

Published on

Openstack is one of the largest OSS projects today with hundreds of commits flowing in daily. This high rate of change requires an advanced CI infrastructure. The purpose of the talk is to provide an overview of this infrastructure, explaining the role of each tool and the pipelines along which changes have to travel before they find their way into the approved Openstack codebase.

Published in: Technology
  • Be the first to comment

How OpenStack is Built - Anton Weiss - OpenStack Day Israel 2016

  1. 1. How Openstack is Built Ant(on) Weiss - Otomato http://otomato.link 1
  2. 2. How Big is OpenStack? 265,850 commits 5,744 developers 54 project teams 585 git repos 2
  3. 3. OpenStack CI System 3 • Developed and managed by the OpenStack Infra team
  4. 4. Ansible+Puppet - two invisible hands. ■ All configuration of CI infra nodes is managed by Puppet manifests ■ The ‘Puppetmaster’ is actually an Ansible control machine ■ Puppetmaster holds playbook + hiera data ■ Ansible playbooks trigger ‘puppet apply’ on the nodes 4 +
  5. 5. Git - the mighty archivist ■ the most widely used distributed open-source VCS ■ Served by ‘cgit’ ■ Changes submitted with the help of ‘git-review’ tool 5
  6. 6. Gerrit – the gatekeeper ■ Originally developed by Shawn Pearce at Google for Android development. ■ Manages git repositories ■ Provides code review and approval rules ■ Collaborates with human and non- human gatekeepers 6
  7. 7. Zuul – the dispatcher ■ Defines and enforces the gating process ■ Listens to Gerrit events ■ Manages the queue and dependencies ■ Dispatches the verification process to Jenkins (through Gearman) 7
  8. 8. Gearman - the outsourcer ■ a framework to farm out work to other machines or processes 8 ■ sends jobs to Jenkins through jenkins gearman plugin
  9. 9. Jenkins - the butler :) ■ Continuous Integration Server ■ Executes testing jobs, records executions results ■ Jobs performed on slaves ■ Jobs created and managed by Jenkins Job Builder ■ Slaves provisioned by Nodepool 9
  10. 10. Jenkins Job Builder ■ Configures Jenkins jobs using YAML files stored in git ■ Defaults, macros and job configs - job: name: example-docs node: node-label triggers: - zuul builders: - git-prep - docs publishers: - scp: site: 'scp-server' files: - target: 'dir/ectory' source: 'build/html/foo' keep-hierarchy: true - console-log 10
  11. 11. Nodepool - the provisioner ■ Manages a pool of Devstack images to use in project testing. ■ Provisions nodes on HP and Rackspace clouds. ■ Makes sure there are always available nodes for each provider type. ■ Tears down old instances when tests are completed. 11
  12. 12. DevStack Gate ■ The integration test that verifies that all projects still work together after the changes ■ Provides a full OpenStack installation on a single machine with Devstack ■ Installs Devstack and runs Tempest 12
  13. 13. Logstash ■ All jenkins job logs are indexed and stored by Logstash ■ Logs pushing also handled by Gearman jobs 13 ■ Elastic Search helps analyze the test results
  14. 14. Wrap Up ■ Git ■ Gerrit ■ Zuul ■ Gearman ■ Jenkins Job Builder ■ Jenkins ■ Devstack ■ Logstash 14 Only 9 out of 32 projects/tools managed by the Openstack Infra Team

×