SlideShare a Scribd company logo
Jenkins, jclouds, and CloudStack
David Nalley @ke4qqq ke4qqq@apache.org
#whoami
• Recovering sysadmin
• Former server hugger, now cloud
addict
• Contributor to a few open source
projects:
– Fedora, Zenoss, CloudStack, jclouds
• VP, Infrastructure for the Apache
Software Foundation
• Employed by Citrix in the Open
Source Business Office
Jenkins + jclouds + Centos + CloudStack
Background: Jenkins
‘Continuous Integration Server’
Formerly Hudson – community forked
Written in Java
MIT-Licensed
Lots of plugins
Jenkins + jclouds + Centos + CloudStack
Background: Apache jclouds
Cloud Abstraction Library
Written in Java
ASLv2 License
Jenkins + jclouds + Centos + CloudStack
Background: Apache CloudStack
Formerly from Cloud.com
Compute- focused IaaS management solution.
Started in 2008
Now a top level project at the ASF
Written in Java
ASLv2
Jenkins + jclouds + Centos + CloudStack
The Problem
Lots of different testing needs – multiple
platforms, multiple versions, multiple
versions of software.
Jenkins + jclouds + Centos + CloudStack
The Problem – at scale
We have nearly 200 software projects each with
their own set of needs, distinct platforms.
Some want CentOS 5, 6, or 7; others want OSX, Win
2012, Ubuntu, FreeBSD, or Solaris.
Demand varies widely – projects getting closer to
release test more; some days we had hundreds of
jobs in the queue, other days we had 20 or 30
Jenkins + jclouds + Centos + CloudStack
The transition for monolith to scalable.
Jenkins + jclouds + Centos + CloudStack
Where we were
Dedicated Jenkins master
Esoteric things had VMs
~30 dedicated machines, strewn across
multiple datacenters.
Jenkins + jclouds + Centos + CloudStack
How we started our move
• Defining what a build slave needs
• Make sure that Jenkins is working
efficiently
(Misconfigured jobs are a huge drain)
• Adopted a LTS version of Jenkins
Jenkins + jclouds + Centos + CloudStack
Building images
Packer – building images
Start with JeOS – declare everything else in
puppet, run puppet as a job.
Create a jenkins job for this – everytime a
job changes – rebuild, reupload.
Jenkins + jclouds + Centos + CloudStack
How you plan to use Cloud
Server huggers wanted to use fixed VMs
(deploy fixed machines)
Jenkins + jclouds + Centos + CloudStack
How you want to use cloud
Fresh, dynamically provisioned VM for each
build
Jenkins + jclouds + Centos + CloudStack
The right answer
Jenkins + jclouds + Centos + CloudStack
The right answer
(for us)
Spin up build slaves in response to demand.
Leave the machine up for 30 minutes after
demand subsides to keep from flapping.
Jenkins + jclouds + Centos + CloudStack
Make it all work
Jenkins + jclouds + Centos + CloudStack
Install the jclouds plugin
Jenkins + jclouds + Centos + CloudStack
Choose a cloud provider
Jenkins + jclouds + Centos + CloudStack
Configure
Jenkins + jclouds + Centos + CloudStack
Configure
Jenkins + jclouds + Centos + CloudStack
Profit?
Jenkins + jclouds + Centos + CloudStack
Bonus
Jenkins + jclouds + Centos + CloudStack
Get Involved
Web: http://cloudstack.apache.org/
Mailing Lists: cloudstack.apache.org/mailing-lists.html
IRC: irc.freenode.net: 6667 #cloudstack
Twitter: @cloudstack
LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859
If it didn’t happen on the mailing list, it didn’t happen.
Jenkins + jclouds + Centos + CloudStack

More Related Content

What's hot

Running OpenShift Clusters in a Cloudstack Environment
Running OpenShift Clusters in a Cloudstack EnvironmentRunning OpenShift Clusters in a Cloudstack Environment
Running OpenShift Clusters in a Cloudstack Environment
ShapeBlue
 
Becoming the master of disaster... with asr
Becoming the master of disaster... with asrBecoming the master of disaster... with asr
Becoming the master of disaster... with asr
nj-azure
 
Datacenter Computing with Apache Mesos - シリコンバレー日本人駐在員Meetup
Datacenter Computing with Apache Mesos - シリコンバレー日本人駐在員MeetupDatacenter Computing with Apache Mesos - シリコンバレー日本人駐在員Meetup
Datacenter Computing with Apache Mesos - シリコンバレー日本人駐在員Meetup
Paco Nathan
 
Thinking in Terms of a Data Center Operating System
Thinking in Terms of a Data Center Operating SystemThinking in Terms of a Data Center Operating System
Thinking in Terms of a Data Center Operating SystemWeston Bassler
 
Sven Vogel: Running CloudStack and OpenShift with NetApp on KVM
Sven Vogel: Running CloudStack and OpenShift with NetApp on KVMSven Vogel: Running CloudStack and OpenShift with NetApp on KVM
Sven Vogel: Running CloudStack and OpenShift with NetApp on KVM
ShapeBlue
 
Running OpenStack in Production
Running OpenStack in Production Running OpenStack in Production
Running OpenStack in Production
Nati Shalom
 
Apache Cassandra Management
Apache Cassandra ManagementApache Cassandra Management
Apache Cassandra Management
Instaclustr
 
Cloudstack: the best kept secret in the cloud
Cloudstack: the best kept secret in the cloudCloudstack: the best kept secret in the cloud
Cloudstack: the best kept secret in the cloud
ShapeBlue
 
Wido den Hollander - building highly available cloud with Ceph and CloudStack
Wido den Hollander - building highly available cloud with Ceph and CloudStackWido den Hollander - building highly available cloud with Ceph and CloudStack
Wido den Hollander - building highly available cloud with Ceph and CloudStack
ShapeBlue
 
Boyan Krosnov - Building a software-defined cloud - our experience
Boyan Krosnov - Building a software-defined cloud - our experienceBoyan Krosnov - Building a software-defined cloud - our experience
Boyan Krosnov - Building a software-defined cloud - our experience
ShapeBlue
 
Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...
Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...
Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...
Cloud Native Day Tel Aviv
 
Wido den hollander cloud stack and ceph
Wido den hollander   cloud stack and cephWido den hollander   cloud stack and ceph
Wido den hollander cloud stack and ceph
ShapeBlue
 
Stratoscale Latest and Greatest
Stratoscale Latest and GreatestStratoscale Latest and Greatest
Stratoscale Latest and GreatestZach Lanksbury
 
Paul Angus – Backup & Recovery in CloudStack
Paul Angus – Backup & Recovery in CloudStackPaul Angus – Backup & Recovery in CloudStack
Paul Angus – Backup & Recovery in CloudStack
ShapeBlue
 
vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28
vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28
vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28
CloudStack - Open Source Cloud Computing Project
 
Giles Sirett: Introduction and CloudStack news
Giles Sirett: Introduction and CloudStack news   Giles Sirett: Introduction and CloudStack news
Giles Sirett: Introduction and CloudStack news
ShapeBlue
 
XenServer HA Improvements
XenServer HA ImprovementsXenServer HA Improvements
XenServer HA Improvements
ShapeBlue
 
Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...
Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...
Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...
DataStax
 
CloudStack news
CloudStack newsCloudStack news
CloudStack news
ShapeBlue
 
PaaS on top of CloudStack
PaaS on top of CloudStackPaaS on top of CloudStack
PaaS on top of CloudStack
buildacloud
 

What's hot (20)

Running OpenShift Clusters in a Cloudstack Environment
Running OpenShift Clusters in a Cloudstack EnvironmentRunning OpenShift Clusters in a Cloudstack Environment
Running OpenShift Clusters in a Cloudstack Environment
 
Becoming the master of disaster... with asr
Becoming the master of disaster... with asrBecoming the master of disaster... with asr
Becoming the master of disaster... with asr
 
Datacenter Computing with Apache Mesos - シリコンバレー日本人駐在員Meetup
Datacenter Computing with Apache Mesos - シリコンバレー日本人駐在員MeetupDatacenter Computing with Apache Mesos - シリコンバレー日本人駐在員Meetup
Datacenter Computing with Apache Mesos - シリコンバレー日本人駐在員Meetup
 
Thinking in Terms of a Data Center Operating System
Thinking in Terms of a Data Center Operating SystemThinking in Terms of a Data Center Operating System
Thinking in Terms of a Data Center Operating System
 
Sven Vogel: Running CloudStack and OpenShift with NetApp on KVM
Sven Vogel: Running CloudStack and OpenShift with NetApp on KVMSven Vogel: Running CloudStack and OpenShift with NetApp on KVM
Sven Vogel: Running CloudStack and OpenShift with NetApp on KVM
 
Running OpenStack in Production
Running OpenStack in Production Running OpenStack in Production
Running OpenStack in Production
 
Apache Cassandra Management
Apache Cassandra ManagementApache Cassandra Management
Apache Cassandra Management
 
Cloudstack: the best kept secret in the cloud
Cloudstack: the best kept secret in the cloudCloudstack: the best kept secret in the cloud
Cloudstack: the best kept secret in the cloud
 
Wido den Hollander - building highly available cloud with Ceph and CloudStack
Wido den Hollander - building highly available cloud with Ceph and CloudStackWido den Hollander - building highly available cloud with Ceph and CloudStack
Wido den Hollander - building highly available cloud with Ceph and CloudStack
 
Boyan Krosnov - Building a software-defined cloud - our experience
Boyan Krosnov - Building a software-defined cloud - our experienceBoyan Krosnov - Building a software-defined cloud - our experience
Boyan Krosnov - Building a software-defined cloud - our experience
 
Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...
Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...
Avishay Traeger & Shimshon Zimmerman, Stratoscale - Deploying OpenStack Cinde...
 
Wido den hollander cloud stack and ceph
Wido den hollander   cloud stack and cephWido den hollander   cloud stack and ceph
Wido den hollander cloud stack and ceph
 
Stratoscale Latest and Greatest
Stratoscale Latest and GreatestStratoscale Latest and Greatest
Stratoscale Latest and Greatest
 
Paul Angus – Backup & Recovery in CloudStack
Paul Angus – Backup & Recovery in CloudStackPaul Angus – Backup & Recovery in CloudStack
Paul Angus – Backup & Recovery in CloudStack
 
vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28
vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28
vBACD - Deploying Infrastructure-as-a-Service with CloudStack - 2/28
 
Giles Sirett: Introduction and CloudStack news
Giles Sirett: Introduction and CloudStack news   Giles Sirett: Introduction and CloudStack news
Giles Sirett: Introduction and CloudStack news
 
XenServer HA Improvements
XenServer HA ImprovementsXenServer HA Improvements
XenServer HA Improvements
 
Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...
Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...
Leveraging Docker and CoreOS to provide always available Cassandra at Instacl...
 
CloudStack news
CloudStack newsCloudStack news
CloudStack news
 
PaaS on top of CloudStack
PaaS on top of CloudStackPaaS on top of CloudStack
PaaS on top of CloudStack
 

Similar to On-demand Continuous Integration with Jenkins, jclouds, and CloudStack

Jenkins, jclouds, CloudStack, and CentOS by David Nalley
Jenkins, jclouds, CloudStack, and CentOS by David NalleyJenkins, jclouds, CloudStack, and CentOS by David Nalley
Jenkins, jclouds, CloudStack, and CentOS by David Nalley
buildacloud
 
Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)
Michael Neale
 
Introduction to Jenkins X - a beginner's guide
Introduction to Jenkins X - a beginner's guideIntroduction to Jenkins X - a beginner's guide
Introduction to Jenkins X - a beginner's guide
Andrew Bayer
 
Continuous Delivery at Wix
Continuous Delivery at WixContinuous Delivery at Wix
Continuous Delivery at Wix
Yoav Avrahami
 
Velocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ NetflixVelocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ Netflix
aspyker
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
Giacomo Vacca
 
Abul-Devops-CloudArch.
Abul-Devops-CloudArch.Abul-Devops-CloudArch.
Abul-Devops-CloudArch.Abulfazl Khan
 
Intro to Node.js (for .NET Developers)
Intro to Node.js (for .NET Developers)Intro to Node.js (for .NET Developers)
Intro to Node.js (for .NET Developers)
David Neal
 
Java Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the CloudJava Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the Cloud
MongoDB
 
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...
Docker, Inc.
 
Docker in Production: How RightScale Delivers Cloud Applications
Docker in Production: How RightScale Delivers Cloud ApplicationsDocker in Production: How RightScale Delivers Cloud Applications
Docker in Production: How RightScale Delivers Cloud Applications
RightScale
 
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
Mark Voelker
 
ContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small businessContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small business
Docker-Hanoi
 
Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in ComparisonJava in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison
adesso AG
 
Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison
Eberhard Wolff
 
Continuous Delivery to Kubernetes with Jenkins and Helm
Continuous Delivery to Kubernetes with Jenkins and HelmContinuous Delivery to Kubernetes with Jenkins and Helm
Continuous Delivery to Kubernetes with Jenkins and Helm
David Currie
 
Demystifying Containerization Principles for Data Scientists
Demystifying Containerization Principles for Data ScientistsDemystifying Containerization Principles for Data Scientists
Demystifying Containerization Principles for Data Scientists
Dr Ganesh Iyer
 
Creating Resilient, Dockerbased Applications using Lattice
Creating Resilient, Dockerbased Applications using LatticeCreating Resilient, Dockerbased Applications using Lattice
Creating Resilient, Dockerbased Applications using Lattice
Dustin Ruehle
 
SQL Server in DevOps Town Hall Webinar
SQL Server in DevOps Town Hall WebinarSQL Server in DevOps Town Hall Webinar
SQL Server in DevOps Town Hall Webinar
Travis Wright
 
Developing in the Cloud
Developing in the CloudDeveloping in the Cloud
Developing in the Cloud
Ryan Cuprak
 

Similar to On-demand Continuous Integration with Jenkins, jclouds, and CloudStack (20)

Jenkins, jclouds, CloudStack, and CentOS by David Nalley
Jenkins, jclouds, CloudStack, and CentOS by David NalleyJenkins, jclouds, CloudStack, and CentOS by David Nalley
Jenkins, jclouds, CloudStack, and CentOS by David Nalley
 
Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)
 
Introduction to Jenkins X - a beginner's guide
Introduction to Jenkins X - a beginner's guideIntroduction to Jenkins X - a beginner's guide
Introduction to Jenkins X - a beginner's guide
 
Continuous Delivery at Wix
Continuous Delivery at WixContinuous Delivery at Wix
Continuous Delivery at Wix
 
Velocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ NetflixVelocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ Netflix
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
 
Abul-Devops-CloudArch.
Abul-Devops-CloudArch.Abul-Devops-CloudArch.
Abul-Devops-CloudArch.
 
Intro to Node.js (for .NET Developers)
Intro to Node.js (for .NET Developers)Intro to Node.js (for .NET Developers)
Intro to Node.js (for .NET Developers)
 
Java Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the CloudJava Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the Cloud
 
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...
 
Docker in Production: How RightScale Delivers Cloud Applications
Docker in Production: How RightScale Delivers Cloud ApplicationsDocker in Production: How RightScale Delivers Cloud Applications
Docker in Production: How RightScale Delivers Cloud Applications
 
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
 
ContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small businessContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small business
 
Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in ComparisonJava in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison
 
Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison
 
Continuous Delivery to Kubernetes with Jenkins and Helm
Continuous Delivery to Kubernetes with Jenkins and HelmContinuous Delivery to Kubernetes with Jenkins and Helm
Continuous Delivery to Kubernetes with Jenkins and Helm
 
Demystifying Containerization Principles for Data Scientists
Demystifying Containerization Principles for Data ScientistsDemystifying Containerization Principles for Data Scientists
Demystifying Containerization Principles for Data Scientists
 
Creating Resilient, Dockerbased Applications using Lattice
Creating Resilient, Dockerbased Applications using LatticeCreating Resilient, Dockerbased Applications using Lattice
Creating Resilient, Dockerbased Applications using Lattice
 
SQL Server in DevOps Town Hall Webinar
SQL Server in DevOps Town Hall WebinarSQL Server in DevOps Town Hall Webinar
SQL Server in DevOps Town Hall Webinar
 
Developing in the Cloud
Developing in the CloudDeveloping in the Cloud
Developing in the Cloud
 

More from ke4qqq

The Tragedy of Open Source
The Tragedy of Open SourceThe Tragedy of Open Source
The Tragedy of Open Source
ke4qqq
 
Cloud Innovation and Risks
Cloud Innovation and RisksCloud Innovation and Risks
Cloud Innovation and Risks
ke4qqq
 
Understanding the CloudStack Release Process
Understanding the CloudStack Release ProcessUnderstanding the CloudStack Release Process
Understanding the CloudStack Release Processke4qqq
 
ApacheConEU Keynote: What is the value of the Apache Software Foundation
ApacheConEU Keynote: What is the value of the Apache Software FoundationApacheConEU Keynote: What is the value of the Apache Software Foundation
ApacheConEU Keynote: What is the value of the Apache Software Foundation
ke4qqq
 
Ceph and Apache CloudStack
Ceph and Apache CloudStackCeph and Apache CloudStack
Ceph and Apache CloudStack
ke4qqq
 
It's not tools, Stupid
It's not tools, StupidIt's not tools, Stupid
It's not tools, Stupid
ke4qqq
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStack
ke4qqq
 
Building a Dev/Test Cloud with Apache CloudStack
Building a Dev/Test Cloud with Apache CloudStackBuilding a Dev/Test Cloud with Apache CloudStack
Building a Dev/Test Cloud with Apache CloudStack
ke4qqq
 
10 Minute Overview of Apache CloudStack
10 Minute Overview of Apache CloudStack10 Minute Overview of Apache CloudStack
10 Minute Overview of Apache CloudStack
ke4qqq
 
CloudStack Intro NYC
CloudStack Intro NYCCloudStack Intro NYC
CloudStack Intro NYC
ke4qqq
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackke4qqq
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStackke4qqq
 
CloudStack - LinuxFest NorthWest
CloudStack - LinuxFest NorthWestCloudStack - LinuxFest NorthWest
CloudStack - LinuxFest NorthWestke4qqq
 
Build a Cloud Day - CloudStack
Build a Cloud Day - CloudStackBuild a Cloud Day - CloudStack
Build a Cloud Day - CloudStackke4qqq
 
Build a Cloud Day - CloudStack
Build a Cloud Day - CloudStackBuild a Cloud Day - CloudStack
Build a Cloud Day - CloudStack
ke4qqq
 
Successfully deploy build manage your cloud with cloud stack2
Successfully deploy build manage your cloud with cloud stack2Successfully deploy build manage your cloud with cloud stack2
Successfully deploy build manage your cloud with cloud stack2
ke4qqq
 

More from ke4qqq (16)

The Tragedy of Open Source
The Tragedy of Open SourceThe Tragedy of Open Source
The Tragedy of Open Source
 
Cloud Innovation and Risks
Cloud Innovation and RisksCloud Innovation and Risks
Cloud Innovation and Risks
 
Understanding the CloudStack Release Process
Understanding the CloudStack Release ProcessUnderstanding the CloudStack Release Process
Understanding the CloudStack Release Process
 
ApacheConEU Keynote: What is the value of the Apache Software Foundation
ApacheConEU Keynote: What is the value of the Apache Software FoundationApacheConEU Keynote: What is the value of the Apache Software Foundation
ApacheConEU Keynote: What is the value of the Apache Software Foundation
 
Ceph and Apache CloudStack
Ceph and Apache CloudStackCeph and Apache CloudStack
Ceph and Apache CloudStack
 
It's not tools, Stupid
It's not tools, StupidIt's not tools, Stupid
It's not tools, Stupid
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStack
 
Building a Dev/Test Cloud with Apache CloudStack
Building a Dev/Test Cloud with Apache CloudStackBuilding a Dev/Test Cloud with Apache CloudStack
Building a Dev/Test Cloud with Apache CloudStack
 
10 Minute Overview of Apache CloudStack
10 Minute Overview of Apache CloudStack10 Minute Overview of Apache CloudStack
10 Minute Overview of Apache CloudStack
 
CloudStack Intro NYC
CloudStack Intro NYCCloudStack Intro NYC
CloudStack Intro NYC
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStack
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStack
 
CloudStack - LinuxFest NorthWest
CloudStack - LinuxFest NorthWestCloudStack - LinuxFest NorthWest
CloudStack - LinuxFest NorthWest
 
Build a Cloud Day - CloudStack
Build a Cloud Day - CloudStackBuild a Cloud Day - CloudStack
Build a Cloud Day - CloudStack
 
Build a Cloud Day - CloudStack
Build a Cloud Day - CloudStackBuild a Cloud Day - CloudStack
Build a Cloud Day - CloudStack
 
Successfully deploy build manage your cloud with cloud stack2
Successfully deploy build manage your cloud with cloud stack2Successfully deploy build manage your cloud with cloud stack2
Successfully deploy build manage your cloud with cloud stack2
 

Recently uploaded

Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 

Recently uploaded (20)

Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 

On-demand Continuous Integration with Jenkins, jclouds, and CloudStack

  • 1. Jenkins, jclouds, and CloudStack David Nalley @ke4qqq ke4qqq@apache.org
  • 2. #whoami • Recovering sysadmin • Former server hugger, now cloud addict • Contributor to a few open source projects: – Fedora, Zenoss, CloudStack, jclouds • VP, Infrastructure for the Apache Software Foundation • Employed by Citrix in the Open Source Business Office Jenkins + jclouds + Centos + CloudStack
  • 3. Background: Jenkins ‘Continuous Integration Server’ Formerly Hudson – community forked Written in Java MIT-Licensed Lots of plugins Jenkins + jclouds + Centos + CloudStack
  • 4. Background: Apache jclouds Cloud Abstraction Library Written in Java ASLv2 License Jenkins + jclouds + Centos + CloudStack
  • 5. Background: Apache CloudStack Formerly from Cloud.com Compute- focused IaaS management solution. Started in 2008 Now a top level project at the ASF Written in Java ASLv2 Jenkins + jclouds + Centos + CloudStack
  • 6. The Problem Lots of different testing needs – multiple platforms, multiple versions, multiple versions of software. Jenkins + jclouds + Centos + CloudStack
  • 7. The Problem – at scale We have nearly 200 software projects each with their own set of needs, distinct platforms. Some want CentOS 5, 6, or 7; others want OSX, Win 2012, Ubuntu, FreeBSD, or Solaris. Demand varies widely – projects getting closer to release test more; some days we had hundreds of jobs in the queue, other days we had 20 or 30 Jenkins + jclouds + Centos + CloudStack
  • 8. The transition for monolith to scalable. Jenkins + jclouds + Centos + CloudStack
  • 9. Where we were Dedicated Jenkins master Esoteric things had VMs ~30 dedicated machines, strewn across multiple datacenters. Jenkins + jclouds + Centos + CloudStack
  • 10. How we started our move • Defining what a build slave needs • Make sure that Jenkins is working efficiently (Misconfigured jobs are a huge drain) • Adopted a LTS version of Jenkins Jenkins + jclouds + Centos + CloudStack
  • 11. Building images Packer – building images Start with JeOS – declare everything else in puppet, run puppet as a job. Create a jenkins job for this – everytime a job changes – rebuild, reupload. Jenkins + jclouds + Centos + CloudStack
  • 12. How you plan to use Cloud Server huggers wanted to use fixed VMs (deploy fixed machines) Jenkins + jclouds + Centos + CloudStack
  • 13. How you want to use cloud Fresh, dynamically provisioned VM for each build Jenkins + jclouds + Centos + CloudStack
  • 14. The right answer Jenkins + jclouds + Centos + CloudStack
  • 15. The right answer (for us) Spin up build slaves in response to demand. Leave the machine up for 30 minutes after demand subsides to keep from flapping. Jenkins + jclouds + Centos + CloudStack
  • 16. Make it all work Jenkins + jclouds + Centos + CloudStack
  • 17. Install the jclouds plugin Jenkins + jclouds + Centos + CloudStack
  • 18. Choose a cloud provider Jenkins + jclouds + Centos + CloudStack
  • 19. Configure Jenkins + jclouds + Centos + CloudStack
  • 20. Configure Jenkins + jclouds + Centos + CloudStack
  • 21. Profit? Jenkins + jclouds + Centos + CloudStack
  • 22. Bonus Jenkins + jclouds + Centos + CloudStack
  • 23. Get Involved Web: http://cloudstack.apache.org/ Mailing Lists: cloudstack.apache.org/mailing-lists.html IRC: irc.freenode.net: 6667 #cloudstack Twitter: @cloudstack LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859 If it didn’t happen on the mailing list, it didn’t happen. Jenkins + jclouds + Centos + CloudStack

Editor's Notes

  1. Do you know what continuous integration is? CI is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day. So Jenkins is merely a testing tool for use in CI. CloudStack maintains its own CI server The ASF has a top-20 in the world, Jenkins instance.
  2. Originally written by Adrian Cole – now a top level project at the ASF Java cloud bindings regardless of what your underlying cloud actually is
  3. Do you know what continuous integration is? CI is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day. So Jenkins is merely a testing tool for use in CI.
  4. And that’s just CloudStack
  5. And that’s just CloudStack
  6. We started this with Ansible – and for many of our build slaves still is the default. The rest of our infrastructure is Puppet – so much of this is moving. Developers – committers on Jenkins as a matter of fact – had us running bleeding edge, unreleased, with custom patches.
  7. The first struggle I faced was explaining how IP addresses were dynamic, and that they couldn’t really reserve a block of them.
  8. A perfectly clean workspace – nothing muddled – and perfect isolation – only a single job running at a time. Downside is that spin up time can be excessive. Depending on the cloud provider you might have several minutes waiting – for the ASF – we discovered that using our own custom images made spin up much slower – a problem our public cloud vendor since solved.
  9. Don’t do what we did and install a personal snapshot. That said – a number of folks have been discussing creating provider-specific plugins.