Presentation and demo on continuous deployment with Jenkins and Saltstack presented at LinuxCon CloudOpen ContainerCon NA 2015

1. Continuous deployment with Jenkins and Salt
LinuxCon CloudOpen ContainerCon
North America 2015, Seattle
Anirban Saha

2. About me
Techie…
Traveller…
Thinker…
Author of ‘Salt Cookbook’

3. Deployment
Should be…
• Simple
• Stable
• Fast (good to have…)
• Reliable
• One click

4. Problems faced
• Code distribution
• Additional tasks (commands, migrations)
• Latency
• Parallel execution
• Batch deployments
• Ensuring uptime

5. Methods available
Deployment
Configuration management tools :
• Puppet, Chef, Ansible, Salt
Remote execution tools:
• SSH, Parallel SSH, Rundeck
Packages:
• Code packaged as RPM or DEB files

6. Code storage
• GIT server (Github, Gitlab, Bitbucket, etc.)
• Object storage (Amazon S3, Swift, etc.)
• Package repositories (YUM/Debian)
• Configuration management repositories
• File servers managed manually

7. Problems
Deployment:
• Remote execution tools :
• Time consuming (over SSH)
• Need to maintain inventory of hosts
Code storage:
• GIT repositories:
• Usually centrally located, latency when deployed from GIT
• Package repositories (RPM/DEB):
• Problems in simultaneously updating repo metadata
Rollback:
• Version manipulation is a problem with most configuration
management and remote execution tools

8. Salt
Provides…
• Agent based communication
• Fast execution
• Orchestration
• Batch execution
• Strong Integration with providers and
services

9. Salt features
Targeting deployment nodes :
• By hostname
• ‘myappweb*’
• By specific grains
• -G ‘node_type:webserver’
• By nodegroups
• -N webgroup

10. Salt features
Task dependencies :
• Salt requisites to make tasks dependent on each other
get_archive:
module.run:
.
.
deploy_app:
module.run:
.
.
- require:
- module: get_archive

11. Salt features
Orchestration :
webgrp2_deploy:
salt.state:
- tgt: ' webgrp2'
- tgt_type: nodegroup
- sls:
- webserver.deploy
- require:
- salt: webgrp1_deploy
webgrp1_deploy:
salt.state:
- tgt: 'webgrp1'
- tgt_type: nodegroup
- sls:
- webserver.deploy

12. Salt features
Batch Execution :
• By number of hosts
• --batch-size 5
• Executes on 5 hosts at a time
• By percentage of hosts
• --batch-size 25%
• Executes on 25% of total target
hosts at a time

13. Salt features
Salt API:
# curl -H "Accept: application/json"
-d secretkey= "mysupersecretkey"
-k https://10.0.0.2:8080/hook/deploy
{"success": true}

14. Tools
To be used…
• GIT for code repository
• Jenkins for CI
• Ant for build and tasks
• Amazon S3 for code archive storage
• Salt for deployment
• Shell scripts (good old bash) for
additional tasks

15. Salt deployment methods
• Salt hosted on the Jenkins server.
Jenkins calls Salt binary
• Salt hosted independently.
Jenkins calls Salt via SSH
• Jenkins calls Salt via Salt API

16. Objective

17. Steps
• salt-cloud used to launch instances
• Post install actions used to synchronize EC2 grains
• Salt reactor used to run states on new instances ,
perform deployment and register with load balancer
• Jenkins build job used to build new code, create tags
and push new deployment ready archive to Amazon
S3
• Jenkins deploy job used to fetch new code version
and deploy code on target servers

18. Steps explained
• Salt cloud is used to spawn instances
• Following parameter is provided in the profile to push custom EC2 grains,
• sync_after_install: grains
• The node is registered and deregistered from the load balancer with the
following module definition,
register:
module.run:
- name: boto_elb.register_instances
- m_name: mywebapp
- instances:
- {{ grains['ec2']['instance_id'] }}
• Here ['ec2']['instance_id'] is one of the custom grains pushed to the node

19. Steps explained
Fetching archive:
Salt s3 execution module used to get code archive from
Amazon S3
fetch_app_archive:
module.run:
- name: s3.get
- bucket: mywebapp-us
- path: mywebapp-{{ app_version }}.tar
- local_file: /tmp/mywebapp-{{ app_version }}.tar

20. Steps explained
Setting App version grain:
After every deploy, a grain is set to record the
app version deployed for tracking,
app_version:
grains.present:
- value: {{ app_version }}
- require:
- cmd: deploy_app

21. Steps explained
Deployment based on App version:
At every deploy, it is checked if the version to
be deployed is already on the node using the
app_version grain,
{% if grains['app_version'] != app_version %}
deregister:
fetch_app_archive:
deploy_app:
register:
{% endif %}

22. Lets do it then !!!

23. Demo repository
Get the demo repository at the
following location,
https://github.com/rosesnthornz/
cloudopen-na-2015

24. Questions ?

25. Contact
Email :
sahaanirban1988@gmail.com
Twitter : @rosesnthornz

26. Thank You !!!