Your SlideShare is downloading. ×
0
Classification 2013/10/3
1
Classification 2013/10/32
What is a system admin?
Don‟t look at me...
I wasn‟t the last one to touch it...
Classification 2013/10/35
Everything the Same
Everything Distinct
Manually
yum install nginx
vi /etc/nginx/conf.d/test.conf
service nginx start
Shell Script
yum install nginx
mkdir -p /etc/nginx/conf.d
cat > /etc/nginx/conf.d/test.conf<<EOF
server {
listen 443;
ssl ...
One Goal:
Revolutionize
System
Administration
Fabric
 command-line tool for streamlining the use of SSH for
application deployment or systems administration tasks
 Ma...
Installation
$ pip install fabric
$ pip install jinja2
$ sudo apt-get install fabric
fabfile.py
@task
def install_package():
run("yum install nginx")
fabfile.py
@task
def update_conf():
if exists("/etc/nginx/conf.d"):
run("mkdir -p /etc/nginx/conf.d")
put(”test.conf", "/e...
fabfile.py
@task
def start_daemon():
run("service nginx start")
fabfile.py
@task
def deploy():
execute(install_package)
execute(update_conf)
execute(start_daemon)
Task Arguments
from fabric.api import task
@task
def hello(name="world"):
print("Hello %s!" % name)
Task Arguments
$ fab hello:name=Alex
Hello Alex!
Done.
$ fab hello:Alex
Hello Alex!
Done.
Template
def update_conf():
context = {
'http_port' : 80,
'https_port' : 443
}
src_path = 'test.conf'
dest_path = '/etc/ng...
Template File
server {
listen %(http_port)d;
}
server {
listen %(https_port)d;
}
Template with Jinja2
def update_conf():
context = {
„ports' : [80, 443]
}
src_path = 'test.conf'
dest_path = '/etc/nginx/c...
Template File with Jinja2
{%- for port in ports %}
server {
listen {{ port }};
}
{%- endfor %}
Execute Model
from fabric.api import run, env
env.hosts = ['host1', 'host2']
@task
def taskA():
run('ls')
@task
def taskB(...
Execute Model
$ fab -l
Available commands:
taskA
taskB
Execute Model
$ fab taskA taskB
taskA executed on host1
taskA executed on host2
taskB executed on host1
taskB executed on ...
Execute Model by Role
from fabric.api import run, env
env.roledefs = {
'web': ['www1', 'www2', 'www3'],
'dns': ['ns1', 'ns...
Execute Model by Role
$ fab -R dns taskA taskB
taskA executed on ns1
taskA executed on ns2
taskB executed on ns1
taskB exe...
Execute Model by Hosts
$ fab -H ns1,www1 taskA taskB
taskA executed on ns1
taskA executed on www1
taskB executed on ns1
ta...
Arbitrary remote commands
$ fab -H ns1,www1 -- whoami
task executed on ns1
task executed on www1
Cuisine
 https://github.com/sebastien/cuisine
 Chef-like functionality for Fabric
 Covers file/dir operations, user/gro...
Cuisine
 text_* : Text-processing functions
 file_* : File operations
 dir_* : Directory operations
 package_* : Packa...
Classification 2013/10/330
Live Demo
Drawbacks
 Not easy to implement by pure operators
 Leak high-level function support
 User, file, package, service mana...
Puppet
 Provides a Domain Specific Language (DSL) to script with
 Classes, conditionals, selectors, variables, basic math...
Classification 2013/10/333
apt-get install nginx
vi /etc/nginx/conf.d/test.conf
service nginx start
Debian
yum install ngi...
An Analogy
Programming SysAdmin
Low-level,
non-portable
Assembly commands
and files
Abstract,
portable
Java / Python / Ruby...
A Partial List of Puppet
typesPackages • Supports 30 different package providers
• Abstracted for your OS automatically
• ...
Dashboard
apt-get install nginx
vi /etc/nginx/conf.d/test.conf
service nginx start
Package
Configuration
Service
Configuration should...
Sample classes
class nginx::server {
$conf_dir = "/etc/nginx/conf.d"
$http_port = 80
$https_port = 443
package {"nginx":
e...
Template
 Puppet templates are flat files containing Embedded Ruby
(ERB) variables
server {
listen <%= @http_port %>;
}
s...
Node
Node definitions look just like classes, including supporting
inheritance, but they are special in that when a node (...
Modules
A module is just a directory with stuff in it, and the
magic comes from putting that stuff where Puppet
expects to...
Module Structure
Network Overview
 Configuration allows for manual synchronizations or a set increment
 Client or server initiated synchro...
Every Client
 Retrieve resource catalog from central server
 Determine resource order
 Check each resource in turn, fixi...
Every Resource
 Retrieve current state (e.g., by querying dpkg db or doing a stat)
 Compare to desired state
 Fix, if n...
Drawbacks
 Hard to prepare the environment
 Install Ruby, puppet packages
 Set up host name, domain name
 Put ssh publ...
Combine Fabric and Puppet
 Fabric
 When
 Operators trigger puppet to deploy packages one by one or parallelly
 Rolling...
Initial functions
 Create EC2 instances (optional)
 Setup SSH keys to all remote hosts
 Configure yum repositories
 In...
Global setup functions
 Mandatory
 Backup
 Clean yum cache
 Sync fabric configurations to puppet pp files
 Restart pu...
Global teardown functions
 Start/stop services across hosts
 Send email/SMS notifications to members
 Do health/sanity ...
Classification 2013/10/352
Questions?
One click deployment
Upcoming SlideShare
Loading in...5
×

One click deployment

2,669

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,669
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "One click deployment"

  1. 1. Classification 2013/10/3 1
  2. 2. Classification 2013/10/32
  3. 3. What is a system admin?
  4. 4. Don‟t look at me... I wasn‟t the last one to touch it...
  5. 5. Classification 2013/10/35 Everything the Same Everything Distinct
  6. 6. Manually yum install nginx vi /etc/nginx/conf.d/test.conf service nginx start
  7. 7. Shell Script yum install nginx mkdir -p /etc/nginx/conf.d cat > /etc/nginx/conf.d/test.conf<<EOF server { listen 443; ssl on; } EOF service nginx start install-nginx.sh scp install-nginx.sh root@server:~/ ssh -o PasswordAuthentication=no -q -t -t “~/install-nginx.sh”
  8. 8. One Goal: Revolutionize System Administration
  9. 9. Fabric  command-line tool for streamlining the use of SSH for application deployment or systems administration tasks  Make executing shell commands over SSH easy and Pythonic  Stop administrating your environment and start developing it...  Re-usable code for managing your software & configurations
  10. 10. Installation $ pip install fabric $ pip install jinja2 $ sudo apt-get install fabric
  11. 11. fabfile.py @task def install_package(): run("yum install nginx")
  12. 12. fabfile.py @task def update_conf(): if exists("/etc/nginx/conf.d"): run("mkdir -p /etc/nginx/conf.d") put(”test.conf", "/etc/nginx/conf.d/test.conf")
  13. 13. fabfile.py @task def start_daemon(): run("service nginx start")
  14. 14. fabfile.py @task def deploy(): execute(install_package) execute(update_conf) execute(start_daemon)
  15. 15. Task Arguments from fabric.api import task @task def hello(name="world"): print("Hello %s!" % name)
  16. 16. Task Arguments $ fab hello:name=Alex Hello Alex! Done. $ fab hello:Alex Hello Alex! Done.
  17. 17. Template def update_conf(): context = { 'http_port' : 80, 'https_port' : 443 } src_path = 'test.conf' dest_path = '/etc/nginx/conf.d/test.conf' files.upload_template(src_path, dest_path, context = context)
  18. 18. Template File server { listen %(http_port)d; } server { listen %(https_port)d; }
  19. 19. Template with Jinja2 def update_conf(): context = { „ports' : [80, 443] } src_path = 'test.conf' dest_path = '/etc/nginx/conf.d/test.conf' files.upload_template(src_path, dest_path, context = context, use_jinja = True)
  20. 20. Template File with Jinja2 {%- for port in ports %} server { listen {{ port }}; } {%- endfor %}
  21. 21. Execute Model from fabric.api import run, env env.hosts = ['host1', 'host2'] @task def taskA(): run('ls') @task def taskB(): run('whoami')
  22. 22. Execute Model $ fab -l Available commands: taskA taskB
  23. 23. Execute Model $ fab taskA taskB taskA executed on host1 taskA executed on host2 taskB executed on host1 taskB executed on host2
  24. 24. Execute Model by Role from fabric.api import run, env env.roledefs = { 'web': ['www1', 'www2', 'www3'], 'dns': ['ns1', 'ns2'] } def taskA(): run('ls') def taskB(): run('whoami')
  25. 25. Execute Model by Role $ fab -R dns taskA taskB taskA executed on ns1 taskA executed on ns2 taskB executed on ns1 taskB executed on ns2
  26. 26. Execute Model by Hosts $ fab -H ns1,www1 taskA taskB taskA executed on ns1 taskA executed on www1 taskB executed on ns1 taskB executed on www1
  27. 27. Arbitrary remote commands $ fab -H ns1,www1 -- whoami task executed on ns1 task executed on www1
  28. 28. Cuisine  https://github.com/sebastien/cuisine  Chef-like functionality for Fabric  Covers file/dir operations, user/group operations, package operations
  29. 29. Cuisine  text_* : Text-processing functions  file_* : File operations  dir_* : Directory operations  package_* : Package management operations  command_* : Shell commands availability  user_* : User creation commands  group* : Group creation commands  mode_* : Configures cuisine's behaviour within the current session.  select_* : Selects a specific option, such as package back-end (apt, yum, zypper, or pacman)
  30. 30. Classification 2013/10/330 Live Demo
  31. 31. Drawbacks  Not easy to implement by pure operators  Leak high-level function support  User, file, package, service management  Built-in environment variables  Leak smart error handling  Would do all things every time (depends on the implementation)  No log, no history  To many SSH communications (keepalive argument would help)
  32. 32. Puppet  Provides a Domain Specific Language (DSL) to script with  Classes, conditionals, selectors, variables, basic math, etc.  Supports Linux, Solaris, BSD, OS X, Windows  Stop administrating your environment and start developing it...  Re-usable code for managing your software & configurations
  33. 33. Classification 2013/10/333 apt-get install nginx vi /etc/nginx/conf.d/test.conf service nginx start Debian yum install nginx vi /etc/nginx/conf.d/test.conf service nginx start Redhat
  34. 34. An Analogy Programming SysAdmin Low-level, non-portable Assembly commands and files Abstract, portable Java / Python / Ruby Resources
  35. 35. A Partial List of Puppet typesPackages • Supports 30 different package providers • Abstracted for your OS automatically • Specify „installed‟, „absent‟, or „latest‟ for desired state • Change from „installed‟ to „latest‟ and deploy for quick Upgrade Services • Supports 10 different „init‟ frameworks • Control whether a service starts on boot or is required to be running always • A service can be notified to restart if a configuration file has been changed Files/Directories • Specify ownership & permissions • Load content from „files/‟, „templates/‟ or custom strings • Create symlinks • Supports 5 types to verify a file checksum • Purge a directory of files not „maintained‟
  36. 36. Dashboard
  37. 37. apt-get install nginx vi /etc/nginx/conf.d/test.conf service nginx start Package Configuration Service Configuration should get modified after package installation Service should restart when configuration changes
  38. 38. Sample classes class nginx::server { $conf_dir = "/etc/nginx/conf.d" $http_port = 80 $https_port = 443 package {"nginx": ensure => installed } -> file {"nginx_conf": path => "$conf_dir/test.conf", content => template("nginx/conf/test.conf.erb"), owner => 'nginx', group => 'nginx', mode => 644, ensure => file } -> service {"nginx": enable => true, ensure => running } }
  39. 39. Template  Puppet templates are flat files containing Embedded Ruby (ERB) variables server { listen <%= @http_port %>; } server { listen <%= @https_port %>; }
  40. 40. Node Node definitions look just like classes, including supporting inheritance, but they are special in that when a node (a managed computer running the Puppet client) connects to the Puppet master daemon. node „www1' { include nginx:server }
  41. 41. Modules A module is just a directory with stuff in it, and the magic comes from putting that stuff where Puppet expects to find it.
  42. 42. Module Structure
  43. 43. Network Overview  Configuration allows for manual synchronizations or a set increment  Client or server initiated synchronizations  Client/Server configuration leverages a Certificate Authority (CA) on the Puppet Master to sign client certificates to verify authenticity  Transmissions of all data between a master & client are encrypted
  44. 44. Every Client  Retrieve resource catalog from central server  Determine resource order  Check each resource in turn, fixing if necessary  Rinse and repeat, every 30 minutes
  45. 45. Every Resource  Retrieve current state (e.g., by querying dpkg db or doing a stat)  Compare to desired state  Fix, if necessary (or just log)
  46. 46. Drawbacks  Hard to prepare the environment  Install Ruby, puppet packages  Set up host name, domain name  Put ssh public key to every client  Configure certificate  Hard to control deployment time (in daemon mode)  Hard to support rolling upgrade  No global view, no service dependency control across hosts
  47. 47. Combine Fabric and Puppet  Fabric  When  Operators trigger puppet to deploy packages one by one or parallelly  Rolling upgrade  Where  Use fab -R or fab -H  Initial functions  Global setup and teardown functions  Puppet  What  Define puppet nodes  How  Define puppet classes and templates  Reporting  Update the status to puppet dashboard
  48. 48. Initial functions  Create EC2 instances (optional)  Setup SSH keys to all remote hosts  Configure yum repositories  Install puppet and ruby packages  Configure puppet and update new hosts to cert list
  49. 49. Global setup functions  Mandatory  Backup  Clean yum cache  Sync fabric configurations to puppet pp files  Restart puppet master service  Optional  Clean the environment if necessary  Put ssh public key  Put yum repo files  Install system development tools  Install ruby and puppet packages  Update puppet patches  Configure puppet environment
  50. 50. Global teardown functions  Start/stop services across hosts  Send email/SMS notifications to members  Do health/sanity check
  51. 51. Classification 2013/10/352 Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×