Chef for Openstack
Mohit Sethi
mohit@sethis.in
Whoami?
Mohit Sethi
Developer, Technical Lead
Senior Engineer at HP R&D
You?
• Developers?
• System Administrators?
• Architects?
Journey so far?
• 2010-11:
• CFEngine,
• Puppet
• Chef
• 2011 - Present
• Contributed to Chef, Knife cloud plugins such as ec2, azure, hp,
openstack, rackspace, google, cloudstack, vsphere, vcloud
• Written extensions for automation tools such as Vagrant, vagrant-
hp, vagrant-vsphere
Goal for today
• Configuration Management Framework – Opscode Chef,
• Principles,
• Automation Constructs
What is Chef?
Chef is a systems integration framework, built to bring the
benefits of configuration management to you entire
infrastructure.
Wait, What?
What is Chef?
Chef is a tool that allows you to define the state your
servers(local or cloud) should be in and then enforces that
state on your servers.
An API for your entire Infrastructure.
A service that exposes data about the state of your
infrastructure
Why should I use Chef?
You have Servers.
You need to configure them.
Why should I use Chef?
But I’ve my AWESOME bash scripts, which already does most of
‘those’ things,
Why else?
We are developers, we write multi-tier applications
Why else?
We like to make things interesting,
Why else?
Application grows,
Why else?
Why else?
Meanwhile 6 months later,
• How did I do that?
• Who changed that?
• Why did I do it what way?
• Then It dies,
• I have to rebuild it
• Did I forget anything
• How did I do it
• And you will be >>
Why else?
And you will be…
Why Chef?
Provides:
Architecture(1000’ view)
Chef Client runs on your servers
Client talks to a Chef Server
Clients authenticate with RSA keys
Knife is used to talk to Chef-Server &
initiate convergence of a server.
Principles
• Idempotent
• Provision Often
• Infrastructure As Code
• Data-Driven
• Thick Clients, Thin Server
#1 Idempotent
#1 Chef rule: Recipes/ Infrastructure code should be
Idempotent.
The number of Chef runs should not affect the state of the
server. The server should converge on the first run. And unless
previously defined state changes, additional runs should not
change anything.
Say “what to do” not “how”
#2 Provision Often
If your recipes are not idempotent refer rule #1.
If they are, you should consider provisioning your servers often.
Possibly every 5 minutes. Seriously.
#3 Infrastructure As Code
Infrastructure should be represented as code,
Server configuration, packages installed, relationships with
other servers, should be modeled with code to be automated.
#4 Data-Driven
• Separate of policy & data (implemented using Attributes &
DataBags)
• Infrastructure code should not have sensitive data. Though it
can have sane defaults.
• Sensitive data should be remain in a secured store, and
should only be shared with authorized clients.
#5 Thick Clients, Thin Servers
As much as possible much work is done by Chef-Client(Nodes)
Pull not Push. Chef-client runs on each node & will interact
with server when it needs to.
Server is designed to distribute data to each node, including
cookbooks, recipes, templates, files and so on.
Server also retains a copy of state of node at the conclusion of
every chef-client.
Okay! let’s write some
infrastructure code…
Vocabulary
• Nodes == Servers
• Attributes ≈ Variables
• Roles can define a Node’s attributes and what Recipes are applied to
that Node
• Clients == Anything that uses the API
• Resources are the basic building blocks to define state
• Related Resources are grouped into Recipes
• Related Recipes are grouped into Cookbooks
Do I need to know Ruby?
A little
It’s a simple syntax
Chef-solo
Chef can also run stand-alone
Nodes == Servers
Nodes have Attributes
Attributes == Variables
Attributes are Searchable
$ knife search node ‘platform:centos’
search(:node, ‘platform:centos’)
Attributes
Attributes == Variables
Attributes are Searchable
$ knife search node ‘platform:centos’
search(:node, ‘platform:centos’)
Nodes have RunList
A RunList defines:
What Roles or Recipes to apply in Order.
$ knife node show ks.ms.openstack.com –r
{
“run_list”: *
“role*os-base+”,
“role*os-identity+”,
]
}
Nodes have Roles
Role: What describes a node
• webserver
• dbserver
• glance-server
• keystone-server
• …etc
Roles have RunList
Roles
• Roles have Run-List
Roles
• Can have other roles!
Roles
• Can override default attributes
Roles
• Roles are Searchable
• To find all roles where attribute: max_children takes value
50.
$ knife search role ‘max_children:50’
search(:role, ‘max_children:50’)
Chef manages Resources on
Nodes
• Resource: Declare a description of the state a part of node should be
in.
• Have a type
• Have a name
• Have parameters
• Take action to put the resource
in the declared state
• Can send notification to other
resources.
Providers
• Resource take action through providers.
• Know how to actually perform the actions specified by a resource,
• Multiple providers per resources type
• Eg. Resource “package” have providers apt, yum, rubygems, portage,
macports, FreeBSD ports, etc
Resources
Platform
Provider
Recipes
• Recipes are list of Resources
• Apply resources in the order they are specified
• Recipes are `import` other recipes,
Recipes are just Ruby!
Cookbooks
• Cookbooks are packages for recipes,
• Distributable
• Versioned controlled.
• Can have dependency over other Cookbooks
Cookbook Structure
• Attributes
• Assets(Files/Templates)
• Providers
• Resources
• Recipes
• Metadata
Cookbook Metadata
• Declares:
• Platform support
• Dependencies
• Recipes
DataBags
• A data bag is a global variable that is stored as JSON data and is
accessible from a server.
• Create a data bag using knife
$ knife data bag create DATA_BAG_NAME (DATA_BAG_ITEM)
users = Chef::DataBag.new
• Can be encrypted
• Data values can be fetched from Recipes
Community Cookbooks
• 1000+ cookbooks for everything
- databases, applications, CMS,
package management, Hadoop,
Cloud deployments
• http://community.opscode.com
• https://launchpad.net/openstack-chef
Platform Support
• Debian
• Ubuntu
• RHEL
• Centos
• OS X
• Windows
• FreeBSD
• SUSE Enterprise
• Solaris
• SUSE
• IBM AIX
Chef Flavors
• Opscode Hosted-Chef
• http://manage.opscode.com
• Hosted SaaS version of Chef.
• Opscode Enterprise/Private Chef
• Private deployments of Opscode Chef Server
• On-Premise deployments
• Open Source Chef
• Installation
Cloud support
• Cloud support by Knife
• EC2, Rackspace, HP, Google, Azure, CloudStack, OpenStack, vSphere, vCloud, Joyent, etc
• Extensible
• Implement own Resources & Providers,
Questions??
self.intro do |mohit|
mohit.twitter = @mohitsethi,
mohit.email = mohit@sethis.in
end

Chef for Openstack

  • 1.
    Chef for Openstack MohitSethi mohit@sethis.in
  • 2.
    Whoami? Mohit Sethi Developer, TechnicalLead Senior Engineer at HP R&D You? • Developers? • System Administrators? • Architects?
  • 3.
    Journey so far? •2010-11: • CFEngine, • Puppet • Chef • 2011 - Present • Contributed to Chef, Knife cloud plugins such as ec2, azure, hp, openstack, rackspace, google, cloudstack, vsphere, vcloud • Written extensions for automation tools such as Vagrant, vagrant- hp, vagrant-vsphere
  • 4.
    Goal for today •Configuration Management Framework – Opscode Chef, • Principles, • Automation Constructs
  • 5.
    What is Chef? Chefis a systems integration framework, built to bring the benefits of configuration management to you entire infrastructure.
  • 6.
  • 7.
    What is Chef? Chefis a tool that allows you to define the state your servers(local or cloud) should be in and then enforces that state on your servers. An API for your entire Infrastructure. A service that exposes data about the state of your infrastructure
  • 8.
    Why should Iuse Chef? You have Servers. You need to configure them.
  • 9.
    Why should Iuse Chef? But I’ve my AWESOME bash scripts, which already does most of ‘those’ things,
  • 10.
    Why else? We aredevelopers, we write multi-tier applications
  • 11.
    Why else? We liketo make things interesting,
  • 12.
  • 13.
  • 14.
    Why else? Meanwhile 6months later, • How did I do that? • Who changed that? • Why did I do it what way? • Then It dies, • I have to rebuild it • Did I forget anything • How did I do it • And you will be >>
  • 15.
  • 16.
  • 17.
  • 18.
    Chef Client runson your servers
  • 19.
    Client talks toa Chef Server
  • 20.
  • 21.
    Knife is usedto talk to Chef-Server & initiate convergence of a server.
  • 22.
    Principles • Idempotent • ProvisionOften • Infrastructure As Code • Data-Driven • Thick Clients, Thin Server
  • 23.
    #1 Idempotent #1 Chefrule: Recipes/ Infrastructure code should be Idempotent. The number of Chef runs should not affect the state of the server. The server should converge on the first run. And unless previously defined state changes, additional runs should not change anything. Say “what to do” not “how”
  • 24.
    #2 Provision Often Ifyour recipes are not idempotent refer rule #1. If they are, you should consider provisioning your servers often. Possibly every 5 minutes. Seriously.
  • 25.
    #3 Infrastructure AsCode Infrastructure should be represented as code, Server configuration, packages installed, relationships with other servers, should be modeled with code to be automated.
  • 26.
    #4 Data-Driven • Separateof policy & data (implemented using Attributes & DataBags) • Infrastructure code should not have sensitive data. Though it can have sane defaults. • Sensitive data should be remain in a secured store, and should only be shared with authorized clients.
  • 27.
    #5 Thick Clients,Thin Servers As much as possible much work is done by Chef-Client(Nodes) Pull not Push. Chef-client runs on each node & will interact with server when it needs to. Server is designed to distribute data to each node, including cookbooks, recipes, templates, files and so on. Server also retains a copy of state of node at the conclusion of every chef-client.
  • 28.
    Okay! let’s writesome infrastructure code…
  • 29.
    Vocabulary • Nodes ==Servers • Attributes ≈ Variables • Roles can define a Node’s attributes and what Recipes are applied to that Node • Clients == Anything that uses the API • Resources are the basic building blocks to define state • Related Resources are grouped into Recipes • Related Recipes are grouped into Cookbooks
  • 30.
    Do I needto know Ruby? A little It’s a simple syntax
  • 31.
    Chef-solo Chef can alsorun stand-alone
  • 32.
    Nodes == Servers Nodeshave Attributes Attributes == Variables Attributes are Searchable $ knife search node ‘platform:centos’ search(:node, ‘platform:centos’)
  • 33.
    Attributes Attributes == Variables Attributesare Searchable $ knife search node ‘platform:centos’ search(:node, ‘platform:centos’)
  • 34.
    Nodes have RunList ARunList defines: What Roles or Recipes to apply in Order. $ knife node show ks.ms.openstack.com –r { “run_list”: * “role*os-base+”, “role*os-identity+”, ] }
  • 35.
    Nodes have Roles Role:What describes a node • webserver • dbserver • glance-server • keystone-server • …etc Roles have RunList
  • 36.
  • 37.
    Roles • Can haveother roles!
  • 38.
    Roles • Can overridedefault attributes
  • 39.
    Roles • Roles areSearchable • To find all roles where attribute: max_children takes value 50. $ knife search role ‘max_children:50’ search(:role, ‘max_children:50’)
  • 40.
    Chef manages Resourceson Nodes • Resource: Declare a description of the state a part of node should be in. • Have a type • Have a name • Have parameters • Take action to put the resource in the declared state • Can send notification to other resources.
  • 41.
    Providers • Resource takeaction through providers. • Know how to actually perform the actions specified by a resource, • Multiple providers per resources type • Eg. Resource “package” have providers apt, yum, rubygems, portage, macports, FreeBSD ports, etc
  • 42.
  • 43.
    Recipes • Recipes arelist of Resources • Apply resources in the order they are specified • Recipes are `import` other recipes,
  • 44.
  • 45.
    Cookbooks • Cookbooks arepackages for recipes, • Distributable • Versioned controlled. • Can have dependency over other Cookbooks
  • 46.
    Cookbook Structure • Attributes •Assets(Files/Templates) • Providers • Resources • Recipes • Metadata
  • 47.
    Cookbook Metadata • Declares: •Platform support • Dependencies • Recipes
  • 48.
    DataBags • A databag is a global variable that is stored as JSON data and is accessible from a server. • Create a data bag using knife $ knife data bag create DATA_BAG_NAME (DATA_BAG_ITEM) users = Chef::DataBag.new • Can be encrypted • Data values can be fetched from Recipes
  • 49.
    Community Cookbooks • 1000+cookbooks for everything - databases, applications, CMS, package management, Hadoop, Cloud deployments • http://community.opscode.com • https://launchpad.net/openstack-chef
  • 50.
    Platform Support • Debian •Ubuntu • RHEL • Centos • OS X • Windows • FreeBSD • SUSE Enterprise • Solaris • SUSE • IBM AIX
  • 51.
    Chef Flavors • OpscodeHosted-Chef • http://manage.opscode.com • Hosted SaaS version of Chef. • Opscode Enterprise/Private Chef • Private deployments of Opscode Chef Server • On-Premise deployments • Open Source Chef • Installation
  • 52.
    Cloud support • Cloudsupport by Knife • EC2, Rackspace, HP, Google, Azure, CloudStack, OpenStack, vSphere, vCloud, Joyent, etc • Extensible • Implement own Resources & Providers,
  • 53.
    Questions?? self.intro do |mohit| mohit.twitter= @mohitsethi, mohit.email = mohit@sethis.in end

Editor's Notes

  • #9 We need Chef because we have servers,
  • #32 Before we move forward lets look more into chef-solo