A Cobbler, Func and Puppet: Tools for Large Scale Environments

1. Cobbler, Func and Puppet:
Tools for Large Scale Environments
Francesco Crippa
fcrippa @ fedoraproject,org
This presentation is licensed under a Creative Commons
Attribution-NonCommercial-ShareAlike (BY-NC-SA) 3.0 license.

2. what is a
“Large Scale Environment”?

3. From this...
Google first WebFarm

4. ...to this!
CC by Johnnie Walker: http://flickr.com/photos/johnniewalker/359440369

5. Large scale environments
● Huge number of machines
● Huge number of environments
– Production
– Pre-production
– Test
– Develop
● Huge number of people
● Need to scale!!!

6. All is complex
● An example: make a web server on line
● In theory it should be easy:
– Buy hardware
– Install operating system (next, next, next...)
– Install a http server (yum install httpd)
– Configure http (vi /etc/httpd/conf/httpd.conf)
– That's all
● ...but...
●

7. All is complex
● In a LSE you need:
– 2 web server to maintain service in high availability
– 2 geographical sites to support disaster recovery
– A testing environment to test configurations
– A development environment to setup machines
– Backup
– Monitoring
– Clustering
● Shared storage (SAN) required for each cluster

8. A lot of web servers
1... ...7 servers!

9. Where is the complexity?
● Web server configuration is simple
● But you need to configure:
– 7 different web server (with exactly the same
configuration!!!)
– Keep all configurations in sync
– 3 clusters (with different configurations)
– Shared storages (maybe with shared filesystems)
● The complexity is not on the “business”,
but on the “infrastructure”

10. Typical work flow
Installing Unconventional
Configurations
O.S. Tasks
● Installing OS to a new hardware
● Configure machines as you need
● Make (sometimes) unconventional tasks

11. Typical work flow
Installing Unconventional
Configurations
O.S. Tasks
COBBLER PUPPET FUNC

12. All under your control!
Copyright by WarnerBras (Matrix)

13. Installing OS...
COBBLER

14. OS Installation
● Usually KickStart driven
● Two main targets for a new installation:
– Real hardware
– Virtual Machine
● Usually expensive because require some
manual human tasks
– Put booting cd-rom into reader, configure network
and location of kickstart...

15. Cobbler
● Cobbler is a Linux provisioning server that
allows for rapid setup of network installation
environments.
● It keeps in sync:
– DHCP
– TFTP
– RPM Repositories
– KickStarts

16. Importing repositories
● Check configuration files
● Import repositories
[root@a~]# cobbler check
[root@a~]# cobbler import
­­mirror=rsync://servergoeshere/path/to/distro
­­name=fedora9

17. Add profiles and systems
● Distros
● Profiles
● Systems
[root@a~]# cobbler distro add
[root@a~]# cobbler profile add
[root@a~]# cobbler system add

18. Keep in sync
● Generate output in:
– /tftpboot
– /var/www/cobbler
– /etc/dhcp.conf
[root@a~]# cobbler sync

19. KickStart templates
● Kickstart is a template
● You can pass values for all your variables
through “cobbler system add”
...
keyboard $KEY
...
[root@a~]# cobbler system add ­­ksmeta=”KEY=us”

20. Koan
● Client side
● Provisioning for virtual machines
●
● koan --server=server.expl.org
--profile=RHEL5-i386
●
● koan -–server=server.expl.org
--profile=RHEL5-i386 --virt

21. Configure all your systems
PUPPET

22. Configuration Management
– Centralize all configurations into a single place
– Defining “templates” to generalize similar
configurations (make one, apply hundreds)
– Manage exceptions
– Make possible for a lot of people to work all
together

23. Working all together

24. Working all together

25. Working all together

26. Working all together

27. Configuration Items
– Files
– Subscribe
– Packages
– Require
– Services
– Notify
– Users
– ...
– Groups
– Arrays
– Mount points
– Conditions
– ...
– ...
– You can define your
own items :-)

28. A formal definition for HTTPD
Class http inherits security-hardening {
file {
“/etc/httpd/httpd.conf”:
owner => apache,
mode => 644,
require => Package[“httpd”]
}
package {
“httpd”:
ensure => “latest”
}
service {
“httpd”:
ensure => “running”,
enabled => “true”,
subscribe => [
File[“/etc/httpd/httpd.conf”],
Package[“httpd”]
]
}
}

29. A formal definition for HTTPD
node 'www.example.com' inherits default {
import httpd
}
node 'default' {
import bacula
import nagios
import bacula-backup
}

30. Puppet architecture
– Client (the puppet) polls the server (the puppet-
master) to know if there are new configuration files
to download (every 30 minutes by default)
– Client is recognized through its hostname
– Server try to find a right configuration for that
hostname, else it uses a “default” config.
– ...caching, md5, checksum and ssl are part of
infrastructure
●

31. Python API for your network
FUNC

32. Goals
● Manage your hosts as python objects
– Groups, ACLs, SSL certificates, ...
>>> from func.overlord import client
>>> client1 = client.Client(“*.lan”)
>>> client1.service.restart(“httpd”)
>>> client1.command.run(“df ­h”)
[root@a~]# func “*.com” call service restart “httpd”
[root@a~]# func “*.lan” call command run “df ­h”

33. Modules and new features
● Func based on modules architecture
● A module support new stuff
● 20 modules (libVirt, jboss, info, process,
command, iptables, nagios, etc)
● Writing a new module is simple.
● When you write a module, it works on both CLI
and PyScripting, no modification on func are
needed.

34. THANK YOU.
Credits: Byte-Code, Tango Icons, Crystals Icons