Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Configuration surgery with Augeas                         Raphaël Pinson                            @raphink              ...
Tired of ugly sed and awk one liners?  or of using tons of different parsing libraries or               common::line trick...
Become a configuration surgeon with                        Augeas        www.camptocamp.com /          3/38
What is the need?●   A lot of different syntaxes●   Securely editing configuration files with a    unified API            ...
A treeAugeas turns configuration files into a treestructure:/etc/hosts -> /files/etc/hosts             www.camptocamp.com ...
Its branches and leaves... and their parameters into branches and leaves:augtool> print /files/etc/hosts  /files/etc/hosts...
Augeas provides many stock parsersThey are called lenses:Access             Cron                Host_ConfAliases          ...
... as well as generic lensesavailable to build new parsers:Build        Sep                  SimplelinesIniFile      Shel...
augtool lets you inspect the tree$ augtoolaugtool> ls / augeas/ = (none) files/ = (none)augtool> print /files/etc/passwd/r...
The tree can be queried using XPathaugtool> print /files/etc/passwd/*[uid=0][1] /files/etc/passwd/root /files/etc/passwd/r...
But also modified$ getent passwd rootroot:x:0:0:root:/root:/bin/bash$ augtoolaugtool> set /files/etc/passwd/*[uid=0]/shell...
Puppet has a native provideraugeas {export foo:    context => /files/etc/exports,    changes => [        "set dir[. = /foo...
It is better to wrap things updefine kmod::generic(  $type, $module, $ensure=present,  $command=, $file=/etc/modprobe.d/mo...
mcollective has an agent$ mco augeas match /files/etc/passwd/rpinson/shell * [ ======================================> ] 1...
... and uses it for discovery$ mco find -S "augeas_match(/files/etc/passwd/rip).size = 0"             www.camptocamp.com /...
Bindings include Perl, Python, Java,       PHP, Haskell, Ruby...require augeasaug = Augeas.openif aug.match(/augeas/load+l...
The Ruby bindings can be used in FacterFacter.add(:augeasversion) do  setcode do    begin      require augeas      aug = A...
Or to write native typesdef ip    aug = nil    path = "/files#{self.class.file(resource)}"    begin       aug = self.class...
The case of sshd_configCustom type:define ssh::config::sshd ($ensure=present, $value=) {    case $ensure {      present: {...
Using the custom type for sshd_configssh::config::sshd {PasswordAuthenticator:  value => yes,}             www.camptocamp....
The problem with sshd_configMatch groups:Match Host example.com  PermitRootLogin no=> Not possible with ssh::config::sshd,...
A native provider for sshd_config (1)The type:Puppet::Type.newtype(:sshd_config) do  ensurable  newparam(:name) do    desc...
A native provider for sshd_config (2)The provider:require augeas if Puppet.features.augeas?Puppet::Type.type(:sshd_config)...
A native provider for sshd_config (3)def self.augopen(resource = nil) aug = nil file = file(resource) begin   aug = Augeas...
A native provider for sshd_config (4)def self.instances  aug = nil  path = "/files#{file}"  entry_path = self.class.entry_...
A native provider for sshd_config (5)def self.match_conditions(resource=nil)  if resource[:condition]    conditions = Hash...
A native provider for sshd_config (6)def self.match_exists?(resource=nil)  aug = nil  path = "/files#{self.file(resource)}...
A native provider for sshd_config (7)def exists?  aug = nil  entry_path = self.class.entry_path(resource)  begin    aug = ...
A native provider for sshd_config (8)def create  aug = nil  path = "/files#{self.class.file(resource)}"  entry_path = self...
A native provider for sshd_config (9)def destroy  aug = nil  path = "/files#{self.class.file(resource)}"  begin    aug = s...
A native provider for sshd_config (10)def value  aug = nil  path = "/files#{self.class.file(resource)}"  begin    aug = se...
A native provider for sshd_config (11)def value=(thevalue)  aug = nil  path = "/files#{self.class.file(resource)}"  begin ...
Using the native provider for        sshd_configsshd_config   {PermitRootLogin:  ensure      => present,  condition   => H...
Errors are reported in the /augeas treeaugtool> print /augeas//error /augeas/files/etc/mke2fs.conf/error = "parse_failed" ...
Other projects using Augeas●   libvirt●   rpm●   Nut●   guestfs●   ZYpp●   Config::Model●   Augeas::Validator            w...
Future projects●   more API calls●   improved XPath syntax●   more lenses●   more native providers●   DBUS provider●   con...
Questions?              http://augeas.net         augeas-devel@redhat.com            freenode: #augeas       www.camptocam...
Configuration Surgery with Augeas
Upcoming SlideShare
Loading in …5
×

Configuration Surgery with Augeas

13,314 views

Published on

Raphaël Pinson's talk on "Configuration surgery with Augeas" at PuppetCamp Geneva '12. Video at http://youtu.be/H0MJaIv4bgk

Learn more: www.puppetlabs.com

Published in: Technology
  • Dating for everyone is here: ❤❤❤ http://bit.ly/369VOVb ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating direct: ❤❤❤ http://bit.ly/369VOVb ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Configuration Surgery with Augeas

  1. 1. Configuration surgery with Augeas Raphaël Pinson @raphink LSM 2012, Geneva 2012-07-11https://github.com/raphink/augeas-talks/
  2. 2. Tired of ugly sed and awk one liners? or of using tons of different parsing libraries or common::line tricks? www.camptocamp.com / 2/38
  3. 3. Become a configuration surgeon with Augeas www.camptocamp.com / 3/38
  4. 4. What is the need?● A lot of different syntaxes● Securely editing configuration files with a unified API www.camptocamp.com / 4/38
  5. 5. A treeAugeas turns configuration files into a treestructure:/etc/hosts -> /files/etc/hosts www.camptocamp.com / 5/38
  6. 6. Its branches and leaves... and their parameters into branches and leaves:augtool> print /files/etc/hosts /files/etc/hosts /files/etc/hosts/1 /files/etc/hosts/1/ipaddr = "127.0.0.1" /files/etc/hosts/1/canonical = "localhost" www.camptocamp.com / 6/38
  7. 7. Augeas provides many stock parsersThey are called lenses:Access Cron Host_ConfAliases Crypttab HostnameAnacron debctrl Hosts_AccessApprox Desktop IniFileAptConf Dhcpd InputrcAutomaster Dpkg IptablesAutomounter Exports KdumpBackupPCHosts FAI_DiskConfig Keepalivedcgconfig Fonts Keepalivedcgrules Fuse Login_defsChannels Grub Mke2fs... www.camptocamp.com / 7/38
  8. 8. ... as well as generic lensesavailable to build new parsers:Build Sep SimplelinesIniFile Shellvars SimplevarsRx Shellvars_list Util www.camptocamp.com / 8/38
  9. 9. augtool lets you inspect the tree$ augtoolaugtool> ls / augeas/ = (none) files/ = (none)augtool> print /files/etc/passwd/root/ /files/etc/passwd/root /files/etc/passwd/root/password = "x" /files/etc/passwd/root/uid = "0" /files/etc/passwd/root/gid = "0" /files/etc/passwd/root/name = "root" /files/etc/passwd/root/home = "/root" /files/etc/passwd/root/shell = "/bin/bash" www.camptocamp.com / 9/38
  10. 10. The tree can be queried using XPathaugtool> print /files/etc/passwd/*[uid=0][1] /files/etc/passwd/root /files/etc/passwd/root/password = "x" /files/etc/passwd/root/uid = "0" /files/etc/passwd/root/gid = "0" /files/etc/passwd/root/name = "root" /files/etc/passwd/root/home = "/root" /files/etc/passwd/root/shell = "/bin/bash" www.camptocamp.com / 10/38
  11. 11. But also modified$ getent passwd rootroot:x:0:0:root:/root:/bin/bash$ augtoolaugtool> set /files/etc/passwd/*[uid=0]/shell /bin/shaugtool> match /files/etc/passwd/*[uid=0]/shell/files/etc/passwd/root/shell = "/bin/sh"augtool> saveSaved 1 file(s)augtool> exit$ getent passwd rootroot:x:0:0:root:/root:/bin/sh www.camptocamp.com / 11/38
  12. 12. Puppet has a native provideraugeas {export foo: context => /files/etc/exports, changes => [ "set dir[. = /foo] /foo", "set dir[. = /foo]/client weeble", "set dir[. = /foo]/client/option[1] ro", "set dir[. = /foo]/client/option[2] all_squash", ],} www.camptocamp.com / 12/38
  13. 13. It is better to wrap things updefine kmod::generic( $type, $module, $ensure=present, $command=, $file=/etc/modprobe.d/modprobe.conf) { augeas {"${type} module ${module}": context => "/files${file}", changes => [ "set ${type}[. = ${module}] ${module}", "set ${type}[. = ${module}]/command ${command}", ], }} www.camptocamp.com / 13/38
  14. 14. mcollective has an agent$ mco augeas match /files/etc/passwd/rpinson/shell * [ ======================================> ] 196 / 196...wrk1saja-map-dev /files/etc/passwd/rpinson/shell = /bin/bashwrk3wrk4 /files/etc/passwd/rpinson/shell = /bin/bash... www.camptocamp.com / 14/38
  15. 15. ... and uses it for discovery$ mco find -S "augeas_match(/files/etc/passwd/rip).size = 0" www.camptocamp.com / 15/38
  16. 16. Bindings include Perl, Python, Java, PHP, Haskell, Ruby...require augeasaug = Augeas.openif aug.match(/augeas/load+lens).length > 0 aug.set(/augeas/load/+lens+incl[last()+1], path)else aug.set(/augeas/load/+lens+/lens, lens+.lns)end (From the mcollective agent) www.camptocamp.com / 16/38
  17. 17. The Ruby bindings can be used in FacterFacter.add(:augeasversion) do setcode do begin require augeas aug = Augeas::open(/, nil, Augeas::NO_MODL_AUTOLOAD) ver = aug.get(/augeas/version) aug.close ver rescue Exception Facter.debug(ruby-augeas not available) end endend (From the augeasversion fact) www.camptocamp.com / 17/38
  18. 18. Or to write native typesdef ip aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) aug.get("#{path}/*[canonical = #{resource[:name]}]/ipaddr") ensure aug.close if aug endend (See https://github.com/domcleal/augeasproviders) www.camptocamp.com / 18/38
  19. 19. The case of sshd_configCustom type:define ssh::config::sshd ($ensure=present, $value=) { case $ensure { present: { $changes = "set ${name} ${value}" } absent: { $changes = "rm ${name}" } default: { fail("Wrong value for ensure: ${ensure}") } } augeas {"Set ${name} in /etc/ssh/sshd_config": context => /files/etc/ssh/sshd_config, changes => $changes, }} www.camptocamp.com / 19/38
  20. 20. Using the custom type for sshd_configssh::config::sshd {PasswordAuthenticator: value => yes,} www.camptocamp.com / 20/38
  21. 21. The problem with sshd_configMatch groups:Match Host example.com PermitRootLogin no=> Not possible with ssh::config::sshd, requiresinsertions and looping through the configurationparameters. www.camptocamp.com / 21/38
  22. 22. A native provider for sshd_config (1)The type:Puppet::Type.newtype(:sshd_config) do ensurable newparam(:name) do desc "The name of the entry." isnamevar end newproperty(:value) do desc "Entry value." end newproperty(:target) do desc "File target." end newparam(:condition) do desc "Match group condition for the entry." endend www.camptocamp.com / 22/38
  23. 23. A native provider for sshd_config (2)The provider:require augeas if Puppet.features.augeas?Puppet::Type.type(:sshd_config).provide(:augeas) do desc "Uses Augeas API to update an sshd_config parameter" def self.file(resource = nil) file = "/etc/ssh/sshd_config" file = resource[:target] if resource and resource[:target] file.chomp("/") end confine :true => Puppet.features.augeas? confine :exists => file www.camptocamp.com / 23/38
  24. 24. A native provider for sshd_config (3)def self.augopen(resource = nil) aug = nil file = file(resource) begin aug = Augeas.open(nil, nil, Augeas::NO_MODL_AUTOLOAD) aug.transform( :lens => "Sshd.lns", :name => "Sshd", :incl => file ) aug.load! if aug.match("/files#{file}").empty? message = aug.get("/augeas/files#{file}/error/message") fail("Augeas didnt load #{file}: #{message}") end rescue aug.close if aug raise end augend www.camptocamp.com / 24/38
  25. 25. A native provider for sshd_config (4)def self.instances aug = nil path = "/files#{file}" entry_path = self.class.entry_path(resource) begin resources = [] aug = augopen aug.match(entry_path).each do |hpath| entry = {} entry[:name] = resource[:name] entry[:conditions] = Hash[*resource[:condition].split( ).flatten(1)] entry[:value] = aug.get(hpath) resources << new(entry) end resources ensure aug.close if aug endend www.camptocamp.com / 25/38
  26. 26. A native provider for sshd_config (5)def self.match_conditions(resource=nil) if resource[:condition] conditions = Hash[*resource[:condition].split( ).flatten(1)] cond_keys = conditions.keys.length cond_str = "[count(Condition/*)=#{cond_keys}]" conditions.each { |k,v| cond_str += "[Condition/#{k}="#{v}"]" } cond_str else "" endenddef self.entry_path(resource=nil) path = "/files#{self.file(resource)}" if resource[:condition] cond_str = self.match_conditions(resource) "#{path}/Match#{cond_str}/Settings/#{resource[:name]}" else "#{path}/#{resource[:name]}" endend www.camptocamp.com / 26/38
  27. 27. A native provider for sshd_config (6)def self.match_exists?(resource=nil) aug = nil path = "/files#{self.file(resource)}" begin aug = self.augopen(resource) if resource[:condition] cond_str = self.match_conditions(resource) else false end not aug.match("#{path}/Match#{cond_str}").empty? ensure aug.close if aug endend www.camptocamp.com / 27/38
  28. 28. A native provider for sshd_config (7)def exists? aug = nil entry_path = self.class.entry_path(resource) begin aug = self.class.augopen(resource) not aug.match(entry_path).empty? ensure aug.close if aug endenddef self.create_match(resource=nil, aug=nil) path = "/files#{self.file(resource)}" begin aug.insert("#{path}/*[last()]", "Match", false) conditions = Hash[*resource[:condition].split( ).flatten(1)] conditions.each do |k,v| aug.set("#{path}/Match[last()]/Condition/#{k}", v) end aug endend www.camptocamp.com / 28/38
  29. 29. A native provider for sshd_config (8)def create aug = nil path = "/files#{self.class.file(resource)}" entry_path = self.class.entry_path(resource) begin aug = self.class.augopen(resource) if resource[:condition] unless self.class.match_exists?(resource) aug = self.class.create_match(resource, aug) end else unless aug.match("#{path}/Match").empty? aug.insert("#{path}/Match[1]", resource[:name], true) end end aug.set(entry_path, resource[:value]) aug.save! ensure aug.close if aug endend www.camptocamp.com / 29/38
  30. 30. A native provider for sshd_config (9)def destroy aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.rm(entry_path) aug.rm("#{path}/Match[count(Settings/*)=0]") aug.save! ensure aug.close if aug endenddef target self.class.file(resource)end www.camptocamp.com / 30/38
  31. 31. A native provider for sshd_config (10)def value aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.get(entry_path) ensure aug.close if aug endend www.camptocamp.com / 31/38
  32. 32. A native provider for sshd_config (11)def value=(thevalue) aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.set(entry_path, thevalue) aug.save! ensure aug.close if aug endend www.camptocamp.com / 32/38
  33. 33. Using the native provider for sshd_configsshd_config {PermitRootLogin: ensure => present, condition => Host example.com, value => yes,} www.camptocamp.com / 33/38
  34. 34. Errors are reported in the /augeas treeaugtool> print /augeas//error /augeas/files/etc/mke2fs.conf/error = "parse_failed" /augeas/files/etc/mke2fs.conf/error/pos = "82" /augeas/files/etc/mke2fs.conf/error/line = "3" /augeas/files/etc/mke2fs.conf/error/char = "0" /augeas/files/etc/mke2fs.conf/error/lens = "/usr/share/augeas/lenses/dist/mke2fs.aug:132.10-.49:" /augeas/files/etc/mke2fs.conf/error/message = "Get did not match entire input" www.camptocamp.com / 34/38
  35. 35. Other projects using Augeas● libvirt● rpm● Nut● guestfs● ZYpp● Config::Model● Augeas::Validator www.camptocamp.com / 35/38
  36. 36. Future projects● more API calls● improved XPath syntax● more lenses● more native providers● DBUS provider● content validation in Puppet (validator)● integration in package managers● finish the Augeas book● ...● your idea/project here... www.camptocamp.com / 36/38
  37. 37. Questions? http://augeas.net augeas-devel@redhat.com freenode: #augeas www.camptocamp.com / 37/38

×