Configuration surgery with Augeas                         Raphaël Pinson                            @raphink              ...
Tired of ugly sed and awk one liners?  or of using tons of different parsing libraries or               common::line trick...
Become a configuration surgeon with                        Augeas        www.camptocamp.com /          3/38
What is the need?●   A lot of different syntaxes●   Securely editing configuration files with a    unified API            ...
A treeAugeas turns configuration files into a treestructure:/etc/hosts -> /files/etc/hosts             www.camptocamp.com ...
Its branches and leaves... and their parameters into branches and leaves:augtool> print /files/etc/hosts  /files/etc/hosts...
Augeas provides many stock parsersThey are called lenses:Access             Cron                Host_ConfAliases          ...
... as well as generic lensesavailable to build new parsers:Build        Sep                  SimplelinesIniFile      Shel...
augtool lets you inspect the tree$ augtoolaugtool> ls / augeas/ = (none) files/ = (none)augtool> print /files/etc/passwd/r...
The tree can be queried using XPathaugtool> print /files/etc/passwd/*[uid=0][1] /files/etc/passwd/root /files/etc/passwd/r...
But also modified$ getent passwd rootroot:x:0:0:root:/root:/bin/bash$ augtoolaugtool> set /files/etc/passwd/*[uid=0]/shell...
Puppet has a native provideraugeas {export foo:    context => /files/etc/exports,    changes => [        "set dir[. = /foo...
It is better to wrap things updefine kmod::generic(  $type, $module, $ensure=present,  $command=, $file=/etc/modprobe.d/mo...
mcollective has an agent$ mco augeas match /files/etc/passwd/rpinson/shell * [ ======================================> ] 1...
... and uses it for discovery$ mco find -S "augeas_match(/files/etc/passwd/rip).size = 0"             www.camptocamp.com /...
Bindings include Perl, Python, Java,       PHP, Haskell, Ruby...require augeasaug = Augeas.openif aug.match(/augeas/load+l...
The Ruby bindings can be used in FacterFacter.add(:augeasversion) do  setcode do    begin      require augeas      aug = A...
Or to write native typesdef ip    aug = nil    path = "/files#{self.class.file(resource)}"    begin       aug = self.class...
The case of sshd_configCustom type:define ssh::config::sshd ($ensure=present, $value=) {    case $ensure {      present: {...
Using the custom type for sshd_configssh::config::sshd {PasswordAuthenticator:  value => yes,}             www.camptocamp....
The problem with sshd_configMatch groups:Match Host example.com  PermitRootLogin no=> Not possible with ssh::config::sshd,...
A native provider for sshd_config (1)The type:Puppet::Type.newtype(:sshd_config) do  ensurable  newparam(:name) do    desc...
A native provider for sshd_config (2)The provider:require augeas if Puppet.features.augeas?Puppet::Type.type(:sshd_config)...
A native provider for sshd_config (3)def self.augopen(resource = nil) aug = nil file = file(resource) begin   aug = Augeas...
A native provider for sshd_config (4)def self.instances  aug = nil  path = "/files#{file}"  entry_path = self.class.entry_...
A native provider for sshd_config (5)def self.match_conditions(resource=nil)  if resource[:condition]    conditions = Hash...
A native provider for sshd_config (6)def self.match_exists?(resource=nil)  aug = nil  path = "/files#{self.file(resource)}...
A native provider for sshd_config (7)def exists?  aug = nil  entry_path = self.class.entry_path(resource)  begin    aug = ...
A native provider for sshd_config (8)def create  aug = nil  path = "/files#{self.class.file(resource)}"  entry_path = self...
A native provider for sshd_config (9)def destroy  aug = nil  path = "/files#{self.class.file(resource)}"  begin    aug = s...
A native provider for sshd_config (10)def value  aug = nil  path = "/files#{self.class.file(resource)}"  begin    aug = se...
A native provider for sshd_config (11)def value=(thevalue)  aug = nil  path = "/files#{self.class.file(resource)}"  begin ...
Using the native provider for        sshd_configsshd_config   {PermitRootLogin:  ensure      => present,  condition   => H...
Errors are reported in the /augeas treeaugtool> print /augeas//error /augeas/files/etc/mke2fs.conf/error = "parse_failed" ...
Other projects using Augeas●   libvirt●   rpm●   Nut●   guestfs●   ZYpp●   Config::Model●   Augeas::Validator            w...
Future projects●   more API calls●   improved XPath syntax●   more lenses●   more native providers●   DBUS provider●   con...
Questions?              http://augeas.net         augeas-devel@redhat.com            freenode: #augeas       www.camptocam...
Augeas @RMLL 2012
Upcoming SlideShare
Loading in...5
×

Augeas @RMLL 2012

385

Published on

Augeas talk at RMLL 2012, Geneva

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
385
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Augeas @RMLL 2012

  1. 1. Configuration surgery with Augeas Raphaël Pinson @raphink LSM 2012, Geneva 2012-07-11https://github.com/raphink/augeas-talks/
  2. 2. Tired of ugly sed and awk one liners? or of using tons of different parsing libraries or common::line tricks? www.camptocamp.com / 2/38
  3. 3. Become a configuration surgeon with Augeas www.camptocamp.com / 3/38
  4. 4. What is the need?● A lot of different syntaxes● Securely editing configuration files with a unified API www.camptocamp.com / 4/38
  5. 5. A treeAugeas turns configuration files into a treestructure:/etc/hosts -> /files/etc/hosts www.camptocamp.com / 5/38
  6. 6. Its branches and leaves... and their parameters into branches and leaves:augtool> print /files/etc/hosts /files/etc/hosts /files/etc/hosts/1 /files/etc/hosts/1/ipaddr = "127.0.0.1" /files/etc/hosts/1/canonical = "localhost" www.camptocamp.com / 6/38
  7. 7. Augeas provides many stock parsersThey are called lenses:Access Cron Host_ConfAliases Crypttab HostnameAnacron debctrl Hosts_AccessApprox Desktop IniFileAptConf Dhcpd InputrcAutomaster Dpkg IptablesAutomounter Exports KdumpBackupPCHosts FAI_DiskConfig Keepalivedcgconfig Fonts Keepalivedcgrules Fuse Login_defsChannels Grub Mke2fs... www.camptocamp.com / 7/38
  8. 8. ... as well as generic lensesavailable to build new parsers:Build Sep SimplelinesIniFile Shellvars SimplevarsRx Shellvars_list Util www.camptocamp.com / 8/38
  9. 9. augtool lets you inspect the tree$ augtoolaugtool> ls / augeas/ = (none) files/ = (none)augtool> print /files/etc/passwd/root/ /files/etc/passwd/root /files/etc/passwd/root/password = "x" /files/etc/passwd/root/uid = "0" /files/etc/passwd/root/gid = "0" /files/etc/passwd/root/name = "root" /files/etc/passwd/root/home = "/root" /files/etc/passwd/root/shell = "/bin/bash" www.camptocamp.com / 9/38
  10. 10. The tree can be queried using XPathaugtool> print /files/etc/passwd/*[uid=0][1] /files/etc/passwd/root /files/etc/passwd/root/password = "x" /files/etc/passwd/root/uid = "0" /files/etc/passwd/root/gid = "0" /files/etc/passwd/root/name = "root" /files/etc/passwd/root/home = "/root" /files/etc/passwd/root/shell = "/bin/bash" www.camptocamp.com / 10/38
  11. 11. But also modified$ getent passwd rootroot:x:0:0:root:/root:/bin/bash$ augtoolaugtool> set /files/etc/passwd/*[uid=0]/shell /bin/shaugtool> match /files/etc/passwd/*[uid=0]/shell/files/etc/passwd/root/shell = "/bin/sh"augtool> saveSaved 1 file(s)augtool> exit$ getent passwd rootroot:x:0:0:root:/root:/bin/sh www.camptocamp.com / 11/38
  12. 12. Puppet has a native provideraugeas {export foo: context => /files/etc/exports, changes => [ "set dir[. = /foo] /foo", "set dir[. = /foo]/client weeble", "set dir[. = /foo]/client/option[1] ro", "set dir[. = /foo]/client/option[2] all_squash", ],} www.camptocamp.com / 12/38
  13. 13. It is better to wrap things updefine kmod::generic( $type, $module, $ensure=present, $command=, $file=/etc/modprobe.d/modprobe.conf) { augeas {"${type} module ${module}": context => "/files${file}", changes => [ "set ${type}[. = ${module}] ${module}", "set ${type}[. = ${module}]/command ${command}", ], }} www.camptocamp.com / 13/38
  14. 14. mcollective has an agent$ mco augeas match /files/etc/passwd/rpinson/shell * [ ======================================> ] 196 / 196...wrk1saja-map-dev /files/etc/passwd/rpinson/shell = /bin/bashwrk3wrk4 /files/etc/passwd/rpinson/shell = /bin/bash... www.camptocamp.com / 14/38
  15. 15. ... and uses it for discovery$ mco find -S "augeas_match(/files/etc/passwd/rip).size = 0" www.camptocamp.com / 15/38
  16. 16. Bindings include Perl, Python, Java, PHP, Haskell, Ruby...require augeasaug = Augeas.openif aug.match(/augeas/load+lens).length > 0 aug.set(/augeas/load/+lens+incl[last()+1], path)else aug.set(/augeas/load/+lens+/lens, lens+.lns)end (From the mcollective agent) www.camptocamp.com / 16/38
  17. 17. The Ruby bindings can be used in FacterFacter.add(:augeasversion) do setcode do begin require augeas aug = Augeas::open(/, nil, Augeas::NO_MODL_AUTOLOAD) ver = aug.get(/augeas/version) aug.close ver rescue Exception Facter.debug(ruby-augeas not available) end endend (From the augeasversion fact) www.camptocamp.com / 17/38
  18. 18. Or to write native typesdef ip aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) aug.get("#{path}/*[canonical = #{resource[:name]}]/ipaddr") ensure aug.close if aug endend (See https://github.com/domcleal/augeasproviders) www.camptocamp.com / 18/38
  19. 19. The case of sshd_configCustom type:define ssh::config::sshd ($ensure=present, $value=) { case $ensure { present: { $changes = "set ${name} ${value}" } absent: { $changes = "rm ${name}" } default: { fail("Wrong value for ensure: ${ensure}") } } augeas {"Set ${name} in /etc/ssh/sshd_config": context => /files/etc/ssh/sshd_config, changes => $changes, }} www.camptocamp.com / 19/38
  20. 20. Using the custom type for sshd_configssh::config::sshd {PasswordAuthenticator: value => yes,} www.camptocamp.com / 20/38
  21. 21. The problem with sshd_configMatch groups:Match Host example.com PermitRootLogin no=> Not possible with ssh::config::sshd, requiresinsertions and looping through the configurationparameters. www.camptocamp.com / 21/38
  22. 22. A native provider for sshd_config (1)The type:Puppet::Type.newtype(:sshd_config) do ensurable newparam(:name) do desc "The name of the entry." isnamevar end newproperty(:value) do desc "Entry value." end newproperty(:target) do desc "File target." end newparam(:condition) do desc "Match group condition for the entry." endend www.camptocamp.com / 22/38
  23. 23. A native provider for sshd_config (2)The provider:require augeas if Puppet.features.augeas?Puppet::Type.type(:sshd_config).provide(:augeas) do desc "Uses Augeas API to update an sshd_config parameter" def self.file(resource = nil) file = "/etc/ssh/sshd_config" file = resource[:target] if resource and resource[:target] file.chomp("/") end confine :true => Puppet.features.augeas? confine :exists => file www.camptocamp.com / 23/38
  24. 24. A native provider for sshd_config (3)def self.augopen(resource = nil) aug = nil file = file(resource) begin aug = Augeas.open(nil, nil, Augeas::NO_MODL_AUTOLOAD) aug.transform( :lens => "Sshd.lns", :name => "Sshd", :incl => file ) aug.load! if aug.match("/files#{file}").empty? message = aug.get("/augeas/files#{file}/error/message") fail("Augeas didnt load #{file}: #{message}") end rescue aug.close if aug raise end augend www.camptocamp.com / 24/38
  25. 25. A native provider for sshd_config (4)def self.instances aug = nil path = "/files#{file}" entry_path = self.class.entry_path(resource) begin resources = [] aug = augopen aug.match(entry_path).each do |hpath| entry = {} entry[:name] = resource[:name] entry[:conditions] = Hash[*resource[:condition].split( ).flatten(1)] entry[:value] = aug.get(hpath) resources << new(entry) end resources ensure aug.close if aug endend www.camptocamp.com / 25/38
  26. 26. A native provider for sshd_config (5)def self.match_conditions(resource=nil) if resource[:condition] conditions = Hash[*resource[:condition].split( ).flatten(1)] cond_keys = conditions.keys.length cond_str = "[count(Condition/*)=#{cond_keys}]" conditions.each { |k,v| cond_str += "[Condition/#{k}="#{v}"]" } cond_str else "" endenddef self.entry_path(resource=nil) path = "/files#{self.file(resource)}" if resource[:condition] cond_str = self.match_conditions(resource) "#{path}/Match#{cond_str}/Settings/#{resource[:name]}" else "#{path}/#{resource[:name]}" endend www.camptocamp.com / 26/38
  27. 27. A native provider for sshd_config (6)def self.match_exists?(resource=nil) aug = nil path = "/files#{self.file(resource)}" begin aug = self.augopen(resource) if resource[:condition] cond_str = self.match_conditions(resource) else false end not aug.match("#{path}/Match#{cond_str}").empty? ensure aug.close if aug endend www.camptocamp.com / 27/38
  28. 28. A native provider for sshd_config (7)def exists? aug = nil entry_path = self.class.entry_path(resource) begin aug = self.class.augopen(resource) not aug.match(entry_path).empty? ensure aug.close if aug endenddef self.create_match(resource=nil, aug=nil) path = "/files#{self.file(resource)}" begin aug.insert("#{path}/*[last()]", "Match", false) conditions = Hash[*resource[:condition].split( ).flatten(1)] conditions.each do |k,v| aug.set("#{path}/Match[last()]/Condition/#{k}", v) end aug endend www.camptocamp.com / 28/38
  29. 29. A native provider for sshd_config (8)def create aug = nil path = "/files#{self.class.file(resource)}" entry_path = self.class.entry_path(resource) begin aug = self.class.augopen(resource) if resource[:condition] unless self.class.match_exists?(resource) aug = self.class.create_match(resource, aug) end else unless aug.match("#{path}/Match").empty? aug.insert("#{path}/Match[1]", resource[:name], true) end end aug.set(entry_path, resource[:value]) aug.save! ensure aug.close if aug endend www.camptocamp.com / 29/38
  30. 30. A native provider for sshd_config (9)def destroy aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.rm(entry_path) aug.rm("#{path}/Match[count(Settings/*)=0]") aug.save! ensure aug.close if aug endenddef target self.class.file(resource)end www.camptocamp.com / 30/38
  31. 31. A native provider for sshd_config (10)def value aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.get(entry_path) ensure aug.close if aug endend www.camptocamp.com / 31/38
  32. 32. A native provider for sshd_config (11)def value=(thevalue) aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.set(entry_path, thevalue) aug.save! ensure aug.close if aug endend www.camptocamp.com / 32/38
  33. 33. Using the native provider for sshd_configsshd_config {PermitRootLogin: ensure => present, condition => Host example.com, value => yes,} www.camptocamp.com / 33/38
  34. 34. Errors are reported in the /augeas treeaugtool> print /augeas//error /augeas/files/etc/mke2fs.conf/error = "parse_failed" /augeas/files/etc/mke2fs.conf/error/pos = "82" /augeas/files/etc/mke2fs.conf/error/line = "3" /augeas/files/etc/mke2fs.conf/error/char = "0" /augeas/files/etc/mke2fs.conf/error/lens = "/usr/share/augeas/lenses/dist/mke2fs.aug:132.10-.49:" /augeas/files/etc/mke2fs.conf/error/message = "Get did not match entire input" www.camptocamp.com / 34/38
  35. 35. Other projects using Augeas● libvirt● rpm● Nut● guestfs● ZYpp● Config::Model● Augeas::Validator www.camptocamp.com / 35/38
  36. 36. Future projects● more API calls● improved XPath syntax● more lenses● more native providers● DBUS provider● content validation in Puppet (validator)● integration in package managers● finish the Augeas book● ...● your idea/project here... www.camptocamp.com / 36/38
  37. 37. Questions? http://augeas.net augeas-devel@redhat.com freenode: #augeas www.camptocamp.com / 37/38
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×