Configuration Surgery with Augeas

Configuration surgery with Augeas Raphaël Pinson @raphink LSM 2012, Geneva 2012-07-11https://github.com/raphink/augeas-talks/

Tired of ugly sed and awk one liners? or of using tons of different parsing libraries or common::line tricks? www.camptocamp.com / 2/38

Become a configuration surgeon with Augeas www.camptocamp.com / 3/38

What is the need?● A lot of different syntaxes● Securely editing configuration files with a unified API www.camptocamp.com / 4/38

A treeAugeas turns configuration files into a treestructure:/etc/hosts -> /files/etc/hosts www.camptocamp.com / 5/38

Its branches and leaves... and their parameters into branches and leaves:augtool> print /files/etc/hosts /files/etc/hosts /files/etc/hosts/1 /files/etc/hosts/1/ipaddr = "127.0.0.1" /files/etc/hosts/1/canonical = "localhost" www.camptocamp.com / 6/38

Augeas provides many stock parsersThey are called lenses:Access Cron Host_ConfAliases Crypttab HostnameAnacron debctrl Hosts_AccessApprox Desktop IniFileAptConf Dhcpd InputrcAutomaster Dpkg IptablesAutomounter Exports KdumpBackupPCHosts FAI_DiskConfig Keepalivedcgconfig Fonts Keepalivedcgrules Fuse Login_defsChannels Grub Mke2fs... www.camptocamp.com / 7/38

... as well as generic lensesavailable to build new parsers:Build Sep SimplelinesIniFile Shellvars SimplevarsRx Shellvars_list Util www.camptocamp.com / 8/38

augtool lets you inspect the tree$ augtoolaugtool> ls / augeas/ = (none) files/ = (none)augtool> print /files/etc/passwd/root/ /files/etc/passwd/root /files/etc/passwd/root/password = "x" /files/etc/passwd/root/uid = "0" /files/etc/passwd/root/gid = "0" /files/etc/passwd/root/name = "root" /files/etc/passwd/root/home = "/root" /files/etc/passwd/root/shell = "/bin/bash" www.camptocamp.com / 9/38

The tree can be queried using XPathaugtool> print /files/etc/passwd/*[uid=0][1] /files/etc/passwd/root /files/etc/passwd/root/password = "x" /files/etc/passwd/root/uid = "0" /files/etc/passwd/root/gid = "0" /files/etc/passwd/root/name = "root" /files/etc/passwd/root/home = "/root" /files/etc/passwd/root/shell = "/bin/bash" www.camptocamp.com / 10/38

But also modified$ getent passwd rootroot:x:0:0:root:/root:/bin/bash$ augtoolaugtool> set /files/etc/passwd/*[uid=0]/shell /bin/shaugtool> match /files/etc/passwd/*[uid=0]/shell/files/etc/passwd/root/shell = "/bin/sh"augtool> saveSaved 1 file(s)augtool> exit$ getent passwd rootroot:x:0:0:root:/root:/bin/sh www.camptocamp.com / 11/38

Puppet has a native provideraugeas {export foo: context => /files/etc/exports, changes => [ "set dir[. = /foo] /foo", "set dir[. = /foo]/client weeble", "set dir[. = /foo]/client/option[1] ro", "set dir[. = /foo]/client/option[2] all_squash", ],} www.camptocamp.com / 12/38

It is better to wrap things updefine kmod::generic( $type, $module, $ensure=present, $command=, $file=/etc/modprobe.d/modprobe.conf) { augeas {"${type} module ${module}": context => "/files${file}", changes => [ "set ${type}[. = ${module}] ${module}", "set ${type}[. = ${module}]/command ${command}", ], }} www.camptocamp.com / 13/38

mcollective has an agent$ mco augeas match /files/etc/passwd/rpinson/shell * [ ======================================> ] 196 / 196...wrk1saja-map-dev /files/etc/passwd/rpinson/shell = /bin/bashwrk3wrk4 /files/etc/passwd/rpinson/shell = /bin/bash... www.camptocamp.com / 14/38

... and uses it for discovery$ mco find -S "augeas_match(/files/etc/passwd/rip).size = 0" www.camptocamp.com / 15/38

Bindings include Perl, Python, Java, PHP, Haskell, Ruby...require augeasaug = Augeas.openif aug.match(/augeas/load+lens).length > 0 aug.set(/augeas/load/+lens+incl[last()+1], path)else aug.set(/augeas/load/+lens+/lens, lens+.lns)end (From the mcollective agent) www.camptocamp.com / 16/38

The Ruby bindings can be used in FacterFacter.add(:augeasversion) do setcode do begin require augeas aug = Augeas::open(/, nil, Augeas::NO_MODL_AUTOLOAD) ver = aug.get(/augeas/version) aug.close ver rescue Exception Facter.debug(ruby-augeas not available) end endend (From the augeasversion fact) www.camptocamp.com / 17/38

Or to write native typesdef ip aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) aug.get("#{path}/*[canonical = #{resource[:name]}]/ipaddr") ensure aug.close if aug endend (See https://github.com/domcleal/augeasproviders) www.camptocamp.com / 18/38

The case of sshd_configCustom type:define ssh::config::sshd ($ensure=present, $value=) { case $ensure { present: { $changes = "set ${name} ${value}" } absent: { $changes = "rm ${name}" } default: { fail("Wrong value for ensure: ${ensure}") } } augeas {"Set ${name} in /etc/ssh/sshd_config": context => /files/etc/ssh/sshd_config, changes => $changes, }} www.camptocamp.com / 19/38

Using the custom type for sshd_configssh::config::sshd {PasswordAuthenticator: value => yes,} www.camptocamp.com / 20/38

The problem with sshd_configMatch groups:Match Host example.com PermitRootLogin no=> Not possible with ssh::config::sshd, requiresinsertions and looping through the configurationparameters. www.camptocamp.com / 21/38

A native provider for sshd_config (1)The type:Puppet::Type.newtype(:sshd_config) do ensurable newparam(:name) do desc "The name of the entry." isnamevar end newproperty(:value) do desc "Entry value." end newproperty(:target) do desc "File target." end newparam(:condition) do desc "Match group condition for the entry." endend www.camptocamp.com / 22/38

A native provider for sshd_config (2)The provider:require augeas if Puppet.features.augeas?Puppet::Type.type(:sshd_config).provide(:augeas) do desc "Uses Augeas API to update an sshd_config parameter" def self.file(resource = nil) file = "/etc/ssh/sshd_config" file = resource[:target] if resource and resource[:target] file.chomp("/") end confine :true => Puppet.features.augeas? confine :exists => file www.camptocamp.com / 23/38

A native provider for sshd_config (3)def self.augopen(resource = nil) aug = nil file = file(resource) begin aug = Augeas.open(nil, nil, Augeas::NO_MODL_AUTOLOAD) aug.transform( :lens => "Sshd.lns", :name => "Sshd", :incl => file ) aug.load! if aug.match("/files#{file}").empty? message = aug.get("/augeas/files#{file}/error/message") fail("Augeas didnt load #{file}: #{message}") end rescue aug.close if aug raise end augend www.camptocamp.com / 24/38

A native provider for sshd_config (4)def self.instances aug = nil path = "/files#{file}" entry_path = self.class.entry_path(resource) begin resources = [] aug = augopen aug.match(entry_path).each do |hpath| entry = {} entry[:name] = resource[:name] entry[:conditions] = Hash[*resource[:condition].split( ).flatten(1)] entry[:value] = aug.get(hpath) resources << new(entry) end resources ensure aug.close if aug endend www.camptocamp.com / 25/38

A native provider for sshd_config (5)def self.match_conditions(resource=nil) if resource[:condition] conditions = Hash[*resource[:condition].split( ).flatten(1)] cond_keys = conditions.keys.length cond_str = "[count(Condition/*)=#{cond_keys}]" conditions.each { |k,v| cond_str += "[Condition/#{k}="#{v}"]" } cond_str else "" endenddef self.entry_path(resource=nil) path = "/files#{self.file(resource)}" if resource[:condition] cond_str = self.match_conditions(resource) "#{path}/Match#{cond_str}/Settings/#{resource[:name]}" else "#{path}/#{resource[:name]}" endend www.camptocamp.com / 26/38

A native provider for sshd_config (6)def self.match_exists?(resource=nil) aug = nil path = "/files#{self.file(resource)}" begin aug = self.augopen(resource) if resource[:condition] cond_str = self.match_conditions(resource) else false end not aug.match("#{path}/Match#{cond_str}").empty? ensure aug.close if aug endend www.camptocamp.com / 27/38

A native provider for sshd_config (7)def exists? aug = nil entry_path = self.class.entry_path(resource) begin aug = self.class.augopen(resource) not aug.match(entry_path).empty? ensure aug.close if aug endenddef self.create_match(resource=nil, aug=nil) path = "/files#{self.file(resource)}" begin aug.insert("#{path}/*[last()]", "Match", false) conditions = Hash[*resource[:condition].split( ).flatten(1)] conditions.each do |k,v| aug.set("#{path}/Match[last()]/Condition/#{k}", v) end aug endend www.camptocamp.com / 28/38

A native provider for sshd_config (8)def create aug = nil path = "/files#{self.class.file(resource)}" entry_path = self.class.entry_path(resource) begin aug = self.class.augopen(resource) if resource[:condition] unless self.class.match_exists?(resource) aug = self.class.create_match(resource, aug) end else unless aug.match("#{path}/Match").empty? aug.insert("#{path}/Match[1]", resource[:name], true) end end aug.set(entry_path, resource[:value]) aug.save! ensure aug.close if aug endend www.camptocamp.com / 29/38

A native provider for sshd_config (9)def destroy aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.rm(entry_path) aug.rm("#{path}/Match[count(Settings/*)=0]") aug.save! ensure aug.close if aug endenddef target self.class.file(resource)end www.camptocamp.com / 30/38

A native provider for sshd_config (10)def value aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.get(entry_path) ensure aug.close if aug endend www.camptocamp.com / 31/38

A native provider for sshd_config (11)def value=(thevalue) aug = nil path = "/files#{self.class.file(resource)}" begin aug = self.class.augopen(resource) entry_path = self.class.entry_path(resource) aug.set(entry_path, thevalue) aug.save! ensure aug.close if aug endend www.camptocamp.com / 32/38

Using the native provider for sshd_configsshd_config {PermitRootLogin: ensure => present, condition => Host example.com, value => yes,} www.camptocamp.com / 33/38

Errors are reported in the /augeas treeaugtool> print /augeas//error /augeas/files/etc/mke2fs.conf/error = "parse_failed" /augeas/files/etc/mke2fs.conf/error/pos = "82" /augeas/files/etc/mke2fs.conf/error/line = "3" /augeas/files/etc/mke2fs.conf/error/char = "0" /augeas/files/etc/mke2fs.conf/error/lens = "/usr/share/augeas/lenses/dist/mke2fs.aug:132.10-.49:" /augeas/files/etc/mke2fs.conf/error/message = "Get did not match entire input" www.camptocamp.com / 34/38

Other projects using Augeas● libvirt● rpm● Nut● guestfs● ZYpp● Config::Model● Augeas::Validator www.camptocamp.com / 35/38

Future projects● more API calls● improved XPath syntax● more lenses● more native providers● DBUS provider● content validation in Puppet (validator)● integration in package managers● finish the Augeas book● ...● your idea/project here... www.camptocamp.com / 36/38

Questions? http://augeas.net augeas-devel@redhat.com freenode: #augeas www.camptocamp.com / 37/38

Configuration Surgery with Augeas

by Puppet Labs on Jul 25, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Configuration Surgery with Augeas Presentation Transcript