augeas logo puppet logo

One of the issues that I have with puppet templates for managing configuration files is that they are fairly rigid in their layout and don’t handle conditional structuring very well. For example if I have a tomcat tomcat-users.xml file that can specify a console/script user and a gui user for tomcat’s manager application then I would need to include placeholders in the form <%= @tomcat_manager_username %> for each value I need.

Now what if I only need these values when a module specifies them? In production I might only want a script user and in development I want a script and a GUI user? I’d have to wrap each value in Ruby style conditionals:

<% if @tomcat_manager_username != nil && @tomcat_manager_password != nil && @tomcat_manager_username != "" && @tomcat_manager_password != "" %>	
	<user username="<%= @tomcat_manager_username %>" password="<%= @tomcat_manager_password %>" roles="manager-gui"/>
<% end %>

So far so good, well so verbose and a bit boiler plate heavy.

This works for files that I have complete control over, yet there are other applications that use configuration files that I cannot say I have absolute control over. The file might have changes required after those made by my module, such as enabling an apache plugin or changes made by other puppet modules. A good example would be changes made to the PHP configuration by a Kanboard module to allow it to run or adding a virtual host to Apache in one puppet manifest and adding the headers module in another. There is no way I can predict these file changes in the main puppet module. So what can be done?

Enter Augeas…

Augeas

Augeas is a tool to replace parts of a configuration file without creating an inflexible erb template or needing to know the contents of the entire file. This allows a PHP module for example to declare a php.ini file and for another module to search and modify or add to the file later on as it sees fit.
This is a first class citizen in puppet’s resource structure.
Augeas treats files as trees of values, a bit like xpath.
A typical puppet declaration of the augeas resource looks like this:

augeas{ "configure-jekins-login":
  incl => "/var/lib/jenkins/config.xml",
  lens => "Xml.lns",
  context => "/files/var/lib/jenkins/config.xml/hudson",
  changes => $jenkins_changes,
}

It should be noted that when using Augeas with puppet in most cases it will tend to fail silently, passing the --debug switch to puppet will give you more visibility as to what puppet is trying to apply via augeas. See a future post on the augtool for more information on debugging with the command line tool for greater control and visibility.

Another gotcha is that it will make these same changes everytime puppet is run so using an onlyif => clause is advisable with a match statement.
Luckily the onlyif can also make use of augeas changes, e.g. match dir[/ = '/foo'] size == 0 or even better you can make use of the context argument to ensure you don’t create duplicates, as per the example above where the hudson prefix is being used in the context field to ensure the change we want to make is fully qualified.

I’ll break down the parameters of the augeas provider next:

incl

This property is used to inform augeas of the location of the file it is making changes to.

lens

A lens is used by augeas to parse a particular file type.
A list of stock lenses can found here
Lenses that I have confirmed to work with puppet in my experience so far are:

  • Xml.lns is used by xml files
  • Php.lns is used by php files
  • Httpd.lns is used by apache httpd config file

context

In order to set the context element for the property you want to change you will typically use the path of the file you wish to change (e.g. /etc/apache2/apache2.conf/). Although you could use the command line augtool with the print command on the file you wish to view to verify, this will provide you with a list of paths and their values as seen by augeas.

The prefix of /files/ is required to be appended to the absolute location of the file you wish to view or operate on. I do not know if there is any other sort of context that can be used with augeas.

changes

These are very similar to the syntax used by the augtool command line program. Bear in mind all changes consist of the context followed by the xpath structure of the file for example:
Directory[1]/arg "/var/www/html"
when setting the first
<directory "/var/www/html"></directory>
value in the apache2 config file and will end up being:
set /files/etc/apache2/apache2.conf/Directory[1]/arg "/var/www/html" in augeas.

A useful tip is to make use of an array to make multiple declarations of changes or use a hiera array.
The quick tour of augeas can provide a patchy overview of the operations available, I’ve summarised a few below:

  • set is used to set the values for a particular path, best used when you want to alter a value for a path
  • ins is used to insert a new value for a particular path
  • rm is used to remove an entry
  • ls is used to list the contents of an entry, if there are children then it will list them too
  • match is used to verify a entry exists without modifying the entry

There is a good write up of making use of xpath expressions on the augeas github site, it will inform you of the different criteria you can use for searching and matching within the hierarchy of entries.

Notes on quotes in the changes string

Handling quotations within the changes string is difficult as it involves both escaping with a backslash and wrapping the text in the alternate quotation, e.g. single quotes (‘) when trying to represent double quotes and double quotes (“) when trying to use single quotes:

  • "set attribute value" will result in attribute value in the file
  • "set attribute \"'value'\"" will result in attribute 'value' in the file
  • "set attribute '\"value\"'" will result in attribute "value" in the file

This is due to having to escape puppet’s string syntax and then adjusting for the command line augeas string syntax, it isn’t pretty so be careful!