Installing Puppet 3 in a BeagleBone or Raspberry Pi

Puppet Labs logoI am a proud owner of a NinjaBlocks device that I use to control my home (blinds, hot water, heater, presence detection,…), welcome to the Internet of Things! But that’s a story for next posts, the important thing is that the device is actually a BeagleBone board running Ubuntu connected to an Arduino board.

raspberry logo

I thought, what would better than managing the NinjaBlock ubuntu with Puppet? there are a number of files and services I added there and it’d be nice to have puppet installed. But the official Ubuntu packages only offer puppet 2.7.11, so I installed the PuppetLabs repository package and tried to install Puppet 3, failing miserably because there is no Facter build for armhf platform,  the same one used in Raspbian.

sudo dpkg -i puppetlabs-release-precise.deb
sudo apt-get update
sudo apt-get install puppet

and got this

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
puppet : Depends: puppet-common (= 3.4.2-1puppetlabs1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

When I tried to install all the required packages

sudo apt-get install puppet puppet-common facter

then I found the actual error

The following packages have unmet dependencies:
facter : Depends: dmidecode but it is not installable
E: Unable to correct problems, you have held broken packages.

Investigating a bit I found that facter has a dependency on dmidecode that should be optional, as dmidecode is not available for arm platform and is not really needed for facter.

The solution? Rebuild the facter package removing that dependency, easily done with this script. When vi opens just delete the dmidecode dependency and you will get the fixed facter_1.6.18-1puppetlabs1_all.modified.deb

curl -O
curl -O
bash facter_1.6.18-1puppetlabs1_all.deb
# remove dependency to dmidecode in the editor that opens
sudo dpkg -i facter_1.6.18-1puppetlabs1_all.modified.deb
# If you want to use ruby 1.9 instead of 1.8
sudo apt-get install libaugeas-ruby1.9.1 ruby1.9.1

# puppet may not install if all the dependencies are not listed
sudo apt-get install puppet puppet-common
# mark dependencies as automatically installed so they are removed when removing puppet
sudo apt-mark auto facter libaugeas-ruby1.9.1 puppet-common

Installing RVM and multiple ruby versions with Puppet

rvm_logoWith the latest version of the Puppet RVM module it is even easier to install multiple versions of Ruby

# install rubies from binaries
Rvm_system_ruby {
  ensure     => present,
  build_opts => ['--binary'],

# ensure rvm doesn't timeout finding binary rubies
# the umask line is the default content when installing rvm if file does not exist
file { '/etc/rvmrc':
  content => 'umask u=rwx,g=rwx,o=rx
                     export rvm_max_time_flag=20',
  mode    => '0664',
  before  => Class['rvm'],

class { 'rvm': }
rvm::system_user { 'vagrant': }
rvm_system_ruby {
    default_use => true;

Hiera can also be used to define what rubies to install, making the Puppet code even less verbose

class { 'rvm': }
# rvm::system_user no longer needed
# rvm_system_ruby no longer needed

The equivalent hiera yaml configuration to the previous example

    default_use: true
  '2.0': {}
  'jruby-1.7': {}

  - vagrant

Continuous Delivery with Maven, Puppet and Tomcat – Video from ApacheCon NA 2013

Apachecon NA 2013A little bit late but finally the video from my session at ApacheCon Portland is available. That was the first version of the talk that I just gave at Agile testing Days which unfortunately was not recorded.

Continuous Integration, with Apache Continuum or Jenkins, can be extended to fully manage deployments and production environments, running in Tomcat for instance, in a full Continuous Delivery cycle using infrastructure-as-code tools like Puppet, allowing to manage multiple servers and their configurations.

Puppet is an infrastructure-as-code tool that allows easy and automated provisioning of servers, defining the packages, configuration, services,… in code. Enabling DevOps culture, tools like Puppet help drive Agile development all the way to operations and systems administration, and along with continuous integration tools like Apache Continuum or Jenkins, it is a key piece to accomplish repeatability and continuous delivery, automating the operations side during development, QA or production, and enabling testing of systems configuration.

Traditionally a field for system administrators, Puppet can empower developers, allowing both to collaborate coding the infrastructure needed for their developments, whether it runs in hardware, virtual machines or cloud. Developers and sysadmins can define what JDK version must be installed, application server, version, configuration files, war and jar files,… and easily make changes that propagate across all nodes.

Using Vagrant, a command line automation layer for VirtualBox, they can also spin off virtual machines in their local box, easily from scratch with the same configuration as production servers, do development or testing and tear them down afterwards.

We will show how to install and manage Puppet nodes with JDK, multiple Tomcat instances with installed web applications, database, configuration files and all the supporting services. Including getting up and running with Vagrant and VirtualBox for quickstart and Puppet experiments, as well as setting up automated testing of the Puppet code.

Infrastructure testing with Jenkins, Puppet and Vagrant at Agile Testing Days

agiletdThis week I’m in Postdam/Berlin giving a talk Infrastructure testing with Jenkins, Puppet and Vagrant at Agile Testing Days. Showing examples of using Puppet, Vagrant and other tools to implement a source code to production continuous delivery cycle.

Slides are up in SlideShare, and source code is available at GitHub.

Extend Continuous Integration to automatically test your infrastructure.

Continuous Integration can be extended to test deployments and production environments, in a Continuous Delivery cycle, using infrastructure-as-code tools like Puppet, allowing to manage multiple servers and their configurations, and test the infrastructure the same way continuous integration tools do with developers’ code.

Puppet is an infrastructure-as-code tool that allows easy and automated provisioning of servers, defining the packages, configuration, services, … in code. Enabling DevOps culture, tools like Puppet help drive Agile development all the way to operations and systems administration, and along with continuous integration tools like Jenkins, it is a key piece to accomplish repeatability and continuous delivery, automating the operations side during development, QA or production, and enabling testing of systems configuration.

Using Vagrant, a command line automation layer for VirtualBox, we can easily spin off virtual machines with the same configuration as production servers, run our test suite, and tear them down afterwards.

We will show how to set up automated testing of an application and associated infrastructure and configurations, creating on demand virtual machines for testing, as part of your continuous integration process.

Testing Puppet and Hiera

Puppet Labs logoAt MaestroDev we have been using Puppet 3 for a quite some time now, and one of the main reasons to upgrade from Puppet 2.x was the ability of using Hiera as a data backend for all the variables that customize the different vms. We don’t have a lot of machines but pretty much all of them have some difference so Hiera allows us to have the same manifests and modules apply to all the machines by just using different parameters in each server.

But testing Hiera is not that simple. With rspec-puppet you can test each class by passing parameters, but how can you test a class that calls another class and so on, and at some point there you need to inject a parameter?

Well, this is possible with the hiera-puppet-helper gem you can stub the hiera data backend

source ''

group :rake do
gem 'puppet'
gem 'rspec-puppet'
gem 'hiera-puppet-helper'
gem 'rake'
gem 'puppetlabs_spec_helper'
require 'puppetlabs_spec_helper/module_spec_helper'
require 'hiera-puppet-helper/rspec'
require 'hiera'
require 'puppet/indirector/hiera'

# config hiera to work with let(:hiera_data)
def hiera_stub
  config = Hiera::Config.load(hiera_config)
  config[:logger] = 'puppet' => config)

RSpec.configure do |c|
  c.mock_framework = :rspec

  c.before(:each) do
    Puppet::Indirector::Hiera.stub(:hiera => hiera_stub)


And then you can use let(:hiera_data) to inject any parameters automatically into the puppet classes from your rspec tests.

require 'spec_helper'

describe 'mymodule::myclass' do

let(:hiera_data) {{
  'mymodule::myclass::myparam' => 'myvalue'

it { should contain_class('mymodule::myclass').with_myparam('myvalue') }

Check out a full module using hiera-puppet-helper at maestro_nodes.

PuppetConf video: How to Develop Puppet Modules

How to Develop Puppet Modules. From Source to the Forge With Zero Clicks (slides)

Puppet Modules are a great way to reuse code, share your development with other people and take advantage of the hundreds of modules already available in the community. But how to create, test and publish them as easily as possible? now that infrastructure is defined as code, we need to use development best practices to build, test, deploy and use Puppet modules themselves.

Three steps for a fully automated process

  • Continuous Integration of Puppet Modules
  • Automatic release and upload to the Puppet Forge
  • Deploy to Puppet master

More about PuppetConf in my previous entry.

PuppetConf recap: How to Develop Puppet Modules. From Source to the Forge With Zero Clicks

Puppet Labs logoLast week I was at PuppetConf in San Francisco, speaking about automating the build-test-release-deploy lifecycle for Puppet modules and how we use them. Check out the PuppetLabs blog The Puppet Forge: Modules From Design to Deployment  and see below the original post at MaestroDev’s blog and some (quite unrelated) pictures I took. More conference pictures in the official Flickr account.

Last week PuppetConf took place at the Fairmont in San Francisco, gathering Puppet users and enthusiasts from all over the world for five days of training, development and sessions.

MaestroDev was present at the event as we are heavy Puppet users, and contributors! We are currently the 3rd most frequent contributor to Puppet modules in the Puppet Forge and publish 30 modules, of the 50 that we use on a day to day basis.

Our architect Carlos Sanchez gave a presentation on How to Develop Puppet Modules: From Source to the Forge With Zero Clicks, showing how to apply an automatic build-test-release-deploy cycle for Puppet Modules, because, when you use Infrastructure-as-Code you have to apply the development best practices to your infrastructure. In a demo, he showed how we use Maestro to automatically build, test, release and deploy our Puppet modules to the Puppet Forge, triggered on each commit for a truly Continuous Delivery experience.

And, not only do we use Puppet to provide module deployment to the Forge, we also integrate Maestro with Puppet and Puppet Enterprise to automate Puppet updates across your Puppet agents.  When you put these capabilities together with other development automation capabilities you achieve Continuous Delivery for both your applications and infrastructure.

Imagine you are building or releasing the latest version of your software and need to propagate an update through all the Puppet agents running, as well as any updates to config files or packages managed by Puppet. Instead of waiting for the next Puppet run to happen, Maestro can automatically deploy the Puppet manifests and modules to the Puppet master after being tested, and propagate those changes as well as the latest application built to all or a subset of the servers running with Puppet agents.

Thanks to all the Puppet Labs guys for a great event, and especially to Ryan Coleman for the work he is doing on the Forge and his help getting these Maestro integrations working.  We look forward to seeing everyone at the next event!

Binary Repository Management refcard on DZone

DZone logoBinary Repository Management refcard on DZone The people at DZone were kind enough to ask me to write a refcard on Binary Repository Management a few months ago, and it’s now available for download.

I wrote about benefits and best practices when using a repository and compare the three tools in the space: Apache Archiva, Sonatype Nexus and JFrog Artifactory.

Testing puppet modules

Puppet Labs logoThere are several steps depending on how much involved the tests are, what parts are tested and, of course, how long it takes to run the tests.

For unit testing we use rspec puppet, and we can check that our manifests and modules compile and contain the expected values. It can be used to test that specific types, classes or definitions are in the compiled catalog and that the parameters math the expectations.

Later on we can do some integration testing starting a new VM with Vagrant and checking that there are no errors in the provisioning, as well as checking that some conditions are met.

For rspec-puppet, PuppetLabs has created a project called puppetlabs_spec_helper that let’s us avoid writing a bunch of boilerplate. A missing point though is that it only allows to use modules for testing from git. If you’re already using librarian-puppet (and you should!) you can easily use the same Puppetfile for deploying modules and to test them. Doing otherwise sounds like a bit of useless testing, you could end with different versions in different development machines, CI server, puppet master,… So just add a call to librarian puppet in your rakefile to populate the rspec-puppet fixtures before running the specs.

Unfortunately rspec-puppet doesn’t work with Puppet 3.0.x and  at least Puppet 3.1.0-rc1 is required. It was a bit of a setback when we moved to Puppet 3 and started using hiera, which is proving to be very useful to have simpler manifests and external data injected for our Maestro installations with Puppet from scratch.

You can also use the same Puppetfile to start Vagrant boxes with the exact same version of the modules. We are using Cucumber and Aruba to execute vagrant, provision the VM with puppet and check several things, like open ports, services up,… but that’s a different story 🙂


In this puppet-for-java-devs project you will find the bits that showcase all these tools integrated. It includes definition of a 3-tier system with Puppet definitions for a postgresql database, tomcat nodes with a war installed and apache nodes fronting them.

Install all required gems

bundle install

Install all Puppet modules with Puppet Librarian

librarian-puppet install

Run the specs with puppet-rspec

bundle exec rake

Start all the vms with Vagrant

vagrant up


require 'bundler'
require 'rake/clean'

require 'puppetlabs_spec_helper/rake_tasks'

CLEAN.include('modules', 'spec/fixtures/', 'doc')
CLOBBER.include('.tmp', '.librarian')

task :librarian_spec_prep do
 sh "librarian-puppet install"
task :spec_prep => :librarian_spec_prep

task :default => [:spec]

Puppetfile for librarian-puppet

forge ''

mod 'puppetlabs/java', '0.1.6'
mod 'puppetlabs/apache', '0.4.0'
mod 'inkling/postgresql', '0.2.0'
mod 'puppetlabs/firewall', '0.0.4'
mod 'tomcat', :git => '', :ref => 'centos'
mod 'maestrodev/maven', '1.x'
mod 'stahnma/epel', '0.0.2'
mod 'maestrodev/avahi', '1.x'
mod 'other', :path => 'mymodules/other'

tomcat_spec.rb with rspec-puppet

require 'spec_helper'

describe '' do
  let(:facts) { {:osfamily => 'RedHat', :operatingsystem => 'CentOS', :operatingsystemrelease => 6.3} }

  it { should contain_class('java').with_distribution /openjdk/ }

  it "configure webapp" do
    should contain_maven('/srv/tomcat/appfuse/webapps/ROOT.war')
    should contain_maven('/srv/tomcat/appfuse/webapps/ROOT/WEB-INF/lib/postgresql-9.1-901.jdbc4.jar')

Vagrantfile do |config| = "CentOS-6.3-x86_64-minimal"
  config.vm.box_url = ""

  config.vm.customize ["modifyvm", :id, "--rtcuseutc", "on"] # use UTC clock

  # db server
  config.vm.define :db do |config|
    config.vm.host_name = "db.acme.local"
    config.vm.customize ["modifyvm", :id, "--name", "db"] # name for VirtualBox GUI
    config.vm.forward_port 5432, 5432 :hostonly, ""
    config.vm.provision :puppet do |puppet|
      puppet.module_path = "modules"
      puppet.manifest_file = "site.pp"

  # tomcat server
  config.vm.define :tomcat1 do |config|
    config.vm.host_name = "tomcat1.acme.local"
    config.vm.customize ["modifyvm", :id, "--name", "tomcat1"] # name for VirtualBox GUI
    config.vm.forward_port 8080, 8081 :hostonly, ""
    config.vm.provision :puppet do |puppet|
      puppet.module_path = "modules"
      puppet.manifest_file = "site.pp"

  # web server
  config.vm.define :www do |config|
    config.vm.host_name = "www.acme.local"
    config.vm.customize ["modifyvm", :id, "--name", "www"] # name for VirtualBox GUI
    config.vm.forward_port 80, 8080 :hostonly, ""
    config.vm.provision :puppet do |puppet|
      puppet.module_path = "modules"
      puppet.manifest_file = "site.pp"


Managing Puppet modules with librarian-puppet

Puppet Labs logoOnce you have your Infrastructure defined as code you need to use coding best practises.

This definitely applies to Puppet code, you have Puppet classes to encapsulate functionality, and modules that group classes together for reuse and redistribution. Modules can be reused from git repositories or from the Puppet Forge, a web archive of tarballed modules in the same fashion as rpm or maven repositories. Modules in the forge are defined by a provider, module name and version, much like Maven groupid, artifactid and version, and a provider has full control over the modules that can be published under its name, with a more open approach, as anybody can upload anything themselves through the forge website, just signup and start uploading them.

But trouble starts when you are using a number of modules from different places, as each module can have its own set of dependencies. How can the dependencies be defined, downloaded and installed automatically as happens with any other package manager?

There are several options actually

  • Git submodules. This is the poor man’s version, where you add the git repositories of other modules as git submodules and use git submodule init and update commands to stay up to date
  • Puppet module tool. Allows the installation of modules from the Puppet forge based on the module name provider/name and version. It will download and unpackage the module in the directory specified
  • Librarian-puppet. It is an extension of ruby’s bundler model to install gems, having a file that defines your dependencies, Gemfile in the case of bundler, Puppetfile in the case of librarian, and a .lock file with the resolved dependencies.

The resolution model is different than those of yum, apt-get or maven, that resolve dependencies at every run on the client, relying on the clients to consistently do that over versions and machines. In the bundler or librarian model, resolution only happens once and it’s saved to the lock file, which is used from there on by the clients until dependencies are changed.

Dependency declaration

The librarian Puppetfile allows defining module dependencies from several sources

Puppet Forge

Using the forge provider and module name, and optionally version it calls puppet module tool to fetch the tarball and extract it in the designed directory

mod "maestrodev/maven", "1.0.0"


Modules can also be cloned from a git repository optionally defining what branch, tag or revision to checkout

mod "maven",
  :git => "",
  :ref => 'v1.0.0'

Path folders

Useful mostly for testing or while transitioning to a full dependency model, local paths can be used as modules

mod 'mymodule', :path => './private/mymodule'

Improvements and issues

Librarian offers several improvements over the puppet module tool. Besides being able to bring in modules from different sources, librarian locks the versions of modules used, which is a must in order to consistently reproduce results. Let’s say you define a dependency on module maestrodev/maven > 1.0.0. If you run it now then librarian will lock the version to 1.0.0. If 1.1.0 is released later your dependencies will stay the same. If you try to run the puppet module tool with

puppet module install maestrodev-maven

you would end with 1.0.0 now and  a newer version later on.

There are some problems, and we hit (and fixed!) a lot of them. We have a fork of the original project where we keep updating the maestrodev branch while our pull requests don’t make it to the original repo. Unfortunately some of the patches have been sitting there for too long and we went ahead and created the librarian-puppet-maestrodev gem  that you can use instead of the original one.

Some of the issues you’ll likely find are the mismatch between puppet module version conventions and gem versions. librarian uses the bundler gemfile resolution mechanism which does not accept some versions such as the ones including dashes 1.0-rc1, or the puppet module dependency type 1.x which our librarian gem will adequately convert to ~>1.0 instead of failing. Check out the rubygems version policies for more details.

Next post I’ll cover how to use librarian in combination with puppetlabs spec helper to make your puppet testing much easier and consistent, and I’ll cover the topic in my upcoming talk on continuous delivery at ApacheCon Portland in February, plus you’ll find me next monday the 28th at the Los Angeles DevOps meetup, and at DevOps Days LA February 22nd as part of the SCALE conference.