Introducing python-jenkins

Over the last 12 months I have made use of a great Python library originally written by Ken Conley in automating the various testing activities that we undertake as part of Ubuntu Development using Jenkins.

Python Jenkins provides Python bindings to the Jenkins Remote API.

I’m pleased to announce that Python Jenkins 0.2 is available in Ubuntu Oneiric Ocelot and the project has now migrated to Launchpad for bug tracking, version control and release management.

The 0.2 release includes a number of bug fixes and new methods for managing Jenkins slave node configuration remotely – this is already being used in the juju charm for Jenkins to automatically create new slave node configuration when slave node units are added to a deployed juju environment (blog posting to follow….)

A quick overview…

It’s pretty easy to get started with python-jenkins on the latest Ubuntu development release – if you are running an earlier release of Ubuntu then you can use the PPA (Natty only ATM but more to follow):

sudo apt-get install python-jenkins

The library is also published to PyPI so you can use pip if you are not running Ubuntu.

Here’s a quick example script:

#!/usr/bin/python
import jenkins
# Connect to instance - username and password are optional
j = jenkins.Jenkins('http://hostname:8080', 'username', 'password')

# Create a new job
if not j.job_exists('empty'):
    j.create_job('empty', jenkins.EMPTY_CONFIG_XML)
j.disable_job('empty')

# Copy a job
if j.job_exists('empty_copy'):
    j.delete_job('empty_copy')
j.copy_job('empty', 'empty_copy')
j.enable_job('empty_copy')

# Reconfigure an existing job and build it
j.reconfig_job('empty_copy', jenkins.RECONFIG_XML)
j.build_job('empty_copy')

# Create a slave node
if j.node_exists('test-node'):
    j.delete_node('test-node')
j.create_node('test-node')

You can find the current documentation here.

Happy automating!

Package rebuild testing using Jenkins and sbuild

I’ve been working on rebuilding the Java packages in the Ubuntu archive that depend on ‘default-jdk’ against OpenJDK 7 to see where we are likely to have issues if and when we decide to upgrade the default version of Java in Ubuntu to OpenJDK 7

You can see the results here (limited time offer only); and here are the details of how I setup this rebuild.

Setup your rebuild server

So first things first; the current development release of Ubuntu (Oneiric Ocelot) includes the most recent LTS release of Jenkins so installing its pretty easy as long as you are using this release:

sudo apt-get install jenkins


You probably want to configure Jenkins with an administrator email address and point it at a mail relay so that Jenkins can tell you about failures.

Next you need to install a few development tools to make it all hang together nicely:

sudo apt-get install ubuntu-dev-tools sbuild apt-cacher-ng

This should get you to the point where you can setup the jenkins account to use sbuild:

sudo usermod -G sbuild jenkins
sudo stop jenkins
sudo start jenkins

Create sbuild environments

I decided that I wanted to compare current build results with openjdk-6 against the rebuild with openjdk-7 so I created two sbuild environments on my server – one for openjdk-6 and one for openjdk-7:

mk-sbuild --name=oneiric-java-6 oneiric
mk-sbuild --name=oneiric-java-7 oneiric


You might have noticed that I also installed apt-cacher-ng – as alot of the dependencies that get downloaded are the same having a local apt cache makes alot of sense to speed things up; its pretty easy to point the schroots we just created at the local instance of apt-cacher-ng:

sudo su -c "echo 'Acquire::http { Proxy \"http://localhost:3142\"; };' > /var/lib/schroot/chroots/oneiric-java-6-amd64/etc/apt/apt.conf.d/02proxy"
sudo su -c "echo 'Acquire::http { Proxy \"http://localhost:3142\"; };' > /var/lib/schroot/chroots/oneiric-java-7-amd64/etc/apt/apt.conf.d/02proxy"

You will also need to setup some self signed keys to make sbuild work:

sudo sbuild-update -k

So we now have two sbuild environments setup – but we need one to point at openjdk-7 instead of openjdk-6 – I stuck a modified copy of java-common (which determines the default-jdk) in a PPA:

sudo schroot -c oneiric-java-7-amd64-source
apt-get install python-software-properties
add-apt-repository ppa:james-page/default-jdk-7

Obviously this could just as easily be an upgrade to mysql, postgresql or any other package of your choosing.

Setting up Jenkins

I split this rebuild into ‘main’ and ‘universe’ components and used a mutli-configuration project with axis for ‘package’ (containing the list of packages I wanted to rebuild) and ‘version’ (java-6, java-7); I’ve included the config.xml for one of the projects here.  You can also see the configuration here.

The build is a very simple script:

#!/bin/bash
export HOME=/var/lib/jenkins
rm -Rf *
pull-lp-source ${package}
sbuild -d oneiric-${version} -A -n ${package}*.dsc

That should do the trick; you can then submit the project for build and it will gradually churn through the packages emailing you with any failures.