Having automated the Ubuntu Server ISO testing process during the Maverick release cycle with Mathias Gug, I was keen to apply a similar automated testing approach to the Ubuntu Server AMI’s for Amazon ec2 this release cycle.
Scott Moser has done some great work on automating this previously using a small framework of scripts. We discussed refactoring to use Python and boto to make the testing a bit easier to maintain and extend going forwards. This also allows re-use the existing Python unittest + subunit approach from the ISO testing project for testing each ec2 instance as it was started.
An initial test release of the code is available for use; you are going to need an AWS account to use this codebase and be aware that running large tests does cost quite alot of $$/££ so be careful!
So the first thing you will need todo is install the ubuntu-server-ec2-testing package; this is available in a PPA and is currently only available for the Natty release:
sudo add-apt-repository ppa:ubuntu-server-ec2-testing-dev/testing
sudo apt-get update
sudo apt-get install ubuntu-server-ec2-testing
This installs the base ec2 testing package onto your machine.
Grab some test cases
Test case definitions and configurations are stored in a bzr branch on Launchpad; this allows configuration and the test itself to be managed outside of the packaged code.
bzr branch lp:~ubuntu-server-ec2-testing-dev/+junk/ec2-automated-tests tests
Take a look in the tests directory this creates:
- the default folder contains the test configuration for the ‘default’ test. This includes the cloud-init configuration that will be injected into each ec2 instance and the Python unittest file which tests the instance.
- *.yaml – these are example test suite configuration files; in this example we will use the test-micro.yaml example file
Setup your ec2 credentials
To access ec2 a key and secret is required; this is stored in another yaml file (~/.ec2.yaml):
AWS_ACCESS_KEY_ID: your access key id
AWS_SECRET_ACCESS_KEY: secret associated with key
Credentials can be setup from your account screens in Amazon Web Services.
Run a test suite
The ubuntu-server-ec2-testing package provides two scripts to execute tests:
- run-ubuntu-ec2-test: runs a single test case; run without any parameters for help.
- run-ubuntu-ec2-multi-test: runs a suite of test cases for multiple sizes, root storage options, regions and availability zones.
So in this example we are going to run the test-micro.yaml test suite. The format should be fairly intuitive:
This suite is a) for maverick b) for the server (desktops later!) c) will run 1 instance of each TestCase d) will run the i386 image on a t1.micro e) in eu-west-1a and eu-west-1b and f) using ebs root filesystem storage. In total this will cost 2 x t1.micro hours (one for each instance).
To execute this test (against the latest maverick published images using the default test):
run-ubuntu-ec2-multi-test -t tests -f tests/test-micro.yaml default
After a short period of time (about 5 minutes) the test suite should have completed. In the current working directory you will find test results (in JUnit XML format) and console output under the eu-west-1a and eu-west-1b folders.
Part 2: integration with Jenkins to follow….