Parallelization and Automation of Drizzle CI infrastructure

This post is all about my work for Google Summer of Code 2013. Drizzle Continuous Integration uses Jenkins CI tool at present. Drizzle had dedicated servers which ran Jenkins server / master with a few slaves. However, the infrastructure did not support parallel running of tests / builds to a scalable extend. So the project aims in bringing parallelism with automation.This can be envisioned as follows.

Drizzle CI does builds using Jenkins. It tests for regression using the available test suites. To bring in parallelism, the CI infrastructure executes  test jobs, which may include a build, a sysbench test, a randgen test, etc., simultaneously. A dedicated system / cloud node is configured for execution of each job in the job queue. Multiple tests + Multiple nodes = Parallelism.

Configuring these nodes is handled in a simple way. It is as simple as writing a config file of specified format, and then issuing a command which configures the nodes accordingly. Cloud nodes + Configuration files = Automated configuration.

To put it in a nut shell, the first work would be to automate the setting up of cloud nodes, and the later half would be to parallelize the test runner. In the following posts, I will be updating about the progress and write about the tools which are involved in the work, how to use them, etc. 🙂