The engineers behind ingenta connect are about to launch into the second iteration of the development which will deliver Connect Compilations. Connect Compilations give publishers the ability to create "virtual" online journals containing articles from any of their publications on ingentaconnect.

Connect Compilations are being developed using an agile development methodology called Scrum. Scrum is a relatively new to us, and so I wanted to share our experiences with it. Prior to the adoption of Scrum we used the best parts of XP and agile methodologies, but did not tie them together formally with an off the shelf process.

What is agile?
Agile development methods are distinguished from other methodologies by a number of traits, but unusually for the software business the name 'agile' is quite expressive. The freeDictionary defines agile thus:
Characterized by quickness, lightness, and ease of movement; nimble.

The lightness and nimble parts of the definition are crucial. Agile methodologies aim to provide development teams a way to react to, or even embrace, change in a rapid but controlled fashion. In other words they need to remain nimble, such that when a change occurs development does not just stop. This is achieved by a tight cycle of software releases, customer review, and revision. This cycle is termed an iteration.

Lightness is another desirable attribute of an agile process: lightness referring to the weight, or effort, needed to adopt and use a process. I wouldn’t recommend this, but if you were to print out the scrum process and rational unified process documentation and put the piles of paper side by side, you’d get a another perspective on agile’s lightness. In general engineers just want to get stuck into the technical elements of a project, and the business would rather they were doing just that, bringing a rapid return on investment, rather than spending long periods learning and interpreting process.

What is different about scrum?
Scrum uses common agile principles, such as continuous integration, short iterations and close customer liaison. It distinguishes itself through a tight integration of development and project management practice.

  • Empirical Management – Rather than trying to steer development according to a long-term plan, progress is tracked, observations are taken and adjustments made accordingly.

  • Self Organised Teams – Although teams are assembled from a selection of people who are likely to play certain roles (developer, tester, leader, architect and such like) the members are expected to organise themselves, rather like the skunkworks concept.

  • Focused Teams – Scrum demands that the development team are allowed to focus on their goals almost exclusively.

  • Team Buy In – The development team create the estimates, and agree the goals they can reach in the time given. This fosters commitment to the project’s goals.

  • Fixed Time – The time element of the development is fixed. The team aims to deliver against its goals by an agreed date. The date cannot slip, bringing it sharply into focus.

  • Prioritised Goals – The team’s goals are listed in a ‘sprint backlog’, the list is prioritised, with an estimated duration associated with each task. There is an overriding goal to create a usable product at the end of each sprint.

Scrum Terminology
Like all good paradigms Scrum is overflowing with jargon and 'in jokes', some of the key terms needed to back the concepts include:

  • Scrum Master – The scrum master is responsible for ensuring the team and the clients play by the rules of Scrum. A crucial part of this role is managing demands on the development team such that they can focus exclusively on the sprint goals.

  • Sprint – A sprint is the fixed time the team have to achieve their goals. As the name implies it’s a period of intense activity with a clear finish time.

  • Sprint Backlog – The prioritised list of goals is termed a backlog, at the start of the project the team can see what they need to achieve in the sprint and how long they have to do so. It is detailed; no task is longer than 14 hours. The backlog is updated daily, so any interested party may accurately track progress.

  • Product Backlog - The sprint backlog is derived from a larger master list – the Product Backlog, this is a continually updated list of prioritised work, in our case encompassing the whole of the ingentaconnect site.

Why is Scrum a good fit?
So why did we choose scrum from the plethora of processes available? For one thing it was close to what we were doing, or had recognised a need to do. It fits well with a rapidly changing business, and a dynamic team structure. Scrum also works well for both the engineering teams – who are able to concentrate on technical aspects - and the business who are able to closely monitor progress.

Why is the second iteration is important?
Entering the second iteration is a crucial milestone, but it was by no means guaranteed that we would reach it; to get this far scrum had to ‘make the grade’. It had to work for engineers and for their managers. The second iteration is also crucial because it represents and evolutionary step, it draws on positive and negative experiences in the first iteration, promoting the former and reducing the later. The first iteration is partly a learning exercise. The second iteration is where we start to see the true value of the process, and in turn that value should be born out when Connect Compilations are released.

Our foray into Scrum has been supported by frequent reference to the book Agile Software Development With Scrum. I’d recommend it to anyone considering Scrum. Process documentation is generally a dry subject but the authors manage to make it quite readable.