Agile methods on software development are very popular. The root of these methods must be sought in the on the Agile Manifesto that says:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
In short, these lines says that the world is always changing, the adaptation to change, the collaboration between people and the transfer of knowledge should be a priority in our professional life.
These objectives fit with software development but also with research because, we software developers and researchers are always looking for new challenges, always updating our professional network and making improvement in science.
The scrum methodology is the most popular of the Agile methods in software development. Like others agile methods, Scrum focus on quick adaptability to changes on the developed products and prefer face to face communications between the members of the team and the customers (from the Agile Manifesto), but also, scrum declares some formal activities, roles on the development team and tools that must be used with that method.
Scrum declares the following activities:
- Sprint planning
- Daily meeting
Scrum also declares the following team roles:
- Scrum master
- Research owner (or in general Product owner)
And finally exposes these tools:
- Research backlog (or in general Product backlog)
- Sprint backlog
- Burndown chart
Scrum defines time periods called Sprint where the team will develop portions of their research (or in general products). On development teams, the work is something deliverable to the client. In a research environment, the final work of a sprint will be something that can be started and finished in the period and something that gives an added value to the research.
In general, the tasks to be done (in software development or research) will be listed in the Research backlog (or in general Product backlog). The list must be ordered in descending order of priority. Important tasks must be done first.
At that point, you must define what is a task on your research. In our experience tasks is something that can be started and finished in a sprint. In a normal case, a full paper is not possible to write and review in a sprint, but parts of it can be done and finished in a sprint.
Examples of tasks can be:
- writing the introduction of a paper
- making a review of the literature of a theme
- write a short blog entry
- writing a questionnaire
- preparing an experiment
Prior to starting a sprint, the Research team will do a meeting in order to plan the work to do in the sprint (Sprint planning). From the Research backlog, the team will select the tasks to be done (those tasks with high priority). After selection the appropriate number of tasks to be done, these tasks must be moved to the Sprint backlog. Here appears the problem of measuring the work to be done. The work (or effort )to be done depends on the capacity of the team, the team size, the difficulty of the tasks and the sprint period. It is always related to the daily work capacity of each team member. Sometimes, effort or work equals to hours of work, but it always depends on how you want to measure your effort.
In our case and for simplicity, we measure our effort in hours. So, for instance, writing a short entry on a blog could be 8 working hours.
Once the tasks are in the Sprint planning, the sprint can start. A typical period for a sprint is 2 or 3 weeks. These short periods let the Research team to add tasks to the Research backlog and change the priority of the already existents tasks. This is what the Agile manifesto call ‘adaptation to change’.
When the sprint is over, the finished work can be released and everybody on the Research team will show their work to the others members on the Review meeting.
Just after the meeting, and this is not a mandatory task, another meeting called Retrospective can be done. On the Retrospective meeting, each team member talks about the difficulties he had faced during the sprint, how he/she deal with it and how to avoid these difficulties in the future.
The daily work is the second important feature of the scrum methodology. On the daily meeting, each team member explains to their fellows what he did last day, what he is going to do the current day and what were the problems the member faced the previous day. With that information, the member can update the Sprint board and the Spring burndown chart.
The sprint board and the burndown chart are something optional but help the team know how they are performing. The Sprint board show the state of the tasks that must be done in the sprint.
Typical scrum boards from software developers have the following states (or a combination of them):
- Todo or Backlog
- Plan or Sprint backlog
- Develop / test / deploy or “In progress”
In our case, a Research team with only research activities and some software development, there are only 3 states: todo, doing, done. Every task in our sprint can be categorized in these states. Of course, every task will achieve the 3 states till being categorized as ‘done’ that is the final state of our board.
The Sprint burndown chart shows how was the work estimation and how we are performing against that estimation. In short, each day a quantity of work must be done and the chart shows if the accumulated work until the current day is done, something is left or more work than the estimated work have been done.
The improvement on the research is clear: the work to be done and their priority is something that must be done prior to start working. This is one of the main problems of research: tasks and priority are not always clear.
The estimated effort (or work) to be done in a period can be estimated without compromising other tasks. Tasks are ordered by their importance, so it helps to focus in a concrete part of the research.
The research and the team can be tracked because is easy to know what has been done, who did the tasks and what remains to be done.
Scrum is a methodology that any research team can use. The researchers do not need to be physically in the same place because this methods helps the team in both aspects: improving the team and improving the individuals.
What is important to understand is that Scrum is open to change. There are no fixed rules, you can modify every aspect. So If you do not need the daily meeting (in the case of working with people on other research centers), you can do a meeting every two or three days. If the Retrospective is not needed, you can skip it… Everything can be changed and adapted to our needs.
Scrum is interesting when tracking different PhD students. Although they do not work on the same research, this methodology helps the tutors to know in a quick way on what have been working every student and how the are performing.
If you want to read more, check out the following references:
I have always been thinking about this and some times, I apply parts of this methodologies in my own research. The challenge that I faced is that some sprint will require you to give a significant amount of time for documentation with the goal that you can write a paper to communicate your results at the end. This conflicts with “Working software over comprehensive documentation”.
Great post! I was waiting for someone to promote this idea.
One of the challenges with the Agile Manifesto is that it requires translation to fields other than software development, and that includes translation to software research. “Working software over comprehensive documentation” means “Something deliverable-that-can-receive-user-feedback over a description-of-what-you-did.”
So, one sprint should be, ideally, a complete paper that might be imperfect, but could be submitted to a journal or conference. The Scrum Guide calls this a “potentially releasable product increment.” You could submit it to a conference, if you are out of time, and then move on to other work. I.e., you are agile, literally speaking.
The funny thing about this is that the software is definitely not a releasable product increment, because your product is research that other users can read, and which helps them advance beyond what you have done. It’s easy to understand how folks can get confused by the Agile Manifesto, since it was designed for commercial software products. Even the part about “customer collaboration over contract negotiation” is a phrase that applies directly to the authors of the Agile Manifesto, who were mostly contractors; you have to do some translation to understand how it applies to full-time employees.
If you are really interested in applying Scrum to research, the Scrum Guide is better written for this purpose (I do not believe it even mentions software). I would be flattered if you might read and provide feedback on my Agile Base Patterns, which take a broad view of agility.