Agile Update

We completed our third Agile iteration last week. Each iteration we've done has been a bit different from the other. Our first goal is to simply learn to iterate. It sounds so simple, right? Just agree on a set of work to try to complete within a 2 week span and get on with it. For the most part, we have been successful at a basic level. Each iteration has started on a Monday and ended the following Friday. However, some observations are worth mentioning.

First, the background of the team was minimally controlled anarchy. It was an approach that has taken them very far. For most of the past history the team has been very small. We have grown the team by 2 or 3 new members over the last several months. The total is 9 developers, 2 test engineers and 1 technical writer. It is a good-size team for experimenting with Agile.

Before starting the first iteration we had trialed and adopted Rally as an Agile project management tool. Having this tool in place made it easier in many respects to be more successful that if we had not done that. For one thing, it allows us to be a "little" Agile in that we actually work more like an iterative development team with very short iterations than a true Scrum or XP approach. Adopting one or both of those approaches would have been a pretty radical culture shift for the team.

The first iteration felt a little frazzled. Some team members were stressed about getting everything done in 2 weeks, even though we had carefully scoped the work to fit. The additional urgency around actually committing to being done at the end of two weeks added energy to the team. Also, there were lots of questions about how to handle process-related things that came up. We learned a lot about Rally and how to use it to enable more efficient communication paths.

The second iteration felt a little more natural. We accepted a very high percentage of the work we had set out to accomplish. There were many less questions along the way and we adapted some of the practices we tried in the first iteration to be more natural in the second.

The third iteration was completely different. First, we started with much more committed work. We ended up only accepting about 45% of the work by the end of the iteration. That was good in that we learned that we can push work if it is not done and not feel too badly about it. Our average productivity over the 3 iterations was respectable. Pushing so much work into the next iteration renewed a commitment toward really being "done" with a story and not just chopping it up to make it fit within an iteration.

We had our first retrospective this past Monday at the beginning of the fourth iteration. The comments from the team were focused on how we get better at doing iterations. One common theme was the desire to do better at transtioning from one iteration to the next. Suggestions for a demo/planning day at the end of the iteration and more time discussing the stories at the start of an iteration were voiced. This will probably be the one thing we work on in this current iteration.

At this point our focus is on using iterations as a synchronization mechanism. We are not having daily standups (although we do send daily email status reports to the entire team with the same format as Agile standups). We do not do a good job of grooming the backlog and having stories fleshed out before we start the iteration. We are using an adjustment in "points estimation" that couples us a little too closely to concrete ideal time estimates. We also have to react to changes mid-iteration due to customer demands or whims within the company.

These are all things that need to be addressed over time if we are successful at getting real buy-in from other departments in the company.

Retrospect: First Agile Iteration

Friday marked the end of the first iteration for the team. We finished respectfully, completing about 84 points of work with 82% story acceptance. These metrics will likely be adjusted Monday morning as we evaluate the status of the last few items to see what might actually be finished in the Monday morning build. The Monday build is the official Iteration 2009.2 build and should be archived for posterity.

Now that we have the first one out of the way, we have a start toward measurement. We can use the points completed in the iteration as a first estimate of how much work the team can finish in a 2 week period. We will get data for about 3 iterations and start averaging the number of points completed to get a baseline team velocity.

There were several learning moments in the first iteration. First, we did not do a great job of defining stories and acceptance criteria. While stories were created for bags of work, and some acceptance criteria was included, we did not create precise expectations around the format or the granularity of stories. Several team members are still struggling with the concept, since there is no such thing as a story template that is universally accepted. Stories are varied and customizable to the particular circumstances.

We also had questions around when the actual finish date for the iteration should be. Since the iteration ends at COB on Friday, many developers talked about needing to be done coding by COB Thursday of the last iteration week. This allows a day to fix any defects found by the test group on the last day of the iteration. Then, the official iteration build is the one created on the following Monday morning.

This week we do it all over again making incremental improvements on what we learned the last two weeks. We will have the team planning meeting tomorrow morning and cover the standard format for story description in Rally. This will include more precise information about what the story should do. We'll do a better job of having acceptance criteria meetings with sub-groups. Here we will refine the description of the story and add test cases as we discuss it. We will also do a better job of hourly estimates and individual capacity so that more of our tracking views will be accurate this time around.

Raising the Urgency Level

I just started using Agile methods in my job of running a software development team. We started our first sprint (iteration) Monday. We are not "both feet in" and only a few days have elapsed, but I already see a difference in the team's sense of urgency and energy level. We are using Rally, an Agile project management tool, to help us run the project. Our first iteration size is 2 weeks. The team was already a reasonably high-productivity team since, in general, everyone on the team is committed to getting things done the right way. What was missing was a little more structure and focus.

We met Monday morning to agree on the list of Stories (collections of tasks) for the iteration and make general estimates of the size of each. By the end of the day most team members had broken down the stories into multiple tasks with concrete estimates in hours for each one. Rally rolls up all hours estimated in tasks into the Story level and provides various views including full iteration and team member allocation. We met the next morning to make any adjustments, agree that the amount of work to accomplish in the 2 weeks seems to fit and clarify any other concepts that were fuzzy.

The team is new to Agile thinking for the most part. As manager of the team I mostly cover the role of ScrumMaster (to use a Scrum term). Scrum is a specific methodology of Agile. When I say that we are not "all in" I mean that we do not do everything recommended in the Agile literature. For example, we use email for daily status instead of daily face-to-face standups. This was a practice already in place when I arrived on the scene late last year. It seems to work well, and we have tweaked it a bit and critiqued it recently to try to make it better.

I subscribe to the method of gradually going Agile. I believe that if you have a team that is already flexible and productive, moving to Agile is a natural evolution. Not being burdened with a heavy Waterfall culture is a real benefit. A small-company environment where there is already fundamental trust and a little bit of development anarchy is much more of a blank slate in which to imprint a particular process, and Agile methods are a natural step.

There is a lot of information that preaches that Agile will fail unless undertaken under the watchful care of expert consultants and training programs. A lot of this information is produced by companies who make their living creating Agile development tools. I am not heeding that advice, and instead trusting that I know what I am doing from reading and leveraging over 25 years of experience developing and managing software teams in a variety of business circumstances and processes.

I may continue to post as the experience unfolds to record the pros, cons and results of this experiment.