From the Agile Transformation Trenches

Agile Scope: A picture and a few words

A couple of weeks ago, I was peer reviewing some work by a colleague. I was struggling to communicate what was and what was not covered by the popular agile processes  - what I call the scope of the process. For example, Scrum and eXtreme Programming essentially focus on a single small, cross-functional, development team. At a fundamental level, they do not cover larger teams or projects or organizations consisting of many small dependent teams. They say little about how customers or product owners decide on the business value and priority of the user stories or backlog items they create. Neither do they say much about working with a formal testing organization.

While there are plenty of suggestions and some popular approaches to extending Scrum and eXtreme Programming to cover multiple dependent teams, they are just that; extensions to the original core set of rules or practices. 

Anyway rather than try and explain this by waving my hands around I resorted to drawing a picture to communicate what was in my head. Although very simplistic, it seemed to help so I thought I'd share it. If I have done it correctly, you should be able to click on the image below to see it full-size (legibly in other words) in a separate, pop-up window.

The circles represent perpetual activities all running concurrently taking input off lists (or queues, backlogs, or whatever) and generating results on subsequent lists/queues/backlogs.

The green dashed box represents the scope of Scrum/XP and supporting tools like Borland TeamFocus today; sets of small independent teams working from their own backlogs.

The red dashed line represents the space where more and more organizations that have been using agile successfully for a while seem to be focusing. In these organizations individual development teams are working well. The result is that inefficiencies in working practices around the individual teams are starting to show up. This enlarged scope is closer to that covered by an approach like Feature-Driven Development (FDD) with broader demand management aspects informing the individual team product backlogs and collaboration across teams around an overall domain and technical architecture where appropriate. It is the space I see the combination of tools like Borland Team Demand and Team Focus being applicable, for example. Of course, we can't forget Borland Togetherfor recording and communicating the overall architecture as it emerges from collaborative design sessions.

Outside of the red dashed line are the activities that trigger new projects or changes to existing systems, and my colleague's interesting contention that this should all be inspired from a business process improvement perspective. He asserts that all IT development work should be about improving one or more steps in a process somewhere and if we understand our processes or our customer's business processes well enough we can use that knowledge to identify and prioritize development work. For example, Borland as a software tools vendor needs to understand the end-to-end processes involved with developing and delivering software. Identifying pain points, bottlenecks, opportunities for increased efficiency, etc,  in those processes enables us to deliver genuinely useful and easy-to-use products.

What the picture does not show explicitly is the current interest in more sophisticated agile testing with tools like the Borland Silk suite. The idea is included implicitly in the Design, Build and VERIFY activities.

Also not shown are the measurement and analytic aspects that I view as another dimension to the picture entirely.

All suggestions for a better picture would be gratefully received.

Can Obsession on Good Practice Become Waste?

One of the driving ideas behind many agile approaches is the removal of waste from traditional software processes. However, we need to be on our guard that we simply do not replace 'traditional waste' with 'agile waste'.

Obsessive Emphasis on Delivering Value

Here's a question for you. I had a recent discussion with someone on the amount of effort that some teams were spending to ensure they delivered user stories with the most business value before anything else. A release is usually a number of iterations (3 or 6 x 1 month sprints). On my backlog, I might have two user stories, A and B, with relatively similar business value. If I am spending a significant amount of effort to calculate whether A or B has the greater business value just to determine whether one is included in iteration one and the other done in iteration two, is that waste?

While, as a Product Owner, I definitely want to deliver as much business value in the next release as possible, my perspective is that I'm not going to lose sleep over whether user story A is completed in iteration 1 or 2. It is highly likely both user stories will end up being completed for the release unless I need to severely reprioritize due to some unforeseen event.

Obsessive Emphasis on Inspection and Adaption

Here's another question. Some people are talking about teams being continuously self-improving through continuous inspection and adaption. However, if the effort of the inspection is costing more than the benefit of the resulting adaption, is that waste?

An overly simplistic example to illustrate: Scrum proposes a few hours be spent at the end of each sprint on a retrospective. What went well? What could be improved? For a team of 7 each costing $Y an hour doing 2 hours of retrospective a month, the cost is of a year's retrospectives is  12 x 7 x 2 x $Y = 168 x $Y. If after a while, a year's worth of retrospectives is only producing improvements delivering an extra 100 x $Y of increased business value, then the cost of holding the retrospectives is more than the benefit they produce. To me that is waste and in those cases perhaps the retrospectives should be shortened, held less frequently or both.

To generalize, if the frequency and duration of the inspection of a process is not appropriate to the amount of process improvement it provides, then that is waste in the process and either the frequency, the duration, or both of the inspection can be reduced.

This is somewhat analogous to closed-loop control in industrial systems. Too high a sample rate is wasteful, as it does not produce a corresponding amount of improved process control.

Great Results from Sub-Optimal Parts

When working with the inventor of Feature-Driven Development, Jeff De Luca, he told me more than once to look out for effort being spent over-optimizing parts of a system that had negligible beneficial effect on the whole ... and this applied to both the software system we were building and the 'system' that was the project developing the software. In software, I had met this before and nicknamed it, Nanosecond Nobby Syndrome.

However, the same syndrome can affect agile software development proponents. To put it another way, it is only worthwhile turning up a best practice to eleven when it makes a significant difference to the process as a whole. The principle of barely-just-good-enough can apply to process improvement efforts as much as to the process itself.

The growing influence of FDD in agile and handling scarce skills in multi-team situations

Feature-Driven Development (FDD) is an agile approach that does not receive the same amount of attention as the likes of Scrum and eXtreme Programming. While Scrum and eXtreme Programming are very similar in many ways, FDD differs quite significantly to both. What is interesting … well interesting to me as an author of a book on FDD … is the gradual acceptance of ideas similar to those found in FDD as agile teams using Scrum and eXtreme Programming scale up and grow in maturity.

Parking Lot Chart

The first one I encountered was FDD’s so-called ‘parking-lot chart’ popping up in books like Agile Planning and Estimating by Mike Cohn. This is an intuitive, color-coded, snapshot view of the status of a larger, multi-team project or portfolio of projects. It communicates well to the senior management level and nicely compliments the usual release-level burn-down charts.

Architecture is not a sin

Then there is the growing acceptance that it is not necessarily an agile software development sin to have some initial, overall idea about the architecture of the software being developed before starting development iterations.  This is especially important when there are multiple teams working on producing a single product or set of integrated products.

Scarce Skills and Feature Teams

Often times a group of teams requires a specific skill for some part of their work - maybe it is knowledge of a specific API, programming language, or a technology. However, sometimes there are fewer people with that skill than there are teams making it impossible to have that skill available on each team full-time.  This is something that I have experienced recently.

One way to solve this would be for the team who needs this skill but doesn't have it present to submit a user story to another team that does have someone with that skill. This should set off much needed alarm bells because of the cross-team dependencies such an approach would create, and the delays in waiting for one team to finish work before another could complete theirs. Despite hiding behind agile terms, this approach is no different in essence to the traditional approach of raising change requests for another team.

This situation is somewhat analogous to the code ownership delays that eXtreme programming sought to eradicate by implementing the practice of collective code ownership (balanced by pair-programming and automated unit testing). However, we cannot solve the problem of a scarce skill using collective ownership if there are simply too little of that specific skill in the collective group to have enough for each team.

FDD proposed a different approach to reducing code ownership delays by recognizing that it is not code ownership per se that cause the delays, but the availability of the code owner. FDD’s simple solution was to change from rigid team composition to a more dynamic model it called ‘feature teams’. In this model, team composition changes after each iteration. A feature team is composed of the code owners required for a specific set of features (user stories) in that iteration.  

The solution to the recent skills constraint problem on my own teams has been similar. Instead of rigid Scrum teams, at the start of a sprint, those with the scarce skill can transfer to another team that requires their skill during that iteration. This changes the nature of problem. Now instead of having user stories in one team that are dependent upon user stories in another team, we have team compositions dependent on particular resources.  This can be resolved by teams coordinating sprint commitments so that only a small enough subset of the teams have work requiring that scarce skill set in any particular iteration.

You can read more about FDD at www.featuredrivendevelopment.org