From the Agile Transformation Trenches

You took such pride in watching me break; you ain't nothin' but a heartache

Continuous integration is a way of speeding up software delivery by making sure every change is introduced without breaking the project. Or, another way of looking at it is that when the build is broken the team is notified immediately and is quickly fixed. Many groups look at breaking the build as a bad thing, something for which the offender is punished.
Photo by Tom Gill Teams that use continuous integration use a build break as a fire alarm; an indication that something has gone wrong that needs immediate attention. This instant feedback is the magic that makes CI so effective.

So is getting a CI tool and pointing it at your project all it takes to get the benefits? Not quite, but it is better than nothing. You also need a robust, reliable build process. If your build gives a lot of false alarms the team won't trust it and won't leap into action when needed. Good, automated tests are another must; without good tests you only know when your project doesn't compile, not when it is actually broken. Note that good tests don't necessarily mean total coverage; high quality tests are always preferred to a simple high coverage metric. Finally, the development process used needs to encourage small incremental changes instead of big-bang commits. If the team only checks-in code occasionally, the benefits of early detection are lost. This last element is often the hardest, as it sometimes requires a change in people on a team, not just tools.

Post title is a lyric from Heartache by Two Tons of Steel.

I was close, gettin' closer, just this far away; come to find out close only counts in horseshoes and hand grenades

Who is responsible for quality? The facile answer is QA, but that doesn't work very well in reality. Quality is the responsibility of the team as a whole, developers included. You can't test quality into a product – if everyone isn't concerned with it, it isn't going to happen. Without quality, then your product is merely okay at best, close to the goal but not quite there. In an agile world, quality is largely determined by acceptance, and stories can't get accepted until they are both coded and tested. A common question is,
Photo by LarimdaME "If QA needs to test what dev creates within the same iteration, what do the developers do at the end of the sprint?" Well, how about... testing?

Unit tests can always be improved, even when practicing test-driven development. And just because QA creates a test case doesn't mean a developer can't execute or automate it. Something our team does periodically is a multi-user test, a "pokeathon" if you will. We get the extended team (support, sales, etc. are invited as well as the engineers) together for an hour, create a conference bridge so we can all talk to one another, and start banging away on a single install of our webapp. This unscientific and undisciplined method of testing has uncovered a wealth of issues for us, improved our overall quality, and turned out to be a bit of fun at times, too. I highly recommend trying it out.

Post title is a lyric from Horseshoes & Hand Grenades by Trent Summar and the New Road Mob.

Agile Development Practices conference take-aways

I've just returned from the superb Agile Development Practices conference in Orlando, Florida.  One of the most inspiring sessions I experienced was Brian Marick's opening keynote.   It set a pretty high bar for the rest of the conference.  I'll try to give you my impressions and take-aways from Brian's talk, but I would strongly encourage you to read the original, which Brian has posted on his blog.

Looking back on the movement christened by the publication of the Agile Manifesto, Brian shared with us his perspective on how that mission statement should be enhanced and extended.  Brian focused on agile values as a way to resist the temptation to chase short-term gains at the expense of long-term benefit.  The eight values he discussed were:

• Courage
• Working Software
• Ease
• Reactive Behavior
• Fast Feedback
• Naiveté
• Visibility
• Joy

Courage is needed to overcome the many kinds of Fear that can distract us from our goal.  It allows us to question the rules, and break or change them when appropriate.  Working Software is our goal.  This means software that fulfills the needs of the people who use the software.  This implies both quality and fitness for purpose.  Brian emphasized that small increments of this goal should be produced very frequently, citing Kent Beck's advice to have no more than 15 minutes of discussion before someone makes the ideas concrete by writing code.  In all the work we do, we must strive for Ease.  Taken in an appropriate context of long-term value, this means making many small investments along the way so that our work becomes more streamlined rather than bogging down under the drag caused by allowing barnacles to accumulate.  Ultimately, this allows us to avoid Yak Shaving, but you'll have to see Brian's transcript for more on that.

A pervasive thread through the presentation was the idea of keeping software Soft.  This was especially emphasized by valuing Reactive (rather than the traditionally valued Proactive) Behavior.  This does not mean that we don't have a plan, but that we retain the Softness required to bend our plans while constantly moving forward.  Fast Feedback helps us become aware of important information that we should integrate into our plans as we react to what we learn.  The faster the feedback, the better we will do at staying on target, so we should strive for feedback on the order of minutes, such as that provided by Test Driven Development and Pair Programming. 

The value of Naiveté really amounts to what Eastern martial artists refer to as Beginner's Mind.  This is the transcendence of what we "know" about what is possible, so that we are open to succeeding with the seemingly impossible.  Visibility is the art of simply exposing the problem and letting our human nature generate a solution.  In this way we find that people can be soft like software.  When the consequences of our choices are visible, we become motivated to change, where we would have resisted exactly the same change if it was imposed on us.  Taken together, these values lead to a sense of Joy.  As David Anderson said in his closing keynote, we can create a place where we want to work, rather than have to work.  From a motivational perspective, this kind of environment captures Discretionary Effort, which is the hallmark of high performance.

At the end of his talk, Brian invited a panel of agilists to offer their thoughts and values.  This list included:

• Simplicity (J.B. Rainsberger)
• Integrity (Linda Rising)
• Questioning (Rachel Davies)
• Angst & Purpose (Jeff Patton)
• Experience (David Hussman)
• Community (Anthony Marcano)

J.B. began with a story about Simplicity that encouraged us to ask ourselves what it would take to start making money sooner.  Linda followed with admonitions to avoid Delusion—a lacking of Visibility, Courage and Ease that leads us to overlook challenging, yet often obvious, opportunities for improvement.  I interpret this as the value of Integrity—doing what we know is right even when it's not easy in the short term.  Rachel asked us to have the Courage to say "No" to taking on too much work, illustrating the value of Questioning. 

Jeff contributed the value of Angst—the motivating sensitivity to things that could be better, and the value of Purpose—to make sure we all understand the strategic vision that guides our tactical decisions.  David offered potential manifesto additions like "Success over Dogma" and "Description over Prescription", ultimately focusing on the value of Experience communicated through sharing our stories.  This really resonated with me as there always seems to be some nuanced understanding to be gained through listening to the struggles and triumphs of others.

Anthony concluded with the value of Community—a kind a Family Spirit that keeps us looking out for each others' best interests.  For me, this relates to the value of Collaboration, which is mentioned in the original manifesto, and seems central to the culture change required for successful agile adoption.  Focusing on Collaboration means replacing win-lose relationships with win-win relationships.  In asking how we all can benefit, we find the path to sustainable success.

What values do you think should be added, removed, clarified or modified?

[Special thanks to Joey McAllister of SQE for allowing me to review the tape of Brian's session and capture the comments of the panelists]

A Good Place to Start

As an Agile Evangelist for Borland, and an officer of Agile Austin, I am often asked for good references on getting started with agile.  This usually goes beyond what can be found by applying your Google-skills, although that's a good way to get a taste of agile thinking.  The best way, of course, is to learn agility from experienced practitioners.  There is nothing more effective than hands-on mentoring.  However, this question is usually asked in a context where direct full-time exposure is not available.  In this case I recommend another old-fashioned technique—reading books.  This narrows my challenge to identifying which books, from the dozens on my bookshelf, to recommend.  Many are excellent treatments of particular aspects of agility.  Some are particularly relevant to specific circumstances.  Over time some newer titles rise above their predecessors.  There are a few that I find myself recommending more frequently as striking an especially good balance between breadth of coverage and depth of details.  I include four such examples below.  Each addresses a somewhat different, although not necessarily distinct, audience.  I leave it to you to self-select which may be right for you.

Agile & Iterative Development (A Manager's Guide) by Craig Larman provides an excellent survey of agile methods and practices.  It is organized for easy self-directed exploration, with extensive margin references to sections that expand on topics mentioned.  It includes an excellent section on the historical foundations of our current software development practices that leads to building a solid business case for agile methods.  About half of the text is devoted to comparative case-studies of Scrum, Extreme Programming, Unified Process and Evo.  This provides a fairly objective overview of each process.  The book concludes with agile practice tips and frequently asked questions (with answers) common to the whole family of agile and iterative development methods.

Implementing Lean Software Development (From Concept to Cash) by Mary and Tom Poppendieck takes a principle-based approach.  The seven principles of Lean Software Development are; Eliminate Waste, Build Quality In, Create Knowledge, Defer Commitment, Deliver Fast, Respect People and Optimize the Whole.  Each of these principles is discussed in detail.  The theoretical foundations are supplemented by real-world examples, case studies and experience reports.  Each section ends with a list of concrete exercises called "Try This", which invites you to put your knowledge into practice.  This book is an excellent way for agile leaders focus on the most significant factors for success.

The Art of Agile Development by James Shore and Shane Warden is an excellent guide for the agile practitioner.  It consolidates years of experience with agile practices based primarily on Extreme Programming (XP), which has long been considered a good set of engineering practices to compliment the management practices of Scrum.  The specific practices are well organized, clearly explained, extensively cross-references and presenting in an appealing "pattern language" format.  The "Getting Started" section provides a solid foundation for understanding agile mechanics and even deciding if it is right for your situation.  The "Mastering Agility" section gives a good description of the values and principles that make agile work.  This makes a good concluding section because it provides a good roadmap for developing more agile maturity once you've had some success applying the specific practices.

Agile Software Development (The Cooperative Game) 2nd Edition by Alistair Cockburn is a tour de force describing the working principles of agility.  Interviews and analysis of dozens of agile projects is distilled into a series of powerful observations and recommendations.  This is the book you need to guide the tailoring of agile methods to your particular environment and circumstances.  Many  inexperienced teams lose much of the value of their agile methods by adapting them in inappropriate ways before they understand the subtle interactions between various agile practices.  This book helps you to stay in the "safe zone" or the "sweet spot" when you are starting out, and to get the more effectiveness from your ongoing process improvements as you gain more experience and maturity.

These references should give you a good place to start.  Then you'll have to put what you've read into practice.  That's where I've found that the real learning begins.