Planview - The Product Pulse | Portfolio Management and Agile Programming

When Kent Beck first articulated the principles of agile programming, he spearheaded the greatest change in software methodology in the last twenty years, and therefore perhaps of all time in such a young art.

As I cannot hope to improve upon his exposition of the synergetic principles of Extreme Programming, I would like to explain how agile programming relates to Portfolio Management and in particular to the critical need to give executives transparency into the status of a software project.

As Pat Durbin and Terry Doerscher write in Chapter 2 of their recent book Taming Change with Portfolio Management under the heading "The Elusive Nature of Knowledge Work":

Knowledge work is difficult to plan, estimate, and complete. Unlike physical work such as construction or component assembly, knowledge-based activities are often one-of-a-kind endeavors with no direct historical comparison for estimating purposes. Knowledge-based work also lacks discernable indicators of progress.

The concept of Velocity in Extreme Programming, or Agile Programming, is an attempt to provide a discernable indicator of progress for software projects. It answers a simple but critical question for an executive of a software company:

How close to being done are we?

Therefore it allows the more interesting question to be answered:

What is the best compromise between what we want (our demand) with what we can do in a period of time (our capacity)?

Velocity is the amount of work completed by a team in a fixed, but repeating, period of time, called a Sprint. At Planview we use two-week sprints. However, the complexity of programming tasks, called Stories, varies widely, so you cannot simply count up the tasks. Rather, you measure the complexity of a completed task in an abstract measure. At Planview we call this measure Story Points, but you could call them "complexitrons" or anything you like. Every task must be consistently estimated in these Story Points.

If you measure the number of Story Points completed in several sprints by a particular team, you have the best measure yet devised for predicting how much work will be successfully completed in the next Sprint by that team. If you have all tasks estimated in Story Points, you can predict, with reasonable accuracy, when the project will be done. To an executive, portfolio manager, or project manager, this transparency enables good decision-making.

There are, however, some subtle keys to Estimation, Story Points, and Velocity that experience has taught us:

The whole team participates in estimation.
The estimate of a story must never change once it is decided. Time may show that the estimate is wrong, but to change the estimate is like telling a debtor he owes twice as much as previously agreed for no reason.
A story may be split into smaller stories, but the Story Points, like money and energy and momentum, must be conserved.
Estimates made quickly, relying on the ineffable thoughts of a skilled programmer, are about as good as detailed estimates.
Expect velocity in any given sprint to vary about 40% around a multi-sprint average.
Story writing is an art, but it is not a black art. A team and their product manager should be able to write 15 reasonable stories in an hour. Perfection is no better than "good enough"-ness. Do not allow the perfect to become the enemy of the good.
When programmers disagree on estimates, resolve the disagreement by consensus and discussion.
"Done" means "Releasable". A task is not done until it is releasable. You may not be willing to ship a single story, but if that story is not done to the level of quality your customers demand, it just doesn’t count

Comments

Hi Rob,
Im would have to challenge you on the use of complexity as a measure when estimating story points. You mention that you can get an reasonably accurate understanding of when a project will be done. Complexity does not have a strong enough relationship with time to be used as a indicator of project length. Effort is the key here. The common example here is the the comparison between licking 1000 stamps and conducting heart surgury. They both are going to take a while but one is far more complicated than the other.
Other than this, your suggestions for estimation are sound.

Posted by Ginn on 08/30/10 at 12:14 AM

Dear Ginn,
Of course you are correct, except for four points that are perhaps unique to computer programming, and the reason that computer programming potentially provides mankind the greatest return on investment of any activity.

If I truly had a simple, repetitive task to do, I would automate it. That is, if I had to lick 1,000 stamps, I would build a stamp-licking machine. This is of course not so easy in the physical world, but in the world of software it is usually possible. This is what Larry Wall (http://en.wikipedia.org/wiki/Larry_Wall) describes as Laziness, one of the three great Virtues of a Programmer:

“Laziness - The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don’t have to answer so many questions about it. Hence, the first great virtue of a programmer. Also hence, this book. See also impatience and hubris.”

Additionally, Complexity tends to be a good measure of how hard something is to test, which is a major portion of the effort in any programming project. In the physical world, you build a brick wall in a week, and check that it is plumb in 15 seconds, but it is rare that code written in a week can be adequately tested in so quickly.

It is easier to reach a consensus on how complex something is than on how much effort it will take to solve. Consider math problems of the class that are sometimes referred to as “open problems”: those that have remained unsolved for some time even when the worth of solving them is generally agreed to. Such problems are “hard” in the following sense. Some genius thinks of a short proof of the solution and now the problem is no longer hard. Now that we can all see the short proof, the problem looks easy—-but before that it looked hard, perhaps impossible.

Many computer programming problems are like that. We can agree on what we might call the “Complexity” before hand, and yet we cannot agree on the effort, because we don’t really know if the problem can be solved or how. We cannot reach a consensus on the effort without investigating, but we don’t want to spend a lot of time coming up with estimates, so it makes sense to estimate based on Complexity rather than Effort.

One weakness in my argument is that “Complex” problems, which programmers tend to call “Hairy”, as opposed to “Smooth”, are more easily estimated than problems that are “Smooth and Hard”. The reason I don’t deal with very many problems that are both Simple and require a lot of effort is that I am not a researcher. Programmers such as myself who work for companies that are not doing blue-sky research should not attempt to solve problems which are so hard as to cross into the realm of research.

In short, because complex tasks can be automated, because complexity corresponds to test effort, because complexity is easier to agree upon than effort, and because most programmers (though not all, thank goodness) don’t work on simple problems that have very hard solutions, it is reasonable to use complexity as a stand-in for effort when estimating.

Posted by Robert Read on 08/30/10 at 04:29 PM