At the same time, new approaches have been introduced regularly and with great fanfare. Each of them has promised to solve our software project problems; to make our software projects predictable and successful. We can't remember the names of most of these approaches because for most of us, the reality was disappointing; the pain remains.
And now along comes "Agile". Why should we pay any attention? Why should we expect this is anything more than the latest brand of snake oil?
The Agile methods are worth adopting precisely because they are not a new approach, and they do not promise to solve all of our problems. They embody a philosophy that, although it has been used in other industries, is foreign to software. And they support that philosophy with a collection of tried and true tools. As we learn to apply this philosophy to software, we can wield those tools (some of which we are already proficient with) to fix own most pressing problems.
There is no magic, and our problems will not just melt away. Rather, we will learn, incrementally, how to succeed on software projects more and more often.
Many of the lean principles are as foreign and scary to the software industry today as they were to the manufacturing industries when they were fist introduced. Doing things "just in time", looking for "wasted motion", deciding "as late as possible", then delivering "as quickly as possible" are not the norm in software.
But the Agile methods are proving that lean software development is not just possible, but it is also practical. And best of all, it provides a foundation for the rest of agility!
If we can't be perfect, our next best option is to learn -- and to learn continually. Learn to avoid repeating mistakes. Learn to make successes routine. Learn to replace imprecise or wrong-headed ideas about what we are working to produce with an accurate vision.
Agility is all about learning. The Agile methods are designed to accelerate learning, and then to capitalize on that learning. They encourage learning, reward learning, and embrace what is learned (even when we learn that we made a mistake).
Agility applies the lean principles discussed above by breaking our project into a series of micro-projects (iterations) of about a month in length. Each of these iterations delivers a little bit of the promised product to our customer for acceptance. Each increment represents a relatively small investment, and allows our customer to validate what we are doing, or to correct us where we have strayed. This helps the development team to learn and mature their vision of the product we are building. At the same time, it allows them to correct any errors when they are cheapest to fix (right after they were made)!
But wait! That's not the only learning that this practice encourages. Our customers are also learning about their own needs. With each increment of the product, they are refining their ideas and discovering what they really needed in the first place.
While traditional approaches brand this learning as "bad" (using phrases like "scope-creep" and "requirements churn"), Agility allows the project to welcome these changes because of the Lean principles. Requirements and designs are elaborated "just in time", so the changes cause much less disruption than with traditional approaches.
It should also be noted that requirements changes from outside of the project (regulatory changes and market shifts) can also be readily accommodated for the same reasons!
Agility means applying Lean principles in this realm as well, by doing "just in time" planning. Dividing the project into a series of mini-projects allows us to defer detailed planning until it is needed. Our course-grained plan for the whole project (based on our course-grained understanding of it) provides a framework and a time-box for the work that will be done. But the details are planned when they are needed, just in time.
This Lean approach accommodates our learning about our project. Rather than fighting against what we have learned because it differs from our plan, we design each new increment of our plan to capitalize on what we have learned so far and to bring us one step closer to a successful project.
There is no such thing as "corrective action" in Agile projects. Agility focuses on adapting the project to what we have learned, not on forcing the project to conform to preconceived notions. When things unfold differently from what we expected, we welcome this new information about the project and exploit it.
Rather than asking, "How can we get back to where we should be (according to our plan)?" we ask, "How can we get from where we are to our final goal?" The difference between those two questions is far more than semantics. The difference is a difference in focus: focus on our plans vs. focus on our goals.
Agility is generative rather than prescriptive. It doesn't tell us the "right" things to do; rather, it leads us to discover what is right for our organization given our context and challenges. This is achieved through regular retrospectives.
Because we break our big project into a series of mini-projects, Agility's focus on learning naturally leads us to end each of them with a mini-lessons-learned. As with any retrospective, we ask about what is working well and what is not with the intent of improving how we work. The difference is that we are doing this multiple times throughout the project, and using the information to incrementally improve our effectiveness.
An Agile team is not defined by the practices they adopt. They define their practices!
Collaborate!
While developers can become somewhat agile on their own, the real benefits of agility come when their customers and managers collaborate with them. When we all embrace the philosophies that under gird Agility, we all benefit.
Customers can collaborate by embracing incremental requirements development and incremental product delivery. We must resist the urge to require a fully developed, gold-plated requirements specification early in the project, and instead articulate our vision for the product. Then we can collaborate with the developers in incrementally converting that vision into the product that will best serve our needs. As both the developers and we learn about the product, we must remain active, re-prioritizing our needs so that the developers deliver the most valuable features in each increment of the product.
Managers can collaborate by embracing incremental planning. We must resist the urge to require exhaustive planning early in the project, and instead ask for a course-grained plan for the whole project, then expect that plan to evolve as the project's unknowns are answered. We should expect detailed plans for only the current increment of the project (about a month), and we should help our developers to become expert at planning and delivering on their promises in that short timeframe.
Developers, of course, must be given the tools, training and coaching that they need to adopt Agile philosophies and become expert with the appropriate practices. Then developers, customers and managers collaborating together will be able to ensure that the most value software that is possible is delivered to the customer within the project's time and budget constraints.