Abstract

With all of the things we must worry about while planning a project, it is easy to miss the one thing that everyone expects so automatically that it goes without saying: Quality! The customer, management, even the development team expects that the software we produce will be "good" quality. But without proper attention to defining what we mean by "good" and then planning for it, achieving the levels of quality we need is far from assured.

Planning for Quality

A good friend of mine (I'll call him "Ralph" to protect the guilty) had just spent several days planning his new software project. "What a job!" he exclaimed. "But it was worth it. I'm not leaving anything up to chance this time! I've got it all planned out: what we must build, when it is needed, how much we will spend, who will do the work, even the skills and knowledge each person brings to the table! I know what is most important to the customer and how senior management will measure our work. In short, I know the definition of "success" on this project!"

"That's great!" I agreed. "What does your Quality Plan look like?" I wondered.

"The testers are putting their plans together; you'll have to ask them" he said absently as he admired his beautiful Project Plan.

"Yes", I replied, "I know they are preparing test plans. But I am asking about your quality plan. How does your plan help you to assure that you will build adequate quality into the system?"

Ralph stared at me blankly for a minute. Then he stammered, "The plan doesn't say it in so many words, but we're going to do our best work and stay on top of things. - Is that what you mean?"

"Ralph, old buddy", I explained, "You planned the scope of work you were going to do, the budget you would stick to, and the schedule for delivery because you don't want to leave any of those things up to chance! The quality of the product you will produce is no less important. And just like those other three things, if you don't plan it, you're leaving it up to chance."

"Whoa!" Ralph mused. "I hadn't thought about it that way." He pondered for a few minutes, and then wrinkled his brow. "I don't think I've ever even seen a Quality Plan before! What does it include? And how do I do it?"

A willing learner! What a pleasure to work with. This is the tutorial I gave him.

Planning to produce a good quality product is a three-step process. First we must agree about what the word "good" means on this project. Then, we must set numerical goals for quality so we can understand our status against those quality goals. And finally, we must determine the specific things we will do on this project in order to achieve the quality goals we have set.

Step 1: Defining "Good" Quality

No system is perfect, and neither our customer nor management expects the product we are building to be the first. So it is important that as we are establishing the goals for the critical aspects of our project, we include quality as one of those aspects. What are we expected to build, when must it be delivered, how much can we spend doing it, and what qualities will make it "acceptable"?

Many of us have little experience with setting quality goals, so trying to do so can be intimidating. But we all have the experience we need to get started. And after setting quality goals for the first time, it will seem less daunting on our next project.

The easiest way to get started on quality goals is to spend some time considering the types of things that have made us (or our customers or management) judge systems to be of "poor quality" in the past. Often, they were buggy (i.e. had too many defects to be used easily). Maybe they were slow (i.e. insufficient response times or throughput capability). They may have been insecure. Or important features may have been overlooked. Try to generate a complete list of the quality dimensions that have proven to be important in the past.

With that start in mind, we can turn our attention to the system this project will build. Which of those items stands out as being important this time? Does this project have other aspects that will be uniquely important? What are the dimensions on which the stakeholders will judge the quality of the system we will build? These are they areas where we need to set quality goals!

"Several of these sorts of things came out as we were defining our project Success Criteria." Ralph ventured. "But on my last project, the things they complained about at the end weren't on the list at all! I can see the value in this sort of historical brainstorming!"

Step 2: Setting Quantitative Quality Goals

Then Ralph got that quizzical look again. "How do you set useful goals for quality? Budgets are set in terms of money. Schedules identify dates. How do you measure quality?"

"Quality" is not something that can be expressed in uniform terms. We must consider the various dimensions of quality that we care about and determine how we can measure each of those dimensions. For example:

If defects are important, then we will want to use a notion like "defect density" to measure it. (Defect density = total defects / product size, where size is measured in lines of code, function points, or some other concrete way.)
If performance is important, then we will want to talk in terms of response times (seconds), throughput (bytes per second), or load (concurrent users).
If features are a key issue, then "requirements stability index" may be more appropriate. (RSI = Number of Requirements Change Requests / Number of Requirements).

"OK" Ralph replied. "I can identify the key quality attributes for my project and determine how each can be measured. But how do I figure out what are reasonable goals to set? Where do I get those numbers from?"

"Actually", I explained, "your best guidance comes from past projects."

If you can identify one or more projects that produced acceptable results for a particular dimension, then get the data from those projects and compute the values they achieved on that metric. Your goal for this project should be similar to the acceptable performance you achieved on those prior projects.

More often, the prior projects you can identify are those on which the key quality attributes fell short of expectations. In those cases you still want to compute the metrics achieved on those projects. But your goals for this project should represent an incremental improvement over past performance. Be sure that the amount of improvement is reasonable. Unless this project will be considerably different from those past projects, you should not expect that you can make dramatic improvements in quality.

"Great!" Ralph beamed. "Quality planning sounds pretty easy. It shouldn't take me too long to bang those numbers out!"

But I burst his bubble: "Establishing quality goals is not the end of the story. In order to achieve those goals, you must do something!"

Step 3: Planning Quality-Related Activities

If your quality goals are no more aggressive than past performance, then the quality-related activities on this project need not be different from prior projects. But more likely, if you are planning for better quality performance this time, then you must plan to do something differently. If there is no change is what you do, then your new more aggressive quality goals are only an exercise in wishful thinking!

There are three types of quality-related activities you can plan for on your projects, detection, correction, and prevention.

Defect detection activities are designed to find defects. Testing is the activity that we usually think of, but it usually happens late in the project when correction activities are likely to take a lot of time. We can improve quality performance by doing more detection activities, doing them earlier in the project, and assuring that we do them well. For example:

Peer reviews of designs and code, when done early in the project tend to have a significant positive impact.
Most programmers can learn to do a much better job of unit testing than they currently do.

Defect correction activities are centered on making sure the defects that are detected are fixed without introducing more problems. We tend to build this sort of time into our plans without consciously calling it a quality-related activity.

But the biggest payback is in defect prevention activities. This usually is a matter of looking at the types of problems you have had on prior projects and improving the technical methods to keep them from happening again. For example:

Many groups find that better design techniques provide the help they need.
Maybe a tool will help the staff to avoid errors in repetitive work.
Perhaps a set of simple standards will assist the developers in avoiding simple mistakes.

"These, Ralph, are the sorts of activities that you must plan if you don't want to leave your quality goals up to chance. Make sense?"

"You bet!" Ralph enthused. "Give me another day or so, and I'll have a complete Project Plan that doesn't leave anything up to chance -- not even quality!"