But Quality Planning is a challenge for many of us. Quality is such a "soft" concept that it is difficult to think about it in concrete terms. This makes it difficult to plan for with the same level of clarity that we can for budget and schedule. This challenge is mainly due to the fact that we have difficulty measuring quality with the same precision we are accustomed to using for other project variables.
If we are to be able to plan for and manage quality with the same focus as we do for budget and schedule, then we must learn to measure it with the same precision. And measuring quality is within our reach.
Defect Density (# of defects divided by product size) is a measure of quality in terms of how many defects a product contains. A 2,000 Function Point system that contains 22 defects has a Defect Density of 11 defects per 1,000 Function Points (11 D/KFP). A system of 25,000 lines of source code that contains a total of 50 defects has a Defect Density of 2 Defects per 1,000 Lines of Code (2 D/KLOC). We can measure the density of defects we detect and correct during any activity in our project, as well as the density we deliver into production.
Defect Removal Effectiveness (# of defects removed divided by # of defects that existed at the time), also called "Yield", is a measure of how well our project activities remove defects. If our Peer Reviews of code detected 375 defects, and a total of 325 additional defects were detected in later testing and reviews and in production, then we can calculate the yield of our Peer Reviews as 375/(375+325) or 54%.
Defect Removal Efficiency (# defects removed divided by the number of engineer hours to find and fix them) is a measure of the cost-effectiveness of our defect removal activities. If our engineers spent a total of 100 hours on the Peer reviews and the resulting bug fixes that removed 375 defects, then the efficiency of our peer reviews were 3.75 Defects per Hour (3.75 D/H).
1) Compute the average defect density achieved on those prior projects. If those projects comprised a total of 150,000 lines of source code, and they delivered 750 defects combined, then the average defect density at delivery was 750/150 = 5 D/KLOC. In addition, if they detected a total of 1500 defects during system testing, we can say that System testing usually detects a density of 1500/150 = 10 D/KLOC. We can make the same computation for any reviews or other testing that we have data for.
2) Compute the average defect removal effectiveness for the various testing and reviews that were done on those prior projects. If those projects removed a total of 5000 defects during peer reviews, 1500 defects during System Test, and delivered 750 defects to the customers, then the average effectiveness of those activities were: 5000/(5000+1500+750) = 69% for Peer Reviews, and 1500/(1500+750) =67% for System test.
3) Compute the average Defect Removal Efficiency of each activity on those prior projects. If those projects spent 1000 hours removing 5000 defects during Peer reviews, then the Peer review efficiency was 5000/1000 = 5 D/H. And if they spent 2000 hours removing 1500 defects during system test, then their testing efficiency was 1500/2000 = 0.75 D/H.
4) Set goals on the current project for delivered defects as well as defects detected at each phase. The project's objective is to reduce delivered defects by 25%, os our target is to deliver (1.0-0.25) x 5 = 3.75 D/KLOC. If the estimated size of the product will be 10,000 LOC, then we must deliver no more than 37 defects.
1) Add more defect removal activities to the project. If you have not been including peer reviews, then add some. If you have been reviewing designs but not code, add code reviews. Of course you need to balance the cost of these new activities with the benefit you expect from them. If you don't have effectiveness and efficiency numbers for these activities from your organization, the best you can do is to base your decision on industry averages.
2) Replace some defect removal activities with more effective ones (or more efficient ones, which would allow you to do more with the same engineering hours). If prior projects reviewed 20% of the code and had good effectiveness and efficiency numbers for those reviews, then do more reviews this time (maybe 75% or even 100% of the code), stealing the hours from a less effective or efficient activity like system test.
3) Improve the efficiency or effectiveness of the activities you do. If your peer reviews are catching only 50% of the existing defects, perhaps some training and a better review process could increase their effectiveness to 60% or 70% this time.
4) Inject fewer defects in the first place. This requires doing causal analysis on the defects that were delivered on those prior projects. You may be able to inject fewer defects by adopting a design methodology, establishing coding standards, installing a tool that will help programmers to be more consistent in their engineering work, or many other actions. Exactly what will pay off will depend on the nature of the defects that have historically been delivered.
With a crisp quantitative understanding of quality, you will be able to make clear and actionable Quality Plans for your projects, and deliver not just the product that is needed, but the quality levels as well.