PSP^SM Industrial Use Data

Reporting of actual industrial use data about the effects of the PSP^SM has been limited by several factors:

• Many companies are reluctant to release specific data that competitors or customers could use to identify actual cost and defect levels.
• Most companies have little or no historical data to compare PSP data against, so it is difficult to quantify the effects that PSP had on their costs and schedule.
• PSP has not yet been widely adopted (as the CMM has), so there are fewer cases available to draw from.

The following data is the most descriptive I can find. If your organization can add to this body of data (even anonymously), please contact me.

The following data shows these striking results:

Software Process Improvement Works!, Advanced Information Services, Inc. (SEI Technical Report 99tr027)

Organization-wide performance in three phases:

1. Before process improvement
2. CMM-based improvement
3. PSP in addition to CMM^®

Acceptance Test Defect Density

Engineer Productivity

Scott D. Griffin, The Boeing Company (2000 SEPG)

The most notable effects of adding PSP to Boeing's process Improvement Program.

• PSP was introduced for Release #9.
• Although it was 2.36 times as large as the prior three releases,
• There were 75% fewer defects detected during system test,
• Resulting in 94% savings in test time.

Jagadish C. Kamatar, Advanced Information Services Inc. (1998 SEI Symposium)

Comparison of the author’s personal performance on two projects.

• Project 1 started before PSP training was complete.
• Project 2 represents improving personal process maturity.

Will Hayes, Software Engineering Institute (1998 SEI Symposium)

Teradyne, Boeing, AIS and Hill AFB (SEI Web Site)

These results were reported on the SEI's web site.

"The first organization to report their results after using both PSP and TSP was Teradyne. Their return on investment (ROI) analysis indicates that by using PSP and TSP on 2 projects totaling 112 KLOC, they have benefited by approximately $5.3 million to date in saved engineering time."

"With code developed using TSP/PSP, they are saving approximately 120 hours/KLOC in integration, system, and field testing. Quality levels improved 20 times over prior projects and actual effort and schedule were within 8% of plan (early). Teradyne estimates the cost of PSP training to be one month per engineer."

"Boeing, AIS, and Hill Air Force Base have also reported receiving significant benefits from PSP and TSP use. The following charts are a compilation of results data from Teradyne, Boeing, AIS and Hill AFB. Data from 18 TSP/PSP projects in these organizations were combined to illustrate before and after performance ranges."

The ranges are the minimums and maximums from this data.

Vast improvements in delivered product quality.

Reduced testing time because of fewer defects to be found.

Remarkable estimating accuracy.

Shannon M. Versaci, Harris FR Communications (1997 SEI Symposium)

Using the TSP on the TaskView Project, David Webb and Watts Humphrey (Crosstalk, February 1999)

"The TaskView team at Hill Air Force Base, Utah used the TSP to deliver the product a month ahead of its originally committed date for nearly the planned costs. Because the engineers' productivity was 123 percent higher than on their prior project, they included substantially more function than originally committed. Testing was completed in one-eighth the normal time, and as of this writing, the customer has reported no acceptance test defects."

"The TaskView project was conducted by the Ogden Air Logistics Center, Technology and Industrial Support Directorate (TI), Software Engineering Division (TIS) at Hill Air Force Base, Utah. TIS is a high-maturity organization with a strong history of software process improvement. In March 1995, TIS was assessed as a CMM Level 3 organization, and the assessment conducted in July 1998 rated them at CMM Level 5. This is the first software organization in the Department of Defense (DoD) to receive this rating, and it is one of the few Level 5 software groups in the world."

"Use of the TSP was found to substantially improve size and effort estimating accuracy. ... When the 9,455 LOC of added function were subtracted, the team's original 14,065 LOC size estimate had an error of 23 percent."

Productivity figures are impacted by many factors. ... The results do, however, suggest that the TSP improves productivity.

With TSP, the TaskView project increased the yield of early defect removal by more than 60 percent ... The benefits of this early attention to quality are apparent from the results of the later test phases.

Typical TIS projects require 22 percent of the project schedule (in days) to perform the final two TIS test phases. The TaskView project, using TSP, sharply reduced this percentage to 2.7 percent. This is a schedule savings of nearly 20 percent. Only one high-priority defect was found in these last two test phases.

Classroom Data

As an alternative, there are a variety of public reports that include data collected from students during PSP classes. Two good sources are:

• "The Personal Software Process (PSP): An Empirical Study of the Impact of PSP on Individual Engineers", Hayes and Over (SEI Techniocal Report 97tr001)
• "Personal Software Process (PSP) in the Workplace: Software Engineering at a Personal level", Hayes (1998 SEI Symposium)

Several companies have report that engineers’ work performance continues to improve beyond their classroom levels, so the student data cited above can serve as a good benchmark for setting expectations.