Software quality assurance on the HP Printed Circuit Design System project - technical

Hewlett-Packard Journal, Feb, 1988 by David E. Martin

Software Quality Assurance on the HP Printed Circuit Design System Project

EFFORTS TO ENSURE SOFTWARE QUALITY must be planned for and kept visible throughout the entire project life cycle. Such efforts were particularly important during the development of Hewlett-Packard's Printed Circuit Design System (HP PCDS). Focusing on quality alone has the inherent danger of never releasing the product because it is not perfect. Developing methods that result in a product that meets high quality standards without sacrificing aggressive goals for a product release date is a monumental challenge for HP as well as the rest of the industry. Two key ingredients in addressing this challenge are quality assurance and engineering productivity.

QA Plan

Often a quality assurance plan is viewed as a necessary nuisance at best and busy work at worst. For a QA Plan to contribute to bettering the quality of the software, engineers and management should view it as a contract among themselves detailing their quality efforts.

The QA plan includes the release criteria for the project, which are the basis for the rest of the document. Although release criteria are usually custom tailored for each project, there are some items that are fairly common, such as a clear downward trend of the defect rate, execution of all of the individual test plans, no known critical defects remaining, etc. The rest of the plan describes how it is anticipated that the release criteria will be met.

An optimal QA plan requires extensive effort from everyone. The management team provides the framework for specifying what areas need coverage and ensuring that the coverage is adequate. The details about how to perform the testing are left to the engineering team. It is vital that everyone knows the plan, believes it, and is committed to fulfilling it. To that end, the QA plan is not a static document, but a dynamic one. It is impossible to foresee all events that might cause certain sections of the document to become infeasible, inadequate, or even unnecessary. Thus the plan becomes much more valuable to a project team if it is kept current, reflecting the real intentions of the team. As the testing progresses, results are entered. Performance measurement results are especially important for comparing later revisions of the product. The results give the management team invaluable information for deciding when the formal QA phase should end. A side benefit of keeping the QA plan current is that it automatically becomes a final document for the product archives.

Defect Tracking

Software defects some in many forms. Many are very innocuous, such as the misspelling of a word or unclear error messages. Others are of such a critical nature that the application software aborts unexpectedly and data is lost.

Throughout the life cycle of any software project a large number of defects are encountered and fixed. Attempting to keep track of thousands of defects in a large software project using a manual method is untenable and especially abhorrent, given the fact that HP is in the computer business. Having an automated defect tracking system is viewed as a necessity for helping the management team balance resources and provide the engineers with data on locating and fixing defects. The two major defects tracking systems used in HP are STARS and DTS. STARS (Software Tracking And Reporting System) runs on HP 3000 Computers under the MPE operating system and is used by marketing and field sales organizations. DTS (Defect Tracking System) runs on HP 9000 Computers under the HP-UX operating system.

HP PCDS Development

Since HP's Printed Circuit Design System was developed in the HP-UX environment, it was felt that the engineers would be more productive using DTS. With other systems for tracking defects, it is not uncommon for the R&D lab to be unaware of defects because the procedure for reporting them is too cumbersome. However, with DTS the ease of entering defect reports facilitated the logging of the vast majority of defects encountered. Furthermore, because DTS was installed on every engineer's workstation, engineers found it very convenient to use DTS as a lab notebook. Notes about what caused a defect and ideas on how to fix it were stored directly with the defect data packet. Reminders to enter copyright notices, say, were entered as "defects." Ideas for new features were entered as enhancement requests.

Throughout the HP PCDS project, the management team was kept up to date with accurate statistics that were used to measure the quality of the software. For example, one measurement that was closely monitored was the number of defects reported versus time. A clear downward trend of this curve is evidence that QA efforts are improving the quality of the software.

Although there were many advantages for the lab in using DTS, the needs of the field, sales, and marketing organizations could not be ignored. Before manufacturing release, the outstanding defects from DTS were copied into the STARS data base. Defects from the field and sales organizations are entered in STARS and then copied over to DTS for the lab engineers. There is now software that automatically provides a link between DTS and STARS which keeps the two data bases synchronized. Although supporting two defect tracking systems simultaneously is certainly not easy, the advantage make the effort worthwhile.