An automated test evaluation tool

Hewlett-Packard Journal, June, 1997 by Jorg Schwering

Results

The manual evaluation time for an overnight test of around one to two hours has been reduced by the use of AutoCheck to less than a minute. This means that the additional effort for the test engineer for writing the expected results in the AutoCheck syntax is compensated after three to five test runs. This depends on the experience of the test engineer with AutoCheck (the normal learning curve) and the nature of the test.

A positive side effect is that it is much easier for another test engineer to evaluate the test.

AutoCheck also leads to bigger tests with an increased number of checks for each test case, such as checks for side effects. Such an automatic evaluation tool is also a prerequisite for statistical testing. It would take too much time to evaluate all these test cases manually. In other words, AutoCheck leads to higher test coverage with lower effort for the test engineer.

Once relieved of a great deal of the more mechanical test execution and evaluation activities, the test engineer has time to work on new and better test approaches or possibilities for an increased automation level. Over time this has led to enhancements of both AutoTest and AutoCheck and to new tools like ATP (see article, page 95).

Acknowledgments

I wish to thank Pascal Lorang for the creation of the prototype and all students involved in the development of AutoCheck.

Reference

[1.] D. Goring, "An Automated test Environment for a Medical Patient Monitoring System," Hewlett-Packard Journal, Vol. 42, no. 4, October 1991, pp. 49-52.

UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company Limited.

X/Open is a registered trademark and the X device is a trademark of X/Open Company Limited in the UK and other countries.

MS DOS is a U.S. registered trademark of Microsoft Corporation.

Jorg Schwering A software quality engineer at HP's Patient Monitoring Division since 1991, Jorg Schwering develops and provides consultations on product generation guidelines. He is professionally interested in product generation processes and software testing techniques. He joined HP in 1988 and worked half-time while attending college until he received a Diploma in computer science in 1991 from Berufsakademie in Stuttgart, Germany. Jorg was born in Steinfurt, Germany, is married, and has an infant son.