The ultimate test - testing year 2000 conversion code - Industry Trend or Event

Software Magazine, Oct 15, 1997 by Axel Kehlenbeck

Testing tools abound, but as in any bet-your-business acquisition -- caveat emptorAs the new millennium approaches, many IT organizations are mired in the process of making their software Year 2000 compliant. In many companies, this process requires programmers to fix tens of thousands of two-digit date fields embedded in millions of lines of code. Unfortunately, that's the easy part. The real Y2K crisis begins with testing the code once it's fixed.

One company that has entered the test phase of its Year 2000 project is Sundstrand Aerospace in Rockford, Ill. Nearly 85% of Sundstrand's applications --10 million lines of code worth -- must be converted and tested. Testing began in February 1996, says Bill Schuyler, project manager. The company plans to spend nearly 50% of its Year 2000 budget on this phase.

Such budget allocation is commonplace. London-based Commercial Union Insurance, for example, is spending nearly 65% of its Y2K budget on testing.

Both Sundstrand and Commercial Union found that creating their own test environments ate up significant resources. Besides establishing the hard- ware environments, setting up the initial test cases to establish the baseline tests for post-conversion regression testing proved to be a major expense. In an effort to keep their projects on schedule, both companies decided to test only for conversion-related failures.

The only real way to minimize testing costs and ensure timely results is to employ automated tools that can consistently execute tests the same way under the same conditions -- both pre- and post-Y2K conversion. Firms must then be able to identify, capture, store, and compare test results for each application.

Before and After

A major question in testing converted Y2K applications is whether to test the entire application or only those areas associated with the date changes. As with any software development or maintenance effort, Year 2000 date changes can introduce new defects -- also known as regression failures -- into the code. While testing only the affected code simplifies the effort, firms should also consider the regression failure issue. Regression defects can be significant time and cost wasters and are a major reason for delays in updates to new applications.

An effective test methodology requires firms to test the entire application completely prior to the start of the code conversion effort. They test again after code conversion with old and new date fields to ensure the application will continue to work this century and into the next. Finally, firms can compare the test results to ensure that no regression failures have been incorporated into the code. This methodology requires that the testing process begin immediately after the inventory and assessment phase of any Y2K conversion effort.

Most consultants, as well as test tool vendors such as Compuware, Mercury Interactive, and Intersolv, recommend this multistage test process to ensure that non-conversion-related defects are eliminated early in the process. According to Sundstrand's Schuyler, the best way is to test the entire application before and after the conversion of each application component. The com- pany has employed this three-step process in its Y2K testing efforts, setting up three distinct conversion test beds.

Both Sundstrand and Commercial Union first create and establish the baseline tests with the existing code prior to conversion. Once the baseline is established, the converted code is checked, modified, and tested in a second test environment using the same tests and dates used to establish the baseline. This retesting ensures that the application functions in the 20th century date formats and that no regression failures have been incorporated into the code. The third step is the actual testing of the converted code with converted date fields to verify that the applications operate with 21st century dates.

Testing applications before and after conversion requires multiple sets of tests to check for old and new date fields. Modifying existing tests is a major programming effort, as is writing new ones. Test tools such as Mercury Interactive's WinRunner 2000, Viasoft's VIA/AutoTest, and Compuware's QA Hiperstation GUI automate the process of ad- ding incremental dates in existing tests.

Resetting the system clock is another important task that's not only time-consuming but can disrupt the entire development and test environment. Without changing the system clock, firms can't be sure that the application will work in the next century. While some companies are touting expensive hardware-based time-shifting capabilities, test tool vendors have developed inexpensive software-based simulation environments. Vector Software's Virtual Date Utility, for instance, enables date variable testing of code and simulates 21st century system clocks.

Management Aid

Once all the required tests and test environments are established, managing them becomes the major issue. For each stage, companies must perform the various tests, and gather and compare the results. Here again, tools can help.