Contractor involvement in operational testing: What is really needed? - Navy OT&E, Industry, and Acquisition - United States Department of Defense

Program Manager, Nov-Dec, 2001 by R. E. Besal, S. K. Whitehead

In the March-April 2001 edition of Program Manager Magazine, retired Army Cal. John Stoddart reflects the voice of industry members of the ICOTE (Industrial Committee on Operational Test and Evaluation) as he describes the attitude within the operational test and evaluation community as secretive. He bases this perception upon a popular myth that "contractors, by law; can not be involved in any aspect of operational testing of their equipment," and that "... application of this myth to all areas of operational testing leads to longer acquisition periods, adds cost to the program, and weakens the close teamwork necessary to meet the challenges of providing the best equipment to the field."

Founded in Law

The "myth" that Col. Stoddart cites has its foundation in 10 U.S.C. Sec 2399 paragraph (d):

"In the case of a major defense acquisition program, no person employed by the contractor for the system being tested may be involved in the conduct of the operational test and evaluation required under subsection (a):"

Missing from Col. Stoddart's quotation was the next sentence, which is important to the complete context: "The limitation in the preceding sentence does not apply to the extent that the Secretary of Defense plans for persons employed by that contractor to be involved in the operation, maintenance, and support of the system being tested when the system is deployed in combat."

Col. Stoddart advocates contractor participation because "Nowhere in the law does it say that a contractor can not have some involvement in the operational test, such as being allowed to observe the test; having access to copies of the Test and Evaluation Master Plan, including the operational test portion; being allowed to participate as an observer in Integrating Integrated Process Teams; or even being provided early test data."

"Long Pole in the Tent"

We will readily second his viewpoint that a closer and more direct working relationship between the operational testers and industry is warranted and necessary. Yet we are concerned that Col. Stoddart and his cohorts believe that operational testing is the "long pole in the tent" in fielding a system, and that industry "observation" of Operational Test (OT) and access to program documentation will, in some manner, significantly improve the acquisition process.

While some small increase inefficiencies might be possible, we'd suggest that industry must look elsewhere for significant gains. Currently, Navy OT averages less than 1 percent of total program cost and takes less than 7 percent of program development time (assuming a five-year fielding effort). By these measures, our Navy operational test process is very efficient and effective.

The Reality

The acquisition process will not be noticeably shorter or cheaper with industry observation of testing and access to documents. What is needed is a shift in the pervasive mindset within the acquisition community that:

THE WARFIGHTER NEEDS IT Now!

This sales pitch is used more often than the ubiquitous "It's New and Improved" commercial marketing technique. What warfighters really need is a system that works reliably when they need it the most -- during combat. A system that works in an unstressed, non-threatening situation is useless if it fails to perform during combat.

WE CAN MAKE IT RIGHT AFTER WE GET IT IN THE FLEET.

Providing warfighters with a system you know does not meet either their needs or requirements is a professional and moral disservice. Planning to "fix it after it's fielded" places the operational tester in a no-win situation. We are charged to evaluate system performance to the level stated in the operational requirements document; if the system can't perform to that level, it fails. If you know it won't perform to that level beforehand, make that known and work the issue out with the requirements sponsor and operational tester. It's frustrating for us -- and expensive for industry -- to find out after planning an operational test and expending funds that someone on the developmental side knew the system was unable to perform, but didn't say anything in time to adjust planning.

"WE CAN DO IT BETTER, FASTER, AND CHEAPER."

We'll defer to the opinion of Edward Comstock, Principal Assistant for Acquisition, Programming and Budgeting in the office of the CNO. "When a program gets down to the brass tacks, there are three principal factors: cost, schedule, and performance. And the statement I often get from my program managers is, 'I can give you two.'" When the best that can be achieved is two out of three of these, stop selling all three and be forthright on which one is not going to make it.

What It Will Take

If we're to achieve "better, faster, and cheaper" acquisition, we need an awareness within the acquisition community that:

SOMETIMES YOU JUST CAN'T GET THERE FROM HERE.

At some point, preferably sooner than later, something may preclude a system from achieving its required capabilities. This could be cost, schedule, or a limitation in current technology. When this occurs, admit the reality of the situation and concede the effort. Invest the remaining resources in areas that offer greater promise of success.