Assessing the transition-to-production risk: a new methodology - Risk Management

Program Manager, Sept-Oct, 2002 by Tamara J. Adams, Steve P. Austin, Robin B. Soprano, Lucinda Matkin Stiene

In recent acquisition streamlining efforts, one process that remains virtually untouched is the method of assessing the risk of transitioning from development into production. Traditionally a formal Production Readiness Review (PRR) is conducted in support of program production decision milestones such as Low Rate Initial Production or Full Rate Production. The PRR team is composed of "independent" subject matter experts covering the gamut of functional areas such as hardware/software design, test, logistics, producibility and production planning, safety and quality assurance. This team generally spends two to three days in each prime and major subcontractor facility evaluating evidence of accomplishments, proper planning, and program stability. Areas of risk are documented and summarized in a PRR report.

A Better Way

In the fall of 1997, a team of engineers from the Production Engineering Division of the Research, Development and Engineering Center (RDEC) at the U.S. Army Aviation and Missile Command (AMCOM) in Huntsville, Ala., set out to streamline this process to better suit the programs they supported. To fulfill the requirement of assuring the decision maker that the risk of transitioning from Development into Production was acceptable, the team identified two programs to use the flagship methodology: Unmanned Ground Vehicle (UGV) and Enhanced Fiber Optic Guided Missile (EFOG-M). The National Missile Defense Ground-Based Interceptor program applied the methodology approximately a year later.

The Traditional Process

The traditional process was costly, both in terms of time and funding. While the ideal evaluation team would be completely independent, it was necessary for the team to have a general understanding of the program. Therefore, PRR team members were usually personnel providing functional support to, and being reimbursed from, the program under evaluation; they were not actually employees of the project/program office. However, in recent years, project office personnel have participated in the review in ever-increasing numbers. Depending on program size and complexity, the cost of conducting this formal examination of a program in the traditional manner might cost several hundred thousand dollars and could take several weeks or months to complete. The culmination of the process was a report that provided a one-time snapshot of the program in support of a milestone decision.

Using the traditional process is inconsistent with the dynamic Integrated Product Team (IPT) approach, where risk assessment and mitigation must be a continuous, real-time activity. In addition, programs are struggling for both government and contractor personnel as well as funding resources. Given the current environment, the need to revisit the way we conduct Production Risk Assessments (PRAs) is obvious. The review or assessment must be conducted in a more "seamless" fashion and provide a results- or performance-oriented view of program production readiness. The concept of using program personnel for the assessment goes hand-in-hand with the integrated relationships of the IPT environment, provides a means to save time and money, and helps to provide insightful, timely information.

The traditional process was also very subjective. Typically, the subject matter expert formulated lists of relevant questions in advance for consistency at each facility However, there was no concrete definition of what the expected outcome should be or what was required for success. The outcome of the review was very much dependent on the personnel selected for the team.

Formal PRR reports are no longer required as mandatory documentation to support a Milestone III decision. The current guidance in DoD 5000.2-R simply states, "Full Rate Production of a system shall not be approved until the system's design has been stabilized, the manufacturing processes are proven, and the production facilities and equipment are in place (or are being put in place)."

Improving the Process?

It seemed that the process could be improved by changing the PRR team structure or make-up and by developing concrete criteria to reduce the subjectivity of the assessment. The team decided to re-think the entire process and to determine what was truly important in determining production transition risk.

Using program personnel, via IPTs or existing functional structures, an iterative and organized PRA should be completed to streamline the process of assessing the risk involved in transitioning from development into production. In keeping with the DoD 5000-series guidance, three logical areas to be evaluated are Design, Manufacturing Processes, and the Manufacturing Infrastructure.

Considering the Design, the team would assess the stability of hardware and software design, looking at issues such as reliability, maintainability, Built-in-Test (BIT), system safety, and average unit production cost. Evaluating the manufacturing processes captures the maturity of the work instructions, manpower, process capabilities, and inspection/test and acceptance procedures. Assessing manufacturing infrastructure would focus on the maturity of the manufacturing facility and support structure. These areas would be assessed for a specified level of the hardware/software Work Breakdown Structure (WBS) or major hardware deliverable products.