End result Auditing

Internal Auditor, June, 2000 by Donald Robitaille

Audits that are limited to compliance tests on existing controls may also be limited in value. Innovative audit approaches--like John Hancock's ERA--focus on the business objectives and end results that the controls are designed to produce.

AT JOHN HANCOCK FINANCIAL SERVICES, end-result auditing (ERA) helps us to reassure our primary customers--the audit committee and senior management--that effective controls are in place and underscores the value of internal auditing. ERA, an audit approach we developed, combines an end-result focused, control-review process with a testing process that identifies the level at which a business unit is currently achieving its business objectives. In other words, end-result control reviews plus end-result testing equals end-result auditing.

CONTROL REVIEWS

End-result control reviews are a natural outgrowth of Internal Control--Integrated Framework, a 1992 report issued by the Committee of Sponsoring Organizations (COSO). The COSO review process broadened the concept of internal control beyond the realm of finance to include all business controls-operating controls, financial reporting controls, and regulatory compliance controls. COSO also established a process for evaluating controls that starts with the identification of a business unit's objectives and the risks that could prevent achievement of those objectives.

The control review phase of our ERA process is based on COSO and starts with an identification of the business unit's objectives. We then identify risks that may prevent achievement of the objectives and identify the control procedures used to reach each goal. After confirming with management our understanding of the business objectives, risks, and control procedures, we perform a preliminary assessment of the control process.

Control reviews can also be performed jointly with management using control self-assessment (CSA). In either situation, operating area management is closely involved with the control review phase.

TESTING

Our testing processes are directed at reaching two distinctly different conclusions. Compliance testing verifies that control procedures are actually in use, whereas end-result testing determines the precise level of achievement for each of the unit's objectives.

Testing each significant control procedure verifies compliance and further enhances our confidence in the control system. But even when the control system "looks" effective and compliance with procedures appears to be occurring, we are still dealing with a theoretical design. Our next step is to verify that the function's objectives, or end results, are actually being achieved. We can't be certain that subtle cracks don't exist in a control system until we examine the end results the control system is supposed to produce.

In addition, when the preliminary assessment indicates a weak control system, we do not abort the testing. End-result testing is critical at this stage to determine whether the weak control system has resulted in failure to achieve the business unit's objectives, as well as to quantify negative results. We believe audit committees and senior management want and need to know the impact of any weak controls. The results of the testing will also help to demonstrate the value of the recommended control improvements.

Having completed an analysis of the control system, compliance testing of control procedures, and end-result testing, the auditor is in a position to conclude on the reliability of the control system with a high degree of confidence. End-result testing will frequently identify business shortfalls or problems of which management is unaware, an important added value derived from the testing.

THE ERA ADVANTAGE

In essence, the ERA process answers two fundamental questions that are at the core of every effective audit: whether or not the business area is meeting its current objectives and whether or not effective controls are in place to achieve future objectives. In addition, ERA enables auditors to provide significant improvements in operations, broaden the scope of audit testing, and enhance the CSA process.

OPERATIONAL IMPROVEMENTS ERA provides value that goes beyond control evaluation. Because ERA is directed at a business unit's objectives, the process frequently leads the auditor to identify operational improvement opportunities. Further, including selective, operational tests with the end-result testing increases the chances that the auditor will identify operational improvements. At John Hancock, we expect our auditors to be alert to possible operational improvements throughout the audit and to add selective testing to specify the monetary value of improvements identified. Operational improvements include both expense savings and revenue enhancements.

Reporting hard-dollar savings from control and operational improvements allows the auditor to demonstrate at least the measurable value produced by the audit function. One cash flow recommendation at John Hancock produced enough in annual investment income to cover one-half of internal auditing's costs for the foreseeable future--but that was just the beginning! In two key audits, we identified significant profit-improvement opportunities. One of those