Getting traction with KRIs: laying the groundwork

RMA Journal, The, Nov, 2003 by Charles Taylor, Jonathan Davies

This article summarizes insights on the incidence of operational risk gleaned from Part I of the KRI Framework Study and discusses the direction of future work.

In recent years, a good deal of the focus in operational risk in large banks has been on quantifying risks and losses as precisely as possible. Behind much of this effort has been the Basel II requirement that by 2007 banks should be able to estimate how much capital they need to hold against their operational risks.

More recently, the banking industry has begun to devote attention to improving tools used in hands-on management of risks and, as a part of that effort, more scrutiny has been given to "indicators" of areas of higher risk and loss--key risk indicators (KRIs).

Some examples of KRIs are:

* Percentage of transactions that have not settled after five business days by product.

* Percentage of transactions requiring some manual input by product.

* Percentage of payments in compliance with OFAC.

* ATM robberies per 1,000 ATMs.

* Numbers of successful hackings into the production environment per year.

* Customer/client satisfaction by product.

KRIs can be useful even if they are only directionally correct most of the time, rising, falling and staying steady with the risks and losses they track. KRIs come in different flavors: coincident indicators that tell us something about losses as they happen; and lagging indicators that serve a purpose by being easier to observe and measure than the losses themselves. But KRIs tend to be most useful when they predict losses--provided they do it with reasonable accuracy.

However, therein lies the problem. KRIs, especially the more predictive ones, do not always track risks well. Some appear to have been defined at too high a level: For example, staff turnover rates for an entire bank don't track any particular loss that well. Others track operational problems well enough, but their relationship to the pattern of dollars lost is not all that well established.

Today, specific KRIs are defined differently from bank to bank and even among departments performing very similar functions in the same bank. Some institutions can count their indicators in the 1,000s as a result. This makes it very difficult to aggregate or summarize information for senior management on how loss and risk are evolving and how well they are being managed.

Up to now the industry has had considerable trouble telling which KRIs work well and which ones don't, and it has been difficult for banks to learn from one another's experience. With so many different definitions out there, it's impossible to know if we are comparing apples to apples.

The Framework Introduced

To help come to grips with these problems, RiskBusiness approached RMA and suggested it sponsor a study to develop a KRI Framework for the banking industry. The aim is to achieve enough standardization, completeness, and consistency to create comparability and enable aggregation, analysis, and reporting at the corporate level, which in turn will set the stage for real improvements in the effectiveness of KRIs.

Is this quixotic? Do KRIs have to be as different from one another as they are? RMA, RiskBusiness, and the banks participating in Part I believe there is a good chance that the answer is no. Certainly, if standardization of loss event data is anything to go by, which the industry is testing in various consortia, a similar effort around KRIs may well be warranted.

Our approach to the task has been methodical:

* Part I--Define a standard risk point matrix and create risk maps.

* Part II--For a small sample of high-risk points, define and specify standard KRIs.

* Then, with a large sample of financial institutions, confirm the risk map and identify, define, and specify standard KRIs for a larger set of risk points.

This standardization should have value in and of itself. Beyond that, however, in due course it should form the groundwork for a Part III industry effort to collect data on KRI values, something that would allow individual banks to benchmark against the industry at large.

Part I. Define a risk point matrix and create risk maps.

RiskBusiness took the seven risk and eight business definitions in Basel II and broke them down into a more granular set of definitions--15 risks and 38 products and services. Working with the participating banks, they then defined 45 functions (high-level processes) that between them account for practically every process supporting the delivery of every product and service a bank offers. With some aggregation of risk points shared across business lines, that yields a 3-D matrix with some 10,000 points where risks of a particular type could arise at a particular stage in the provision of a particular product or service. (1)

This 3-D approach--risk type, business type, and function type--is very promising. Up to now, a 2-D approach--just risk type and business type--has dominated thinking in the industry. Bringing in the third dimension to create a risk cube as opposed to a risk square provides the structure to test and improve KRI effectiveness at a practical level of granularity (see Figure 1).