Statistical interpretation of key comparison reference value and degrees of equivalence

Journal of Research of the National Institute of Standards and Technology, Nov-Dec, 2003 by R.N. Kacker, R.U. Datla, A.C. Parr

Key comparisons carried out by the Consultative Committees (CCs) of the International Committee of Weights and Measures (CIPM) or the Bureau International des Poids et Mesures (BIPM) are referred to as CIPM key comparisons. The outputs of a statistical analysis of the data from a CIPM key comparison are the key comparison reference value, the degrees of equivalence, and their associated uncertainties. The BIPM publications do not discuss statistical interpretation of these outputs. We discuss their interpretation under the following three statistical models: nonexistent laboratory-effects model, random laboratory-effects model, and systematic laboratory-effects model.

Keywords: interlaboratory evaluation; measurement uncertainty; variance components.

**********

1. Introduction

Key comparisons are interlaboratory comparisons that serve as technical bases for Mutual Recognition Arrangements (MRA) between national metrology institutes (NMIs) [1]. Key comparisons carried out by the Consultative Committees (CCs) of the International Committee of Weights and Measures (CIPM) or the Bureau International des Poids et Mesures (BIPM) are referred to as CIPM key comparisons. Key comparisons carried out by regional metrology organizations (RMO) are referred to as RMO key comparisons. The guidelines for carrying out CIPM key comparisons are given in reference [2].

The objectives of a CIPM key comparison are described in reference [1]. We consider two interpretations of these objectives. A common interpretation is summarized by Nielsen [3] as follows: "The purpose of measurement intercomparisons between NMIs is to test, whether measurements performed in the participating countries are consistent taking into account the uncertainties assigned to the measurements. If an inconsistency is detected, the participating countries should take the corrective actions needed to obtain consistency. Otherwise, measurement results exchanged across borders cannot be considered equivalent without adding a 'between countries uncertainty,' which would be in disharmony with the concept of the SI system of units."

This paper is based on a second interpretation of the objectives of a CIPM key comparison: Generally, the participants of a CIPM key comparison are NMIs that are members of the appropriate Consultative Committee; at least some of these NMIs provide realizations of the SI values to establish the traceability of measurements made in their countries. The purpose of a CIPM key comparison is to establish the key comparison reference value (1), the degrees of equivalence (2), and their associated uncertainties on the basis of the data provided by the participants.

This paper is limited to a simple CIPM key comparison where the common measurand is a physical quantity of stable value during the comparison. Many CIPM key comparisons are not simple because it is often impractical or impossible to realize exactly the same measurand for or by all participants. We use the symbol Y for the stable value of the measurand. The data provided by the participants of a simple CIPM key comparison are paired results and standard uncertainties [[x.sub.1], u([x.sub.1])],..., [[x.sub.n], u([x.sub.n])], where the results [x.sub.1],..., [x.sub.n] are measurements of Y. The outputs of a statistical analysis of these data are the key comparison reference value [x.sub.R], the degree of equivalence [d.sub.i] = [x.sub.i] - [x.sub.R] of the result [x.sub.i], the degree of equivalence [d.sub.i,j] = [d.sub.i] - [d.sub.j] = [x.sub.i] - [x.sub.j] of the results [x.sub.i] and [x.sub.j], and their associated standard uncertainties u([x.sub.R]), u([d.sub.i]), and u([d.sub.i,j]), respectively, for i,j = 1, 2,..., n and i [not equal to] j [1]. The key comparison reference value [x.sub.R] is an estimate for Y. An estimate for Y is a combined result of measurement determined from the data [[x.sub.1], u([x.sub.1])],..., [[x.sub.n], u([x.sub.n])].

An understanding of the difference between sampling probability distributions, used in classical (frequentist) statistics, and state-of-knowledge probability distributions, used in Bayesian statistics, is necessary for proper analysis and interpretation of the data from a key comparison. Briefly, they are defined as follows. In classical statistics, the value of the measurand is assumed to be an unknown constant, often called the true value, and each result of measurement is regarded as a realization of a random variable with a sampling distribution. A sampling distribution is a probability distribution that describes the relative frequencies of occurrence for all possible results of measurement when the conditions of measurement are hypothesized to be fixed at the intended levels [4]. The metrologist relates the expected values of the sampling distributions for the results of measurement to the value of the measurand. A classical (frequentist) statistical interpretation is a statement that relates the realized measurements to what one might expect if the key comparison could be repeated infinitely many times and throughout these repetitions the hypothesized sampling distributions continued to apply.