How Real Are Statistics? Four Possible Attitudes - )

Social Research, Summer, 2001 by Alain Desrosieres

REFERENCE to "reality" is a commonplace among both producers and users of statistics. This "reality" is understood to be self-evident: statistics must "reflect reality" or "approximate reality as closely as possible." But these two expressions are not synonymous. The very notion of "reflection" implies an intrinsic difference between an object and its statistics. In contrast, the concept of "approximation" reduces the issue to the problem of "bias" or "measurement error." Thus even these two common expressions, generally used without regard to consequences, tell us something important: a critical reexamination of this notion of "reality" is, for statisticians, an efficient way to reconsider the deepest-rooted but also the most implicit aspects of their daily work--precepts, tricks of the trade,justifications provided to users, and so on. This paper argues that the way in which producers and users of statistics talk about "reality" is informed by the fairly unconscious intermingling of several attitudes to reality. The mix of these attitudes and the links between them vary according to the circumstances--or, rather, according to the specific constraints prevailing in different situations. As it happens, the field of business statistics offers a representative spectrum of these various possible attitudes to reality. However, we cannot say--despite their differences--that some are better than others since each is so closely associated with situational constraints specific to particular phases of the statistician's technical, administrative, or managerial work.

Four possible attitudes to reality (among others) will be discussed. First, we will describe them in a "pure" (hence certainly exaggerated) form. Next, we will see how they are applied to concrete situations. Each attitude has its language, that is, a register of words, requirements, and arguments; these are consistent, but difficult to interlink if one shifts from one attitude to another. Here is a list of the four attitudes ranked by "obviousness" (at least for statisticians; for other communities the order would, no doubt, be different):

* metrological realism;

* pragmatism of accounting (which may be "national" accounting);

* use of material from a database for argumentative purposes in social life or in quantitative economics and social sciences; and

* the explicit admission that the definition and coding of the measured variables are "constructed," conventional, and arrived at through negotiation.

Of these four attitudes, the first three may be qualified as "realistic," but each has different reality tests--that is, ways of verifying and articulating the substance of that reality and its independence from observation. The fourth attitude, instead, emphasizes the conventional and social character of statistical variables, and may thus be labeled as "constructivist." It mainly comes into play in situations marked by discontinuity, controversy, and innovation. We will now examine each attitude individually, spelling out their languages, origins, and conventions.(1)

Sampling and Confidence Interval

Metrological realism derives from the theory of measurement in the natural sciences that is complemented, in the social sciences, by the sampling method. The object to be measured is just as real as a physical object, such as the height of a mountain. The vocabulary used is that of reliability: accuracy, precision, bias, measurement error (which may be broken down into sampling error and observation error), the law of large numbers, confidence interval, average, standard deviation, and estimation by the least-squares method (Stigler, 1986; Hacking, 1990). This terminology and methodology was developed by eighteenth-century astronomers and mathematicians, notably Gauss, Laplace, and Legendre. The core assumption is the existence of a reality that may be invisible but is permanent--even if its measurement varies over time. Above all, this reality is independent of the observation apparatus. In a sense, this is the dream of the statistician and the specialist in quantitative social sciences: the possibility of making the metrology of these sciences equivalent to the proven methodologies of the natural sciences. This may be seen as a benchmark, an ideal to which statisticians aspire, despite an awareness that their objects do not display all the properties assumed by the methodology. We could describe this as the lost paradise of the social sciences, which would have liked to have been endowed with the same persuasiveness as the natural sciences of the nineteenth century.

In this endeavor to connect the methodologies of the social sciences and the natural sciences, one element plays a crucial role: the law of large numbers, which is the basis for probability formulas and the resulting convergence theorems. The "law" serves as an operator for the transformation and transition from the world of observations to the world of generalization, extrapolation, and forecasting. Its hybrid nature is summed up in the famous and revealing quip dating back to the nineteenth century: astronomers and physicists believe the law to be a theorem demonstrated by mathematicians, whereas mathematicians think the law has been proved by the results of repeated testing. In fact, the possibility of conducting multiple and mutually independent observations of comparable objects is the foundation of a statistical methodology initially developed to study populations of human individuals or households.