Social Measurement: What Stands in Its Way?

Social Research, Summer, 2001 by Martin Bulmer

When you cannot measure * your knowledge is * meager * and * unsatisfactory.

-Lord Kelvin (inscription carved in 1929 below the bay window of the Social Science Research Building of the University of Chicago)

who cares if some one-eyed son of a bitch invents an instrument to measure spring with.

-e. e. cummings

Measurement

MEASUREMENT is any process by which a value is assigned to the level or state of some quality of an object of study. This value is given numerical form, and measurement therefore involves the expression of information in quantifies rather than by verbal statement. It provides a powerful means of reducing qualitative data to a more condensed form for summarization, manipulation, and analysis. Classical measurement theory argues that numbers may perform at least three purposes in representing values: (1) as tags, identification marks, or labels; (2) as signs to indicate the position of a degree of a quality in a series of degrees; and (3) as signs indicating the quantitative relations between qualities. On some occasions, numbers may fulfill all three functions at once (Cohen and Nagel, 1934).

One of the most influential twentieth-century statements of the classical approach was that of psycho-physicist S. S. Stevens, who proposed four scales or degrees of measurement: nominal, ordinal, interval, and ratio measurement (1946, 1975). Nominal and ordinal measurement is nonmetric; interval and ratio measurement is metric. These theoretical standards are translated in measurement standards in the physical world through organizations such as the United States National Bureau of Standards (NBS). The NBS provides state, county, and local officials with technical and operational guides that set out measurement specifications, standard tolerances, and model laws designed to support the physical measurement system (Hunter, 1980: 869). The primary standards are those of the International System of Units (SI units) and are seven: length (meter, m), mass (kilogram, kg); time (second, s); electric current (ampere, A); temperature (kelvin, k); luminous intensity (candela, cd); and amount of substance (mole, mol) (Zebrowski, 1979).

This scientific paradigm of physical measurement provides a model the social sciences, or some social scientists, seek to emulate. The quotation from scientist Lord Kelvin carved on Chicago's Social Science Research Building reflects that aspiration. The poet e. e. cummings's skepticism reflects doubts as to whether the aspiration is worthwhile in the first place. The place of measurement in social science research is a contentious issue; this tension runs through social science disciplines such as sociology and political science. It is reflected in the ambivalence with which many social scientists look upon research methods such as social survey research. The aim of this article is to consider some of the hindrances to improved measurement of the social. There has been a notable failure to agree on standards for social measurement (as distinct from psychological or economic measurement), whether in terms of social indicators and conceptual unification, or at the practical level of operationalizing variables.

The Scope of Social Measurement

Social scientists take up differing positions in relation to the value of what is involved in social measurement. Some part of this may be due to resistance to, or ambivalence about, the place of numbers in the realm of knowledge, coupled with inability to appreciate the role that number may play (cf. Paulos, 1988). But the issue cannot be reduced, in T. D. Weldon's phrase, to "like a taste for ice cream." The merits of measurement in social science, and the obstacles to measurement, need to be set out and debated. In this way, some of the passions the subject enflames may be restrained and cooled.

Relatively few of those who have approached thoughtfully the issue of social measurement subscribe to classical measurement theory as outlined at the beginning of this article. A great deal of social measurement is nonmetric, and uses the assignment of numbers to qualities of an object of study as a way to label characteristics or make statements of more or less. A common definition of the properties of social measurement is the following:

   Whenever we classify a number of units we shall talk of measurement. This
   is a rather broad use of the term, but it leads to no difficulty; if we
   classify a set of units by a quantitative variate (variable) we have the
   special case of conventional measurement (Lazarsfeld, 1970: 66).

In terms of Stevens's four levels of measurement (1946), much social measurement is of a nominal or ordinal kind, lacking the properties of interval and ratio measurement. But this creates difficulties. How one characterizes the state of the health of a population, or the level of crime in a particular area, is by no means a straightforward matter, given the wide variety of measures of each that are available.

Recognition of the complexity and provisional character of much social measurement comes from a variety of positions that in other respects may not share much in common. For a time, definitional operationism was in vogue, particularly as put forward by the physicist Bridgman (1927). Donald Campbell, however, criticized this as failing to do justice to the complexity of social constructs, and argued instead for a multiple operationism: