Assessing construct validity in organizational research - includes appendices

Administrative Science Quarterly, Sept, 1991 by Richard P. Bagozzi, Youjae Yi, Lynn W. Phillips

One objective of the present article is to provide a systematic framework for examining construct validity in organizational research. We begin with the classic criteria proposed by Campbell and Fiske (1959) and provide four empirical illustrations that are used later in the article to show that Type I and II errors are possible when the criteria are applied. Next, we consider the confirmatory factor analysis (CFA) model, point out its advantages over Campbell and Fiske's (1959) criteria, and illustrate its features on the same four data sets noted above. Following this, the issue of confounded random error with specific variance is discussed, and three solutions are investigated: the second-order confirmatory factor analysis (SOCFA) model with measures loading directly on trait and method factors (Anderson, 1987(, the hierarchical confirmatory factor analysis (HCFA) model with measures loading indirectly on trait and method factors (Marsh and Hocevar, 1988), and the first-order multiple-informant, multiple-item (FOMIMI) model with separate factors for traits, methods, and measure specificity (Kumar and Dillon, 1990).

All the approaches mentioned so far proceed from the assumption that traits and methods independently produce variation in measures; the effects are presumed to be additive. When traits and methods interact in their impact on measures--such that the higher the association among traits, the greater the method effects--a new approach is needed. In this regard, we introduce the direct product model (Browne, 1984) and show how it applies to organizational data in certain instances. Although treatments of construct validity in the organizational behavior literature have emphasized method biases, we draw special attention throughout our presentation also to convergent and discriminant validity and how trait, method, and error effects relate to these properties. Our purpose is to show that the analysis of construct validity is a very complex and multifaceted endeavor, and a number of pitfalls reside throughout the leading apdproaches to the issue. Before we explore the various methods used to investigate construct validation, we provide a brief analysis of research in organizational studies, with particular focus on the use of key informants to measure properties of organizations. Key informants are one of the most frequently used vehicles for examining construct validity of concepts in organization research, and the principles discussed are equally applicable to other methods, such as using archival reports.

Key Informants and Construct Validation

The key-informant method is a technique for collecting information on organizations and collectivities (e.g., Seidler, 1974). Informants are chosen on the basis of particular qualifications such as specialized knowledge or position in an organization. Rather than reporting on their own personal feelings or opinions, key informants provide information on the properties of organizations, their relationships with other organizations, or other group or aggregate data (e.g., Campbell, 1955). Organizational researchers have used the technique to obtain knowledge on organizational structure (e.g., Hage and Aiken, 1967), technology (e.g., Hage and Aiken, 1969; Khandwalla, 1974), environment (e.g., Lawrence and Lorsch, 1967; Downey, Hellriegel, and Slocum, 1975), internal power relations (e.g., Hinings et al., 1974; Salancik and Pfeffer, 1974), and external exchange relationships (e.g., Schmidt and Kochan, 1977; Molnar and Rogers, 1979).