Assessing construct validity in organizational research - includes appendices

Administrative Science Quarterly, Sept, 1991 by Richard P. Bagozzi, Youjae Yi, Lynn W. Phillips

The results (see first row of Table 1) show that convergent validity is achieved in that all four monotrait-heteromethod correlations (i.e., .46, .55, .82, .74) from the validity diagonal (see Appendix A) are large and significantly different from zero (p < .01). The first discriminant validity criterion is also satisfied in that each monotrait-heteromethod correlation is greater than each correlation lying in its row and column of the heterotrait-heteromethod triangles (indicated with dashed lines in Appendix A). This criterion involves a total of 24 comparisons, and all the differences in correlations are statistically significant (p < .05). For example, the validity diagonal value of .46 for the first trait is significantly higher than its corresponding values in the heterotrait-heteromethod triangles (i.e., it is greater than .25, .07, .12, .17, .13). Application of the second discriminant validity criterion shows that 23 of 24 (95.8 percent) comparisons of each validity diagonal correlation to its corresponding entry in the heterotrait-monomethod triangles (the solid triangles) produce statistical significance (p < .05) in the proper direction. For example, the validity diagonal value of .74 for the fourth trait is significantly higher than .27 and .38 in the heterotrait triangle for method 1 but is actually less than the correlation of .75 between the measures of the third and fourth traits by method 1, thus showing one violation in the second criterion for discriminant validity.

However, a potential problem with Campbell and Fiske's criteria should be apparent at this point. Much ambiguity exists in the criteria as to what constitutes satisfactory results. We are left with the task of determining whether the proportion of failures is acceptable. A rule of thumb is needed for deciding what constitutes a failure to meet the criteria. Because there is no widely accepted standard, we will adopt the .05 level as the rule of thumb in this article; that is, if the proportion of failures is higher than 5 percent, we conclude that the violations cannot be attributed to chance. (2) Applying this rule of thumb to the second discriminant validity criterion, we find that the lone violation is less than what might occur by change (p = 1/23 = .04 < .05); therefore, the criterion is satisfied.

Finally, the patterns of correlations in all heterotrait triangles in both monomethod and heteromethod triangles are compared by using Kendall's Coefficient of Concordance. The results show that the third criterion for discriminant validity is satisfied. That is, one can reject the hypothesis that the pattern of correlations is discordant: [X.sup.2](5) = 17.8, p < .001.

In the second study, Phillips (1980) used the key-informant methodology to ascertain the extent of usage of computers by 205 distribution companies in forecasting, accounting, and financial practices. Two informants--the CEO and subordinate--provided information on each of the three traits. But unlike Bagozzi and Phillips (1982), who aggregated across multiple items for their measures of traits, Phillips (1980) used the items directly as multiple indicators of each trait (see Appendix A). As we show below, these data permit the application of a new model (the FOMIMI model) to the MTMM, which has not been applied in organizational research.