A central question in cross-cultural research: do employees of different cultures interpret work-related measures in an equivalent manner?

Journal of Management, Fall, 1994 by Christine M. Riordan, Robert J. Vandenberg

1. The way my supervisor listens when I have something important to say.

2. The way my supervisor sets clear work goals.

3. The way my supervisor treats me when I make a mistake.

4. My supervisor's fairness in appraising my job performance.

5. The way my supervisor is consistent in his/her behavior toward subordinates.

An understanding of cross-cultural diversity is critical to the development of international business strategies and to the management of multicultural organizations (Cox & Blake, 1991). Yet, cross-cultural diversity as an area of study is in its early stages. To date, the majority of cross-cultural research has focused primarily upon comparative differences between cultural groups. These identified differences, in turn, are commonly used as vehicles for explaining and understanding the influence of culture on work-related attitudes, behaviors, and values (see the reviews by Barrett & Bass, 1976; Price-Williams, 1986). Within these comparative studies, however, there has not been a comparable devotion to investigating the equivalence of the measures used to operationalize organizational constructs between the culturally diverse groups (Drenth, 1985). Specifically, the ability to meaningfully interpret findings (e.g., differences in mean values) from studies directly comparing cultures "... depends among other things upon: (1) the construct validity of measures within each population; and (2) the equivalence of measures and constructs across populations being compared" (Cole & Maxwell, 1985, pp. 389-390). The concern here is that while the construct validity of measures is commonly established in single populations (most often the U.S.), little research has been conducted on whether the measures operationalize equivalent constructs across cultural groups. For example, both older (e.g. Luthans, McCall & Dodd, 1985) and contemporary (e.g. Earley, 1989; Ralston, Gustafson, Elsass, Cheung & Terpstra, 1992) cross-cultural studies have failed to fully examine the degree to which self-report measures are equivalent across the cultural groups prior to using them in tests of a substantive nature from which meaningful inferences about cultural differences are made.

Underlying the general concern for full measurement equivalence in cross-cultural research are two issues: (1) the comparability of constructs; and (2) the calibration of true scores (Labouvie, 1980). In brief, the first issue is whether research instruments elicit the same conceptual frame of reference in culturally diverse groups. It is possible that members of distinct cultures do not use a common frame of reference when responding to the items of a given instrument. The use of different frames of reference by the diverse cultural groups renders comparisons between cultures impossible because scores on the instrument refer to different constructs for each group (Millsap & Everson, 1991; Millsap & Hartog, 1988). The latter finds its basis in the fact that general value characteristics often differ between cultures (Hofstede, 1980). These general values, in turn, define to a large extent the specific values and goals people display within their work contexts, and thus, provide a frame of reference against which they define work experiences (Meindl, Hunt & Lee, 1989). Since many research instruments are designed to tap some aspect of work experience, it is not readily apparent that respondents from diverse groups will complete the measure using the same definitions of organizational concepts. Any comparison between cultural groups is only appropriate if construct equivalence between them is established first.

The second issue is whether there is true-score equivalence between the cultural groups. This concern is whether respondents calibrate the intervals anchoring the measurement continuum in the same manner. Cultural backgrounds may also alter how the rating scales on measurement instruments are interpreted, and thus, how the groups perceive differences between the intervals underlying the scale (Millsap & Hartog, 1988). As an illustration, a 3 [neither agree nor disagree] on a 5-point Likert scale may mean "no opinion" to a group of American employees, while it may be interpreted as "mild agreement" in a group of Korean employees. As with conceptual (construct) equivalence, the lack of true-score equivalence may also result in an inappropriate interpretation of the data.

The purpose of the present study was to examine the degree to which the equivalence of measures is truly problematic for cross-cultural research. Given the increasing prevalence and interest in cross-cultural research within the management field, an examination of these issues from a cross-cultural perspective is both timely and appropriate. Richard Steer's (1989) most succinctly states the importance of promoting the examination of equivalence issues within the domain of cross-cultural research: "The instruments used are typically existing American instruments hastily translated (and often not back-translated) into another language. The results too often come straight from a computer printout, with little effort to get behind the data to understand process dynamics". The process dynamics under examination in the present cross-cultural study are the problems that occur when assumptions of measurement equivalence are not fulfilled.