A meta-analysis of the situational interview: an enumerative review of reasons for its validity

Canadian Psychology, Feb 1999 by Latham, Gary P, Sue-Chan, Christina

Abstract. The premise underlying the situational interview (SI) is that intentions predict behaviour. The distinguishing characteristics of the SI include the depiction of a dilemma in the interview question, the use of an item scoring guide, and summative scoring of items to yield a total interview score. A meta-analysis indicated that the weighted mean observed criterion-related validity of the SI is .35 (n = 1010), and the mean corrected criterion-related validity is .47. An enumerative review of the literature revealed that the SI is applicable across a diversity of jobs, participants, performance criteria, and countries. The criterion-related validation evidence for the SI is explained in terms of content, construct, and incremental validity, freedom from interviewer biases, and practicality. The article concludes with a comparison of the SI to interviews that measure either past behaviours or future intentions where the interview question does not contain a dilemma or scoring guide, and performance dimensions rather than each individual item are scored.

The situational interview (SI) is structured in that all applicants are asked the same questions. The questions, derived from a job analysis, are evaluated by a panel of two or more interviewers. The premise underlying the SI is that intentions are the immediate, though not sole, regulators of human action (Locke & Latham, 1990). Thus the technique is based on goal-setting theory (Latham, 1989; Latham & Sue-Chan, 1996).

Two core aspects of the SI are the development of situational dilemmas that employees encounter on the job, and a scoring guide to evaluate responses to each dilemma. A dilemma is defined as having to choose between two or more mutually exclusive courses of action. The purpose of the dilemma is to "force" interviewees to state their intentions as to what they would do in various job situations by presenting them with questions that minimize the probability of socially desirable responses. The purpose of the scoring guide is two-fold. First, it requires organizational decision-makers to articulate the organization's culture or values and hence the behaviour desired of job incumbents. Second, it facilitates agreement among interviewers as to what constitutes an appropriate response to an interview question. An example of a SI is as follows:

You are in charge of truck drivers in Toronto. Your colleague is in charge of truck drivers in Montreal. Both of you report to the same person. Your salary and bonus are affected 100% by your costs. Your colleague is in desperate need of one of your trucks. If you say no, your costs will remain low and your group will probably win the Golden Flyer award for the quarter. If you say yes, the Montreal group will probably win this prestigious award because they will make a significant profit for the company. Your boss is preaching costs, costs, costs, as well as co-operation with one's peers. Your boss has no control over accounting who are the score keepers. Your boss is highly competitive; he or she rewards winners. You are just as competitive; you are a real winner! What would you do in this situation?

A meta-analysis of the interview revealed that interviews which are structured, based on a job analysis, and use a panel have a mean corrected validity coefficient of .60 (Wiesner & Cronshaw, 1988). Although this analysis provided strong support for key steps used in developing a SI, it did not single out the SI from other structured interview techniques.

Wright, Lichtenfels, and Pursell (1989) focused on the SI in their meta-analysis. They found a mean corrected validity of .39. However, the studies in their meta-analysis included the Kennedy (1985) studies which, as discussed in a subsequent section, are methodologically flawed.

Huffcutt and Arthur (1994) found that the mean corrected validity of interviews with the highest degree of structure, namely the SI, is .57. A limitation of this meta-analysis is that it is unclear whether the interview questions were derived from a job analysis even though they reported that the interview questions and response scoring was standardized. A second limitation is that the studies in their meta-analysis examined only entry-level jobs. Other limitations of their meta-analysis are discussed in a subsequent section.

r n Ryy SD Ratio

----------------------------------------------------------------------

.34 149 .76 65

----------------------------------------------------------------------

.39 70 .92 --

----------------------------------------------------------------------

.32 47 -- --

----------------------------------------------------------------------

.53 29 .86 --

----------------------------------------------------------------------

-.07 34 -- --