Effects of information sequence and irrelevant distractor information when using a computer-based decision aid

Decision Sciences, Winter 2001 by Reneau, J Hal, Blanthorne, Cindy

Further tests of H1 and H2 were conducted using the subjects' categorical (fail/not fail) judgments. Nine separate two (information sequence) by two (irrelevant distractor information) hierarchical log linear models were run. Significant effects were observed for five of the nine firms. A summary of the significant effects and within-cell proportions are shown in Table 5. All significant differences are in the direction predicted by H1 and H2.

Hypothesis 3 concerns subjects' confidence in their judgments of continued existence. A two (information sequence) by two (irrelevant distractor information) MANOVA was run using the nine confidence scores as the dependent measures. As predicted, the main effect for information sequence (p = .43), main effect for irrelevant distractor information (p = .22), and interaction (p = .29) were not significant. Because no significant effects were observed in the multivariate analysis, no follow-up univariate analyses are reported.

In summary, the multivariate analyses provide support for all three predictions. Univariate analyses provide additional consistent support for H1 and H2. Univariate analysis was not completed for H3 since the multivariate analysis did not suggest an effect on confidence (as predicted).

DISCUSSION

Designers of decision aids must make choices concerning both the form and content of information to display to decision makers. This study reveals effects on auditor judgments in a failure prediction task resulting from differences in both the form and content of information displayed by a computerized decision aid.

Concerning form, a recency sequence effect was observed. Judgments were more accurate when information was displayed such that the more highly diagnostic information appeared late rather than early in the information sequence. Recency effects in auditors' judgments have been observed in other tasks and settings (e.g., Messier & Tubbs, 1994), but have not previously been associated with auditors' judgments in a prediction task supported by a computerized decision aid.

Concerning information content, an effect of irrelevant distractor information was observed. Judgments were more accurate when irrelevant distractor information was not presented than when irrelevant distractor information was presented to the auditors. Effects of irrelevant distractor information have been previously observed in non-business tasks and settings.

Neither the information sequence manipulation nor the irrelevant distractor information manipulation was systematically associated with judgment confidence. This finding supports normative models which predict that judgment confidence should not be affected by either information sequence or irrelevant distractor information.

Although judgment accuracy was higher when diagnostic information was presented at the end of the sequence and when no irrelevant distractor information was presented, auditors' judgments did not closely approximate the model's predictions. Further, judgments did not converge on the model's predictions over the experimental trials; judgments were conservative in the sense that subjects' failure probabilities generally exceeded those of the decision aid. Previous research has found similar patterns of lack of convergence on a model's prediction in tasks involving inconsistent cues (e.g., Peterson & Pitz, 1988), and two decades of research has documented conservatism in auditors' judgments (reviewed in Smith & Kida, 1991). Although the causes of this conservatism are not fully understood, the explanation is probably complex, with variables including technical features of the decision aid, motivational and cognitive attributes of auditors, consequences of incorrect judgments, and intentional strategic auditor behavior playing a role (Boatsman et al., 1997). Conservatism in auditors' judgment may simply be rational, having been characterized as a "functional heuristic that serves to minimize economic losses" (Smith & Kida, 1991, p. 485).