The ganzfeld debate continued: A response to Milton and Wiseman - 2001

Journal of Parapsychology, The, March, 2002 by Lance Storm, Suitbert Ertel

**********

The thrust of our critique of Milton and Wiseman's (1999a) article is condensed in our (Storm & Ertel, 2001) article's abstract:

J. Milton and R. Wiseman (1999[a]) attempted to replicate D. Bern and C. Honorton's (1994) meta-analysis, which yielded evidence that the ganzfeld is a suitable method for demonstrating anomalous communication. Using a database of 30 ganzfeld and autoganzfeld studies, Milton and Wiseman's meta-analysis yielded an effect size ES of only 0.013 (Stouffer Z = 0.70, p = .24, one-tailed). Thus they failed to replicate Bem and Honorton's finding (ES = 0.162, Stouffer Z = 2.52, p = 5.90 x [10.sup.-3], one-tailed). The authors [Storm & Ertel] conducted stepwise performance comparisons between all available databases of ganzfeld research. Larger aggregates of such studies were formed, including a database comprising 79 ganzfeld/autoganzfeld studies (ES = 0.138, Stouffer Z = 5.66, p = 7.78 x [10.sup.-9]). Thus Bem and Honorton's positive conclusion was confirmed .... The ganzfeld appears to be a replicable technique for producing psi effects in the laboratory. (p. 424)

By way of a reply to our (Storm & Ertel, 2001) article, Milton and Wiseman (2001) criticized some of our assumptions and procedures. Milton and Wiseman's (2001) response was as follows (taken from their abstract):

[Storm and Ertel] ignored the well-documented and widely recognised methodological problems in the early studies, which make it impossible to interpret the results as evidence of extrasensory perception. In addition, Storm and Ertel's meta-analysis is not an accurate quantitative summary of ganzfeld research because of methodological problems such as their use of an inconsistent method for calculating study outcomes and inconsistent inclusion criterion. (p. 434)

We address Milton and Wiseman's (2001) criticisms below in the order that they appeared in their reply, which is not always in the order of importance.

ARE QUOTES TAKEN OUT OF CONTEXT ADMISSIBLE As EVIDENCE?

We criticize Milton and Wiseman (2001) for selectively picking quotes that feature "psi-questioning" content while ignoring "psi-supporting" accounts. That is, they started off by (a) strategically quoting, at length, skeptic Ray Hyman (see Milton &Wiseman, 2001, p. 434), (b) failing to represent Honorton's (1985) and Hyman and Honorton's (1986) positive views, and (c) using a misleading passage from Hyman and Honorton (see Milton & Wiseman, 2001, p. 435). Milton and Wiseman thus quoted Hyman and Honorton (1986) out of context (Milton & Wiseman, 2001, p. 435) and misrepresented Hyman and Honorton's (1986) joint overall conclusion, which was: "we agree that the overall significance observed in these studies cannot reasonably be explained by these selective factors [i.e., "multiple testing, retrospective experiments, ... the file-drawer problem," etc.]" (p. 352). Two years later, after further testing, Harris and Rosenthal (1988b) reiterated this conclusion: "Our analysis of the effects of flaws on study outcom e lends no support to the hypothesis that ganzfeld research results are a significant function of the set of flaw variables" (p. 3).

However, Milton and Wiseman (2001, p. 435) placed greater credence on the following statement from Hyman and Honorton (1986): "the final verdict awaits the outcome of fixture experiments--ones conducted by a broader range of investigators and according to more stringent standards" (p. 353). The fact is that Hyman and Honorton also "agreed that the significant outcomes have been produced by a number of different investigators" (p. 352) and that the argument over "stringent standards" was largely rhetorical (p. 353). (Note that we do not object to Milton and Wiseman's appeal to desirable future research for further evidence, but they make their point as if past research had been inconclusive.)

Thus, Milton and Wiseman (2001) clouded the waters and misled the unsuspecting reader into thinking that the statistically significant result of Honorton's (1985) database taken at face value contributed little, if anything, to the evidence for psi because the methodological issues (see Milton & Wiseman, 2001, pp. 434-435) were of greater concern. Yet, it is common knowledge that significance testing, aside from assessments of effect size, is an indispensable way of finding out whether experimental effects should be regarded as existent. The undoubtedly justified demand for replication, within and between investigators, cannot replace the equally important demand for statistical confidence of independent studies. Even if psi would seem to entirely disappear, like an ice-age climate in earth history, previous significant observations--ice formations as in our analogy--would not become invalid.

Even Hyman and Honorton (1986), it seems, disregarded this logic to some extent when they made the distinction between significant effects, on the one hand, and evidence for psi (i.e., a communications anomaly), on the other. Yet at a very early stage, Rosenthal (1986) insisted that the accumulated evidence should not be neglected: "At any point in time some judgment can be made. . . . We feel it would be implausible to entertain the null given the combined p from these 28 studies" (p. 333). Paraphrasing Rosenthal, our judgment is that psi effects have been evidenced by significant results so that we may rightfully defend our (Storm & Ertel, 2001, p. 424) quotes taken from Hyman and Honorton (1986).