Exploring the limits of science and beyond: research strategy and status

Journal of Parapsychology, The, March, 1994 by J.E. Kennedy

Experimental research will be much more difficult if goal-oriented experimenter effects are valid. Conceivably, an experimenter with psi abilities may obtain support for virtually any hypothesis he or she wants. Research on basic issues such as the relationship between psi and other variables, optimum conditions for psi operation, and how psi works would be dominated by the experimenter's expectations or goals.

The goal-oriented psi hypothesis can be tested by meta-analysis examining the effect of sample size on experimental outcome and by analysis of signal enhancement with majority-vote procedures. Data to date appear to support the hypothesis, but further work is needed. The hypothesis explains findings that are very difficult to explain (i.e., are anomalous) with more simple models.

Sample Size and Goal-Oriented Psi

The statistical significance level of an experiment is directly related to the number of trials if psi operates on individual trials. The nature of this relationship is shown in Figure 2A. The z score is directly proportional to the square root of the number of trials, all else being equal. The direct relationship between the z score and square root of the number of trials is easily seen from the equation for the z score:

z = [N.sub.hits] - NP/[square root of NPQ] = N[P.sub.hits] - NP/[square root of NPQ] = N [P.sub.hits] - P/[square root of N] [square root of PQ] = [square root of N] [P.sub.hits] - P/[square root of PQ]

where N is the number of trials, [N.sub.hits] is the number of hits, P is the a priori probability of a hit, [P.sub.hits] is the scoring rate or proportion of hits, and Q is (1-P). Other statistical tests also follow this principle.

Traditional experimental design and statistical analyses are based on this assumption. As shown in Figure 2A, different psi strengths have lines with different slopes, but they all start at zero and increase directly with the square root of the number of trials.

However, statistical significance level is not related to number of trials under the goal-oriented experimenter-effects hypothesis. Because the overall experiment is viewed as one complex random event with the a priori probability of success equal to .05, the z score would be constant, regardless of such details as the number of subjects or trials in the experiment. This relationship is shown in Figure 2B. Different psi strengths have lines that are constant and parallel. The differences between Figures 2A and 2B are clearly testable in principle.

A meta-analysis of electronic RNG experiments supported the goal-oriented psi hypothesis. Using 332 z scores from 56 reports, May et al. (1985) tested several models, including both the goal-oriented psi model and the model that assumes that psi operates on individual trials. The data fit the goal-oriented psi model and were significantly different from the model that assumes that psi operates on individual trials. However, several aspects of the analyses need clarification before these results can be interpreted with confidence regarding goal-oriented psi.(3)