With enough cases, why do you need statistics? Revisiting Kinsey's methodology

Journal of Sex Research, May, 1998 by Julia A. Ericksen

It is appropriate to reexamine Kinsey's methodology on the 50th anniversary of the publication of his landmark Sexual Behavior in the Human Male (Kinsey, Pomeroy, & Martin, 1948). Kinsey's work is important in a number of ways but particularly because it legitimated the sex survey in American academia. Although data collection for the first sex survey started in 1892, it was never published. Between 1892 and 1938, when Kinsey started data collection, fewer than 24 surveys appeared. After Kinsey's books appeared, sex surveys began to appear with regularity, and over 750 have been published since Kinsey's day, many of them by readers of this journal.

Although Kinsey's methodology received early criticism from three world-renowned statisticians (Cochran, Mosteller, & Tukey, 1953), most writers have ignored this aspect of his work and have focused instead on the large number of interviews and the path-breaking nature of the data. Cochran et al. (1953) undertook their critique because the Rockefeller Foundation, which funded Kinsey's research through the National Research Council's Committee for Research in the Problems of Sex, was stung by critiques of Kinsey et al.'s (1948) methodology (Christensen, 1971). Cochran et al. found much to praise in Kinsey's work, and they acknowledged that Kinsey could not have been expected to use more sophisticated methods when he began. It is important to remember this point when reviewing Kinsey's methodology from the vantage point of 60 years' additional experience with sex surveys. Today we look back at Kinsey with the hindsight afforded those who have access, not only to these many sex surveys, but also to many improvements in survey practice.

Today, we raise questions about the nature of science, but Kinsey believed that proper science was completely objective. Its role, he thought, was to discover the facts about a topic and to present them to others. These facts would be unaffected by either the historical moment or the complexities of respondents' sexual identities. Although some accuse Kinsey of a greater interest in fighting ideological battles than in searching for the truth (Jones, 1997), there is little doubt that he viewed himself as an objective scientist. Kinsey was certainly an advocate for sexual tolerance, but he saw science as his ally in this because he thought that the scientific facts he uncovered would liberate Americans from years of "Victorian repression." Thus, he worked diligently to reveal the truth about the varieties of sexual behavior in America. Kinsey did not recognize that his own social location created class, gender, and education biases that shaped his observations. Nor did he recognize that his personal sexual interests, including his desire to validate sexual variety, would influence his results. Jones' (1997) recent biography of Kinsey makes much of this last point and argues that, had Kinsey had a less personal stake in sexual variety, he would have been a neutral observer. An individual with more conservative sexual tastes, however, would have had as much of a sexual interest, albeit a different one. A more conservative researcher might have discovered a more sexually conservative America, but this too would be a product of the researcher's view of the world and not of more truthful results.

What was Kinsey's research question? Kinsey apparently thought the answer was obvious,,so he provided only a brief account of his research goals. In Kinsey et al.'s (1948) book, these appear at the beginning of a 30-page introduction that criticizes earlier studies, reviews the data sources, and describes Kinsey's approach to science. The research question appears on the first page: "[the study] is a fact-finding survey in which an attempt is being made to discover what people do sexually, and what factors account for differences in sexual behavior among individuals, and among various segments of the population." In Sexual Behavior in the Human Female, Kinsey, Pomeroy, Martin, & Gebhard (1953) posed a similar research question along with two additional questions: "how [individuals'] sexual experiences have affected their lives, and what social significance there may be in each type of behavior" (p. 3).

Kinsey et al. (1948) acknowledged that such data might be used by physicians, psychiatrists, and other professionals in need of accurate information. By bemoaning the lack of information available to answer sexual questions, they even revealed concerns about the limits of science in a sexually restrictive society. Yet, they insisted that they could not solve this problem, given that their task as scientists was to obtain "an objectively determined body of fact about sex" with no "social or moral interpretations" (p. 5). They could not interpret the data in terms of their moral values, they argued, because "this is not part of the scientific method and, indeed, scientists have no special capacities for making such evaluations" (p. 5). Kinsey's faith in science and his assumption that its revelations would be uncontaminated by the persona of the scientist are typical of his day. We cannot expect him to have lived outside of his historical context. However, we can examine the kind of science created by this historical context, and we can question the reliability of his results. There are four major areas to which we can attend: defining and sampling the population to be surveyed; questionnaire design; interviewer training and interviewing; and data reduction and analysis.