How Will We Know "Good" Qualitative Research When We See It? Beginning the Dialogue in Health Services Research

Health Services Research, Dec, 1999 by Kelly J. Devers

Objective. To lay the foundation for an explicit review and dialogue concerning the criteria that should be used to evaluate qualitative health services research. Clear criteria are critical for the discipline because they provide a benchmark against which research can be assessed.

Data Sources. Existing literature in the social sciences and health services research, particularly in primary care and medicine.

Principal Findings. Traditional criteria for evaluating qualitative research are rooted in the philosophical perspective (positivism) most closely associated with quantitative research and methods. As a result, qualitative research and methods may not be used as frequently as they can be and research results generated from qualitative studies may not be disseminated as widely as possible. However, alternative criteria for evaluating qualitative research have been proposed that reflect a different philosophical perspective (post-positivism). Moreover, these criteria are tailored to the unique purposes for which qualitative research is used and the research designs traditionally employed. While criteria based on these two different philosophical perspectives have much in common, some important differences exist.

Conclusion. The field of health services research must engage in a collective, "qualitative" process to determine which criteria to adopt (positivist or post-positivist), or whether some combination of the two is most appropriate. Greater clarity about the criteria used to evaluate qualitative research will strengthen the discipline by fostering a more appropriate and improved use of qualitative methods, a greater willingness to fund and publish "good" qualitative research, and the development of more informed consumers of qualitative research results.

Key Words (MeSH Terms). Health Services Research (Methodology or Health Services Research, Methodology; Research--Methods or Research, Methods; Reproducibility of Results; Research Design; Evaluation Studies; Health Care Evaluation Methods

Other Key Words. Qualitative research and methods, mixed methods, standards, evaluation criteria

Nothing provides more opportunity to strengthen a discipline than an explicit examination and clarification of the criteria by which research is assessed. Such criteria represent a discipline's collective perspective on fundamental issues, including the kinds of questions that are worth asking; the ways of answering questions that are legitimate; the constituent elements of evidence and the weighting of those elements; the appropriate role of the researcher; and, the ways in which results should be presented and used. In essence, criteria provide a benchmark against which research can be evaluated.

The purpose of this article is to lay the foundation for an explicit review of, and dialogue about, the criteria that should be used to evaluate qualitative health services research. [1] The question of how to assess the rigor of a qualitative study has engendered significant debate within basic (e.g., anthropology and sociology) and applied (e.g., evaluation research) social science disciplines and the clinical sciences (e.g., medicine). Moreover, this debate can be extraordinarily difficult to map and understand for two reasons. First, qualitative research is neither monolithic nor static. The particular issues of concern in each discipline and the language (i.e., the disciplinary jargon) used to discuss them are somewhat unique and evolving given the discipline's tradition of qualitative research. The philosophical and theoretical perspectives [2] that inform the use of particular qualitative methods (e.g., observation, interviewing), and the extent to which qualitative methods are accepted as legitimate modes of inquiry, shape the debate in any field. As a result, it can be very difficult to follow developments within and across disciplines.

Second, different ways of using the term "qualitative research" often create confusion. Sometimes the term denotes a paradigm that competes with quantitative research and the philosophical perspective with which it is associated (i.e., positivism). Building on the work of Kuhn (1970), Patton (1990) defines a paradigm as:

a world view, a general perspective, a way of breaking down complexity of the real world. As such, paradigms are deeply embedded in the socialization of adherents and practitioners: paradigms tell them what is important, legitimate, and reasonable. Paradigms are also normative, telling the practitioner what to do without the necessity of long existential or epistemological consideration.

At other times, the term "qualitative research" refers to a diverse set of methods for conducting social research that are appropriate for answering particular types of research questions and, therefore, are capable of being integrated with quantitative research. [3] Authors often vacillate between the philosophical and methodological meanings without being clear about the one to which they are referring or without specifying the relation between the two. As discussed further on, the link between paradigms, theoretical perspectives, and particular qualitative methods is real but imperfect.