Qualitative research in health care: assessing quality in qualitative research

British Medical Journal, Jan 1, 2000 by Nicholas Mays, Catherine Pope

In the past decade, qualitative methods have become more commonplace in areas such as health services research and health technology assessment, and there has been a corresponding rise in the reporting of qualitative research studies in medical and related journals? Interest in these methods and their wider exposure in health research has led to necessary scrutiny of qualitative research. Researchers from other traditions are increasingly concerned to understand qualitative methods and, most importantly, to examine the claims researchers make about the findings obtained from these methods.

The status of all forms of research depends on the quality of the methods used. In qualitative research, concern about assessing quality has manifested itself recently in the proliferation of guidelines for doing and judging qualitative work.[2-5] Users and funders of research have had an important role in developing these guidelines as they become increasingly familiar with qualitative methods, but require some means of assessing their quality and of distinguishing "good" and "poor" quality research. However, the issue of "quality" in qualitative research is part of a much larger and contested debate about the nature of the knowledge produced by qualitative research, whether its quality can legitimately be judged, and, if so, how. This paper cannot do full justice to this wider epistemological debate. Rather it outlines two views of how qualitative methods might be judged and argues that qualitative research can be assessed according to two broad criteria: validity and relevance.

Two opposing views

There has been considerable debate over whether qualitative and quantitative methods can and should be assessed according to the same quality criteria. Extreme relativists hold that all research perspectives are unique and each is equally valid in its own terms, but this position means that research cannot derive any unequivocal insights relevant to action, and it would therefore command little support among applied health researchers.[6] Other than this total rejection of any quality criteria, it is possible to identify two broad, competing positions, for and against using the same criteria.[7] Within each position there is a range of views.

Separate and different: the antirealist position

Advocates of the antirealist position argue that qualitative research represents a distinctive paradigm and as such it cannot and should not be judged by conventional measures of validity, generalisability, and reliability. At its core, this position rejects naive realism--a belief that there is a single, unequivocal social reality or truth which is entirely independent of the researcher and of the research process; instead there are multiple perspectives of the world that are created and constructed in the research process.[8]

Those relativists who maintain that assessment criteria are feasible but that distinctive ones are required to evaluate qualitative research have put forward a range of different assessment schemes. In part, this is because the choice and relative importance of different criteria of quality depend on the topic and the purpose of the research. Hammersley has attempted to pull together these quality criteria (box).[7] These criteria are open to challenge (for example, it is arguable whether all research should be concerned to develop theory). At the same time, many of the criteria listed are not exclusive to qualitative research.

Using criteria from quantitative research: subtle realism

Other authors agree that all research involves subjective perception and that different methods produce different perspectives, but, unlike the antirealists, they argue that there is an underlying reality which can be studied.[9 10] The philosophy of qualitative and quantitative researchers should be one of "subtle realism"--an attempt to represent that reality rather than to attain "the truth" From this position it is possible to assess the different perspectives offered by different research processes against each other and against criteria of quality common to both qualitative and quantitative research, particularly those of validity and relevance. However, the means of assessment may be modified to take account of the distinctive goals of qualitative research. This is our position.

Assessing the validity of qualitative research

There are no mechanical or "easy" solutions to limit the likelihood that there will be errors in qualitative research. However, there are various ways of improving validity, each of which requires the exercise of judgment on the part of researcher and reader.

Triangulation

Triangulation compares the results from either two or more different methods of data collection (for example, interviews and observation) or, more simply, two or more data sources (for example, interviews with members of different interest groups). The researcher looks for patterns of convergence to develop or corroborate an overall interpretation. This is controversial as a genuine test of validity because it assumes that any weaknesses in one method will be compensated by strengths in another, and that it is always possible to adjudicate between different accounts (say, from interviews with clinicians and patients). Triangulation may therefore be better seen as a way of ensuring comprehensiveness and encouraging a more reflexive analysis of the data (see below) than as a pure test of validity.