Measurement in veterans affairs health services research: veterans as a special population

Health Services Research, Oct, 2005 by Robert O. Morgan, Cayla R. Teal, Siddharta G. Reddy, Marvella E. Ford, Carol M. Ashton

The U.S. Department of Veterans Affairs Health Services Research and Development (VA HSR&D) service has been a leader in focusing on system-wide measurement excellence, in part because veterans represent a distinct and special population. Veterans are a large and highly selected group with an increasingly recognized cultural identity. Further, eligible veterans are served by a large, nationally integrated health care system. Thus, the veteran population and the VA health system embody a microcosm in which measurement quality must be assessed and addressed. In this article, we briefly discuss the consequences of poor measurement and the importance of assessing measurement quality across the varied populations in which health services research is conducted, using the veteran population as a case example. We then introduce a series of related articles, each of which focuses on a measurement issue important in current health services research.

MEASUREMENT IN HEALTH SERVICES RESEARCH

Data that are unreliable or have poor validity can lead to erroneous and nongeneralizable study results through a combination of low statistical power and lack of sensitivity in data analyses, biases in statistical conclusions, and biases in estimates of prevalence and risk (Skinner, Teresi, and Holmes 2001). These errors can affect our understanding of therapeutic effectiveness by restricting our ability to detect an intervention's effect, and distort our assessments of the epidemiology of medical conditions by biasing our assessment of different subpopulations of patients.

It is widely recognized that measurement properties such as reliability and validity are both sample- and purpose-dependent (Anastasi 1998). That is, they vary across the populations and settings in which measures are used. Typically, researchers are most familiar with these issues in the context of measurement with self-report instruments, surveys, or scales. On scales, for example, individual items may differ across populations in terms of how they relate to the underlying constructs being measured, and the constructs themselves may shift across populations. Measures may be affected by differences in demographic characteristics (e.g., age, socioeconomic status, location), illness burden, psychological health, or cultural identity. Consequently, a scale developed to assess communication ability in Anglo Americans may not be as effective when used with African Americans or Hispanic Americans; a scale may not work as well with individuals raised in a rural setting as with those raised in an urban one; or the properties of a scale developed in a sample of young female patients may not generalize when the scale is used with older males. Similarly, the measurement properties of scales may vary according to how they are used. For example, a measure developed for assessing cross-sectional group differences in health status may be inadequate as an instrument for measuring change over time for a particular individual. When measurement is conducted via survey methodology, these vulnerabilities may be compounded by biased nonresponse to the survey or partial completion of survey items.

The need to verify measurement properties extends beyond "traditional" psychometric applications (e.g., reliability or validity of survey or other self-report measures) and beyond the characteristics of the population we are attempting to study. For the U.S. population in general, there are substantial differences among the health care systems in which individuals seek care. These differences may affect entry into the system (e.g., access), therapeutic decisions (e.g., quality), and availability of end-points (e.g., outcomes). Thus, as health services researchers, measurement and our resulting research findings are influenced by features of the health care system. Health services research incorporates measurements obtained through direct observation, self-report, or from administrative or medical records (e.g., illness classification, health care use, morbidity, mortality). Attention to measurement quality necessarily includes design issues (e.g., formatting and administration of measurement instruments), settings in which measurement is conducted (e.g., at a physician's office versus a hospital setting, or at home), and the source from which the measures are obtained (e.g., self-report by an individual, observer rating, administrative or medical record).

VETERANS AS A SPECIAL POPULATION

Research with veterans and within the VA health care system serves as a case example of how measurement can be affected by the issues raised above. The population characteristics of veterans reflect the characteristics of the armed forces in which they served. As a group they are predominantly male, and more educated and better off financially than the general U.S. male population (Klein 2001; Klein and Stockford 2001). The male veteran population is projected to decline substantially (approximately 27 percent) between 200,5 and 2020. In contrast, the female veteran population is projected to increase by 12 percent over the same period, reflecting the changing gender composition of the military (U.S. Department of Veterans Affairs 2000). How these changes will be altered by the conflict in Iraq is unclear. Like the Gulf War, the war in Iraq has seen substantial mobilization of military reserve units; and although training for the Reserves or National Guard unit does not entitle an individual to veterans' benefits, activation for service does.