Comment on the recent work of Kwon, Scott, Safranski, and Bae: no, your evidence dosen't prove what you think it does! - response to Ik-Whan Kwon, The American Journal of Economics and Sociology, vol. 56, p. 41, 1997

American Journal of Economics and Sociology, The,  July, 1998  by Tomislav V. Kovandzic

• E-mail
• Print
• Link

I

Introduction

Kwon, Scott, Safranski, and Bae (1997, p. 41) use a cross-sectional research design to assess the effectiveness of gun control laws on firearm related deaths. According to their assessment: "The multivariate statistical regression model suggests that the existence of gun control laws indeed have a deterrent effect on firearm deaths" (p. 41). Although some gun control laws indeed might have a deterrent effect as claimed by Kwon et al., their evidence does not support this conclusion. In fact, the Kwon et al. study suffers from many methodological flaws and this renders any substantive conclusion impossible. Most of what is reported by Kwon et al. can be attributed to methodological artifacts. Furthermore, the authors omit the mention of a large number of published books and articles about the relation between gun control laws and violence.

Previous Research

The authors in this study are simply wrong when they state, "In spite of charged emotional debates and passage of numerous laws and regulations, no empirical studies have been done to evaluate the effectiveness of gun control laws in this country. The debate on the Brady Bill could have been better informed by scientific research" (p. 41). At the time this article was published in the AJES, there were at least forty-five empirical studies that evaluated the impact of gun laws on violent crime, suicide, and gun accidents, the bulk of which suggest that gun laws have no impact on rates of violence (Kleck 1991, pp. 251-255, 302-303, 390-392; Kleck, 1995).

More specifically, Kleck and Patterson (1993) completed a Study that evaluated the effectiveness of nineteen gun laws at both the state and city level; they found that most gun control restrictions have no net effect on violence rates. Criminologists on both sides of the gun control debate have cited the Kleck and Patterson study widely, demonstrating a technically more sophisticated research methodology.

Kwon et al. compare their results to a study conducted by Mauser and Holmes (1992). They then erroneously inform their readers that the Mauser and Holmes study found a deterrent effect for the 1977 Canadian gun law on homicide rates. In fact, Mauser and Holmes (1992) stated in their conclusion, "The results are consistent with the findings of previous studies that the 1977 Canadian firearms legislation did not have a significant effect on homicide" (p. 613).

III

Which Gun Laws were Evaluated?

The authors claim to examine the effectiveness of gun control laws prior to the passage of the Brady Bill in 1992. Unfortunately, they do not make clear which laws they attempt to evaluate. Instead the authors state, "The purpose of this study is to investigate the effectiveness of laws and regulations prior to the passage of the Brady Bill in 1992. A multivariate statistical technique is proposed to establish the relationship between the number of gun related deaths by states and sets of determinants including state laws and regulations on firearm use" (p. 42).

This passage suggests that all current state gun law statutes prior to the passage of the Brady Bill in 1992 were to be evaluated. As of 1990 there were at least nineteen different types of gun laws in the United States ranging from gun registration requirements to assault weapon and handgun bans (Kleck and Patterson 1993, p. 262). However, Kwon et al. focus on only three types of gun laws: those requiring (1) background checks, (2) licensing requirements, and (3) mandatory waiting periods.

IV

Tests of Statistical Significance

The most serious problem with this work is the authors' disregard for tests of statistical significance. Kwon et al. state, "According to the model, states with gun control laws had almost 3 fewer deaths per 100,000 than states without any such laws. The relationship, however, is not statistically significant" (p. 46). Kwon et al. may have misunderstood what significance tests mean. They found no statistically significant negative relationship between gun laws and firearm-related deaths, but they continually refer to their findings as if they did. In fact, the relationship was not even significant at the more generous .10 level or even the .15 level for that matter. However, Kwon et al. simply ignore these tests and rely on the negative beta coefficient that their regression results produced. They do not seem to appreciate that by chance alone they had a 10 percent chance of obtaining a negative coefficient, given that the coefficient had to be positive or negative, even if the true coefficient was exactly zero. Thus, the notion that states with restrictive gun laws had almost three fewer deaths per 100,000 is completely unfounded and can be attributed mostly to noise.

V

Unit of Analysis

Kwon et al. use U.S. states as their unit of analysis. This is problematic for two reasons. First, states are arbitrary statistical aggregations, including vastly divergent ecological units (such as large cities and rural fanning areas). Thus; states are more heterogeneous with respect to levels of violence and variables that affect violence rates. Second, the most restrictive gun laws in the United States are at the local level. Thus, residents of a particular state might be subject to very strong gun laws at the city level, but they might face little or no state regulation (Kleck 1993, p. 253). As Kleck (1991) noted, "A state could have one area with high violence rates but little local gun regulation, while the rest of its component areas have moderately low violence rates and severe local gun regulation.... Yet when areas are lumped together in the entire state unit, the high violence areas could dominate the violence measure so much that the state showed a higher-than-average violence rate despite generally severe local gun restrictions. The aggregation would thus conceal a causal effect of gun laws evident at lower levels of aggregation" (Ch. 10, p. 389). Thus, Kwon et al. are unable to measure the degree to which the majority of state residents are subject to restrictive gun control.