Some comments on the robustness of student t procedures

Journal of Engineering Education, Jan 2003 by Tebbs, Joshua M, Bower, Keith M

ABSTRACT

Many business and engineering courses stress the use of confidence interval and hypothesis testing procedures based on the Student t distribution. However, students are often unaware of the underlying assumptions that govern these procedures and of the consequences of misapplying them. In this exegetic account, we discuss the concept of robustness by exploring violations of the Student t assumptions and the effects of those violations on the resulting inferences. We also discuss the use of randomization methods as possible alternative methods. Our goal is to familiarize readers with the underpinnings of the t procedures, to summarize their use in practice, and to offer words of caution as to when they should not be used.

I. INTRODUCTION

When conducting an experiment to investigate treatment differences or potential process changes, experimenters frequently use hypothesis tests or confidence intervals based on the t distribution. These model based inferential techniques have a well known history of successful application and are widely used in engineering, business, medicine, and other areas. Because of their broad applicability, these procedures are emphasized in business and engineering curricula and are used extensively in industry and research.

Assumptions for the usual t tests and confidence intervals indude that (a) the data from the experiment are well represented by normal distributions, and that (b) the samples are drawn randomly from specific populations. In practice, these assumptions may be reasonable, but they need not be. For example, other probability models such as the Poisson, exponential, or Weibull, may more accurately represent the true process distribution. Also, the random sampling assumption may not be viable by design. In fact, in many manufacturing settings, it is not uncommon for the experimenter to randomize a nonrandom sample of individuals (e.g., parts, products, batches) to treatments rather than taking a random sample from distinct populations.

It is important to be cognizant of those assumptions that underpin the t procedures; in addition, it is also necessary to understand what occurs when the normality and random sample assumptions are violated. The primary focus of this manuscript is to descant these issues and to discuss randomization methods and their potential uses in industrial and research contexts.

II. DEPARTURES FROM NORMALITY

In introductory level statistics courses, students are often taught that if Y^sub 1^, Y^sub 2^, ..., Y^sub n^ represents a random sample of size n from a normal population, then hypothesis tests of the form H^sub 0^:(mu) = 0 versus H^sub 0^:(mu) -= (where (mu) denotes the population or process mean) are based on the one sample t statistic given by t =(square root)n(Y - (mu)^sub 0^)/S, where Y and S denote the sample mean and sample standard deviation, respectively, and (mu)^sub 0^ represents the value of mu under the null hypothesis. Values of t are then compared to the t distribution with n - 1 degrees of freedom, hereafter denoted t^sub n-1^, since this represents the distribution of t in repeated sampling.

When Y^sub 1^, Y^sub 2^, ..., Y^sub n^ do not follow a normal distribution, the t statistic does not have a t^sub n-1^ distribution. However, it may be approximated by a t^sub n-1^ distribution for large n, a result due largely to the Central Limit Theorem. The implication is that, in practice, experimenters may still use the t statistic to make inferences regarding a process mean in large samples. In light of this, students frequently pose the obvious question, "How large should n be?"

Attempting to answer this question, authors often provide the ubiquitous n > 30 guideline for the approximation to be useful in practice. However, this guideline can be misleading. In truth, the goodness of this approximation depends not only on the sample size, but rather in conjunction with the skewness of the original process distribution. For distributions that are symmetric, n does not need to be that large (e.g., n = 10 may suffice). On the other hand, for heavily skewed or bimodal distributions, one may need n to be much larger than 30. Unfortunately, there is no "one size that fits all" in terms of the minimum sample size needed.

Since the t procedures may only be approximate, one might ask, "In what situations would I not want to use a t procedure?" Mathematics can show that convergence to a t distribution happens faster (for smaller n) when the underlying process distribution is less skewed. Thus, if the sample size is small, say n = 5, and the true process distribution is heavily skewed (like in the exponential case), then the t procedures are probably not very good. This is because the true distribution of the t statistic will most likely be markedly different from a t^sub n-1^ distribution. Smaller departures in the normal model do not affect the sampling distribution of t too greatly, even when n is small, and hence these model departures do not affect resulting decisions to a large degree. Of course, one can check the normal assumption by using normal probability plots and goodness of fit procedures.