More on gaging statistics

Modern Machine Shop, March, 1996 by George Schuetz

Deviation, as a statistical term, tells how much a given piece of data diverges from the mean. (For example, if the mean value of a sample is 0.010[inches], and the part in question measures 0.012[inches], deviation is 0.002[inches].) Standard deviation, when designated as S, is a measure of how much the values of all the individual items in the sample diverge from the mean value of the sample. Standard deviation can also be designated as [Sigma] (the small Greek letter sigma), which is an estimate, based on a sample, of how much the values of the individual items in the total population from which the sample was drawn will diverge from the mean value of the population [ILLUSTRATION FOR FIGURE 1 OMITTED].

We won't go into how to calculate S or [Sigma] here (the methods appear in any basic textbook of statistics, or you can just punch up the values on a scientific calculator). Suffice it to say that once you've found the value of [Sigma], further standard deviations are, by definition, simply multiples of [Sigma]. (For example, two standard deviations = 2 x [Sigma].) Some of the data in a sample will fall beyond one standard deviation from the mean. The second standard deviation tells us how much additional deviation exists among those parts that do not fall within the first standard deviation; the third standard deviation tells how much deviation exists among parts that don't fall within the first two deviations, and so on.

Standard deviations can be displayed on a distribution curve as pairs of positive and negative bands on either side of the mean value, as shown in the figure.

In any random sample showing a normal, bell-curve distribution, one standard deviation (that is, one positive and one negative band) will encompass roughly 68 percent of the values in the population; two standard deviations will encompass roughly 95.5 percent; and three standard deviations will encompass roughly 99.75 percent of the workpieces. It always works out this way, because of the laws of random distribution and statistics: wider bell curves naturally exhibit larger standard deviations, and narrower bell curves exhibit smaller ones, and the two are always in proportion to one another.

By comparing the width of the standard deviation bands with the specified tolerance limits, it is possible to calculate, from the sample, how many bad parts will be produced for the production lot from which the sample was drawn. For example, if the tolerance limits are equal to plus or minus three standard deviations, and the mean of the sample is perfectly centered on the specification, one could expect 25 bad parts out of every 10,000 produced (100 percent 99.75 percent). It's simply a matter of calculating the mean and three standard deviations, and comparing the results against the tolerance specification. If the "[ or -]3 sigma" spread falls entirely within the tolerance limits, as in the figure, then the process appears to be under control. If part of the spread falls above or below the tolerance limits, then the process must be adjusted to make the bands narrower (that is, reduce variation), change the location of the mean, or both.

In any precision manufacturing operation, dimensional variation is very tightly controlled, and the sigma limits are monitored to keep them within the tolerance limits. The span can be compared to the tolerances in various ways. This is known as process capability, and can be discussed in future columns.

In summary: it is possible, based on the laws of statistical probability, to effectively monitor a process and maintain high levels of quality control using a sampling method. In most applications, this tends to be far more cost effective than 100 percent inspection. Rather than drawing histograms, one can calculate the "control limits" - that is, the upper and lower boundaries of the standard deviation bands, and the acceptable range of variation of the mean - mathematically.