Understanding Student Differences

Journal of Engineering Education, Jan 2005 by Felder, Richard M, Brent, Rebecca

II. A NOTE ON VALIDATION

Much of this paper describes assessments of various student attributes and inferences that have been drawn from the data. Before too much stock is placed in such inferences, the instrument used to collect the data should be shown to be reliable (consistent results are obtained in repeated assessments) and valid (the instrument measures what it is intended to measure).

In another paper in this issue, Olds, Moskal, and Miller [9] offer a good introduction to reliability and validity analysis. Some of the measures of reliability and validity they discuss that are applicable to instruments of the types we will describe are the following.

* Test-retest reliability-the extent to which test results for an individual are stable over time.

* Internal consistency reliability-the homogeneity of items intended to measure the same quantity, that is, the extent to which responses to the items are correlated.

* Scale orthogonality-the extent to which the different scales of the instrument (if there are two or more scales) are independent.

* Construct validity-the extent to which an instrument actually measures the attribute it purports to measure. The instrument scores are said to have convergent validity if they correlate with quantities with which they should correlate and divergent or discriminant validity if they fail to correlate with quantities with which there is no reason to expect correlation.

Reliability and validity data of these types are readily obtainable for some of the instruments to be discussed, while for others (notably several of the learning style assessment instruments) they are difficult or impossible to find. At the end of each of sections III (Learning Styles), IV (Approaches to Learning), and V (Levels of Intellectual Development), we offer lists of potential research questions. To each list might be added the following two-part question: If an assessment instrument is used to study any of the preceding questions, what reliability and validity data support its use (a) in general, and ft) for the population studied?

III. LEARNING STYLES

Students are characterized by different learning styles, preferentially focusing on different types of information and tending to operate on perceived information in different ways [10, 11]. To reduce attrition and improve skill development in engineering, instruction should be designed to meet the needs of students whose learning styles are neglected by traditional engineering pedagogy [12-14].

Several dozen learning style models have been developed, five of which have been the subject of studies in the engineering education literature. The best known of these models is Jung's Theory of Psychological Type as operationalized by the Myers-Briggs Type Indicator (MBTI). Strictly speaking, the MBTI assesses personality types, but MBTI profiles are known to have strong learning style implications [14-16]. This instrument was the basis for a multi-campus study of engineering students in the 1970s and 1980s and a number of other engineering-related studies since then [17-24]. Other models that have been applied extensively to engineering are those of Kolb [12, 14, 25-31], and Felder and Silverman [13, 14, 32-40]. We discuss these three models in the sections that follow. Two other models that have been used in engineering are those of Herrmann [14, 41-43], and Dunn and Dunn [44-46]. Relatively little assessment has been performed on the applicability of these models to instructional design in engineering, and we do not discuss the models further in this paper. For information about them, see the cited references.