Experiential Learning Environments: Do They Prepare Our Students to be Self-Directed, Life-Long Learners?

Journal of Engineering Education, Jul 2006 by Jiusto, S, DiBiasio, D

IV. RESULTS

A. IDEA System Results

The data used to assess the effectiveness of the Global Program preparation in meeting the SDL and LLL related learning objectives against other courses nationally and at WPI was from self-reported student assessments of development on a five-point scale ("My progress on this objective was" 1 = Low, 5 = High). For all three objectives analyzed, the average student responses were IDEA system 3.8 ( /- 0.1), WPI 3.6, and the Global Program preparation 4.3 as shown in Table 2. In all cases, average ratings were substantially higher for the Global Program preparation than for WPI as a whole.

To understand the significance of differences in means between traditional WPI courses and the preparation phase of the experiential program, Table 3 shows for each learning objective how these means rank compared against all other classes in the IDEA system national database that targeted the same educational outcome. The differences are dramatic. On average, the Global Program preparation courses ranked in the 67th percentile of all courses seeking to develop student capacities for research skills, critical thinking, and expression, far higher than the 29th percentile average of all WPI classes seeking these outcomes.

Another way to analyze collective student progress is to consider the frequency with which students report meeting certain thresholds of progress. Figures 2 to 4 show for each objective and each population the percentage of classes that reached three average ratings thresholds (3.50, 3.75, and 4.00 on the five-point scale described above). Figure 2, for example, shows that student research skills ratings averaged at least 3.5 in all (100 percent) of the Global Program preparations, twice the WPI average and appreciably more than the 70 percent rate for all IDEA courses. In all cases, ratings were substantially higher for the Global Program preparation than for WPI as a whole.

One key to these comparatively high ratings was the degree to which the Global Program succeeded in improving the SDL-related abilities of many student participants. Of the nine preparation classes assessed, all achieved an average student response of 3.5 or greater on all three objectives, save for one class on one objective. These data indicate the project preparation experience strongly improves student acquisition of core SDL capacities, both deeply (individual gains are sizable) and broadly (most students benefit).

B. Faculty Review Results

Table 4 summarizes the results from the 2004 final report evaluation cycle. The sample size is somewhat lower than the total SDLRS numbers because we analyzed only a random sample of final reports to reduce expenses. Note that there are three or four students per team and sample sizes represent numbers of teams. The numerical rating score is 1 = Poor (or absent), 3 = Acceptable, and 5 = Excellent. The 2004 results are from the same offcampus cohort reported above. We have data for several years that show the same result. That is, that the off-campus project experience provides better evidence of LLL abilities (in the dimension assessed) than the on-campus experience. For example, in 2002 the off-campus average for the LLL item was 4.1 while that for oncampus was 3.1. Off-campus cohort averages are consistently higher than on-campus and there is a striking difference in the percentage of reports rated "below acceptable".