Editorial: Teaching and learning innovation: Is something missing?

International Journal of Electrical Engineering Education, Apr 2001

There is no doubt that changes in teaching methods and laboratory experiments are becoming widespread. The impetus for this transformation comes from two sources: first, the desire to improve students' learning experience; secondly, the question of cost efficiency, based on savings in money or time, or both.

Having identified the need, project leaders can diversify into a whole range of responses. As reports in this journal and elsewhere testify, these make fascinating reading. But there is a further point which should not be overlooked: once changes have been implemented, how do we know how effective they have been?

If changes are to be refined into an effective mechanism for improving the learning experience, feedback on them is essential. Likewise, if they are aimed at improving efficiency, some way of measuring their impact is required. Yet in many reports of innovations in teaching, evaluation features as only a hazy outline, and in some cases is missing from the picture entirely. Admittedly, neither designing questionnaires and conducting interviews, nor analysing the responses, is easy. However, examples of good practice already exist. Projects being run by the Fund for the Development of Teaching and Learning (FDTL) and Teaching and Learning Technology Programme (TLTP) with funding from the UK government's higher education funding councils, among others, incorporate strategies for evaluation.

There are many possible approaches to evaluation. Students' examination performance is only one, indirect, indicator of how understanding has been improved. Questionnaires and interviews, although more descriptive, can also provide more direct evidence. Questionnaires can take the form of tick boxes or open-ended questions; they can be presented on paper, on the Web, or by e-mail. Interviews can be individual or group based, and take place face to face, by e-mail, or by telephone, for example. Other evaluation strategies include diaries, logs, observation and use of focus groups. For the evidence to be meaningful at all, some comparison has to be made before and after changes are implemented.

Whichever evaluation strategy they choose, project designers need to remember one essential point: evaluation of final outcomes is something to think about right from the start.

Web links and further information * For information about FDTL and TLTP projects see http://www. ncteam.ac.uk

* The report TLTP Guidelines for Project Evaluation, Version 1.2, compiled by the Tavistock Institute in London and the Open University in 1998, usefully summarises concepts which apply equally to engineering and social science disciplines. The Tavistock Institute is at http://www.tavinstitute.org/

* Projects which consider how best to conduct evaluations have been run at Glasgow and Heriot-Watt Universities. See for example http://www.gla.ac.uk,/TILT/TILT.html and for some hands-on ideas, http://www.icbl.hw.ac.uk/ltdi/cookbook/contents.html