Evaluating Digital Libraries for Teaching and Learning in Undergraduate Education: A Case Study of the Alexandria Digital Earth ProtoType - ADEPT

Library Trends, Fall, 2000 by Christine L. Borgman, Anne J. Gilliland-Swetland, Gregory H. Leazer, Richard Mayer, David Gwynn, Rich Gazan, Patricia Mautone

ABSTRACT

THIS IS A DISCUSSION ON THE RESEARCH DESIGN FOR AN educational evaluation of the Alexandria Digital Earth ProtoType (ADEPT), a digital library of geo-referenced information resources. ADEPT is being studied in undergraduate classrooms at the University of California, Los Angeles, and the University of California, Santa Barbara. The article provides a brief review of the deployment of digital libraries in educational settings, the role of information technology in developing students' scientific thinking, and the evaluation of digital libraries. We outline the overall research design, report on progress to date, and describe plans for the remainder of the five-year project. The article concludes with initial observations about classroom environments for using ADEPT and about the initial deployment of ADEPT prototypes.

INTRODUCTION

Digital libraries offer a wealth of opportunities to improve access to information resources in support of both "traditional" on-campus instruction and distance-independent learning (Borgman, in press). We are still at the early stages of realizing the potential of digital libraries in educational contexts, however. Few of the technological, logistical, and economic aspects of integrating digital libraries into university education have yet been assessed, much less the curricular and pedagogical challenges (National Research Council, Center for Science, Mathematics, and Engineering Education, 1998). Relatively little work has been done on evaluating the usability of digital libraries in any context, and minimal work has been done on assessing learning outcomes associated with the implementation of digital libraries in instruction. Many complex research design questions remain to be addressed, such as what to evaluate, by what methods, and how to determine if learning is occurring.

We report here on the research questions, research design, and preliminary observations from the first year of a five-year project (1999-2004) to develop and deploy a digital library of geo-referenced information resources ("geolibrary") in undergraduate courses at the University of California, Los Angeles, and the University of California, Santa Barbara. This study is part of the Alexandria Digital Earth ProtoType (ADEPT) project funded by the U.S. Digital Libraries Initiative, Phase 2 (National Science Foundation, 1999). ADEPT is an emerging digital library that will provide instructors and students with the means to discover, manipulate, and display dynamic geographical processes. The ADEPT system provides an interesting case study to observe the deployment of a digital library in instructional settings. Our thesis is that digital library services will contribute positively to undergraduate instruction and to student learning of scientific processes. To examine this thesis, we employ a variety of qualitative and quantitative methods to investigate the impact of ADEPT in undergraduate instruction. This article extends our initial reports on the education and evaluation component of ADEPT (Leazer, Gilliland-Swetland, & Borgman, 2000; Leazer, Gilliland-Swetland, Borgman, & Mayer, in press). Continuing reports will be provided on the ADEPT Web sites at UCLA (http://dlis.gseis.ucla.edu/adept/) and UCSB (http://www.alexandria. ucsb.edu/adept/).

DIGITAL LIBRARIES AND UNDERGRADUATE EDUCATION

Educational applications of digital libraries range from primary school through graduate school and across all disciplines. One of our chief interests is how the use of digital libraries can promote thinking processes associated with problem domains (e.g., science, social sciences, humanities) at the undergraduate level. The first stage of the ADEPT educational evaluation focuses on scientific thinking and is being conducted in physical geography courses. This section provides a brief literature review of the role of digital libraries in education and in scientific thinking to set the context for the case study of ADEPT. The literature is reviewed relating to the evaluation of digital libraries in general.

Educational Applications of Digital Libraries

Faculty and librarians alike are concerned about ways to implement digital libraries in education. The Council on Library and Information Resources (1999) held a meeting "to consider changes in the process of scholarship and instruction that will result from the use of digital technology and to make recommendations to ensure that libraries continue to serve the research needs of scholars." Among their recommendations was that institutions of higher education should "place more emphasis on training and support for faculty use of information and instructional technologies."

The University of Michigan Digital Library Project (Wallace, Krajcik, & Soloway, 1996) posits that the main benefit of digital libraries in the classroom is improved means and opportunity for inquiry-based learning. A component of this research, the Middle Years Digital Library project (Soloway et al., 2000), allowed science students in grades six through nine to learn and explore topics in a less-regimented manner than traditional textbook learning. The few outcome-based studies that have been conducted have suggested a positive correlation between integrating electronic information sources into the classroom and increased scholastic success. Newnham, Mather, Grattan, Holmes, and Gardner (1998), for example, gave geography students access to Internet source material downloaded onto a local network file server. Students were encouraged to make use of the material and communicate among themselves via electronic mail. The study found that access to electronic geo-information sources enhanced student learning.