User-based data collection techniques for evaluating networked information services - Libraries and the Internet: Education, Practice & Policy

The logistics associated with administering data collection instruments can oftentimes be formidable. And since the notion of a user perspective is to make the study participants at ease and able to relate social and behavioral concerns in arranging for a focus group session, one has to not only identify and obtain the cooperation of participants, one also has to (among other things):

* arrange for a pleasant setting to conduct the focus group and,

typically, provide some refreshments and amenities;

* consider the order and development of topics to be discussed in

the session;

* have a technique for recording and analyzing the content of the

session while it is occurring;

* manage and moderate the session in a positive and productive

manner; and

* provide follow-up thank you notes. Similar logistical concerns affect the use of other data collection techniques such as surveys, transaction logs, interviews, and so on. Once again, it is essential that these logistical concerns are considered and resolved as part of the data collection process.

Presenting Study Results and Findings

Evaluations incorporating user-based data collection techniques typically fall under the heading of "action research"--i.e., research that is intended to assist in the decision-making process or assist in policymaking. Thus, if the evaluation is to be successful, the findings have to be presented to decision makers in such a manner that: (1) the decision makers are aware of the findings, (2) the findings are adequately explained and made understandable, and (3) specific implications and recommendations are made explicit.

INTEGRATING EVALUATION INTO THE PLANNING PROCESS

One of the most important challenges facing the development of successful networked information services is to assess these services in light of a user perspective. One of the best possible strategies for meeting this challenge lies in the development of a comprehensive strategic planning approach that integrates evaluation with strategic planning. Such a comprehensive and integrated approach to network development is essential if evaluators are to provide leadership in accessing, managing, and disseminating networked information to users effectively in the future.

Strategic planning is a disciplined effort to produce fundamental decisions and actions that shape and guide the networked information services being provided currently as well as those being designed. At its best, strategic planning requires broad-scale information gathering, an exploration of alternatives, and an emphasis on the future implications of present decisions (Bryson, 1988). Strategic planning and ongoing evaluation is a critical management process for the development of networked information services if they are to be effective, if designers are to allocate resources wisely, and if user information needs are to be met. More specifically, strategic planning and evaluation:

* provides a rational response to uncertainty and change: We will