The Six Year Test - National Consortium for Recreation and Youth Development evaluates program effectiveness

Parks & Recreation, March, 2000 by Peter A. Witt

PROGRAMS FOR AT-RISK YOUTH GO UNDER INVESTIGATION

For the past six years a group of academics, graduate students and practitioners have been involved in a unique effort to evaluate the outcomes of recreation programs for at-risk youth. The National Consortium for Recreation and Youth Development (originally called the National At-Risk Youth Recreation Consortium), a cooperative effort of faculty and graduate students from eight universities, along with practitioners from approximately 15 cities across the United States, was created with funding from the National Recreation and Park Association (NRPA) and National Recreation Foundation (NRF). Funding was made available and the Consortium created out of a concern that park and recreation departments were not commanding deserved respect and attention from political leaders and funders who were trying to find ways to deal with high levels of juvenile crime, drug use, school dropouts and failure, and teen pregnancy.

In the early 1990s, while money was forthcoming to deal with increasing youth risk behaviors--and the life-long consequences of these behaviors--park and recreation professionals were increasingly being asked to provide evidence that the programs they were undertaking made a difference in the lives of youth. Institutional structures such as schools, law enforcement agencies, churches and various non-profit youth serving organizations were being called upon to provide programs and services that would help to ameliorate these problems. Park and recreation departments wanted to be major players in these efforts, but they did not have evidence of their effectiveness that could justify funding. In many instances, recreation was perceived by funders as fun and games--activities without impact, designed to keep children busy. Park and recreation professionals correctly saw that in order to be players in a youth services system, they would have to change public perceptions about the value and potential impact of park and recreation programs. In order to accomplish this goal, the park and recreation field would have to do a better job of telling its story and produce the kind of information necessary to make the story credible. This latter step would require greater focus on documenting outcomes and contributions being made by park and recreation programs. While the park and recreation community toted the value of late night, after-school, weekend and other non-school-hour programs, little evidence existed beyond anecdotes that park and recreation programs made a difference.

In response to the need for improved documentation, two NRPA-funded three-year grants were received from NPRA/NRF. For the first three years (1993-1996), four universities were involved: Texas A&M, Arizona State (West and Main campuses), Clemson, and Penn State. For the second three years (1996-1999), these institutions were joined by Illinois, Georgia, North Carolina, and San Francisco State.

An initial report on studies undertaken by the Consortium appeared in the Journal of Park and Recreation Administration (JPRA: Witt & Crompton, 1996) and was reprinted in Parks and Recreation (P&R: Witt & Crompton, 1996). Many of the outcome reports published from 1993 to date are available on the Consortium's web site: wwwrpts.tamu. edu/witt/consort.htm. More will be added as projects are completed. Additional articles are scheduled to appear in both P&R and JPRA.

Evaluation Challenges

The research teams have faced several challenges while undertaking the evaluation efforts. A few issues have been prominent in almost every evaluation that has been undertaken. First, evaluation is dependent on a clear set of expected or desired outcome. However, programs often have no clearly articulated goals or have goals that do not lend themselves; easily to measurement. Secondly, even when goals have been articulated, programs are not always designed to effectively achieve them. And lastly, appropriate means are not available to measure whether certain targeted outcomes have been achieved.

Need for Clearly Delineated Goals

Our work suggests that many park and recreation programs are loosely constructed efforts to occupy children's time without a concrete plan for increasing skills (e.g., interpersonal, activity) or helping to develop or change attitudes necessary to decrease risk behaviors. While children and leaders report that participants have fun, this diversionary approach to service provision does not fully meet the expectation of funders who have more lofty crime prevention, risk behavior reduction or improved school performance goals in mind.

Need for Programs that Meet Stated Goals

In many communities, there still is a belief that any program developed and implemented will achieve a wide variety of outcomes. While NRPA has promoted the benefits of recreation participation, these benefits will probably only occur in programs designed to achieve them. In many instances, the benefits catalog is quoted without an understanding that programs must have goals consistent with the stated benefits and be designed to achieve the desired outcomes.