Content Validity of Preservice Teacher Portfolios in a Standards-Based Program

Journal of Instructional Psychology, March, 2000 by Beverly M. Klecker

The introduction of a standards-based teacher education program in Kentucky was framed by eight New Teacher Standards adopted state wide. At Eastern Kentucky University, the Teacher Education Portfolio was implemented to (1) provide more meaningful, valid indicators of what preservice teachers know and can do, (2) enhance both teaching and learning, and (3) provide useful assessment information for reporting to people outside the classroom or local context. This article presents the eight New Teacher Standards and describes the connection between the content in an assessment course and the standards. A course assignment resulted in a portfolio entry designed to demonstrate the students' knowledge and skills for Standard four, "The teacher assesses learning and communicates results to students and others."

The introduction of a restructured standards-based teacher education program at Eastern Kentucky University mandated the inclusion of portfolios for preservice teachers. There were three major expectations for the preservice teacher portfolio. The portfolio was to: (1) provide more meaningful, valid indicators of what preservice teachers know and can do, (2) enhance both teaching and learning, and (3) provide useful assessment information for reporting to people outside the classroom or local context. Recommended practices related to portfolios (e.g., After & Spandell, 1992; Wiggins, 1989; and Wolf, 1991) were reviewed during the design stage of the Teacher Education Portfolios. If preservice teacher portfolios are to be used as assessment, issues of validity and reliability (Ghiselli Campbell & Zedeck, 1981; Mehrens, 1992) must be addressed. The first reliability consideration was the consistency of scoring of both the parts and the whole portfolio. This issue is still being resolved. Portfolio entries are all scored with a four-point rubric, however, the scoring is done by individual instructors/professors. Clearly, the most valid measures of whether or not a student can perform a certain task is to design a performance assessment that requires the student to "show what you can do." However, even readers with only tangential knowledge of assessment can recall the relationship between reliability and validity. Reliability is a necessary but not sufficient condition for validity. That is, a test cannot be valid unless it is reliable. Feldt (1997) recently examined this relationship by exploring the frequently quoted dictum of classical test theory which states that validity can never be greater than reliability (classical test theory offers an equation for calculating the relationship). Felt concluded, "If test authors allow themselves to be overly swayed by the desire to attain high reliability, they might easily favor those items that measure lower order skills" (p. 386). The Teacher Education Portfolio was planned to assess higher order thinking skills (Bloom, 1956). The preservice teacher is required to write a reflection for each portfolio entry, thus including "evaluation," Bloom's highest cognitive category. This article will address two validity issues: (1) the measures taken to assure the content validity of the overall preservice teacher education portfolio and (2) the alignment of the performance based assessment in a newly-created "Assessment in Education" course to assure content validity. Teacher education in Kentucky is Standards based. The Kentucky Standards Board was established with the comprehensive Kentucky Education Reform Act (KERA) of 1990. KERA also mandated an internship year as teacher certification requirement. The eight New Teacher Standards are used as evaluative criteria for the Kentucky Teacher Internship Program (KTIP). Because these standards define what teachers should know and be able to do, they were used as content guides for the Teacher Education Portfolio. The eight standards are:

1. The teacher designs/plans instruction and learning climates that develop student abilities to use communication skills, apply core concepts, become self-sufficient individuals, become responsible team members, think and solve problems, and integrate knowledge.

2. The teacher creates a learning climate that supports the development of student abilities to use communication skills, apply core concepts, become self-sufficient individuals, become responsible team members, think and solve problems-and integrate knowledge.

3. The teacher introduces/implements/ manages instruction that develops student abilities to use communication skills, apply core concepts, become self-sufficient individuals, become responsible team members, think and solve problems, and integrate knowledge.

4. The teacher assesses learning and communicates results to student and others with respect to student abilities to use communication skills, apply core concepts, become self-sufficient individuals, become responsible team members, think and solve problems, and integrate knowledge.

5. The teacher reflects on and evaluates specific teaching/learning situations and/or programs.