Assessing Prospective Elementary School Teachers' Beliefs About Mathematics and Mathematics Learning: Rationale and Development of a Constructed-Response-Format Beliefs Survey

School Science and Mathematics, Feb 2004 by Ambrose, Rebecca, Clement, Lisa, Philipp, Randolph, Chauvot, Jennifer

Because of the role beliefs play in the teaching and learning of mathematics, mathematics educators need to consider ways to assess beliefs and belief change. Beliefs, because they must be inferred, can be difficult to measure, particularly with a common metric that enables a comparison of individuals. Because of the limitations of Likert scales, a computer-based survey was developed to assess beliefs. In this survey, prospective teachers interpret scenarios in a free-response format. The survey, used with more than 150 participants, captures qualitative data that are later quantified for purposes of comparison. A systematic method for creating research rubrics was developed in an effort to quantify the qualitative data. Results from a pre/post administration of the survey demonstrated that it is an effective tool for assessing belief change. In this article the theory behind the development of the survey is discussed, as well as some specific information about the survey and the way responses are coded and a description of the process used for developing rubrics. Recommendations are included for researchers interested in developing similar surveys.

Because of the important role beliefs play in the teaching and learning of mathematics (Leder, Pehkonen, & Torner, 2002; Pajares, 1992; Thompson, 1992), mathematics educators need to consider ways to assess beliefs and belief change. Beliefs, because they must be inferred, can be difficult to measure, particularly with a common metric that enables a comparison of individuals. As part of a large-scale research project, Integrating Mathematics and Pedagogy (IMAP), a survey was needed to assess the belief change for a large number of prospective teachers. (A browse version of the IMAP Web-Based Beliefs Survey and a manual explaining how to code the survey are available at http://www.sci.sdsu.edu/CRMSE/IMAP/ pubs.html.)

In this article, first, the limitations of Likert-scale surveys are outlined. Then the web-based instrument is described, along with a rationale for its benefits. Next, the process of rubric development is delineated. Finally, data are provided to show that this survey is general enough to capture a range of positions on beliefs and sensitive enough to capture change.

Two components of beliefs account for the critical role beliefs play in teaching and learning and are, thus, important for the way beliefs are measured. First, beliefs influence perception (Pajares, 1992). That is, beliefs serve to filter some complexity of a situation to make it comprehensible, shaping individuals' interpretations of events. Teachers and students are constantly faced with uncertain situations requiring interpretations. In the survey described in this article respondents are provided with complex situations they are asked to interpret.

Second, beliefs might be thought of as dispositions toward action, having a motivational force (Cooney, Shealy, & Arvold, 1998; Rokeach, 1968). When teachers face challenging decisions that often must be made spontaneously, their beliefs compel them to act in particular ways. In measuring beliefs, the survey provides respondents with scenarios in which they are called on to make teaching decisions. Their dispositions to act in these situations provide evidence from which their beliefs can be inferred.

Beliefs are not all-or-nothing entities; they are, instead, held with different intensities (Rokeach, 1968, as cited in Pajares, 1992). When measuring beliefs, the survey provides tasks that offer multiple interpretation points; a variety of questions are asked, first general questions and then more specific questions about the complex situations. When scores are assigned to responses, differentiation is made between strong evidence, evidence, weak evidence, and no evidence of a belief to allow for the different intensities with which individuals hold beliefs. We are careful not to claim that an individual lacks a particular belief but instead state that no evidence was found for the belief in the responses the individual provided. Beliefs tend to be context specific, arising in situations with specific features (Cooney et al., 1998), and hence, survey segments are situated in contexts, and a respondent's belief is inferred on the basis of his or her interpretation of the context.

The Problem of Assessing Beliefs of Large Numbers of Students

"For the purposes of investigation, beliefs must be inferred" because individuals can be unaware of some the beliefs shaping their actions (Pajares, 1992, p. 315). Mathematics education researchers have typically used case-study methodology to infer teachers' beliefs related to mathematics teaching and learning (e.g., Clarke, 1997; Cooney et al., 1998; Raymond, 1997). Using this approach, researchers provide rich descriptions of the beliefs of a small number of prospective teachers by relying on thick data sets that include multiple observations, interviews, and surveys collected over a long period of time. The findings from this research provide details of the conceptions of small numbers of teachers, with conclusions that have multiple data points to support findings. These rich reports are important for theory building, but theory testing often requires tools for studying larger groups of individuals. Given the nature of our work, we faced two problems in assessing beliefs: The beliefs of prospective elementary school teachers needed to be assessed years before they were in the classroom, and an assessment was needed that could be administered to more than 150 prospective teachers.