Credible Evaluation: Not Yet State-of-the-Art - assessment of superintendents study shows weak criteria

School Administrator, Feb, 2001 by Michael F. Dipaola, James H. Stronge

The authors' nationwide study of superintendent assessment finds weak criteria and few comprehensive models

The centerpiece of the current reform wave in education is the setting of standards with state legislatures and state boards of education mandating standards for curriculum, school accreditation and student learning. As a result, educators at every level, from classroom teachers to superintendents, are feeling the tight grip of accountability.

Several years ago, legislation was enacted in Illinois linking superintendent contracts to student performance. In 1999, the Virginia General Assembly took standards setting to the next level when it enacted legislation requiring uniform performance standards be used by local school boards in evaluating superintendents, other administrators and teachers. One mandatory criterion for evaluation captured our attention as Virginia educators: improving student achievement.

The inclusion of "improving student academic progress," as the legislature phrased it, as a mandatory standard of superintendent performance highlights the critical role of the district's chief executive in curriculum planning and development, instructional leadership and, ultimately, student performance results. We wondered if the action of the Virginia legislature was unique or trend-setting. How does it compare to what is happening in the rest of the country?

About a year ago, these questions prompted our investigation of the current state of superintendent evaluation nationwide. We collected and analyzed superintendent evaluation procedures, guidelines and instruments issued by state departments of education.

In addition, we compiled documents from state affiliates of AASA and the National School Boards Association that provide recommended frameworks or instruments for superintendent evaluation. However, the professional affiliates in eight states (California, Delaware, Florida, Indiana, Iowa, Kansas, Maine and Nevada) reported they had neither state guidelines nor recommendations on superintendent evaluation for their constituents.

Our national study examined descriptions of evaluation guidelines and/or processes, copies of paradigm evaluation instruments, and copies of state-mandated evaluation procedures (where applicable).

Shared Standards

It seems only sensible that evaluations of educators, including superintendents, should be grounded in the duties of the particular professional groups. Thus, basing performance assessment on the professional competencies and job duties should be a logical and necessary initial step in designing a superintendent evaluation system.

Unfortunately, attempts to define the role of the superintendent and establish priorities for performance by local or state boards often fail to include many dimensions of the role. Local school boards traditionally have defined the responsibilities of the superintendent in terms of a job description--a generalized overview that often is loosely related to actual job responsibilities and even more loosely connected to the superintendent's performance evaluation. Most superintendents have a job description, but in AASA's 2000 Study of the American School Superintendency only 50.2 percent said they are evaluated according to the criteria in the job description.

In an attempt to define the profession of the superintendency, AASA established a commission in 1992 that developed a set of eight profess anal standards and a corresponding set of competencies. When coupled with defined professional responsibilities, the standards can serve as a framework for defining the role of superintendent and serve as a basis for meaningful evaluation (see Table 1, page 21).

The superintendent performance competencies embedded in the evaluation materials collected in our 1999 national study were compared to the AASA's "Professional Standards for the Superintendency." When language in a state document wasn't identical but clearly fell under a standard, it was considered to be a partial match. When language in state performance indicators mirrored an AASA standard, we counted it as a complete match. If no language met the two criteria, the indicator was not counted as a match in this analysis. Only the recommended evaluation materials from Hawaii, Oregon and Texas fully matched all of the AASA professional standards.

The AASA standards are reflected in materials gathered from all states. However, the manner in which they are incorporated varies greatly from state to state. Organizational management was the most common standard reflected in state evaluation information (88 percent). The least common standard reflected was values and ethics of leadership (26 percent). More recently, AASA's 2000 study of the superintendency found that the criteria most frequently used to evaluate superintendents were "periodic/systemic accountability" and "assessing performance of district attempts to meet standards." Clearly, the national trend of high-stakes testing has had an impact on how superintendents are being evaluated.