Teaching portfolios: Another perspective

Academe, Jan/Feb 2000 by Burns, Candace W

Are teaching portfolios a scam? They're time-consuming to put together, and we don't know if they improve teaching.

IN THE CONTEMPORARY DEBATE OVER ACCOUNTABILITY in higher education, leaders of a well-funded campaign assert that college professors have neglected and now must attend to teaching. Supported by the American Association for Higher Education (AAHE), Jossey-Bass, the William and Flora Hewlett Foundation, the Carnegie Foundation for the Advancement of Teaching, and the Pew Charitable Trusts, the campaign pushes teaching portfolios as one way to improve teaching. Teaching portfolios are a collection of materials meant to document teaching performance. They typically include statements by professors about their teaching effectiveness, along with supporting information, such as sample syllabi, sample student work, student ratings, and comments from students and colleagues. Unfortunately, in their zeal to remedy what may or may not be a problem, proponents of portfolios ignore over twenty years of research into what makes teaching effective.

Peter Seldin, professor of management at Pace University, is the most prominent champion of teaching portfolios. For nearly a decade, his books and workshops have spread the word on using portfolios in educational reform. In 1993 Seldin marketed them chiefly as a device for fine-tuning teaching: "It is important to keep in mind that use of the portfolio for personnel decisions is only occasional. Its primary purpose is to improve teaching performance." By 1997, however, the subtitle for his book on portfolios read, "A Practical Guide to Improved Performance and Promotion/ Tenure Decisions."

Even though the title of Seldin's most recent book implies a separation between using portfolios for teaching improvement and relying on them for personnel decisions, in practice these two functions intertwine. In post-tenure reviews, for example, improvement of teaching has become a major factor in determining conditions of retention. Some institutions have even begun to include portfolio programs in their policies on posttenure review. A recent AAHE booklet about post-tenure review offers samples of such policies, including Colby-Sawyer College's, which explicitly mentions their use, and which allots deficient professors one year to improve their teaching. Among sanctions the college imposes for failing to improve are "frozen salaries," "career redirection," faculty "buyouts," and the ultimate sanction--dismissal.

Dearth of Research

BECAUSE FAILURE TO IMPROVE can have such serious consequences, professors deserve recommendations based on sound research. Seldin says that "we know that preparing a teaching portfolio helps to improve teaching," while AAHE leaders speak of the "special power of teaching portfolios to involve faculty in reflection on their own practice and how to improve it." Yet I have found no controlled experiments that support such confident, causal statements. Indeed, the only experiment that I could locate that compared teaching ratings before and after portfolio construction concluded that these ratings did not improve significantly.

What is needed is careful research. At the least, such research should be based on experimental or quasiexperimental designs that strive to reduce threats to interpreting cause-effect relationships. In the absence of such designs, problems such as sampling bias can occur. It can arise, for example, when researchers select subjects who are likely to support a particular outcome. As a group, the professors who supply portfolios for Seldin's books credit their perceived improvement to the "reflection" that portfolios occasion; few of them consider extraneous factors, such as possible placebo effects. Seldin maintains that "tens of thousands" have benefited from portfolios. Such breezy assertions about the value of portfolios raise a number of questions.

Unanswered Questions

FIRST, ARE THE PROFESSORS WHO RECEIVE "PORTFOLIO training" and who appear in Seldin's books randomly selected, or are they volunteers? Although he doesn't specify how he selects the professors who develop portfolios for his books, Seldin started." So it seems likely that he also relies on at least some volunteers, even though doing so leads to a potential for sampling bias. Volunteers tend as a group to be more motivated toward a particular method than nonvolunteers. We should not be surprised, therefore, if Seldin's volunteers speak highly of teaching portfolios.

Second, do institutions handpick some professors to participate in teaching reforms? If so, sampling bias may infect the discussion of reform. Seldin suggests that institutions involve their .most respected professors from the start" to gain faculty members' acceptance for portfolios. More than likely, Seldin has chosen the portfolios of some of these "respected" professors for his books.

Faculty members who have participated in two other heavily promoted reform projects, the AAHE's Teaching Initiative, which has encouraged and nurtured portfolio development, and the Carnegie Foundation's Academy for the Scholarship of Teaching and Learning (CASTL), have all been preselected. For the AAHE's Teaching Initiative, provosts from twelve universities selected three departments, which then chose faculty volunteers. And the Carnegie Foundation's Web site actually uses the word "tapped" when referring to the selection of CASTL's "Carnegie Scholars," whose mission is to "advance the practice and profession of teaching." According to the Web site, professors are selected based "not only on individual qualifications but also on the need to assemble a group of scholars with ... complementary strengths, interests, and experiences." One example of a qualifying experience is prior use of teaching portfolios.