Introduction

NUS is a state university, for which teaching remains a very important remit, even while research and entrepreneurship are emphasised. In this context, I have been invited to contribute my thoughts on how to develop a culture in which teaching evaluation is responsibly and helpfully carried out.

To create such a culture, I believe there are three crucial factors. First, there must be faculty members who care about teaching to take the time to develop good approaches to teaching, as well as know how to develop good portfolios, thus putting their best case forward. Second, it is helpful to have facilitators and mentors who assist with the development of good teaching and good portfolios. Third, there must be evaluators who do their jobs honestly, fairly and in an informed manner.

In the following paragraphs, I will first focus on those key players who are directly involved in the teaching evaluation process: the individual academic, the peer reviewer and the student evaluator. Next, I will discuss those others who can and should play important facilitating roles in the development of teaching evaluation, particularly the colleague-mentor.

The Individual Academic

As the one being evaluated, a healthy perspective for the individual academic to adopt is to view the teaching evaluation process as a formative one, and therefore to participate in the craft of teaching and teaching evaluation positively. Self-improvement can be achieved if he/she puts aside some time to prepare an evolving portfolio, which is then seen as a growing document paralleling the evolution of teaching, rather than an end summary of efforts.

However, a lack of educational mission in the individual academic might hamper the successful use of teaching evaluation as a way of improving teaching. Fundamentally, if we believe that teaching is unimportant, and that what gets us promotion and tenure is research, then the culture of teaching is lacking (let alone a culture of teaching evaluation). But assuming that we understand from the start that our educational mission is at least as important as our research and entrepreneurial drives, then a key factor in developing a culture of teaching evaluation is to begin emphasising the formative value of teaching evaluation, as opposed to a purely summative one, and for individuals being evaluated to take that stance as one step towards contributing to a useful process.

The Peer Reviewer

In developing a culture of evaluation, peer reviewers clearly play a central role. They must understand a two-fold responsibility: responsibility to the candidate who deserves to have his/her best qualities accurately documented, and responsibility to the Head, Dean and promotion and tenure committees who have to use the information to make proper decisions.
The following issues are not confined to NUS, but certainly observable here. One example is a lack of honesty. In a 2000 Chronicle of Higher Education article, a writer commented on letters of recommendation for promotion and tenure, which might well describe peer review evaluations: “Puffering is rampant. Evasion abounds. Deliberate obfuscation is the rule of the day.” Indeed, when inflationary rhetoric becomes normative, it is very difficult for individual reviewers not to follow suit.

Yet, it is not difficult to understand why inflationary rhetoric may creep in: there is an inevitable lack of anonymity when peer reviewers are obviously sitting in the room observing a class; there is an understandable unwillingness to be responsible for someone else’s job and an unwillingness to criticise a colleague; and there is the potential “You scratch my back, I scratch yours” syndrome. The consequences are simply that inflation is unfair to those who deserve praise and makes peer review meaningless. Such evaluations create self-sustaining and systemic pressures that may then force reliance on student feedback.

The converse of inflationary rhetoric is that of brevity, where peer review reports contain so little qualitative information that they are unhelpful. Other peer reviewers offer such detail (e.g. that the individual they are evaluating offered a 16-minute break as opposed to the announced 10 minutes) that the risk of missing the woods for the trees is high. Yet, inconsequential details are less problematic than vindictive reports, which leave the users of peer review reports without the value of good feedback.

Peer reviewers are probably not born; they are made. Heads, and units such as CDTL, can play important roles in facilitating best practice sharing. Equally, the work of peer reviewers can be enhanced and facilitated if good instruments are provided for their use. Effective peer review report forms can go a long way to prompt reviewers to make appropriate observations.

The Student Evaluator

Students do not always value the same things that a peer reviewer would, thereby offering different, but important, perspectives about the classroom process. So how can Heads, Deans and promotion and tenure committees use student feedback reliably and judiciously? It is imperative to recognise that students are appropriate sources of evidence for some things but less so others, and to reflect on the reliability of student evaluation.

Students are probably more suitable sources of evidence of student-instructor relationships; the instructor’s professional and ethical behaviour; what they have learned in the course; the instructor’s ability to communicate clearly and to enhance interest; and comparative workload. In contrast, students are less fitting sources of evidence for quality of course content (e.g. teaching the latest ideas) and the instructor’s scholarship in the field.

When reflecting on the reliability of student evaluation, some factors to consider include the following: Do more favourable ratings/good feedback suggest that students are grateful for an easy course? Or that students are grateful for high continuous assessment grades/anticipated high finals grades? Or do they indicate that students have learnt a lot in the class because of effective teaching? Research in this direction in other contexts is not conclusive. Two studies (conducted in North American institutions in 1980) suggest that lecturers cannot purchase favourable student ratings through easy grading (see reports in Braskamp & Ory, 1994). But other studies have illustrated that grading leniency often correlates with teacher ratings.

The Colleague-Mentor

Apart from those directly involved in the evaluation process, the colleague-as-mentor has an important role to play. The Head could serve as mentor, but it could as well be a colleague (not necessarily in the same discipline, to avoid focusing on content) or a ‘faculty development specialist’. Why is there a need for a mentor and what role should he/she play? There are multiple roles:

• Reflecting on the value of teaching portfolios for self-improvement: The mentor can help to guide a junior faculty member in setting goals and developing plans/objectives through teaching portfolios. In other words, portfolio creation need not be done only for promotion and tenure, but as part of the evolution of the teaching process. It can help an academic to monitor his/her own progress against standards he/she sets from the beginning for himself/herself.
• Focusing on areas in the teaching-learning process to be examined: The mentor can help a faculty member to focus from the start on what aspects of the teaching-learning process to develop and eventually present these as strengths. For example, faculty members might be advised to think about level of content, type of assignments, enhancement of collaborative learning, teacher-student interaction, etc.
• Advising on presentation and analysis: Unused as we are in presenting portfolios, advice on how to present an organised portfolio, and a ‘balanced’ one with materials from oneself (personal statement of teaching goals and representative course syllabi), materials from others (student evaluations, alumni feedback), and products of good teaching (examples of student work with comments) could be very useful.
• Developing early information-collection system: The portfolio is a growing, evolving document, and mentors can urge colleagues to think about developing an early information-collection system, dedicating a drawer to relevant material, with appropriate headings, such as teaching goals, student evaluations, and alumni feedback.

Summary

Developing a strong culture of teaching relies in part on a healthy culture of teaching evaluation, in which such evaluation is as much a formative as it is a summative process. All this is possible only if different participants in the process contribute positively: the individual academic must be committed to the educational mission and want to seek improvements to his/her teaching contributions, and be open to feedback and suggestions. The peer reviewers’ honest work can be facilitated if there is some best practice sharing and if they are offered effective, user-friendly instruments. Student evaluators must be assured that their responsible views will be taken seriously and there must be judicious use of their feedback. The mentoring role is an equally important one, but this will only be possible and effective if we see ourselves as a community, not a set of atomistic individuals looking out purely for ourselves; only then will there be willing mentors who can benefit other colleagues. Heads can play this role, and additionally, act as information brokers, alongside Deans, in effecting best practice sharing.

References

Braskamp, Larry A. & Ory, John C. (Eds.). (1994). Assessing Faculty Work: Enhancing Individual and Institutional Performance, San Francisco: Jossey-Bass Publishers.

This essay is a summary of a presentation made at a CDTL workshop for Deans and Heads on 19 April 2002.