Examinations, not module introductions, guide students. In a similar way but to a lesser extent, teaching evaluation guides teachers. Among the five sources that provide the bulk of information for teaching evaluation at NUS, Peer Review and Teaching Portfolio have not yet been systematically put into practice university-wide. This leaves Student Feedback, Examination Questions, and Module Folders to meet much of the practical need. But the operational mechanism of these three still has room for improvement.¹

Student Feedback

By far, Student Feedback is the most visible and prevailing component of the system, serving many purposes. It reflects certain teaching qualities, saves users the trouble of reading vague words, makes unambiguous comparison among teaching staff, gives the appearance of numerical accuracy, empowers students and compensates their frustration, etc.
However, the method in calculating Student Feedback often distorts the collective voice of a student body. Each student is allowed to rate the teacher on a scale of 1 to 10. This would be fair if the average were 5.5. Yet we know from experience that the average is often 8 and above. Thus the voting power of a student who totally rejects a teacher with a score of 1 is much greater than that of a student who absolutely adores the teacher with a score of 10. For example, when the average score is 8.2, it will then take four ‘adorers’ to balance a single ‘rejecter’.²

This skewed voting power has repercussions: the most critical students (sometimes the least successful ones) have the loudest feedback voice; teachers learn not to challenge students in order to avoid radical feedback. Accordingly, intellectual stimulation is not encouraged or rewarded by this mechanism.

A fairer calculation, I propose, is to discard a certain percentage of the highest and lowest votes in Student Feedback. In a class of 25, for instance, when 8% of the votes at each end are excluded, four votes—the two highest and the two lowest—are not counted. The other 21 middling votes are then averaged as the Feedback score. This method of ignoring extreme votes, a common practice in sports compe-titions such as diving and gymnastics, is fair play. The purpose is to avoid one judge’s radical score severely distorting other judges’ decision.

Examination Questions

Do Examination Questions successfully reflect the quality of teaching? Questions that test only familiarity with knowledge content and mechanical application suggest that students need only memorise facts or operational procedures as a result of taking the module. Such modules are usually not judged as having high quality. In contrast, questions that require the thinking abilities of a higher order suggest a stimulating process of learning and an intellectually successful module.

The problem in teaching evaluation, however, lies in the fact that the nature of Examination Questions is not self-evident. A critical- or analytical-looking question may demand no critical thinking or analytical skills on the students’ part at all. We have seen Examination Questions that trouble a teacher while posing no challenge to an average student. The secret? These questions have been lectured on or discussed in class/tutorial sessions. What is being tested is still the students’ ability in memorising the answers.

In order to determine if higher order thinking has been achieved from a module that appears to promote higher order thinking, we probably have to look beyond Examination Questions. I propose that a comparison of a few Examination Answers will suffice. Uniformity in wording, examples given, and details suggest a memory competition, while diversity in feasible approaches testifies for an academically vibrant module. Given the quality of teaching is judged by the learning outcomes, evaluation of Examination Answers is more justifiable than that of Examination Questions.

Module Folders

Module Folders, to a certain extent, demonstrate a teacher’s attitude. But they are unreliable sources of data for judging the quality of teaching.

First, a significant part of a Folder is teaching plans. They bear no witness as to what will be taught, which only takes place after their composition. As a result, they probably will not give more valid testimony to a module than a textbook does.

Second, a Module Folder, conventionally filed before the semester ends, is not responsive to how the module has been taught.

Third, a Module Folder, usually a teaching but not learning record, gives little evidence if the teacher has enhanced students’ learning experience. Not only may teaching not lead to learning, but learning may also occur without teaching.

As Module Folders are thus thrice remote from teaching results, I would urge caution to be exercised when it comes to using them as a reference base in teaching evaluation. It is quite possible that a dynamic semester appears dull at its humble planning stage. It is also possible that a brilliant teaching manifesto never takes physical shape in the classroom. Consequently, it can be legitimately said that only a good teacher, not the best Module Folder in NUS, will guarantee a successful module.

Endnotes

1. The following thoughts were inspired by an email discussion in August 2000, triggered by Prof K.P. Mohanan, in the Department of English Language & Literature.
   
2. I was recently informed of a new Student Feedback scheme, in which the scale ranged from 1 to 5. Such a scale would reduce the rejecter’s voting power by 25% at the expense of 50% precision. It would need three adorers to offset one rejecter.