Teaching Evaluation at NUS

The university’s guidelines on teaching evaluation for Promotion & Tenure, Re-appointment, and Outstanding Educator Awards (HR 094/02), distinguish between:

1. Minimal threshold of competence in and commitment to teaching expected of all faculty members (such that falling below this level would be grounds against promotion and tenure). Basic teaching competence includes adequate knowledge, communicative ability, and the skills of conducting tutorials. Commitment to teaching manifests itself as the willingness to devote time and energy to teaching.

2. Value-added ingredients that contribute towards excellence in teaching (as the grounds for promotion and tenure based on teaching). The value-added ingredients that raise a teacher’s level above the threshold level towards excellence includes the following parameters:

   1. Accomplishment of higher order learning outcomes,
   2. Module development,
   3. Project/research supervision,
   4. Quality of teaching materials, and
   5. Educational reform activities

The Office of Human Resource’s (OHR) documents on teaching evaluation refer to exceptional qualities that make a teacher stand out from others. The following are some possible ways to identify exceptionality in teaching:

1. Given the available evidence presented in the teaching portfolio does the candidate stand out from the typical “very good” teachers in the discipline, particularly in terms of the learning outcomes facilitated by him/her?

2. What makes his/her teaching special?

3. How valuable are these special features (2) from the point of view of:

1. The given discipline or sub-discipline, and
2. University education in general?

If the answer to (3b) is “considerable”, the candidate is an outstanding educator; a similar answer to (3a) makes the candidate an exceptional teacher.

The types of learning outcomes that make a teacher exceptional depend on the standard pedagogical practices in the discipline. For instance, the valuable outcome of providing evidence for knowledge propositions is the standard practice in mathematics since theorems are accompanied by proofs in the subject’s conventional teaching, but not for the physical sciences. Hence, a teacher who closely attends to issues of justification stands out in the physical sciences, but not in mathematics. Likewise, a teacher who engages in argumentation and debates stands out in biology and psychology, but not in law and philosophy.

Exceptionality may also be a function of the nature of the discipline. Medicine is about solving the practical problems of health and illness that concerns everyone. Therefore, we would expect a neuroscience module in medicine to centre on problems of medical illnesses. Conversely, for a neuroscience module in psychology, it would be desirable but not essential for students to be able to relate to practical problems and illnesses. Not all problems in astronomy have a strong practical component; the subject cannot be easily applied to solve practical problems that students care about, except perhaps in the hands of highly imaginative teachers. Hence, it may be necessary to view intellectual problems in astronomy as the counterparts of practical problems. Yet another example, the ability to help students understand abstract concepts that are not easily applicable to everyday experience may be a pedagogical challenge in particle physics and philosophy, but perhaps not in history and social work.

The examples above demonstrate that there are different ways of achieving exceptionality. For instance, a body of knowledge in a discipline can be made more meaningful by showing how it can be applied to solve practical or intellectual problems, how it has evolved from the history of the discipline, how it can be constructed, how its concepts and statements are justified, how it is connected to other disciplines and so on. One teacher’s primary strength may lie in guiding students towards innovative applications, thereby facilitating creativity; while another’s may reside in doggedly pursuing matters of evidence and justification, facilitating open-minded scepticism as a result; yet another’s forte may be inherent in explaining recalcitrant abstractions, consequently facilitating deep understanding. Not every subject lends itself equally well to all the strengths. Likewise, not all teachers need to exhibit all the qualities to be called exceptional.

According to OHR’s guidelines, statements about the quality of the faculty member’s teaching are expected be justified with concrete examples in the teaching portfolios. How do we accomplish this?

The Teaching Portfolio

We would like to suggest that it is useful to treat the teaching portfolio as an extended “research paper” that demonstrates the quality of the candidate’s teaching. The document presents specific evidence within the context of a coherent, integrated narrative. Mere displays of data (e.g. teaching materials such as PowerPoint slides, exams, syllabi) without an interpretative text to support the claim of high quality teaching are not particularly useful in convincing the committees about teaching excellence.

To help justify the claim of teaching excellence, we should:

1. Provide evidence and an explicit argument for the claim,

2. Present material so that:
   1. Salient points are noticeably highlighted,
   2. It is easy for a busy DEC/FRC/UPTC member to read the document, and
   3. Clear tabs are used for quick reference.
3. Situate the claim within the value system outlined in the NUS guidelines on teaching evaluation; this necessitates analysis and interpretation of the presented material.
   Teaching portfolios are evolving documents, not final products meant solely for submission at the time of review. They are both formative and evaluative. As teachers, it is useful to develop our portfolios almost as soon as we begin teaching. The portfolios are be periodically revised, updated and supplemented with accumulating evidence. Revising reflections about our teaching is important as they form the basis of a coherent and integrated document
   

Some Recommendations

This section provides some hints for writing selected parts of the portfolio. Owing to space limits, only two parts, teaching philosophy and module folders, are highlighted as examples of the approach taken to develop the whole portfolio.

1. Teaching Philosophy

   While the scope of individual philosophies varies, we recommend elaborate discussion of at least three central, interrelated issues. First, it is useful to discuss the educational goals that we feel should be promoted in a university context. Common pitfalls in this section include:

   1. Overly brief discussion,
   2. Overuse of buzzwords without expanding upon their meaning in capturing educational processes, and
   3. Incongruence between philosophical statements and actual practices.

   Next, it is useful for us to address how these educational goals are optimally promoted by certain teaching approaches. Writing about the issue helps us to reflect upon what we do in the classroom and explain how our teaching approaches have advanced the goals. (Incongruence between our philosophy and practices is a cause for concern).

   The final issue is on how students interact with our approaches to develop the cognitive abilities reflected in the above educational goals. However, the issue is often intertwined with the previous one and frequently written concurrently with it rather than as a separate sub-section. The issue requires that we take the students’ perspective, to explain exactly how they learn. It is also useful for us to describe the role of students in the learning process.
   Overall, the teaching philosophy outlines our educational goals and builds a bridge between the abstract goals and the actual classroom activities undertaken to achieve those goals.

2. Module Folders

   Like the teaching philosophy, the scope of an individual’s module folders varies. However, it is most useful present the module folder as more than mere reproduction of teaching materials (e.g. syllabi, lecture notes, exams). Explicit explanations and interpretations will help convince committees about the quality of one’s teaching. The following issues that deserve such analysis on our part do not represent separate subsections, but rather are intertwined issues permeating the module folder:
   

   1. The planning of our modules flows directly from, and is coherent with our teaching philosophy. Choices concerning course content, teaching approach and assessment reflect the educational goals outlined previously in the philosophy. The focus here is the promotion of specific learning outcomes via the choices.
   2. Execution addresses how various aspects of planning are manifested in the classroom. It is desirable to give the reader a sense about what goes on in our classrooms by drawing a narrative picture. The narration is especially important if we use non-traditional approaches (i.e. other than unidirectional lecturing). We make the strongest case when we explicitly link particular activities with specific learning outcomes.
   3. Outcomes draw our attention to the definitive goal of teaching—actual learning outcomes—the ultimate criterion by which teaching is judged. Perhaps, the easiest way to demonstrate the outcomes is via our assessment tasks. Continuous assessments include both assignment handouts and sample papers completed by students. In the assignment handouts, it is useful to highlight briefly how the structure of the assignment has promoted specific learning outcomes. Sample papers can also display concrete manifestations of the outcomes. Reviewing them could be made easier by highlighting selected parts and providing a brief narrative explaining the specific cognitive skills illustrated by each part. In addition, our written feedback to students on the sample to papers could also bear evidence to the furthering of the desired learning outcomes. Finally, it would be desirable for us as teaching professionals to make the sample papers anonymous.

   In terms of exams, it is best to include the complete exam. It is valuable to demonstrate that the exam questions are not repeats from previous terms’ exams and tutorial questions (and the like) within that term. Dissecting some exemplary exam questions, highlighting the specific cognitive skills required by the questions, will help indicate the quality of the assessments.

   Finally, when we use discussions or group work in our classrooms, it is extremely helpful to include brief transcripts of such as evidence of the learning outcomes promoted by such techniques. Again, the review of the data can be facilitated by highlighting parts of the transcript and providing a brief narrative outlining the specific learning outcomes illustrated by each highlighted part.

Conclusion

It is most beneficial to present the teaching portfolio like a research paper such that the data are not only presented, but also analysed within a larger interpretative framework. Without such analysis and explicit arguments, the portfolio will be incomplete. An incomplete portfolio will underestimate the hard work put forth into teaching. Thus, a well-presented portfolio allows us to receive full and proper credit for the professional activity of teaching.

For a full schedule of CDTL’s seminars, workshops and the teaching portfolio seminar, visit: http://www.cdtl.nus.edu.sg/cdtlhome/calendar.htm. Look up the following pages on the CDTL website http://www.cdtl.nus.edu.sg/Ideas/iot17.htm and http://www.cdtl.nus.edu.sg/handbook/evaluate/portfolio.htm for more information on the teaching portfolio. Links to OHR’s documents on teaching evaluation in NUS can be found at: http://www.cdtl.nus.edu.sg/cdtlhome/policies.htm. The FAQs on P&T on the FASS website: https://ap1.fas.nus.edu.sg:8001/servlet1/admin_page contains helpful suggestions.