On Teaching Evaluations (with a bonus guest essay…)

A long post with some thoughts on student evaluations of teaching, and a guest essay on similar themes.

Part 1: I suggest that such evaluations should be done a year after a course, rather than immediately at its end.

Part 2: (Guest post) On the difference between “good” and “effective” teaching, and our failure to differentiate them.

I am continually amazed that though teaching occupies a good fraction of my time, and though the largest chunk of my university’s income comes from student tuition [1], so little effort is spent assessing how well I teach or how well students learn. This isn’t particular to me or to the University of Oregon. At least at moderately sized U.S. universities it’s nearly universal that the primary mechanism for assessment comes from end of term evaluations by students. The validity of these evaluations, or their correlation with actual teaching quality, are highly suspect.

Everyone knows this. The internet is full of critiques of the practice of student evaluations of teaching [2], and I’ve never heard a passionate defense of it. I myself am very critical, even though my evaluations tend to be quite high [3], something I’ve put a lot of work into. High evaluations could be a sign of inspiring, well-organized, stimulating teaching, or they could be a result of cheap theatrics, an easy grading scale, or personal charm. Low evaluations could result from logistical ineptness, poor communication skills, or capricious grading, or they could stem from presenting challenging but eventually rewarding material that pushes students more than they are used to. Numerical evaluations can’t distinguish between these possibilities, and I doubt that even written responses would work unless students were taught to reflect on what learning should entail.

Though a lot has been written about the flaws of student evaluations, there seems to be relatively little on what a good evaluation scheme would look like, especially with respect to when such assessment should occur. (Another good question is who should be assessing teaching, for which I’d argue that both students and other faculty should be evaluating classes, but that’s another essay…)

Longer-term evaluations

I’ve long thought that evaluations shouldn’t come at the end of the term, but rather much later — the following year, perhaps, when one can actually reflect on whether the class prepared one for whatever came later, or on whether the things one learned changed one’s views of the world.

End-of-term evaluations are, of course, convenient — all the students are there. But in 2018, it shouldn’t be hard to track and contact students. Except for senior-level classes, the students will likely still be students the following year. Those who have graduated or left can provide an email address for future contact. (University fundraisers seem quite capable of finding them!) Perhaps the response rate for later polling will be less than for end-of-term surveys, but a smaller number of meaningful responses is better than a larger number of meaningless ones. And, of course, there could always be an immediate survey as well for simple questions of fact. (Did the course meet when it was supposed to? How many hours per week did you spend studying for this course?)

From personal experience, it’s great to hear from students long after a course ends who report that a class had an impact on them. I’ve heard from students who went on to graduate school, but also from random former students on the streets of Eugene. My favorite was a cashier at a local natural foods store, who happily commented on topics from my Physics of Life course, despite it being two or three years in the past, and despite getting a D in it.

These encounters rare, and subject to a large selection bias in that anyone who didn’t like my classes would keep their mouths shut! Institutionalized year-after evaluations would make meaningful feedback more common, and more useful for assessing the success of a course.

I’ve been thinking of writing about this topic for months, and I was spurred by a colleague who sent me a “guest post” on similar themes, most of which I’ll reproduce below. It’s excellent, and I’ll especially point the reader to the student quote.

Before that, I’ll conclude by stating that I can’t think of a good argument against longer-term evaluations. If you’ve got one, I’d love to hear it!

Guest post: “Professor X” reflecting on good vs. effective teaching

After 40 years, there are still 3 things that I have no clue about:

1. How students learn?
2. What defines good teaching?
3. How to motivate dis-interested students?

But once upon a time, life was simple and I knew everything about teaching success even before I taught my first college class. I would come to class, highly prepared, organized and knowledgeable and give the students an efficient, but high density lecture from my perceived position of authority. In turn, the students will eagerly digest this beautifully presented content and they will be motivated, learn, and I will be a Good teacher. But hey, isn’t that what textbooks already provide on any subject? Am I more knowledgeable than a textbook? Wait a minute, what value do I actually add to the teaching enterprise? Zero?

I taught my first college course in spring term 1978 at the University of Washington. Although I was just in my first year of graduate school, I needed the extra money associated with teaching as the primary instructor so I lobbied my department for the opportunity by creating a brand new course titled “Cosmology and the Origin of Life”. It is now December of 2017 and I have just graded my final exam in a course titled “Cosmology and the Origin of Life”. Have I learned anything by teaching the same course for 40 years – a course in which the content has completely changed due to discoveries? I don’t know, but I think I am now keenly aware of two issues a) the physical nature of the classroom “learning” environment has seen no evolution, for the most part, over this 40 years (indeed the classroom I most recently taught in is 60 years old), b) that there is a real difference between “Good Teaching” and “Effective Teaching”. After 40 years, my conclusion is that I am NOT a Good Teacher but I am an Effective Teacher.

To back up this conclusion I need to define the difference between good and effective teaching. In the operational real world Good teaching and recognition of that usually is based on student evaluations. In my opinion student evaluations are primarily based on

• how efficient the instructor used class time
• how comfortable the students felt in the class
• how comfortable the students feel in approaching the instructor outside of class

Going 3 of 3 here guarantees that one is a Good teacher and academia generally assumes that a Good teacher is an Effective teacher.

But what is an effective teacher?

Simply put, I believe the measure of teaching effectiveness is how the students interact with the material. I, the professor, should never be the “center of attention in the room”, the material should always be at this center. The one thing I can say for sure over my 40 year career is that I have consistently made up interesting assignments and group assignments for student navigation and interaction with the material. Yet, never once have these materials ever been considered in my teaching evaluations. Indeed, there is no way for the evaluation process to even know how the students interacted with the material – I am never asked for example assignments, exams (and I have given many creative finals), etc. So really, am I just a talking head that interprets textbook material for the students, and if I tell a few jokes along the way, then I can be a Good teacher by their evaluations? Is that all you need to do to be a Good teacher? Wow, in that case I have really screwed up.

Hmm, let’s see what happens if I go beyond being the interpreter: Let’s try raising some nuanced issues related to the material. Let’s try to engage the students with complexity and uncertainty. What happens? Well, students do not like this because the professor is “teaching less” and, worse still, they are actively off-loading the responsibility to critically think about the material onto the students, rather than simply telling them (er, excuse me, teaching them) what it all means. Thus, while student engagement with the nuanced issues in some course might be an example of effective teaching, the consequently worse student evaluations under this approach are a strong reminder that Good teaching cannot be achieved in this manner. Hence, BAD teachers like me that challenge students and directly confront student biases and pre-conceptions (so that they can actually learn) always get a tail of strongly negative evaluations as I am perceived as not teaching the material.

I am a big believer in “reflective learning” and I approach all my courses in different subjects from this basis. Experience shows that this works for about 80% of the students but the 3 for 3 requirements listed above means that 100% of the student evaluations need to be positive – then you are a Good teacher. Indeed, the only evaluations I ever pay attention too are the longitudinal ones in which students, after several years of reflection write back and generally say the following:

I really couldn’t stand your approach to teaching while I was actually taking the class. But now, after some real life experience and some reflection I realized that your class was the rare class that consistently presented big picture complexity and offered tools and skills to deal with that. I certainly did not appreciate it at the time, but now I realize this was perhaps the most valuable course I took in college.

By now, I have enough similar statements that make me believe that I have been an effective teacher over my career. I also now realize how hard it has been to intentionally strive to be an effective teacher. Forces work against you, peers never acknowledge your efforts at effective teaching, students initially disdain effective teaching, and for the most part, the physical classroom still treats students like sardines in a can which certify you, the professor, as sardine Master. So my experience at trying to be effective, for the most part, amounts to pushing an increasingly massive dung ball uphill; when I finally get to the top, it will simply run me over.

But what can be done about this? Well, I will close this blog-rant on a positive note because there are things that can be done. The use of various forms of IT and communication tools do help to support the concept of the Learning Community. As a definition, I will excerpt that from Harvard University:

Learning communities provide a space and a structure for people to align around a shared goal. Effective communities are both aspirational and practical. They connect people, organizations, and systems that are eager to learn and work across boundaries, all the while holding members accountable to a common agenda, metrics, and outcomes. These communities enable participants to share results and learn from each other, thereby improving their ability to achieve rapid yet significant progress.

Operationally I try to have a class in which a group of interdisciplinary scholars (you call them students) and I form an equal basis learning community. Classes I teach in areas of climate change, energy policy, and data science are good vessels, but I find this approach can be used in most any class, but likely not in classes over 80 students, where, in such classes, expediency generally replaces learning.

So in closing, while teaching students as isolated sardines can lead to Good teaching, truly effective teaching lays in forming a learning community that asks probing and critical questions about the nature of the subject and its supporting evidence in order to collectively synthesize a better understanding. On that score, I believe I have been successful, and while no one is ever going to characterize me as a Good teacher, that same voice will never convince me that I have not been an effective teacher. In the end, that is my contribution to academia.

Note added: see also the next post, a brief addendum to this one.

Today’s illustration

Myoglobin, which I drew for something else I’m writing. In 1959, Myoglobin became the first protein to have its 3-dimensional structure determined.

Notes

[1] Tuition makes up about 39% of the University of Oregon’s revenue. The state of Oregon provides about 7%. Source: http://ir.uoregon.edu/expenditures

[2] For example, “Everyone Hates Course Evaluations,” “Zero Correlation Between Evaluations and Learning,” “Can the Student Course Evaluation Be Redeemed?”

[3] For example, see the second-to-last graph here. This is typical.