Reading the Evals

Although we got the statistical reports on evaluations a few weeks ago, the actual forms with narrative comments did not come back to my mailbox until today.

I'm becoming a fairly popular teacher here, and so affirming narrative comments are embarrassingly common here. Not because I think I'm this amazing teacher, but because I'm beginning to think that the way students fill out evaluations is as much a function of their overall learning across many courses, as well as their general critical thinking acumen, more than what happens in one course. And that relationship is not necessarily a direct relationship, either.

Take the 300-level commonwealth course: The statistical scores were a bit more varied than other courses I've taught, with a few lower scores than I've gotten at this institution (though not other places--more about that in a sec). And yet the narrative comments say so much more than a 4.04 statistical mean on one score or the other. For example, the evaluation that answered the question, "Course material challenged students intellectually" with "usually" (3 out of 5), wrote in her narrative comments: "Horace is the absolute best professor I have ever had at BRU. He masters his subject matter, encourages critical thinking, and his enthusiasm makes you want to learn and participate. He is also incredibly helpful and has a great personality."

So wow. That's a dream comment, and I strive to live up to that kind of vision that some students have of me. But her decision to evaluate the questions individually instead of doing a straight-ticket evaluation tells me more about my teaching, and affirms my teaching more than the student who assigns all 5s. Why? because one of the higher levels of Bloom's taxonomy of learning is "evaluation." Which is to say that this student has (or at least demonstrates) a clearer sense of what the critical task of evaluation is than a student who assigns straight 5s, or, for that matter, straight 3s, and then leaves no narrative comments at all. The straight 3s, by the way, come from the only student who anticipated a C in the course.

In fact, I find that this institution generally has a bit of grade inflation when it comes to evaluation scores. Listen, I think many of my colleagues are likely fantastic teachers. But I also thought that my colleagues at my last institution were also amazing teachers as well. And while I don't have access to anyone else's evaluations, I can guess that eval scores and Rate-my-professor scores probably correlate pretty generally. And I know that here, RMP scores for my department are really generally quite positive, and at my last instituion (andexpensive, private, USNews-top-50 school) my colleagues RMP scores (and mine, too) were pretty average, even mediocre.

I fear that what this means is not that here we're better, or here the students are nicer, but rather at the last institution, there was a culture of critical thinking that is not strong here. Now this is a tenuous conclusion, I know, and it privileges all sorts of things (class and geographic location among them) in drawing that conclusion. But I fear that indirectly, at least, the most uniformly good scores that I get on evaluations may be potentially a sign of a secondary failure, by me and by my institution generally, to really make critical thinking (including but not limited to evaluation, but also analysis, creativity, and other higher-order thought) a priority and an automatic response.

Furthermore, I am both selfishly happy and a little disturbed to see how often "enthusiasm," "personality" and "humor" are listed as characteristics of excellent teaching in these evaluations. Not disturbing in and of themselves: I know I tell some decent jokes, and I like to think of myself as a nice person. But the fact that evaluations generally have become so important in terms of faculty evaluation makes me concerned for colleagues who are, say, low key, or not very funny, but brilliant, and who actually provoke a great deal of learning without necessarily being, say, entertaining, or lovable. This kind of customer-service teaching sends us in many of the wrong directions; It's a lucky (and not necessarily skilled) teacher, who can strike a balance between rigor and likability.

In the short term, these evaluations give me some nice data for my annual report. But they also send up red flags about the culture of critical thinking at this institution (and perhaps many others), and more generally, for the reliability of teaching evaluations for determining teaching efficacy and faculty merit.