I'm pretty skeptical of the claims in this - it'd be interesting to see actual data.
Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses.
Though the worst professor I had at RPI held our grades hostage until the evaluations were in, so they're not perfect.
In America teachers get tenure so your feedback has no impact on them improving.
Also if you look at literally any study of them TAs for harder courses get harsher ratings, they get harsher ratings from weaker students.
I also spent a number of years as a TA I can say anecdotally that is my exact first hand experience - good reviews in easy courses like graphics, atrocious scores in design of concurrent systems. In fact for concurrent systems it was well known that the TA for the course would literally always get the lowest review rating in the department, and that was independent of their rating in any other courses that they TAd.
TAs also get punished if the actual instructor is bad - because students blame everyone in the course.
I mean the American system has its own slew of problems - lecturers run the tests themselves, they by design don’t provide prior exams (hint: if knowing the prior years exam questions tells you enough to be considered your exams are bad and you should feel bad)
Very few professors are granted tenure, as was the case in the past. It's weakly non-zero: times have changed. There are far more adjunct professors today. Universities are more run like businesses than in decades past.
I don’t have data, but I think most tenure track professors get tenure (if not at their current university, than another).
Adjunct professors are not tenure track, so there is really no expectation that they would get tenure. Though I have seen the numbers that say the number of adjuncts is increasing.
Almost every lecturer my wife has is tenure track or already tenured.
One of them just plays videos of himself in his lectures. He doesn't actually go to them. He is actually around during less than half of his scheduled office hours. He complains to (and at least once has shouted at) the TAs about how students keep asking him questions. He has tenure.
As a former TA for one of the "bad" professors, you can somewhat rescue your rating, but you basically have to be a hero, and the professor needs to not be malicious (if the prof wants to fail everyone, not a lot you can do).
You have to attend every lecture (I did anyway), redirect / fix during discussion / lab, help them learn what they need to know to pass the exams, and if you have time, what they should remember when the term is over for their future academic / work careers.
Only bad ratings I recall were from students who only attended the last discussion section; it's hard to teach 10 weeks of theoretical CS in 50 minutes.
Not my experience - being a TA for a bad professor or bad course involved more work but gets one better evaluations. You're the "good guy". If the professor is good, the standards are higher and you can't leave their shadow.
The way to ensure that they’re different is to require all prior assessment be publicly available - if someone having access to an old exam gives them an unfair advantage over people who don’t then that’s bad design.
Oh sorry I interpreted it as a hypothetical or something :)
My interpretation of the way US lecturers work is that they may reuse entire exams. Especially as a lot just get exams from book publishers.
In NZ (at Canterbury at least) all exams are archived and available in the libraries, and most also as PDFs. All of them. Shelf after shelf stretching back decades. At least back when I was there - maybe it’s different now? Publishers selling exam material as part of their text books seems to be moderately recent
I’ve found that the best professors tend to come with higher standards for grading, more involved assignments, and just generally more work involved. This is great if you’re actively interested in learning the subject; most students are not. In which case, its far worse than the shitty teacher, and its reflected in the feedback.
From the perspective of most students, the ideal professor has low grading standards, “good enough” teaching, and most importantly, a “fun” class (primarily by means of humor). Most students are there to get a degree, to get a job. The best teacher according to students is the one that best fits that model (which is generally at odds with the goals of classical academia).
One of the best classes that I had was very hard in homeworks but was easy on the grading. It was literature and art class taught by a lawyer. I felt that I grew up a bit after that class. The class was also rated high by students
I was thinking primarily of my history with stem (really, specifically the t and e); I feel like the arts are still more classically oriented, because a degree isn’t anywhere near as required, and theres weaker guarantees on income streams, such that its been relatively unaffected by the university trying to fulfill the role if trade schools. The same would be true of mathematics, phds in general, and probably most of the hard sciences.
Undergrad feedback I wouldn’t trust at all regardless of major, Masters is major-dependent (ce, cs are particularly screwed) and phd candidates are probably fine. Simply by looking at the why the general population is even in the major (eg ask an undergrad and the reason is probably not actual interest in the subject: parents, jobs, money, dropout-major, couldn’t decide, best grades in hs, etc). Their feedback will naturally reflect the misaligned incentives
If your interest in feedback is to make sure the teachers are actually good at teaching, anyways.
Tying them directly indeed seems like a bad idea (like pretty much any other naive reputation score), but that doesn’t mean you have to give up on the feedback mechanism.
Just set a threshold - such as “scores an average of < 4/10”. If the score is below that threshold, invest a little more time and effort into getting more detailed evaluations and figuring out why the score is low and whether it’s a genuine problem with the professor vs something like their having high standards and lazy students. Then train or fire as appropriate.
Strict thresholds don't usually work. My school had public metrics, and teachers preferred classes made a huge difference. The teachers that taught the harder classes (component design rings a bell) always got significantly lower scores than the ones teaching easier classes (physics I, circuits for Mechanical Engineers)
That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And sure, tune the threshold to the department. But also consider that maybe teaching is better on average in some departments than others. (For instance, the drop-off in teaching quality from high school to college in my experience was enormous in math and arts but slight in both sciences and humanities. Which seems to me more like a problem with the math and art departments’ approaches to hiring or teaching than with math or arts as subjects.)
That’s why you don’t use the threshold to trigger anything more punitive than further investigation.
And by "you", you mean "no one ever". Teacher evaluations have immediate and universal impact. There is essentially no filter and no interpretation on this. It's like tweets. Once a social signal is out-there, public, everyone is acting on the implications regardless of hypothetical mitigating factors.
A lot of people talk about this stuff in the abstract, as some hypothetical, like we'll do this and we'll have safeguards in place. The news is, this is how things have been for college teachers for a while. You get an evaluation and it has an impact and there's no mitigation and no contextualizing.
Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses.
Now that I think about that.. I did have a professor in my undergrad who held grades until the evaluations were done. Suffice to say, I regret the evaluation I gave.
Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses.
Though the worst professor I had at RPI held our grades hostage until the evaluations were in, so they're not perfect.