Students rating profs: Waste of time?
April 15, 2010 by Geneva ReidPosted in: Academics, In this week's e-newsletter, Latest News & Views
At the end of each semester, students are asked to evaluate their professors. But is it a useful measure of the best teachers — or simply a popularity game?
Some colleges certainly take students’ feedback seriously.
So much so, they’re going to extreme measures to ensure students participate in the evaluation process.
Currently, paper surveys are being replaced at most schools by online evaluations. Why? It’s faster and cheaper in the long run.
But it seems it’s one thing to ask students to fill out paper surveys on the last day of class — and another to ask them to give their feedback online.
For example, 80% of students at Northeastern University filled out paper evaluations, compared to 54% online, according to The Boston Globe.
So the school swung into action and decided to offer a little incentive. During the two-week period designated for filling out professor evaluations, students have a chance at winning iPods and meal vouchers.
At Harvard, students are allowed to see their grades a few weeks earlier than usual — if they fill out an online survey.
But are these evaluations worth all the effort? Some say absolutely, since they help determine which classes students take and which professors get tenure.
But others believe the feedback from students isn’t much more than a popularity game, with the highest ratings going to professors who are well-liked and not necessarily the best teachers.
What do you think of student evaluations? Let us know in the comments section below.
HigherEdMorning delivers the latest HigherEd news once a week to the inboxes of over 200,000 HigherEd professionals.
Click here to sign up and start your FREE subscription to HigherEdMorning!
Tags: Harvard, Northeastern, professor evaluation
Loading ...
April 14th, 2010 at 8:54 am
Student evaluatioin of classroom instruction can be a valuable resource if the form used is one that has been well written and meets the requirements of a valid survey. Many of the forms used by schools are self-created and do not lead to any meaningful data, but merely indications of a faculty member’s popularity.
April 14th, 2010 at 9:25 am
Student evaluations help me, as an instructor, to know what is working and what is not working. For instance, I teach a course that requires me to apply my concepts to diverse populations. I had a student comment that I put too much diversity in the course. Perversely I knew I had done my job that semester. So I use the comments as tools to help me be a better teacher. I also know which comments to take seriously and which ones are not helpful.
April 14th, 2010 at 10:06 am
This is a topic on which there is some research. It turns out there are some things students are good at assessing, and some things … not so good. They are not very good at assessing content — what the prof knows and wants to impart. They are good at assessing style — how much the prof holds student attention, how warm s/he is, how enjoyable the class is. One question is “how important is is style?” It should count for something, but content isn’t chopped liver, and since content is seldom assessed by departments or administators (at least not in a rigorous fashion), teaching assessments based on student course evaluations are woefully incomplete.
April 14th, 2010 at 10:39 am
I think it’s all in the way the survey is set up. If you want feedback, ask pertinent questions for students to answer. You can usually figure out if you have a whiney student versus a poor instructor based on several surveys from a particular course. My daughter recently used the ratings to pick her upcoming course instructors. I looked at them with her and felt they were valuable. She actually chose courses that were less advantageous for her schedule based on better course reviews. I felt the reviews had substance and were not just about how easy/difficult the instructor was. Several reviews bashed the instructor, but some were language barrier problems with a foreign instructor or that the instructor often didn’t show up for class, etc.
April 14th, 2010 at 11:09 am
I believe they have value, if used properly. That is, administrators can look for outliers (and have reward/sanctions/constructive feedback accordingly) and faculty can get some useful feedback. But, the problem comes when faculty “compete” for high ratings (for ego, rewards, etc.) and administrators “split hairs” over relatively fine gradations. Also, it would be helpful if the top and bottom 10% of all respondents were dropped–at times, students “retaliate” if they receive a low grade in a course.
April 14th, 2010 at 11:29 am
Students tend not to trust campus based surveys. Mainly this is because the are never sure of the anonymity of the responses. This is especially true of on-line surveys. They are tech-savvy enough to know codes are easily embedded.
My daughter uses ratings of professors and courses overall (looks at responses for the course as a whole) to help guide her decisions. So far after two years she has been pleased. This is compared to another daughter who has not been as thorough and has would up in some terrible situations related to course, content and professors.
April 14th, 2010 at 11:37 am
Students are not in a position to judge the quality of the content presented. They can judge whether or not they were entertained, whether they thought the prof was too demanding or (in their view) inflexible about assignments, whether or not the course syllabus was clear (to them), and so forth. Students’ comments ought to be considered in the overall evaluation of the quality of teaching, but too many administrators consider those comments as the ONLY evaluation, and that’s the problem.
April 14th, 2010 at 11:41 am
Mostly a waste of time. The only part I find at all valuable is the comments section, and many students don’t take the time to say anything there. As one professor told me about a decade ago when I started at his school, “Take donuts the day of the eval….”
April 14th, 2010 at 11:45 am
Why do all this talking about student evaluations when it’s obvious that they are a popularity contest, how could they not be? Do you think a student who dislikes a professor (for whatever reason-style, race, look) can/will offer a “fair” assessment? If you think so, then ask yourself why African American professors virtually always get lower scores (erase all racist hogwash from your brain before pondering). They get lower scores because students are biased (see Harvard Implicit Association Test).
April 14th, 2010 at 12:18 pm
My experience is mixed. I have found the common 1 – 5 rating of X number of characteristics to be pretty useless. On the other hand, evaluations that incorporate actual written responses have been of value. The written response not only often tell what is or isn’t working but also can point out problems and misconceptions about the course. These problems and misconceptions (generally about course requirements etc) can generally be successfully addressed. These are also the sorts of things that do not show up in the 1 -5 ratings. My current institution uses the 1 – 5 ratings. I supplement with an in-class anonymous survey on my own to get real feedback.
April 14th, 2010 at 12:41 pm
Student evaluation of professors can be a good reference point of professors’ teaching; however, it should not be the only reference point. Student evaluation is not the only evaluation that has biases. Similar biases also apply to peer review and even promotion and tenure review. It is always a popularity contest.
April 14th, 2010 at 12:47 pm
I think we’re forgetting that we ourselves can *learn* from them. Leaving aside the question of how useful they are to the administration (I accept that “popularity” plays an unfortunate role here), let’s bear in mind that student responses help us to change the way we teach. This is especially true when questions are not of the form “was X good or bad?” but rather of the form “what in X was particularly helpful?”. As a result of such feedback, I’ve dropped texts from a syllabus, kept texts I thought I’d have to drop, increased the number of handouts I use, etc., etc. If enough students report on something specific, they’re probably right.
April 14th, 2010 at 12:55 pm
Not only are evaluations a popularity contest (in which “traditional” white, male professor at an advantage), but worse, they put pressure of professors to ease their courses, to accept late assignments, to give EXTRA credit to students who haven’t worked hard at the assignments outlined in the syllabus. If we do not adjust for the needier students, they can not only lower their evaluation of teaching, but also complain enough to their peers and have them come away less satisfied as well.
My department used SETs as the sole criterion for rating teaching performance for years. Obviously, this is an ignorant approach. Now that we have moved away from that strategy, we are left with a small but entrenched group of students who are well accuctomed to bullying their way to getting their late work accepted, to demanding extra credit, and to wielding their power over their professors’ career tracks.
Although I like the feedback from evaluations, I think it should be kept private between the class and the prof. The university should spend time and effort evaluating teaching through classroom visits by the Deans who will rate teaching and other peer-to-peer or Dean-to-professor techniques.
April 14th, 2010 at 1:17 pm
I often peruse the “Rate your professor” online service where the disgruntled tend to predominate, but unfortunately that is what prospective students see. I think a valuable component for any student evals might be to include the expected or actual grade received. I would imagine the students who do poorly often give the more negative comments.
What I have noticed is how students react when they see me after the term has ended. On campus many of them avoid eye contact (I’m like the parish priest who knows their inner secrets revealed during the “confession” of their classroom performance), but surprising enough some who didn’t get outstanding grades tend to greet me warmly. Obvioulsy, they feel they learned something and they noticed my concern for their educational progress. Thus, I would find positive remarks coming from a C+ student or negative comments from an A student to be most significant.
April 14th, 2010 at 1:27 pm
It is my perception that student comments on Instructor evaluations are based on what they percieve their final grade will be. My technical college switched from paper evals to online a few years ago. We went from a participation rate of almost 100% to about 20%.
April 14th, 2010 at 1:45 pm
I do not often believe most of it. I generally get reasonably good reviews, but I do not know whether it is because I am a good teacher or whether the students think I am a nice guy.
April 14th, 2010 at 2:33 pm
I was deeply involved in the committee in the chemistry department at the UW-Madison in the late 60′s that was one of the first academic institutions, if not the first, to come up with a student evaluation form. At that time, it clearly was done as a response to the “student power” platform as one way of quieting the campus unrest, at least a bit. There was only one question we used that we, as professors, felt could be used for anything as serious as tenure input. That became the infamous “question #9, The professor was an effective teacher”. In this context, we felt that the word “effective” meant “get the job done”. After all the modifications that have taken place since then, various evaluation forms have queried course content, professor style, some issues of apparent fairness of exams, and, yes, professor likability. After I received my tenure, I paid little attention to the popularity issues, but quite a bit of attention to those that addressed my effectiveness as a teacher. Before my tenure promotion, all that mattered was my research.
April 14th, 2010 at 2:40 pm
I find student evaluations, as practiced in my department, generally accurate but unhelpful. I can imagine scenarios under which they might actually be counterproductive.
The evaluations correctly identify the better and the worse teachers – but this is usually known, even to the teachers themselves. They also often reflect the overall grades earned by a particular class, illustrating the reciprocity between students and professors’ impressions: if you give x% A’s, you can expect x(plus or minus delta) % excellent evaluations. None of this teaches us anything.
Occasionally, I value some of the written comments – but since our system went on-line whereas the written comments are still handled with paper, those now come from less than 20% of the class and are not representative. Conversely, student evaluations put pressure on the teachers to dumb down the content, to emphasize the entertainment aspects of the lectures, and to decrease the rigor and objectivity in grading. I can imagine how professor who are up for tenure, and who know that their SEI grades will be part of their dossier, might feel what I estimate is undue pressure.
April 14th, 2010 at 3:25 pm
Student evaluations in theory could provide useful information. Unfortunately they are used as a tenure criterion and as a former chair who has had to read a lot of them are pretty useless- apart from some surprisingly racist and sexist comments. The existence of such comments is for me sufficient to believe the entire process is potentially tainted and certainly biased.
For actually improving the courses, meeting with a small group of students after grades are in helps a lot more. Of course that fails to provide a number that we can pretend means something.
April 14th, 2010 at 3:35 pm
The comments made David Beckingham are offensive and should be removed. He has said anyone who rates a black professor low is a racist. This blatant use of racism (when there is none) has got to stop and the perpetrators exposed.
April 14th, 2010 at 3:53 pm
I usually get good reviews but I am more excited to find out that the class has performed well on a national exam. I really think that the only way to determine how a professor has done in a course is by content taught and lessons learned by the student. The only way to really assess this is by giving a pre-test at the beginning of the course and a post-test at the end of the course. It is difficult to measure what student evaluations measure about a professor.
April 14th, 2010 at 4:00 pm
I had the opportunity to evaluate the evaluation process at two very different institutions, one a large public university with commuter students and the other a small, selective, residential college. The State U procedure was a questionnaire during the last week of the term, just before finals — stress time. That was definitely a popularity and entertainment-value contest. If a class criticized a prof for a certain practice or procedure, the next class criticized him/her for doing the opposite in response to the previous criticism. The small college queried students during the year following a particular course, so that they could evaluate their learning when applied to a subsequent, related course. This is much more informative and useful in evaluating the preceding prof, but (I grant) impossible to use in Big State U because of the turnover of registered students even when there is a pair of sequential courses.
April 14th, 2010 at 4:01 pm
I consider the comments useful–usually if a student has a strong enough opinion + or – they will write a comment. The rest is only GENERALLY useful to me as an instructor. I have to turn in my evals to my administrator once a year, and I really don’t know why she would want to look at them. I’ve only been observed by peers or administrators a handful of times in 15 years at either of the schools where I teach. To me, that would be a much more solid evaluation of my skills and whether or not I’m covering the content of the course.
April 14th, 2010 at 4:10 pm
Last semester I read several hundred evaluations other than my own. It appears that students do not know how to assess faculty as the answers on the front of the form and those on the back did not correlate. Why give a Professor <3.0 (1-4 scale) and then go on to say how much one learned and how great this teacher was in helping students? In general those with larger classes seem to be involved in a popularity contest, with many Professors getting the highest scores who 1) entertain students, and 2) provide a "study guide" of all possible exam questions. I do not provide exams questions to students. Next year I am going to spend a little time explaining what the scale on the one side means and how it should be consist with the boxes checked and statements made on the other side of the form. I agree about the "traditional white male" syndrome that we have to operate under. My male colleagues never get the nasty, mean-spirited comments that my female colleagues and I get. And some of the men are very harsh with the students and tolerate no bull from them! Yet no one complaints about it, even when they humiliate them in class. I have to say disrespect for female faculty has become more frequent in the last 8yrs or so as students realize that they can say and do just about anything to a Professor and get away with it because of many Universities fear law suits. Actually, before taking my current position 11yrs ago, students never made nasty personal comments about me and they were quite respectful. This seems to be part of the new generations view of their Professor.
April 14th, 2010 at 4:55 pm
Student ratings of professors are an absolute waste of time. My better-performing students tend to rate me highly, and my lowest-performing students tend to blame their poor performance on me (a form of denial?). I stopped wasting my time reviewing the ratings a long time ago.
April 14th, 2010 at 4:57 pm
This is a topic where personal experience and opinion often trump data. The IDEA Center has some good reviews of the voluminous research on student ratings of instruction. Go to http://www.theideacenter.org/ and click on “IDEA Papers”. Bottom line, as others have commented — student ratings are a useful component of a faculty evaluation process, but shouldn’t be weighted too heavily or used to make fine distinctions. The forms used need to be validated.
April 14th, 2010 at 5:11 pm
I actually have to type the comments on 4000+ evaluations submitted by the students in my department, so that the instructors have possibly one (sometimes two) pages of comments along with the data rather than seeing the handwritten version (which also cuts down tremendously on storage). It’s interesting to me to see that sometimes the trend is did the instructor make the course FUN and was it FUN and did they have FUN and was coming to class FUN. The next semester the trend word was respect. Did they feel the instructor respected them and was the instructor respectful and were they treated with respect. Our department takes these comments seriously, and they are considered when tenure/promotion/reappointment decisions are made, and adjunct instructors with low overall performance are not rehired. On the other hand, the administration does not take seriously glowing evaluations for instructors who pass out A’s like candy. I can always tell when the instructors hand out the surveys. Those who distribute them at the beginning of the class have LOTS of comments; the ones where they are done at the end of class when the students can complete them and leave are lucky if one person writes one comment.
April 14th, 2010 at 5:30 pm
I am, based on grades given, one of the toughest teachers in our department. I also get some of the highest evaluations. Colleagues who I know are not the best, either in their discipline, or in teaching, get some of the worst.
Of course, in our evaluations, we always look at the courses in question. Business and education majors tend to be over-critical and negative, engineering and science majors tend to be much more objective.
April 14th, 2010 at 6:13 pm
I teach a chemistry course for BS nursing majors. Multiple choice evaluations (like multiple choice exams) are oftern not too useful. The results usually correspond to the student’s expected grade, how little work is required by the teacher, and the student’s thoughts as to whether or not “the teacher is on my side.” I have from time to time given my own essay evaluation that asks students to discuss 3 questions: 1) What would you recommend to your friends who need to take the course next semester with the same teacher? 2) What were the most useful and most interesting aspects of the course? 3) What recommendations do you have for the teacher for improving the course BUT you cannot suggest decreasing the amount of chemistry in the course.
April 14th, 2010 at 6:19 pm
The value of course ratings are similar to the value of the comments here. They are limited by all sorts of bias. Selection bias being the biggest one. You typically hear from readers (here) or students (in class) at either end of the spectrum. If you use incentives to try and improve response rate, will you get a true evaluation, or only what’s needed to get the incentive. Until we can ensure that the ratings and comments are free of bias, the value of the ratings will be in question. How do we get there?
April 15th, 2010 at 8:47 am
Evaluations are an utter waste of time. In my experience, if the student “liked” you/the class/the grades he or she received, a good evaluation will come. If the student didn’t “like” you/the class/the grades, a poor evaluation will come. As for students assessing content, I agree with Paul: the students are in no position to judge content, especially when their judgement is often that books are “too long” or “too complicated.” Too often this leads to instructors dumbing down courses, teaching to the least motivated and/or lowest of the low. On the other hand, grades are then inflated, SEI’s get better, and all’s right with the administrative world. Then again, perhaps I’ve simply grown jaded . . .
April 15th, 2010 at 10:25 am
I believe that student evaluations are more useful than not. Administrators love them because the results are a
‘countable set’, i.e. it gives them (administrators) numbers on which to base salary, promotion, tenure, etc.
thus they have numerical ‘justification’ for whatever they do. Still, I found evaluations useful in pointing up my own shortcomings as well as what I was (presumably) doing right.
I’ve thought for 30+ years (I retired after 50) that a really cheap (oops!….er….inexpensive!) and reliable evaluation system would be as follows:
Hand each graduating student in the pre-ceremonial commencement line a 3 x 5 card and ask them to print the name(s) of their best and worst instructors, labeled, of course. If they can’t remember, so be it: a large fraction of students never bother to learn their instructors’ names. This would yield a quick revelation of the top and bottom strata of the instructional staff!
April 15th, 2010 at 10:39 am
As a dept. chair I use student evals of other faculty for very few purposes – identifying outliers is one, and thankfully it’s rare (and I usually have found out about this well before evaluations are done). The other information I find useful, and think students can comment upon, is whether class sessions meet, begin and end on time, and whether graded material is handed back within a week. That’s it. The evaluation we use asks the student to rate the “knowledge” of the instructor – yeah, right. I don’t put any stock in that one.
As most of us know, the WORST evaluation you can receive from a student is when they say “The prof knows his/her subject, they just can’t TEACH it!”.
April 15th, 2010 at 12:01 pm
I am hesitant to use the word “evaluation” because students are not really trained or qualified to “evaluate” faculty; instead the word ratings seem more applicable. Student ratings of instruction has been shown in the research to be highly related to teaching effectiveness and learning, as determined by other measures; in short, research says that students are able to tell the difference between good teaching and bad teaching.
We use a nationally-normed tool, IDEA, which is valuable because of the comparison of ratings for each faculty member to our own University faculty and to faculty around the country. We have moved to online administration without lowering the participation rate or offering incentives to students, by having them still complete the ratings in the class using their own laptops. While we do see the student ratings of instruction as being valid and reliable, we stress that they should only be used as part of the evidence used to evaluate instruction; there are also chairperson and/or peer evaluations, classroom visits, personal reflection, scores on standardized examinations, and many other measures of teaching effectiveness and learning.
What distinguishes the IDEA system from others is that it asks faculty to select general learning objectives that are important in the course, and then asks the students to report their progress on those objectives. The measure of teaching effectiveness then becomes, in part, the degree to which students report progress on the objectives that the faculty member stresses. This is significantly different than only asking a student if he thinks the teacher is good or if he liked the course. IDEA also uses correlations between objectives and specific teaching methodologies that support progress on objectives to create a diagnostic tool that offers concrete feedback that can be used to support an action plan for improved teaching. That’s what this should really be all about…improved teaching that leads to improved learning.
April 18th, 2010 at 1:02 pm
I agree with a lot of the above comments about the mixed value of the student evaluations. Yes, they can point out areas of concern for faculty –especially when there is a large number of comments about a particular concern; in these cases, faculty can reflect on what caused this specific response from students. Sometimes, the cause may not be within the faculty control; sometimes it may be. So in these cases, the evaluations can be useful for faculty as a reflective practice. Even negative evaluations, those harsh disparaging comments, if analyzed and reflected upon can yield some results.
Yes, it is true that minorities, female faculty, and faculty who are not what students perceive the average, mainstream (whatever that is in different regions) usually get evaluations that can be lower that whatever those mainstream perceptions are. And I want to make a distinction here- not quality of the course or performance- just mainstream acceptance. This is important because yes, it does taint the value of one’s teaching.
Are student evaluations a fair way to evaluate faculty performance? Perhaps if they are done properly. Yes, the IDEA center has forms that may be better, (and that’s what we use in my institution) but still they are not accurate or perfect. Interestingly, the forms are appropriately labeled as student impressions- they are just that. It’s their impression of whether or not they met a learning objective. Sometimes this means to them that they progressively earned a higher grade in their assignments. But what if those assignments were also progressively more challenging, and the tasks were increasingly more difficult, and their grades may not necessarily got better but remained the same? Do students have time to differentiate on that form the level of difficulty and the grade they got (i.e. meeting the learning objective at various levels)? Not really. Their overall impression may actually be a neutral response, an in between, a 3. So what we see there is an average.
The same happens with student perceptions of teaching effectiveness. Again, the total number (the average score) can hurt a faculty member whose average is low because of the distribution of these scores. To come across as an above average in effectiveness (i.e to have at least a 4 score), you need about 2/3 of the students to mark you with 5 or 4 — the more 5s the better. You may be an ‘effective’ teacher but you may end up as a very average to poor instructor on these forms. And yes, there are the strange paradoxes (even on the IDEA forms) where the front page numbers show a poor to mediocre picture, yet the written comments speak about real learning… Go figure.
But departments do use numbers- they set benchmarks which often start at the upper end of the 50% comparative group. And yes, although departments do say that they also take into consideration other material when they make decisions for retention, merit, and promotion, the numbers do end up weighing a lot more heavily on those who are exclusive assessed by their teaching– i.e. the non-tenure track faculty who in most cases in the large public institutions teach the labor intensive courses where the disparate groups of students get sorted out.
So, what are the lessons and the big picture about student evaluations: well, yes a very mixed and unpleasant picture.
I have studied the research about student evaluations for a number of years now –especially after been hired in a position that is full time but non-tenure track and student evaluations weigh very heavily in my professional life (50% of my job depends on them)– and I also bring the experience and observations (and reflection) I have been doing on this issue for the past 25 years I have been teaching first year courses (that general ed required boot camp course– English Comp 1 and 2), and what I am seeing is that they punish sometimes good teachers (for being good), they tend to be unfair for the non-tenure track faculty (i.e they become manipulation mechanisms- unfortunately), they tend to have the effect of stroking the ego of students (especially in the last 10-15 years — for very complex reasons — one of the respondents mentioned how different evaluation results were 11 years ago –) and keeping them pleased rather than challenged (especially in the first year required courses), and help institutions boast about ‘performance’ and ‘productivity.’
Are they useful? Yes- they reflect how higher education works. It works just like any other corporation.
April 21st, 2010 at 1:28 pm
My campus is caught up in the business model of “student as customer” with the view that the student’s autonomy is the key factor. Since the students are paying the money, they can decide how valuable the course and its content and instruction are to them.
I see this as similar to the views of people in entertainment, marketing, etc. who say, “We just produce it. If people don’t want or need it, they won’t patronize it.” We’re just giving them what they want. However, where is the line between freedom and responsibility? I’d rather not be back in Nazi Germany, but we do seem to be at the front lines in a type of culture war.
I teach at a community college and the students are not my bosses; the community is whom I need to answer to. Thus, I need to be cognizant of community standards rather than individual student preferences. This isn’t so easy in our post-modern culture where constant change is here to stay and values seem to be as fluid as celebrity relationships.
As the third grader and the custodian in the comic strip Frazz recently noted, the perspective should be “student as product” instead. The community is the “customer” and we should be producing students who can be community contributors.
April 21st, 2010 at 2:35 pm
It is often not very helpful, but some of the more civil comments can be put to constructive use. (One has to be thick-skinned when screening reviews.)
However, if administrators in their infinite wisdom decide to use student reviews as a factor for promotion & tenure—all bets are off!
The ultimate outcome of such folly would be:
1)Students will receive perfect grades.
2)Professors will receive perfect reviews.
3)Students will perfectly graduate knowing nothing perfectly.
April 23rd, 2010 at 6:40 pm
It is a system to take revenge on a professor who has given me a low grade: Just fire him of her for the sin of not giving me the grade I crave to get through these for years of boring edutainment. Fire that prof, get an easier one!
April 23rd, 2010 at 6:45 pm
Well at the time students evaluate, which is always later in the session, those who were disatisfied likely dropped out. Therefore, those who remain are likely to give the tacher a positive score. If the teacher is funny and likeable and I am getting a good grade, I am going to give him or her a positive score. The opposite is likely true.
My point being is that this process is not a very good one for many reasons. I am not saying it should be excluded but it should be carefully weighted amongst all the other criteria for which the teacher is evaluated including but not limted to peer review.
April 23rd, 2010 at 6:52 pm
Student evaluations are better at revealing problems than certifying effectiveness in teaching, which is the most significant parameter. Students are in a good position to complain about shortcomings and a careful assessments of free-form comments over the simple numerical evaluation of relative scores can point to specific problems. Very low scores can also identify that problems exist even if they don’t point precisely to the details. However, much more difficult assessments of effectiveness, such as following students into subsequent courses that use the course in question as a serious prerequisite and comparing the performance of students of one faculty member to another can be revealing. This includes attempting to compare the subsequent student performance with the evaluation scores in the former courses. Efforts (admittedly low statistics) at this kind of comparison tend to show a lack of correlation with respect to evaluation scores in the upper acceptable ranges and a stronger correlation of poor performance with very low scores. My point being again that it is probably not wise to split hairs over differences in evaluation scores at the top end, while it is reasonable to take action in cases at the bottom end.
April 23rd, 2010 at 7:12 pm
Student evaluations can be useful as personal feedback for professors, but quickly lose their utility when seen by administrators for tenure, promotion, and other purposes. More outliers (on either positive or negative extremes) are prone to fill them out voluntarily on line than those in the mainstream, thus skewing data. Filling them out in class increases sample size–a more accurate report the outcome. Reporting the mean (instead of the median) as measure of central tendency also allows outliers in small sample sizes to skew the outcome. Bottom Line: Let’s keep evaluations as the property of professors who receive them as an aid to improving their performance. We respect the privacy of student grades. The same right should be afforded their professors.
April 23rd, 2010 at 7:19 pm
If Ludwig Wittgenstein were being evaluated by students on an American campus today he would presumably earn a score of zero and promptly be fired by the administration
April 23rd, 2010 at 7:50 pm
To ask students how well a professor did at teaching a course presupposes the student has experience with other professors teaching that same course. Rarely does a student have more than an N of 1 on which to base an opinion.
To bluechip: the “student as customer” model is absurd. If doctors used the “patient as customer” model, they’d be dispensing all manner of unnecessary and even dangerous medicine and surgical procedures. The same is true of faculty. We should be giving students what they NEED, not what they think they WANT.
April 24th, 2010 at 2:31 am
See “Student Evaluations of Teaching: Are They Related to What Students Learn? A Meta-Analysis and Review of the Literature, ” by Dennis E. Clayson in Journal of Marketing Education, 2009, 31(16) DOI: 10.1177/0273475308324086. He concludes, “on balance, universal and group weighted SET results should not be utilized, or they should be interpreted with great care. First, little information may be
obtained about how much students are learning by looking only at the evaluations. Second, instructors who are teaching students to think, and to stretch mentally and professionally, could actually be penalized. As summarized by Paswan and Young (2002) at the conclusion of their study of business students, “Instead of asking instructors to improve teaching evaluations, schools should be asking
themselves whether they should be asking instructors to make the course more or less demanding, interactive, or structured and organized” (p. 200).
April 24th, 2010 at 8:26 am
Individual evaluations are not often useful themselves, but if you look at trends in evaluations (as do some department chairs, evaluation groups), these usually tell a ‘story’…
April 24th, 2010 at 10:08 am
We have been looking very carefully at summary data of student evaluations since their validity has often been questioned by both faculty and administrators. One of the most prevalent beliefs has been that faculty who are generous graders get the best evaluations, that the number of A’s positively correlates to the teacher’s rating. We have correlated teachers’ ratings with grades distributed for two terms now in a very large College and find no relationship. In fact, the teachers who give all A’s (and yes we have a few) often get the lowest ratings and the most comments on what needs to be changed. I think that student evaluation data, if mined carefully, can provide enomously useful data for faculty and administrators in improving the quality of both the curriculum and instruction.
April 24th, 2010 at 10:57 am
I agree completely with Bluechip – he put his finger on the essential fact: the students should not be the customers we are required to satisfy. Rather we should be concerned with our responsibility to society to educate those students.
Although I think the written comments provide valuable feedback for the instructor, the numerical data usually does not. In many cases such data obtained from teacher evaluations can be truly detrimental to education. At our university many instructors are not on tenure tracks. Renewal of their contract is based on teacher evaluation (actually only one number for one question, namely what grade would you give your instructor?). Teachers who are “understanding” (i.e. easy graders) are especially well liked and obviously will receive a better grade. I was on a teaching awards committee and was shocked to see that we have instructors with very good teaching evaluations whose grade distributions are absurd. There were even some who, semester after semester, gave almost every student an A. This is higher education?
Many students, by the time they reach the upper division courses (taught by tenured faculty) find themselves unprepared and, what is worse, accustomed to to being able to pass courses without having to study outside of class.
April 24th, 2010 at 11:15 am
Student evaluations are indeed a mixed bag-most instructors get evaluations that range from “Why did you ever allow this person in a classroom?” to “Socrates had nothing on this person as a teacher.” One complicating factor with the change to online evaluations is that the respondants become self-selected from among the class. This definitely skews the results, typically in the direction of students who have complaints about the course. With paper evaluations, the students are (more-or-less) all there in class the day that evaluations are distributed. They may not give written comments, but usually they will fill in the numerical ratings, typically done by filling in circles in response to a Likert statement, and you have a good sample of the students. I noted that Northeastern had a 54% response rate to online evalautions – that is high compared to ours, which oftern runs from 20-40%. It is difficult to imagine that any truly useful data can be obtained from such a low response rate, especially if the group is self-selected in this way. This becomes critical if awards, tenure and promotion decisions are based on this skewed data. I have gotten some useful comments from the written sections but the numerical data, which I dutifully include as required in my annual report, I largely discount as not very meaningful (even though I get fairly high numbers).
April 24th, 2010 at 11:35 am
It is true that students are not in a position to judge the content of a course. Unfortunately, neither are we. There is an Irish proverb, “The young don’t know what it is like to be old, but the old have forgotten what it is like to be young”. Some of what we perceive as absolutely critical content is actually no longer relevant, some never was (usually our research). On the other hand, students, without the context of related information, cannot see the interrelationships and understand the relevance of specific content. It is only by drawing on both students’ judgement and professors’ judgement that we can come closest to ideal course content.
Historically, student evaluations were critical. In the original universities, interested students banded together and hired professors to teach them. That was before the days of University Administrations. One cannot say that students’ perceptions were not important then. Even in modern day, we need student feedback to assess our teaching so that we can constantly improve.
This is not to say that all student evaluations are prefect instruments. Many are deeply flawed. A poor evaluation instrument will not yield tremendous value. Even worse, if the University Administration is given a weapon against the faculty, it would be beyond foolish to expect them to use that weapon appropriately. (No matter how low one sets one’s expectations, the Administration will always surprise and astonish).
Ultimately, it is somewhat ironic that, when we were formally students (I hope that we all are still students), we all complained at some time or another about the imperfections of our evaluations, and how the grade received did not always coincide with out performance. Now that we are professors, we are still grousing about the fairness of “the test”. Surely we have common ground with our students.
April 24th, 2010 at 12:07 pm
i’ve found a strong correlation between student exam scores and ratings, with the higher grade students generally giving a more favorable evaluation. It’s not really useful but gives the students a (bogus) sense of “say” in their education. It mirrors the (bogus) results of Presidential elections, in that it’s all about the appearance of actually meaning something, but (as we’ve just witnessed with Obama – who i volunteered in his campaign and voted for) in reality, is a sham.
April 25th, 2010 at 8:27 am
I have mixed reviews about course evaluations. As others have said, those students who know they are not getting an “A” tend to automatically give poor feedback despite all other factors (i.e. the instructor was readily available at all times, offered help sessions throughout the semester, etc.) I do value feedback from my students, so tend to do a feedback form of my own throughout the semester (along the lines of Stephen Brookfield’s feedback) and this seems to tell me “how I am doing” as far as reaching all of my students and addressing various learning styles. I require my early childhood education students to be reflective of their own theoretical choices, think critically, and demonstrate proficiency (in the form of skill application-not multiple choice testing) in their content and these are things they really REALLY dislike. In undergraduate courses, my course evaluations tend to be more negative the more I ask them to really think critically and be reflective. Graduate students appreciate critical thinking and reflective practice much more and my evaluations tend to be extremely positive for the same types of lessons. I therefore agree with several others who have already posted that students are good at evaluating how well a professor “entertains” the students (engages them) but not good at all at evaluating content. One recent example involves a wonderful field-based project I set up with a local elementary school (something that is highly supported by research as “best practice”). My undergraduates had the opportunity to work with ESOL students in the field (actually learning through real-world activities with students). On course evaluations, at least 4 students said this was “useless” and they “did not learn a thing” and yet my department chair said (in my yearly evaluation) that he was “impressed” with my ability to create such valuable learning experiences. The students were unhappy that they had to drive off-site and that “their peers did not have to go” to this same field experience. This is just one of many examples of the lack of value of certain areas on student evaluations.
April 25th, 2010 at 1:02 pm
A point that needs to be made is that evaluations affect different levels of faculty in different ways. It is a brave adjunct who will hold students feet to the fire, knowing that “bad” evaluations might mean no classes the next semester.
April 26th, 2010 at 8:47 am
The usefulness depends on how well the evaluation is designed, and how many of the students completed the survey. A well designed eval will lead the students in a manner to get the desired feedback. Unfortunately most often the students who lost interest in the course do not complete the survey, so it’s hard to say that they are truly useful as an evaluation to determine tenure or teacher effectiveness. All the students who dropped the course should be allowed to fill out an “exit survey” as well to find out why they dropped.
Students also know that profs who are tenured can’t be fired so they often feel that the evaluation of those profs is pointless.
April 26th, 2010 at 10:59 am
What Gloria Donnelly says is comforting to hear, but the point is not whether or not giving high grades results in higher teacher evaluations. The problem, I think, is that many (most?) instructors feel that that is the case, act upon that premise, and the message they thus send to the students is that they can pass courses with very little effort on their part.
April 26th, 2010 at 1:18 pm
The fact that faculty may give higher grades and/or easier coursework or exams as a result of pressure from the need to have good evaluations may be a very real contributing factor to grade inflation, a topic discussed in this forum a while back. Administrators do tend to overrate the numbers on evaluations. Tenure and promotion (or for non-tenure instructors, rehiring) may be based on these numbers. Often, it is not until semesters or years later, even after graduation, do the students fully realize the value of a difficult course or the value of the demanding instructor to which they might have given low evaluation scores. By then it is too late for an instructor that has been let go due to low evaluation scores based on student perceptions.
April 26th, 2010 at 2:10 pm
In the 1980s and over much of the 1990s, I was a very demanding professor, and I tried to keep my grades at a B- to B average. My student evaluations were consistently low. Then I woke up to the fact that my standards were quite abnormally high, and that I was being subject to a great deal of pressure from students as a result. Accordingly I relaxed my demands, and my average grade is now A-. My student evaluations since this change of heart are among the highest in my department. Make of it what you will.
April 26th, 2010 at 5:17 pm
Guermantes;
If you consider an average of a B- to B average in the 1980′s and 1990′s to be demanding, and now average an A-, there are several reasonable possibilities:
a) You teach at a very select institution, say one where the incoming ACT score is typically 30+
b) You only teach graduate students
c) The grades where you teach are horribly inflated. I could be wrong, but grade scales that you describe are not usually seen outside of high school, and colleges of education.
April 26th, 2010 at 5:40 pm
Guermantes, it is obvious: your teaching skills have dramatically improved. Congratulations! I am sure you are well on your way to winning one of your university teaching awards!
April 27th, 2010 at 5:51 pm
In my experience, there are numerous problems with student evaluations. One that frequently is overlooked is the fact that you’re asking students to evaluate how well the prof taught that particular course, when those students rarely have any basis for comparison, since they usually haven’t taken the same course under any other professor.
The best use of these evaluations is in reading the verbatim remarks. Administrators generally take the “scores” and fail to look at the individual remarks. When you see remarks such as, “I hate the ties he wears” or “My friends took this course with another professor and didn’t have to write a paper,” you know that these students’ evaluations might be suboptimal input into assessing teaching ability.
April 28th, 2010 at 12:27 pm
At the end of the semester, students are asked to evaluate the effectiveness of the professor’s efforts. It seems to me that excellent teaching will affect a student for years to come. We are hoping to have a permanent, long term impact on our students.
In my experience, when students have had their most precious assumptions toppled, they are disgruntled and they cannot say enough negative about the course, and about the professor. However, these students have moved out of their comfort zone into a zone in which they will transform. I would find useful their responses to the same question five or ten years after they complete the course.
May 4th, 2010 at 2:12 pm
Having sat on an evaluation form committee for two years! (yes two years) it was clear that nobody was sure how to word them and if they were effective. If they are administered properly by somone other than the faculty member and given to the appropriate offices without faculty handling them, the results may be different. In my experience as a former chair instructors would give them out,collect them and then turn them in (sometimes days after they were submitted.) Some with many less filled out than members in the class. I personally found them helpful when followed over a year they would/could reflect similar issues. Issues I could address and correct if necessary.
May 10th, 2010 at 8:49 am
The general consensus among students is that the evaluations are a waste of time, and the ones that ask students to predict their grades usually meet the most resistance. While some students dig deeply and write solid and useful comments, most feel that it is their only opportunity to vent their frustration or any issues they had; while content is crucial, the way a teacher instructs and guides students is also very important, and I believe it is extremely unfair to ignore the fact that students clearly know what they think and what they experience, as well as what they have maintained through the experience. So many instructors insinuate that students don’t discuss content, but they are not asked to evaluate content on some forms, but rather the experience they had in a teacher’s class. For those forms that include a myriad of questions that specifically address content, style, expected grades, and so forth, there seems to be a more consistent level of feedback whereby teachers and the administration can better hone in on the best and worst elements and re-craft accordingly. By the same token, many instructors denounce important feedback, especially negative feedback, if they feel it came from a student who has not achieved high grades, it is often discounted.