I learn a heck of a lot from Andrew Gelman’s blog–not only his own posts, but the many interesting and substantial comments. It’s one of my favorite places on the internet right now (granted, I have low tolerance for “surfing” and tend to focus on a few sites). That said, I find myself questioning some of his arguments and views, particularly about measurement in education. Now, I am not about to say “learning can’t be measured” or “tests are unfair” or anything like that. My points are a bit different.
In an article for Chance, vol, 25 (2012), Gelman and Eric Loken observe that, as statisticians, they give out advice that they themselves do not apply to their classrooms; this contradiction, in their view, has ethical consequences:
Medicine is important, but so is education. To the extent that we believe the general advice we give to researchers, the unsystematic nature of our educational efforts indicates a serious ethical lapse on our part, and we can hardly claim ignorance as a defense. Conversely, if we don’t really believe all that stuff about sampling, experimentation, and measurement—it’s just wisdom we offer to others—then we’re nothing but cheeseburger-snarfing diet gurus who are unethical in charging for advice we wouldn’t ourselves follow.
They acknowledge the messiness and complexity of education but maintain, all the same, that they could improve their practice by measuring student learning more systematically and adjusting their instruction accordingly. “Even if variation is high enough and sample sizes low enough that not much could be concluded,” they write, “we suspect that the very acts of measurement, sampling, and experimentation would ultimately be a time-efficient way of improving our classes.”
I agree with the spirit of their argument; yes, it makes sense to practice what you proclaim, especially when this can improve your teaching. Of course assessment and instruction should inform and strengthen each other. Still, any measurement must come with adequate doubt and qualification. I think they would agree with this; I don’t know, though, whether we would raise the same doubts. I see reason to consider the following (at the college level, which differs substantially from K-12):
While still moving toward independence, students are more in charge of their own learning than before. Ideally they should start figuring out the material for themselves. What is the class for, then? To introduce topics, organize the subject matter, illuminate certain points, and work through problems … but perhaps not to “produce” learning gains, at least not primarily. On the other hand, the course should have adequate challenge for those at the top and support for those at the bottom (within reason). Introductory courses may include additional supports.
Also, a student might deliberately choose a course that’s too difficult at the outset (but still feasible). Some people thrive on difficulty and are willing to let their grade drop a little for the sake of it. The learning gains may not show right away, but this does not mean that the teacher should necessarily adjust instruction. If the student puts in the necessary work and thought, he or she will show improvement in good time. Students should not be discouraged from the kind of challenge that temporarily slows their external progress.
In addition, there are inevitable mismatches, at the college level, between instruction and assessment. (This may be especially true of the humanities.) If you are teaching a literature, history, or philosophy class, your students will probably write essays for a grade, but your teaching will address only certain components of the writing. Students have to learn the rest through practice. Thus you will grade things that you haven’t explicitly taught. (Your course may not deal explicitly with grammar, but if a paper is full of ungrammatical and incoherent sentences, you still can’t give it an A.) This may seem unfair–but over time, through extensive practice and reading, students will come to write strong essays.
Since September 2015 I have been taking classes part-time, as a non-matriculated student, at the H. L. Miller Cantorial School at JTS. In my first class, I was far below the levels of my classmates. That was what I wanted. I studied on the train, in my spare moments, and at night. (I was teaching as well.) I flubbed the final presentation, relatively speaking, not because I was underprepared, but because I prepared in the wrong way. I ended up with a B+ in the course. The next semester, my Hebrew had risen to a new level; the course (on the Psalms) enthralled me, and I did well. This year, I have been holding my own in the course I longed to take all along: a year-long course in advanced cantillation. If the professors had worried too closely about my learning gains, I wouldn’t have learned as much.
On the other hand, in the best classes I have taken over the years, the professors did great things for my learning. I wouldn’t have learned nearly as much, or gained the same insights, without the courses. The paradox is this: to help me understand, the professors also let me not understand. To help me progress, they sometimes took me to the steepest steps–and then pointed out all the interesting engravings in them. It wasn’t just fascination that took me from step to step–I had to work hard–but they trusted that I could do it and left it largely in my hands.
Granted, not all students are alike, nor are all courses. In an introductory course, students may be testing out the field. If they are completely lost, or if the course takes extraordinary effort and time, they may conclude that it’s not for them. A professor may need to respond diligently to their needs. There are many ways of looking at a course; one should work to become alert to its different angles.
In short, college should be where students learn how to teach themselves and how to gain insights from a professor. While helping students learn, one can also hope, over time, to simulate Virgil’s last words to Dante in Purgatorio, “I crown and miter you over yourself” (or to accompany them to the point where, like Alice, they find a crown atop their heads.)
Image: Sir John Tenniel, illustration for the eighth chapter of Lewis Carroll’s Through the Looking Glass (1865).
Note: I revised the fourth paragraph for clarity and made a minor edit to the last sentence.