Teacher Ratings and Rubric Reverence

Some seven years ago, when I was taking education courses as a New York City Teaching Fellow, we had to hand in “double-entry journals”—that is, two-column pages with a quotation or situation on one side and our response on the right. On one occasion, I needed far more room for my response than for the quotations, so I adjusted the format: instead of using columns, I simply provided the quotations and my comments below each one.

The instructor chided me in front of the class. She said that this was a masters program and that I should learn to produce masters-level work. (She wasn’t aware that I already had a Ph.D. from Yale.) If the instructions specified a double-entry journal, well, then I was supposed to provide a double-entry journal. She had no quibbles with my commentary itself, which she found insightful. She just took issue with my flouting of the instructions. I have no grudges against the instructor, who meant well and knew her stuff. But it was an eye-opener.

Up to this point, I had not encountered such rigidity regarding instructions. In high school, college, and graduate school, we were expected to use certain formats for term papers, publishable work, and dissertations. But on everyday assignments, it was substance and clarity that mattered most. The teacher or professor even appreciated it when I departed from the usual format for a good reason. I did so judiciously and rarely.

The double-entry-journal incident was part of my induction into New York City public schools. There, the rubric (which usually emphasized appearance and format) ruled supreme; if you did everything just so, you could get a good score, while if you diverged from the instructions but had a compelling idea, you could be penalized. I saw rubrics applied to student work, teachers’ lessons, bulletin boards, classroom layout, group activities, and standardized tests. I will comment on the last of these—rubrics on standardized tests—and their bearing on the recent publication of New York City teachers’ value-added ratings (their rankings based on student test score growth).

A New York Daily News editorial asserts that teachers with consistently high value-added ratings are clearly doing something right. (This is the argument put forth by many value-added proponents.) But that’s not necessarily so; all we really know is that their students are making test score gains.

In New York State, on the written portion of the English Language Arts examinations, it matters little what the students actually say or how well they argue it. What matters is that they address the question in the prompt and follow the instructions to the letter. A student may make erroneous or illogical statements and still receive a high score; a student may make subtle observations and lose points for failing to do everything exactly as specified.

Here’s an essay prompt from the 2009 grade 8 ELA exam. (For an example at the high school level, see my blog “A Critical Look at the Critical Lens Essay.”)

Bill Watterson in “Drawing Calvin and Hobbes” and Roald Dahl in “Lucky Break” discuss their approaches to their work. Write an essay in which you describe the similarities and differences between the work habits of Watterson and Dahl. Explain how their work habits contribute to their success. Use details from both passages to support your answer.  In your essay, be sure to include

  • a description of the similarities between the work habits of Watterson and Dahl
  • a description of the differences between the work habits of Watterson and Dahl
  • an explanation of how their work habits contribute to their success
  • details from both passages to support your answer 

To get a good score, a student would only have to write one paragraph about similarities, one paragraph about differences, and one paragraph about how their work habits led to their success. By contrast, a student who began by considering definitions of “success” (as G.  K. Chesterton does) would not fare so well, even though that might be the more thoughtful essay. Likewise, a student who questioned the direct link between work habits and success (as Mark Twain does) would be at a disadvantage. Students are better off if they write a predictable essay, even a bland one, that meets the criteria. Their teachers are better off, too; every point counts when it comes to value-added scores. 

I have scored ELA exams. Human judgment has little place in those scoring rooms. To maintain consistency, everyone is supposed to follow the rubric, and, if there’s any doubt, the state’s own interpretation of the rubric. It comes down, in the end, to following instructions rather than judgment. On the one hand, this is fair and justified. If teachers were to use their own judgment when scoring, two essays of similar quality could receive wildly different scores. On the other, it means that there’s no way to acknowledge the student who struggles with the question becausethe question is tricky or problematic—that is, the student who pushes beyond the obvious response. 

Now let’s consider the consequences in the classroom. Teachers A and B teach at a relatively high-performing school. Teacher A tells students that to write well, you should have something to say and should take care with words. Her students read G. K. Chesterton, Ralph Waldo Emerson, Mark Twain, Jonathan Swift, and others. They discuss these essays, look at their structures, respond to favorite passages in them, and write essays inspired by them. Teacher B, within the same school, has a different approach. She brings in reading passages like those on the tests. She teaches students how to read essay prompts and produce the expected responses. She has them do this every day. Now, arguably, one can teach students to write thoughtfully and follow directions precisely. But the latter has the greater test score payoff.

So, teacher B’s students make more test score gains than Teacher A’s students. Teacher B gets rated “high”; teacher A, “below average.” (This is a plausible scenario in an unusually high- or low-performing school, where a slight difference in points can account for a large difference in ratings.) Then the ratings appear in the New York Times and elsewhere. Many readers will assume, even with caveats galore, that teacher B does better work than teacher A. Teacher A then finds herself under pressure to do what teacher B is doing. That means ensuring that her students follow directions.

How do you get teachers to teach in this manner? Train them in education school. Impress upon them the sacrosanctity of instructions. Teach them that if the assignment is a double-entry journal, then that is what they must produce, period.

What David Brooks Doesn’t Get

In his New York Times op-ed “Testing the Teachers” (April 19), David Brooks warns that “an atmosphere of grand fragility” hangs over America’s colleges. The grandeur, he says, comes from the colleges’ increased application rates, new facilities, and international reputation; the fragility, from increased tuition combined with uncertain results. What must we do? Hold colleges accountable for results—through value-added testing. That’ll show who’s teaching and who isn’t!

Brooks is wrong. Accountability systems would drag down our colleges. The best would be made mediocre, and the worst would rise to mediocrity at most.

Having put forth the idea, Brooks waxes dreamy about it. “There has to be some way to reward schools that actually do provide learning and punish schools that don’t,” he muses. “There has to be a better way to get data so schools themselves can figure out how they’re doing in comparison with their peers.”

What Brooks doesn’t understand is the difference between accountability and responsibility. It is the latter, not the former, that will help and sustain colleges.

Responsibility is an internal sense of duty; accountability, an external show. The professor who who puts full thought into lesson preparation, corrects student work, holds office hours, challenges students in class, and takes them, day by day, into the subject—this professor has a deep sense of responsibility but may or may not “produce” test score gains. A professor who focuses on showing results to outsiders (an accountable professor) may be less immersed in the subject, less concerned about navigating tricky points—but may raise test scores. If schools must foster the latter sort of teaching, they will glide into a monotone.

But why should accountability and responsibility be at odds? They are not always opposed, but there’s ongoing friction between them. To honor one’s best thinking and conscience is not the same as to do what others want and recognize. The best instruction does not absolutely and consistently produce test score results.

For one thing, course content may not match the content of standardized tests (and it would be dreary if it did). Second, if students take especially difficult courses, they may go an entire semester without showing visible progress. A grade of “C” may be honorable in such cases. Third, each subject has its language, structure, and logic; these are not always easy to convey to those outside the field. In their presentation “Assessment on Our Own Terms,” delivered at the 2007 Annual Meeting of the National Association of Schools of Music, Mark Wait and Samuel Hope draw attention to the difficulty of translating “musical logic” into “speech logic.”  Fourth, the higher the level of study, the more complex the assessment becomes. (That’s not to say that assessing kindergarteners is a straightforward matter.)

This leads to another flaw in Brooks’s suggestion. He assumes that it is the colleges’ duty to “produce” visible signs of learning. But even today, with the tuition hikes, many students go to college to be challenged, to explore many subjects, to dedicate themselves to a major, and to work on something of beauty. Getting top grades isn’t necessarily their first priority. Some would rather take more courses, or more difficult courses, at the risk of lower grades than take easy courses and get all A’s. Some find themselves immersed in a particular course or subject and let the other ones slide a bit. Some follow an idea or a project only to discover that they are on the wrong track. This is their prerogative, and they must take the consequences.

True, not all students are so serious–many  skip class repeatedly, go to party after party, and fret over relationships. If they slip too far, a good hard “F” can shake them up. Deans and advisors should watch for students in danger of failing, but students must learn to make choices and take responsibility for them. It does not help students—especially college and graduate students—to make someone else responsible for their performance.

Now, of course I am assuming a liberal arts college or school of art (or music or drama), and a high-level one at that. I am not referring here to colleges where most of the students need remedial courses. Nor am I talking about vocational and technical schools, whose mission is to prepare students for a concrete profession or trade. These are colleges with specific, standardized goals—and they should make good on their promises, provided the students do their part.

But it is not nostalgic, romantic, or naive to insist that college also be about something else: about pursuing interests, enjoying a life of the mind, making and learning from mistakes, being around intensely knowledgeable and interesting people, studying a subject at a high level, and yes, allowing for imbalances between receiving and giving. Education is a gift in a troubling sense, a sense that recalls Robert Frost’s lines about a star, “It asks a little of us here. / It asks of us a certain height.” This is no trivial demand. Students, receiving a fine education, do not immediately show the height required. Sometimes this takes years, even decades. Sometimes we think back on something learned long ago and see how it honed our thinking and our lives. That’s a result worth defending to the end. We must not treat such learning as a lie.

The Solitude of Good Collaboration

Not long ago, I attended a meeting where a teacher presented her own definition and explanation of “analysis.” She suggested that other teachers do the same: think about analysis on their own, put their best definitions and explanations together, and then bring these ideas to the next meeting. As I listened, I understood what she was after. She realized that the discussion would be more productive if the teachers first thought alone about the matter. In other words, she saw that collaboration requires an element of solitude—an idea that seems obvious but is often forgotten.

We hear, over and over, about the need for cooperative learning and collaborative planning. In a recent article in the Atlantic, Jeffrey Mirel and Simona Goldin put forth the familiar argument that teachers want to collaborate, bless their souls, but end up spending most of their non-instructional time alone, in their isolated rooms. It is time, they say, to create more opportunities and resources for collaboration. But why do Mirel and Goldin pit solitary work against collaboration? Take away the former, or reduce the time for it, and the latter will lose meaning. Meetings will gravitate toward the lowest common denominator—that which everyone can readily understand and accept.

In many schools, teachers are required to spend time in teams every day, but there is no protection of solitary time. Most of the day is taken up with instruction, meetings, and various other tasks and duties. Even when alone in the room, the teacher is usually gathering materials and correcting student work. One of the most important parts of teaching—mulling over the subject itself—gets pushed out to the edges of the day.

Yet is this very mulling, this solitary relationship with subject matter, that preserves the integrity of teaching. When we bring our own work and thought to the group, the group does not hold us back; it does not reduce what we have to say, since we have already worked it out in our minds. “Conversation will not corrupt us,” wrote Ralph Waldo Emerson, “if we come to the assembly in our own garb and speech, and with the energy of health to select what is ours and reject what is not.” To have “our own garb and speech,” we must know what it is; to have the “energy of health” for selection, we must be able to restore and strengthen ourselves alone.

When teachers have insufficient time for thinking alone, they are more susceptible to group errors and group jargon. Suppose teachers were trying to come up with a definition of “analysis.” If they did this as a group, without thinking alone first, they would end up with a collection of scattered thoughts, which they would then try to cobble together. They might arrive at something like, “Analysis is a higher-order critical investigation in which a thesis is substantiated with evidence and clear connections are made between the evidence and the thesis.” That is difficult to detangle, as many group statements are. But if they took the question into their minds, played with it, figured something out, and then brought their thoughts to the table, they could arrive at a good working definition. (Note: dictionaries offer multiple definitions of “analysis”—so even after looking it up, one must think it over.)

One teacher might say, for instance, that analysis is the act of breaking something into its elements. Another might say that it is the act of inferring generalities from specific details. Still another might say that it is both: that it involves relating details to the whole and vice versa. Still another might define it as the examination of a phenomenon’s structure. As they considered the ideas that had been presented, they might see truth in all of them. Analysis, they might conclude, is the systematic explication of a relationship—for instance, between a part and a whole of a literary text or between a historical event and its possible causes. Having arrived at a plausible general definition, the teachers might supplement it with specific definitions to suit the situation at hand. This is not likely to happen without solitary thought.

By bringing solitary thought into their collaboration, teachers not only enhance their own work but set an example for students. Students, too, will learn more from each other if they know how to think and work alone. Let us suppose that, in a music class, students are considering how the sonata form plays out in the first movement of Beethoven’s Sonata No. 8 (“Pathétique”). To do this well, they are best off listening to it alone, without distractions, several times. The form (introduction, exposition, development, recapitulation, coda) is not difficult to discern. When alone, they will hear its particularities: the key changes, the textures, the transitions from one theme into another, and the subtle, less tangible changes of color and mood. Then, in class, they can point out what they found; one student may have noticed something that others did not. The teacher will be able to alert them to still more subtleties and patterns, which they will be able to appreciate. It is not only time with the material that they need; they need private, nonsocial time with it, time without peers nearby to condition what they think and say.

We must halt the collaboration screech-wagon and pursue greater thougthfulness instead. The visible signs of collaboration are not the only ones; taken too far, they impede good work. There is something vast in a bit of quiet: a chance to absorb, practice, and tinker. At its best, it takes us past our narrower selves, allowing us to see our mistakes and misconceptions. There is no need to shove it aside, no need to disparage the thing that allows us to bring something to others.

Twitter and Loss of Solitude

Last March, during a book talk,  Jonathan Franzen committed the scandalous act of criticizing Twitter. An audience member took issue not with his points, but with his failure to admit to his own privilege. Franzen, she argued, doesn’t have to  worry about promoting himself. His publicist probably dreams about him every night. Many lesser-known writers have to go on Twitter and Facebook if they want to reach readers.

But do we? Isn’t there a way to reach people without reducing yourself? If you genuinely enjoy Twitter—and many do—then it can have benefits. It can serve as a good source of information, if nothing else. But if you aren’t drawn to it, why force yourself to use it? One of the most grating aspects of Twitter (and other social media) is the subjection of everything to a popularity vote and public display.

Publicity is not evil; writers and others need to reach an audience. Still, there are many ways of doing this, each with its benefits and costs. One must choose carefully, resisting pressure to join the crowd.

Promoting your work through Twitter is no mere thumb movement. It isn’t enough, from what I have seen, to toss out a tweet now and then. No, you have to build a following (which you can check moment to moment). This requires time and strategic activity. What’s more, it requires that you look somewhat friendly and accessible. You tweet about how great it was to meet so-and-so for lunch. You tweet that you’d love to come to so-and-so’s reading but—alas—are about to board the plane to LA, where you will be giving a reading of your own. Too bad! Another time!

Now, some claim that this sort of online socializing actually preserves privacy. Susan Cain suggests that it appeals to introverts because it relieves them of the pressure to socialize in person. According to Cain, it is more comfortable for them to tweet and blog than to speak in public or introduce themselves at parties. Clearly there’s some truth to that. It’s probably less draining in some ways to send a hundred tweets than to meet ten people in a day. Introvert or extrovert, a person gets tired.

Let’s set aside the question of introversion for the moment and consider solitude instead. (Introvert-extrovert distinctions are a bit messy, in my view.) If you value solitude—that is, time apart in the mind, even time alone with a friend—do you really want to muddy it up with tweets of “great to see you” and “say hi to Nancy”? Are your conversations really mass entertainment pieces? Some will argue that such communications aren’t special or intimate–so nothing is lost in making them public. But I consider even acquaintanceships important enough (and, in a sense, private enough) to keep to myself.

In my book, Republic of Noise, I define solitude as the apartness we have at all times, which we may honor and shape or not. There is solitude in friendship, because friendship requires a certain aloneness of spirit, a willingness to take the other person on his or her own terms. Each friendship has its special language, history, and rituals, which are understood by the two friends alone. Conversations between friends do not have to become public property.  Something’s corrupted when they do.

I like to separate public from private. When in public–for instance, when giving a speech or teaching at school—my words are for all, and my focus is mainly on ideas, not on personal relationships. When in private, alone or with others, a mixture of ideas, rumblings, and affections (or, in some cases, antipathies) comes into play. Although it is impossible to separate the public and the private completely, I find meaning and respite in such division.

Twitter and other social media erode the distinction between public and private. They create a zone that is neither one nor the other. Of course, this erosion is not new, nor is social media the sole cause. Hannah Arendt considered it a feature of modernity;  she gives a fascinating analysis of the problem in The Human Condition. There is something perturbing about the zone that is neither this nor that—the extension of our selves into arenas that do not care for us.

Now, one can use Twitter in a purely formal manner: sending out links and announcements with no personal content. But unless you have a large following, this will likely have little effect; moreover, you still have to deal with tweets from others. I’d rather stay off the whole thing.

My upstart abstinence may cost me a host of readers. So be it; I’d rather have a thousand readers and independence of mind, than a hundred thousand and twice as many tweet-intrusions. I do not have to broadcast what is private or mundane (or even what is not). Some say social media is the wave of the future, but that does not obligate anyone to ride it. An age contains far more than its trends; a life, far more than avatars and “likes”; a book, far more than its surrounding chatter.

Note: I made a few edits to this piece after its initial posting.

Follow

Get every new post delivered to your Inbox.

Join 151 other followers