District Gives Students Teacher-Rating Gadgets

Benchmark, OH—In order to facilitate the accumulation of teacher data and to entice students into 21st-century technology use, Benchmark Unified School District has given out 35,000 teacher evaluation gadgets, equipped with skin conductance bracelets, to students in grades K-12. The leveled, adaptive software provides a user-friendly interface for real-time evaluation of teachers while the lesson is in progress.

“Nothing could be a greater boon for us right now,” said Superintendent Bret Elony, who recently signed a multi-million-dollar contract with Quicker Data, Inc., the creator of the software. “We need to know what kids think, but a great deal of the time, they’ve forgotten the lesson as soon as it’s over. So what could be better than to have a way for them to rate it on the spot?”

The software provides students with a series of animated “prompts” followed by multiple-choice (usually yes-or-no) answers. For instance, at the start of the lesson, a young character of adjustable race pops up and asks, “Is the aim on the board?” After the student responds with “yes” or “no,” a tiger or other animal (adapted to the student’s preferences) appears and asks: “Do you understand exactly what you are expected to learn today?” and then: “How close is this to what you want to be learning?”

The next questions pertain to the teacher’s appearance and organization: “Does the teacher have her papers in order? Does she have efficient routines for collection and distribution of student work?” The next questions have to do with the minilesson: “Are you happy with the length of the minilesson? Were you bored or confused at any point? Did the teacher provide you with enough information for your group work task?” (If there is no group work task, the student must select “n/a,” which will automatically trigger an administrator visit.

Over the course of the lesson, the skin conductance bracelet sends signals to the software, which translates them into “engagement” levels. “We didn’t want students to have to rate their own engagement while they were being engaged, or not,” said Elony. “That could get confusing.” If the overall engagement goes below a certain level, a red light goes on at the front of the room; if the engagement level is high, a green light goes on. An administrator passing by can easily spot these lights.

During the group work portion of the lesson, the animated characters (now butterflies and birds) ask questions such as, “Is the teacher moving around the room to help the various groups? Do you have the materials you need to complete your task? If you want to change your grouping, do you feel comfortable taking the initiative?

“I find this incredibly distracting,” complained Hecate Loomis, an eighth grader. “How am I supposed to get anything out of the lesson if I have to rate it every few seconds?” She was swiftly booed by a few others, who said they liked the software precisely for that reason.

“Class is a lot less boring now,” said her classmate Bob Tull. “Plus, for every three evaluations we complete, we get a free video game. I have better video games at home, but this is cool because I can play it during lunch.”

“And we’re generating data,” piped in Abby Lombardo. “We’ve been told that the more data we generate, the more we’ll be able to customize our own world.”

“That’s exactly the spirit,” affirmed a representative of Quicker Data, Inc. “While the main goal is to rate teachers swiftly, we’re all about customization. We’re helping to build a world where kids like everything they’re doing, all the time, where adults like their jobs, and where work gets done efficiently and pleasantly, with the help of pleasant cartoons.” Of course, he added, some teachers, students, and workers do not fit in with such a world; the software helps identify and remove them.

A few teachers have tried in vain to outwit the gadgets. One teacher rewired the lights at the front of the room so that the green light would always shine. A hidden camera filmed her in the act. Another teacher told her students to turn off the devices; she was fined (over ten thousand dollars, according to rumor) by both the school district and Quicker Data, Inc. Still others wrote letters of protest to the district; the superintendent’s reply reminded them that they needed to be open to change.

“This really is all about change, basically speaking” said Elony proudly. “We’re moving into a world where there isn’t time to think, where studies of the humanities and liberal arts are merely holding us back. We can’t keep holding onto the slow stuff. We need to get with it. Speaking of which, I know you have a lot of questions, but I’ve resolved ahead of time not to answer them. I have too much to do. Another contract is waiting to be signed.” He strode out of the room. Our questions (impostor ghosts) stayed behind; even after we left, they hung in waiting for a more inquisitive era.

The Low-Inference Room

seatbeltOne Tuesday morning in June, a cohort of one hundred novice principals was escorted into a classroom, ordered to fill out a checklist, and then herded into the Low-Inference Room, where they were assigned seats by number and told to fasten their seat belts. “These seats come with seat belts because systemic change is quite a ride,” said a booming voice through the loudspeakers. “You probably don’t want to change, and the teachers are even more resistant than you. So be prepared for some discomfort.”

Bewildered, the principals looked up, down, and around them. Their escort, a consultant named Gil Hines, walked slowly around the room, inspecting the seat belts.

“Which component in the Danielson rubric are we evaluating today?” the voice fairly screamed. “Number 24, provide the answer.”

“Number 24, that’s you, Nina Perotta,” said Hines. “Everyone check your seat numbers, because there won’t be any excuses next time.”

“Which component are we evaluating? It’s on the tip of my brain, it’s something about assessment…”

“Unprepared!” bellowed the voice. “Number 24 receives zero and a warning in the file. Number 96, which component are we evaluating today?”

“Today we are evaluating Component 3c, ‘Engaging Students in Learning’; specifically, the second item, ‘Grouping of students.’”

“Correct! Now, everyone say that together. “Component 3c, second item, ‘Grouping of Students.’”

“Component 3c, second item, ‘Grouping of Students.”

“Are we looking at anything else today? Number 57, are we looking at anything else?”

“Well, I always try to see what’s going on in a lesson when I….”

“Zero for number 57. This brings us to our lesson for today. Today we are going to learn low-inferencing. By the end of the lesson, all principals will be able to deliver a five-second low-inference evaluation of component 3c, item 2.” The objective suddenly appeared on a large screen. “Number 63, read the aim out loud for us.”

“All principals will be able to deliver a five-second low-inference evaluation of component 3c, item 2.”

“Correct. Number 15, read the aim aloud again.”

“All principals will be able to deliver a five-second low-inference evaluation of component 3c, item 2.”

One principal was sobbing. A few others tried to turn around but found they could not move. Two more straps had somehow emerged and fastened their shoulders to the chair.

The voice in the loudspeaker was now accompanied by a steady, quiet electronic drumbeat. “Now, I am going to deliver the lesson on low-inferencing,” the voice said. “Low-inferencing goes against everything you have been taught. You have been taught to interpret the world around you, to make judgments about what you see, to fill in the blanks in order to make sense of things. Low-inferencing is just the opposite. Low-inferencing is about reporting what you see, no more, no less, and refraining from any and all interpretation. Now, I am going to give you an example, and then I will give you an activity.”

A video of a classroom appeared on the screen. Students were working in groups; the teacher, who had been circulating from group to group, paused and said, “You have been grouped today according to your proficiency in graphing, but if any of you feel that you have been put in the wrong group, you are welcome to move to a different table right now.” The video stopped.

“Now, did anyone see the aim on the board?” the voice continued. “Number 88, did you see the aim?”

“It was to graph the sine function, I think,” said Number 88, the one who had been sobbing earlier.

“Good. Therefore, is the grouping appropriate to the aim of the lesson?”

“It appears so.”

“Not ‘appears.’ Yes or no.”

“Yes.”

“So I am going to rate the teacher as ‘effective’ in this category, at least. Now, what about students’ own control over their grouping? I am going to answer this for you, in the interest of time. For a teacher to be rated ‘highly effective’ in this category, the students would have to initiate changes of grouping. Here we see the teacher giving the students the opportunity to change groups, but this did not come from the students. Therefore, the teacher will be rated ‘effective’ but not ‘highly effective.’”

“But—“ Several principals began to speak up at once, and then shrieked in pain.

“This is for you to implement, not to question. Everyone must implement it in exactly the same way. Now, think back on the classroom you visited this morning. What did you see in terms of grouping and in terms of grouping only? Number 7, what did you see?”

“This doesn’t seem fair because—“ Suddenly Number 7 let out a long, agonized scream, through which the throbbing of his chair could be heard.

“Number 10, what did you see?”

“I saw no grouping at all. It was a whole-class lesson.”

“Wrong, but you get partial credit. A whole class is still a group. The question is, is this grouping appropriate to the students and to the instructional outcomes, and do the students have control over it? Number 34?”

“It’s impossible to tell, because…” Number 34 winced.

“Wrong. You may have seen a student looking out the window. This means the student was not engaged. This means the grouping was not appropriate for that student.”

“But that’s an interpretation!” Number 34 exclaimed. Her chair began to shake and buzz, and she didn’t speak again.

Engagement is engagement,” said the voice. “There’s no interpretation there. Either the students are engaged or they aren’t. If even one child is unengaged, we can low-inference that something is at least slightly wrong with the grouping. Now, using your clickers, give this teacher a rating. If you get it wrong, your chair will tell you, and you may try again. The results will show in a bar graph on the screen.”

At first the bar graph showed many “effective” ratings. Then, after many screams, cries, and groans, the “Ineffective” bar grew to 100 percent.

“I am very pleased with you,” said the voice, as a soothing melody began to play. “You have all achieved the instructional aim of the lesson. What’s more—pardon the joke—you were all engaged.”

The principals were then escorted into a lavishly decorated room with appetizers and champagne. “Change is painful at first,” the voice said over the loudspeaker, “but as you see, there are rewards. After this reception, you will all go together to a spa for the rest of the day; the buses are waiting outside.”

Later that morning, the teacher received notice that one hundred principals had rated her ineffective and that she no longer had a job.

The Biometric Bracelet and the End of Daydreaming

Children won’t be able to get away with daydreaming much longer. If their mind wanders “off task,” a sensor will catch them.

News broke recently that the Bill & Melinda Gates Foundation has awarded $1.4 million in grants to researchers who will experiment with “biometric” bracelets in middle schools around the country. (These bracelets send a small electrical current across the skin and then measure the electrical changes as the wearer responds to stimuli.)  The researchers intend to use them to measure student “engagement” and to determine which parts of a lesson (or reading or other activity) show higher engagement levels than others. Supposedly, through analyzing engagement levels in this manner, the researchers can deliver recommendations for raising engagement overall. The Reuters article explains:

Teachers could, for instance, use the bracelets to monitor student response to a video or a reading, then use that data to spark a lively discussion by zeroing in on the most engaging points, said Rosalind Picard, a computer scientist at MIT and a co-founder of Affectiva, which makes the sensors.

Such use of sensors in educational experiments is by no means new. Researchers at MIT, Arizona State University, and UMass Amherst have been developing “affect-aware tutors”—cartoon characters that respond to the students’ moods. Various sensors (including a mental state camera, posture analysis seat sensor, pressure mouse sensor, and skin conductance bracelet) detect the user’s state of mind; the cartoon character then responds. If a student shows frustration with a math problem, for instance, a cartoon might pop up with an expression of concern and say, “Gee, that was difficult. Would you like to try something easier?” (I discuss this in the eighth chapter of my book.)

Now, something is deeply wrong with all of this—in fact, there’s so much wrong with it that it’s difficult to get it all into a short space. But I’ll give it a try. Many more responses can be found on Diane Ravitch’s blog (for instance, here and here).

First of all, these bracelet sensors are invasive. Students (and people in general) have a right to their own thoughts and thought patterns. Yes, a teacher may demand attention in the classroom, but what goes on inside a student’s head remains his or her own business. Yes, sometimes doctors use sensors to test us, but they do this with our consent, for medical reasons. Privacy is a complex subject; what belongs to each of us alone, and what belongs to society? The answer cannot be determined through science; it is an ethical and philosophical matter. We must use our best judgment and conscience when drawing the line.

Second, engagement in itself is not necessarily a good in the classroom; higher levels do not necessarily mean more learning. Engagement comes in many forms and has complex rhythms. There is fleeting engagement—entertainment—that fades as soon as object moves away. There are behaviors that do not look like engagement but actually are (a student may look off to the side in order to think about something the teacher just said). A student working at home on a difficult problem will have ups and downs of engagement—puzzling over the problem, trying this approach, ending up in a rut, shaking the head, getting up and walking around, sitting down again and trying another approach, and finally figuring it out. All in all, engagement is secondary to what’s actually going on (which we must interpret with full mind).

Boosting engagement could even degrade instruction. Rosalind Picard (mentioned in the quote above) imagines teachers using the bracelets to determine the most “engaging” points of a reading. They can then zero in on these points in class discussion. Have the researchers spoken with teachers and professors of literature? Do they know how literature works? The most engaging points are not necessarily the most important ones. Sometimes subtle details prove essential to the story. Sometimes the ending confuses the reader at first and then suddenly makes sense. When selecting points of a story (or essay or other work) for discussion, one should think about the story itself, not the engagement levels. A “lively” discussion driven by “engagement data” could be supremely shallow.

Finally (for now), these efforts to neasure and boost engagment may rip up the last remnants of daydreaming. Some might say, “so what?” but there’s a lot at stake here. Many of us need to daydream in order to solve problems, try out possibilities, imagine scenarios, puzzle over words, or even just be by ourselves now and then. Much pedagogy discussion assumes that students should always be “on task,” that they should be hard at work toward a specified goal. When I was in school, this wasn’t so; for one thing, there weren’t so many tasks. You came into class to listen to the teacher and take part in discussion. Your mind could drift now and then. Sometimes the teacher would say, “What’s on your mind?” and you could say, “oh, nothing” or else divulge your thoughts.

One of my favorite daydreaming scenes occurs at the end of the first chapter of Charlotte’s Web by E. B. White:

The children ran out to the road and climbed into the bus. Fern took no notice of the others in the bus. She just sat and stared out of the window, thinking what a blissful world it was and how lucky she was to have entire charge of a pig. By the time the bus reached school, Fern had named her pet, selecting the most beautiful name she could think of.“Its name is Wilbur,” she whispered to herself.

She was still thinking about the pig when the teacher said, “Fern, what is the capital of Pennsylvania?”

“Wilbur,” replied Fern dreamily. The pupils giggled. Fern blushed.

The researchers and their funders may have forgotten the gentle wisdom of this story. We need to defend such wisdom against all things that push it away. Researcher or salesperson, if you come to my classroom with a biometric bracelet, I will invite you to read Charlotte’s Web with me. Or Seneca’s letter “On the Shortness of Life,” which is about “idle busyness”—that is, empty engagement. Or Tennyson’s “The Lotos-Eaters” (“Why should we only toil, the roof and crown of things?”). As we read, neither of us will wear a bracelet or make graphs of our engagement levels. That shrill, simplistic science will stay out of the room.