What Does “Predict” Mean in Research?

In her TED talk, Amy Cuddy says,

Nalini Ambady, a researcher at Tufts University, shows that when people watch 30-second soundless clips of real physician-patient interactions, their judgments of the physician’s niceness predict whether or not that physician will be sued.

This is quoted all over the place, yet I have been unable to track down the study. Maybe it is unpublished or in press, or maybe it is published under a title that doesn’t mention physicians or videos.

In the meantime, I wonder whether Cuddy might have conflated two separate studies by Ambady: one of surgeons’ tone of voice (2002), and another of soundless clips of teachers (1993).

But my greater concern is with the word “predict.” As Cuddy puts it, the “judgments of a physicians’ niceness” actually predict whether or not that same physician will be sued in the future. To determine this, a researcher would have to follow the physicians over the long term and compare their subsequent lawsuit patterns (if any) to the initial ratings.

That would be terrifically difficult to accomplish. First, doctors with malpractice litigation histories are a small percentage of the whole, so the sample size would be tiny. Second, how long do you wait for a doctor to be sued? Two years? Five? Ten? Indefinitely?

Instead, I suspect the study followed a procedure similar to that of “Surgeons’ Tone of Voice.” (If I find out I am wrong, I will post a correction.) That is, the ratings of the videos were related to doctors’ existing lawsuit history. If there was any prediction, it was retrospective; as the authors state in the surgeon study, “Controlling for content, ratings of higher dominance and lower concern/anxiety in their voice tones significantly identified surgeons with previous claims compared with those who had no claims” (emphasis mine).

Why does this matter? There’s a big difference between predicting past and future events. People are easily dazzled by the idea that a thirty-second video clip (or sound clip, or whatever it may be) can predict a future lawsuit. The idea that it might predict a person’s existing history is perhaps interesting (if it holds up under scrutiny) but less dazzling.

When it comes to doctors, I imagine those with a lawsuit history might be a little grumpier than the others. I can also see how tone of voice could affect patients’ sense of trust and comfort. But I know of no study that demonstrates that ratings of soundless video clips of physicians predict whether they will one day be sued.

To avoid turning scientific research into a magic show, use the word “predict” carefully and precisely. Also, give a little more detail when referring to research, so that those interested can look up the study in question.

Update: I just learned that Ambady died in 2013. See the comments below. Thanks to Shravan Vasishth for the information.

Another update: Thanks to Martha Smith for explaining various  kinds of “prediction” in statistics. (See here and here.)

  • “To know that you can do better next time, unrecognizably better, and that there is no next time, and that it is a blessing there is not, there is a thought to be going on with.”

    —Samuel Beckett, Malone Dies

  • Always Different

  • ABOUT THE AUTHOR

     

    Diana Senechal is the author of Republic of Noise: The Loss of Solitude in Schools and Culture and the 2011 winner of the Hiett Prize in the Humanities, awarded by the Dallas Institute of Humanities and Culture. Her second book, Mind over Memes: Passive Listening, Toxic Talk, and Other Modern Language Follies, was published by Rowman & Littlefield in October 2018. In February 2022, Deep Vellum will publish her translation of Gyula Jenei's 2018 poetry collection Mindig Más.

    Since November 2017, she has been teaching English, American civilization, and British civilization at the Varga Katalin Gimnázium in Szolnok, Hungary. From 2011 to 2016, she helped shape and teach the philosophy program at Columbia Secondary School for Math, Science & Engineering in New York City. In 2014, she and her students founded the philosophy journal CONTRARIWISE, which now has international participation and readership. In 2020, at the Varga Katalin Gimnázium, she and her students released the first issue of the online literary journal Folyosó.

  • INTERVIEWS AND TALKS

    On April 26, 2016, Diana Senechal delivered her talk "Take Away the Takeaway (Including This One)" at TEDx Upper West Side.
     

    Here is a video from the Dallas Institute's 2015 Education Forum.  Also see the video "Hiett Prize Winners Discuss the Future of the Humanities." 

    On April 19–21, 2014, Diana Senechal took part in a discussion of solitude on BBC World Service's programme The Forum.  

    On February 22, 2013, Diana Senechal was interviewed by Leah Wescott, editor-in-chief of The Cronk of Higher Education. Here is the podcast.

  • ABOUT THIS BLOG

    All blog contents are copyright © Diana Senechal. Anything on this blog may be quoted with proper attribution. Comments are welcome.

    On this blog, Take Away the Takeaway, I discuss literature, music, education, and other things. Some of the pieces are satirical and assigned (for clarity) to the satire category.

    When I revise a piece substantially after posting it, I note this at the end. Minor corrections (e.g., of punctuation and spelling) may go unannounced.

    Speaking of imperfection, my other blog, Megfogalmazások, abounds with imperfect Hungarian.

  • Recent Posts

  • ARCHIVES

  • Categories