A Lesson from the Power Pose Debacle

Amy Cuddy’s TED talk on power posing has thirty-seven million views. Its main idea is simple: if you adopt an expansive, authoritative pose, your actual power will increase. For evidence, Cuddy refers to a study she conducted in 2010 with Dana Carney and Andy Yap. Holes and flaws in the study have since been revealed, but Cuddy continues to defend it. Doubt fuels scientific inquiry, but in an era of TED-style certainty and snappiness, it gets short shrift. It is time to tap the reserves.

Recently TED and Cuddy appended a note to the summary of the talk: “Some of the findings presented in this talk have been referenced in an ongoing debate among social scientists about robustness and reproducibility.” In other (and clearer) words: The power pose study has not held up under scrutiny. At least two replications failed; Andrew Gelman, Uri Simonsohn, and others have critiqued it robustly; and Carney, the lead researcher, detailed the study’s flaws—and disavowed all belief in the effect of power poses—in a statement posted on her website. Jesse Singal (New York Magazine) and Tom Bartlett (The Chronicle of Higher Education) have weighed in with analyses of the controversy.

Very well, one might shrug aloud, but what should we, irregular members of the regular public, do? Should we distrust every TED talk? Or should we wait until the experts weigh in? Neither approach is satisfactory. When faced with fantastic scientific claims, one can wield good skepticism and follow one’s doubts and questions.

Before learning of any of this uproar, I found Cuddy’s talk unstable. Instead of making a coherent argument, it bounces between informal observations, personal experiences, and scientific references. In addition, it seems to make an error early on. Two minutes into her talk, Cuddy states that “Nalini Ambady, a researcher at Tufts University, shows that when people watch 30-second soundless clips of real physician-patient interactions, their judgments of the physician’s niceness predict whether or not that physician will be sued.” Which study is this? I have perused the Ambady Lab website, conducted searches, and consulted bibliographies—and I see no sign that the study exists. (If I find that the study does exist, I will post a correction here. Ambady died in 2013, so I cannot ask her directly. I have written to the lab but do not know whether anyone is checking the email.)

In separate studies, Ambady studied surgeons’ tone of voice (by analyzing subjects’ ratings of sound clips where the actual words were muffled) and teachers’ body language (by analyzing subjects’ ratings of soundless video clips). As far as I know, she did not conduct a study with soundless videos of physician-patient interactions. Even her overview articles do not mention such research. Nor did her study of surgeons’ tone of voice make inferences about the likelihood of future lawsuits. It only related tone of voice to existing lawsuit histories.

Anyone can make a mistake. On the TED stage, delivering your talk from memory before an enormous audience, you have a million opportunities to be fallible. This is understandable and forgivable. It is possible that Cuddy conflated the study of physicians’ tone of voice with the study of teachers’ body language. Why make a fuss over this? Well, if a newspaper article were to make such an error, and were anyone to point it out, the editors would subsequently issue a correction. No correction appears on the TED website. Moreover, many people have quoted Cuddy’s own mention of that study without looking into it. It has been taken as fact.

Why did I sense that something was off? First, I doubted that subjects’ responses to a surgeon’s body language predicted whether the doctor would be sued in the future. A lawsuit takes money, time, and energy; I would not sue even the gruffest surgeon unless I had good reason. In other words, the doctor’s personality would only have a secondary or tertiary influence on my decision to sue. On the other hand, it is plausible that doctors with existing lawsuit histories might appear less personable than others—if only because it’s stressful to be sued. Insofar as existing lawsuit histories predict future lawsuits, there might be a weak relation between a physician’s body language and his or her likelihood of being sued in the future. I suspect, though, that the data would be noisy (in a soundless kind of way).

Second, I doubted that there was any study involving videos of physician-patient interactions. Logistical and legal difficulties would stand in the way. With sound recordings—especially where the words are muffled—you can preserve anonymity and privacy; with videos you cannot. As it turns out, I was flat-out wrong; video recording of the doctor’s office has become commonplace, not only for research but for doctors’ own self-assessment.

It matters whether or not this study exists—not only because it has been taken as fact, but because it influences public gullibility. If you believe that a doctor’s body language actually predicts future lawsuits, then you might also believe that power pose effects are real. You might believe that “the vast majority of teachers reports believing that the ideal student is an extrovert as opposed to an introvert” (Susan Cain) or that “the whole purpose of public education throughout the world is to produce university professors” (Ken Robinson). The whole point of a TED talk is to put forth a big idea–but an idea’s size has little to do with its quality.

What to do? Questioning Cuddy’s statement, and statements like it, takes no special expertise, only willingness to follow a doubt. If TED were to open itself to doubt, uncertainty, and error—posting corrections, acknowledging errors, and inviting discussion—it could become a genuine intellectual forum. To help bring this about, people must do more than assume a doubting stance. Poses are just poses. Insight requires motion—from questions to investigations to hypotheses to more questions.  This is what makes science interesting and strong.  Science, with all its branches and disciplines, offers not a two-minute “life hack,” but rather the hike of a lifetime, full of doubt and vigor.

Update: TED has changed the title of Cuddy’s talk from “Your Body Language Shapes Who You Are” to “Your Body Language May Shape How You Are.” In addition, the talk’s page on the TED website has a “Criticism & updates” section, last updated in August 2017. Both are steps in the right direction.

Note: I made some revisions to this piece long after posting it.

Research Has Shown … Just What, Exactly? (Reprise)

A few years ago, I wrote a piece with this title, minus the “(Reprise).” (And here’s a piece from 2011.)

It seems apt today (literally today) in light of Dana Carney’s statement, released late last night, that she no longer believes  “power pose” effects are real. She explains her reasons in detail. I learned about this from a comment on Andrew Gelman’s blog; an hour and a half later,  an article by Jesse Singal appeared in Science of Us (New York Magazine).

Dana Carney was one of three authors of the 2010 study, popularized in Amy Cuddy’s TED Talk, that supposedly found that “power poses” can change your hormones and life. (Andy Yap was the third.)

The “power pose” study has been under criticism for some time; a replication failed, and an analysis of both the study and the replication turned up still more problems.  (For history and insight, see Andrew Gelman and Kaiser Fung’s Slate article.) Of the three researchers involved, Carney is the first to acknowledge the problems with the original study and to state that the effects are not real.

Carney not only acknowledges her errors but explains them clearly. The statement is an act of courage and edification. This is how research should work; people should not hold fast to flawed methods and flimsy conclusions but should instead illuminate the flaws.

 

Update: Jesse Singal wrote an important follow-up article, “There’s an Interesting House-of-Cards Element to the Fall of Power Poses.” He discusses, among other things, the ripple effect (or house-of-cards effect) of flawed studies.

 

Science ≠ Community

communityI enjoy Andrew Gelman’s blog; it’s a great place if you have heard too many false and flashy “research has shown” and “science tells us” statements and want to know (a) where such research goes wrong and (b) why these errors don’t get attention. The blog attracts readers and commenters from many fields and perspectives; some of the comments could be pieces on their own.

Recently there was an enlightening post followed by lively discussion of Susan Fiske’s call for an end to the reign of “destructo-critics” and “self-employed data police” in social media—that is, those who employ “methodological terrorism” in criticizing others’ research. She doesn’t name names or give any concrete examples, so it isn’t clear who the “destructo-critics” are. She does suggest, though, that the legitimate channels for criticism are peer review or “curated” discussion. That is, she opposes not just nasty tweets, vicious personal attacks, and so forth, but (possibly) any unsanctioned critical commentary on research. In her conclusion, she says, “Ultimately, science is a community,  and we are all in it together.”

A community, eh? That rings a bell….

A commenter (“Plucky”) seized this sentence this and gave it a good shaking:

The key problem with Fiske is this sentence: “Ultimately, Science is a community, and we are all in it together.”

That is just flat-out wrong, and wrong in the ways that result in all that you have criticized. Science is not a community, it is a method.

That’s just the beginning; “Plucky” goes on to explain the dangers of the “community” metaphor. I recommend reading the whole comment. Here’s another choice quote:

My main criticism of this post is the stages of the metaphor—you’re nowhere near six feet of water in Evangeline. If Science devolves into merely a community, then it’s just another political interest group which will be treated as such.

I then remembered the many times I had heard the phrase “the consensus of the scientific community,” along with references to Thomas Kuhn, who supposedly coined it. Kuhn actually said, “What better criterion could there be … than the decision of the scientific group?” He explained what he meant by this, but I consider the statement problematic at best, even in context.

Kuhn aside, I use the word “community” sparingly and cautiously. Many entities that call themselves communities are not communities or should not be. As “Plucky” notes, “communities do not generally value the truth over their members’ well-being.” They exist to support their members.

In fact, someone who wishes to challenge a prevailing idea must often speak independently, without waiting for “community” approval. Dana Carney has just done this in relation to “power poses.” (This is big news, by the way.)

I do not disparage communities overall. Communities of various kinds have a place in the world, and I belong to a few. Still, even the best communities can ask themselves, “To what extent is this a community, and whom do we leave out?” and “What goals does this community not serve, and where does it even counter them?” The point is not to make the community all-encompassing but rather to recognize its limitations.

When it comes to science, I’ll take an open forum over a community any day.

 

Note: I added the paragraph about Carney after posting this piece.
Update: Writing for
Science of Us (New York Magazine), Jesse Singal reported this morning on Carney’s statement and explanation.