Bad Lemon Logic (Updated)

Back in the 1960s, Hans Eysenck and Sybil Eysenck conducted an experiment that suggested that introverts (identified through a questionnaire) salivated more than extraverts when exposed to lemon juice, presumably because they have a higher baseline level of cortical arousal. The results were widely popularized; in an interview with Scientific American, Susan Cain said, “Introverts even salivate more than extroverts do if you place a drop of lemon juice on their tongues!” If you Google “lemon introvert salivation” you will see thousands of mentions of the study and minimal critical discussion.

Now, this study has problems; later studies called its findings into question. (What nerve! I think of Andrew Gelman’s “Enough with the Replication Police.”) I intend to dust off my own police uniform and look into all of this. For now, I will focus on the error that comes up again and again in interpretations of this test. People now claim that you can find out how introverted you are by conducting the lemon experiment. That is not only preposterous but illogical.

It is one thing to say that a study suggests that introverts tend to salivate more than extraverts in response to lemon juice. I question such a claim and the rigor of the study that led to it, but that’s what you’re supposed to do with such studies anyway. Now, to claim the reverse—that you can find out how much of an introvert you are by putting lemon on your tongue and measuring your saliva output—is to succumb to the famous fallacy of affirming the consequent.

Here’s why it’s wrong. Studies like the lemon juice experiment draw general conclusions from an array of individual results. Within the experiment, there may have been introverts who salivated less than extraverts. There may have been quite a few introverts and extraverts who salivated at similar levels. It might even be the case that if you divided salivation levels into two groups, a “low salivation group” and a “high salivation group,” you would find comparable numbers of introverts and extraverts in each. In no way does the test even suggest that if you salivate a lot, then you are an introvert.

Who is claiming such a thing, anyway? The BBC declares, “The amount of saliva you produce after putting a drop of lemon juice on your tongue might tell you something about your personality.” (Shame on them!) But that article has no listed author; it’s possible an intern wrote it. I give the BBC the benefit of the doubt it failed to cast on itself.

I see no excuse, though, for the famous TED-talking scholar Brian Little, who writes in Me, Myself, and Us: The Science of Personality and the Art of Well-Being (2014):

One of the more interesting ways of informally assessing extraversion at the biogenic level is to do the lemon-drop test. [Description of experiment omitted from present quote—DS.] For some people the swab will remain horizontal. For others it will dip on the lemon juice end. Can you guess which? For the extraverts, the swab stays relatively horizontal, but for introverts it dips. … I have done this exercise on myself a number of times, and each time my swab dips deeply. I am, at least by this measure, a biogenic introvert.

Someone of Little’s stature and renown should exercise more responsibility. He not only generalizes “introverts” and “extraverts” but suggests that you can conduct this experiment on yourself and find out who you are. The media (with exceptions) drools over this sort of thing; perhaps bad reasoning is a lemon, and perhaps the press as a whole has high cortical arousal.

Update (September 27, 2018): For a long time, I thought that the main published “lemon introvert test” study was Eysenck and Eysenck, “On the Unitary Nature of Extraversion,” Acta Psychologica 26 (1967), 383-390. I later (i.e., today) realized that their earlier study, also published in 1967, was the famous one: “Salivary Response to Lemon Juice as a Measure of Introversion,” Perceptual and Motor Skills 24 (1967), 1047-1053. (In “The Unitary Nature,” they refer to the earlier study as unpublished, but it was published not long afterward.)

In the famous study, they show a linear relationship between introversion and lemon-juice-induced increase in salivation. But when I look at the graph, I question the linearity; the middle region looks a bit messy even though it has already been smoothed out by averages. (They divided the subjects into five groups, according to their extroversion scores, and plotted the mean results for each of these groups–first for men and women separately, then for both together.) The study provides no raw data; I would like to see the scatterplot of all the results. I suspect some unevenness in the middle ranges. Here is the graph, from p. 1049 in Perceptual and Motor Skills 24 (1967).Pages from eysenck and eysenck 2

While this study replicated results found by D. W. J. Corcoran in 1964, it also raised some new questions and doubts. When the authors performed the same test using a commercial preparation of lemon juice (rather than fresh lemon juice), they found no significant correlation at all between increased salivation and introversion. (Corcoran had found no correlation when using citric acid instead of lemon juice). The authors were not sure of the reasons for this;

The related study (“On the Unitary Nature of Extraversion”) invites even more skepticism; there they analyze lemon juice test results in relation to individual items on an extraversion test. Overall, one can see a relationship between lemon test results and factor loadings for test items–but there are some anomalies and irregularities as well.

My argument remains the same: these studies say little, if anything, about the relationship between introversion and lemon-juice-induced salivation at an individual level. But in making this case, I regret focusing on an ancillary study rather than the one that gave rise to the lemon juice takeaway in the first place.

Also, Cain’s and Little’s summaries are not so far from those of Eysenck and Eysenck in “Salivary Response to Lemon Juice as a Measure of Introversion.” Cain and Little were not misrepresenting this research; they just appear to believe its conclusions much more readily than I do.

Leave a comment

1 Comment

  1. How to Educate Against Pseudoscience? — Joanne Jacobs

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: