Facebook and Mortality: Final Post

Throughout the Facebook/mortality study published in PNAS (“Online Social Integration is Associated with Reduced Mortality Risk“), the researchers walk a fine line: they present the results as “observational” but also represent them in dramatic graphs and read unwarranted meaning into them. In other words, the patterns look much stronger in this paper than they probably are.

One of my chief complaints was that the researchers lumped all deaths together. In the last part of the study, they separate them out, so I’ll finish by looking at that. This is the “nitty-gritty” part of the study; the question is, does the “grit” actually get in the way?

They focus on death causes that are predicted relatively strongly by levels of social support: cancer, cardiovascular disease, drug overdose, and suicide. They write: “We present cause-specific estimates in order, from least expected to be predicted by social support to most expected.”

They find that “the number of online friendships is not significantly related to decreased mortality due to cancer but is for cardiovascular disease (91%; 95% CI: 87–96%) and even more so for drug overdose (78%; 95% CI: 70–87%) and suicide (73%; 95% CI: 66–80%). Moreover, when we separately analyze initiated and accepted friendships, the results suggest that accepted friendships are driving the overall relationship, as we previously showed in Fig. 1.” Here are the graphs:

f3-medium

I see no reason to believe that “accepted friendships are driving the overall relationship.” Rather, the three friend-related activities (friend count, friendships initiated, and friendships accepted) are clearly interrelated. The difference in the relative mortality risk is not as great as the graph makes it seem; moreover, for drug overdose and suicide, there are all sorts of confounding factors that could affect the figures (including situations where their online access was restricted).

What about the two figures below? The most important point here is that the researchers distinguish, first between statuses posted and photos received, and then among photos/messages sent, photos/messages received, and photo tags received. The authors interpret the results:

Fig. 3C shows that sent text-based communications are generally unrelated to mortality risk for these causes, but received communications tend to predict higher risk of mortality due to cancer (108%; 95% CI: 104–112%) and lower risk due to drug overdose (88%; 95% CI: 80–96%) and suicide (82%; 95% CI: 74–90%). Once again, this association suggests that social media is being used by cancer victims to broadcast updates, which elicit received messages, and the contrast between cancer (a positive relationship) and other causes (a negative relationship) may help to explain the nonlinear relationship observed with all-cause mortality in Fig. 2. Meanwhile, received photo tags, our strongest indicator of real-world social activity, are strongly inversely associated with all types of mortality except those due to cancer, and the inverse relationship is strongest with drug overdose (70%; 95% CI: 64–77%) and suicide (69%; 95% CI: 63–76%).

I realize that the researchers controlled for age; even so, I imagine photo tags are more common among younger users (where the mortality risk is lower) than among older users (who may consider the practice tacky, or who may worry about privacy). The researchers state that “received photo tags, our strongest indicator of real-world social activity, are strongly inversely associated with all types of mortality except those due to cancer, and the inverse relationship is strongest with drug overdose.” But this is just one of many possible interpretations; moreover, it’s possible that we are looking at noise.

First, I question the assertion that received photo tags are “strongly inversely associated” with deaths due to cardiovascular disease; the association looks quite small in fact. As for suicide and drug overdose, once again, I suspect the presence of confounding factors; in addition, I wonder about the sample size and the distribution over age groups.

I wonder, also, whether received photo tags really indicate “real-world social activity” and whether there isn’t a severe mismatch between tagging and suicide demographics. Suicide rates are higher for older age groups (highest for 85 or older, and next-highest for 45-64)—and tagging (I suspect) much more common for younger age groups; so, even with controls for age, there could easily be some false correlations here. (Also, a lot of tagging is automated, and many people take time to remove their name from photos. The researchers didn’t consider deletions at all.)

Enough! I have had more than my fill of this study. Thanks to Shravan Vasishth for the link to two papers he co-wrote with Bruno Nicenboim on statistical methods for linguistic research. They explain statistical issues clearly and sequentially, starting with hypothetical data and building up to analyses. Some of the errors they bring up seem especially pertinent here. For instance, on p. 29 of the first paper, they note that “multiple measures which are highly correlated … are routinely analyzed as if they were separate sources of information (von der Malsburg & Angele, 2015).”

A statistician would have been able to take one quick look at this study and see its flaws. I suspected some serious problems but needed time to see what they were. This leads to the ethical question: is one obligated to read a study from start to finish before critiquing it? No, as long as you are forthright about what you have and haven’t read, and as long as you focus on essentials, not trivial matters. Just as a poet or literary scholar can easily spot a bad poem (of a certain kind), someone with statistical knowledge and insight can tell immediately whether a study is flawed in particular ways. A promising study can take longer to assess.

On the other hand, it’s important to recognize what the researchers are trying to do. If their point is not to offer answers but rather to explore patterns, then one can read the study with appropriate caution and accept its limitations. Here it’s a mixture; the authors acknowledge the study’s “observational” and tentative nature but at the same time claim strong findings and back them up with questionable interpretations. It is up to the reader, then, to cast the study in appropriate doubt. I hope I have accomplished this here.

(For the four previous posts on this study, see here, here, here, and here. I made a few minor edits and two additions to this piece after posting it.)

Facebook and Mortality: A Look at the First Figure

I was going to make this the fourth post on the Facebook/mortality study (“Online Social Integration is Associated with Reduced Mortality Risk“) but am instead making it the third. (See Part One, Part Two, and Interlude.) After this, I will look at the relation (according to the study) between Facebook activities and specific causes of death.

I question the implications of the patterns in the two graphs of Figure 1. The first graph illustrates the association between Facebook friend requests and relative mortality risk; the other, between Facebook friend acceptances and relative mortality risk.

f1-medium

There’s a slight similarity between the two graphs. They both dip downward, hover a little in the middle, dip down some more, and then dip a little upward again. This is no coincidence; the authors acknowledge that they estimated the two friendship categories separately “due to high collinearity between them.”

The first graph (of friend requests) seems to show no significant relation between the activity and mortality risk; the other (of friend acceptances), a possible relation. But Facebook friend acceptances are fewer than requests, almost by definition. (Not all request get accepted, and one can only accept a request that has been made.) Couldn’t this magnify their effects? In any case, I see a lot in common between the two figures; the main difference is in the range of y-values.

But what is going on here, anyway? Both figures show relative mortality risk in relation to the reference category (the mean for each activity, I presume). The second figure has the wider range (between 1.383 and 0.852 times the reference category); to determine whether the differences have meaning, we would need to see the original numbers. In terms of visual effect, the range looks dramatic; in terms of actual numbers, it may be tiny.

I was hoping to get to the bottom of the Cox proportional hazards model, from which these figures are derived; but even the appendix doesn’t reveal the initial numbers or the specific application of the model. The numbers in the two graphs are “adjusted for age, gender, device use, and length of time on Facebook.” Just how does this work? I don’t know, but the adjustments seem somewhat arbitrary. An adjustment for education level or income might be more illuminating. (Update: Actually, they did control for “type of device used,” which was supposed to correlate with socioeconomic status. They also tried controlling for “highest education levels listed by friends on Facebook” but found that this did not substantially alter the results.)

There’s no adjustment, apparently, for length of time between activity and death. In the appendix, the authors state: “To be clear: we are testing associations between 1) social media usage over a six month period and 2) mortality over a subsequent 24 month period, with a 6 month gap between these two measurement periods.” So there’s no distinction between people who were already ill, people who became ill after that period of Facebook usage, and people who died suddenly, with no intervening illness.

Given the “high collinearity” between the two activities and their relation to mortality risk, I also wonder why there isn’t an adjustment for the presence of the collinear activity. It doesn’t seem quite right to separate the two activities when they are so closely related. I suspect that the apparent difference between them is deceptive.

Why does this matter? I take this time with the study because I see in it a combination of good intentions and profound nonsense. Facebook activity isn’t necessarily meaningful or positive; it has a good deal of banality and meanness. Is this the right place to look for insight on the relation between social activity and mortality? Is it the right way? I doubt it. But even if I put aside my initial doubts, I find problems in the logic of the study.

Recently some interesting articles about friendship have appeared; while imperfect, they seem more promising to me than studies of Facebook activity. I will write one more post about the Facebook/mortality study and then proceed to friendship.


Note: I made minor revisions to the fourth paragraph after posting this piece.

Facebook and Mortality: Interlude

On Sunday or Monday I will write Part Three, which discusses the last part of the Facebook study (“Online Social Integration is Associated with Reduced Mortality Risk“). Having examined the mortality rate of Facebook users (discussed in Part One) and the relative mortality risk of specific Facebook activities (discussed in Part Two), the authors now relate specific Facebook activities to specific causes of death.

I think there will be a Part Four as well; I want to look more closely at the Cox Proportional Hazard Model and a few other details.

For now, I will bring up a problem that has stayed on my mind. Does it make sense at all to compare the dead and the living in this case? We aren’t looking at a disease or harmful exposure. We’re looking at activities of dead Facebook users and living Facebook users. We don’t know how healthy the living Facebook users are; we don’t know whether any of them are at substantial risk for any of the deaths under consideration. It seems highly unlikely to me that Facebook activity (or lack thereof) contributed significantly to any of these deaths or non-deaths. A possible exception is suicide; some sort of contact with the outside world could keep a person from self-destruction, though it could also push someone over the edge. (Not all Facebook interaction is kind.)

Death is polymorphous; it seems to have the same end but has many causes and circumstances. In other mortality studies I’ve seen, there was a similar cause or circumstance. Here, with a stratified sample of living Facebook users and the full group of deceased Facebook users (in California over a two-year period), it seems we’re lacking essential information. The problem is not necessarily with the case control study approach but with the nature of the comparison; the dead and the living may or may not have mortality risks in common.

Another big problem: There’s no way to know, from these data and this study, to what extent any of the effects (real or not) are Facebook-specific. Any Facebook activity might be combined with or influenced by other online activity. The authors do consider a possible relation between Facebook activity and offline activity, but not between Facebook and other online activities.

I will think and write more on this later.

Facebook and Mortality: Part Two

In the previous post, I examined the first part of the study on Facebook and mortality rates. Here’s the citation again:

William R. Hobbs, Moira Burke, Nicholas A. Christakis, and James H. Fowler, “Online Social Integration is Associated with Reduced Mortality Risk,PNAS, Early Edition, published ahead of print on October 31, 2016, doi: 10.1073/pnas.1605554113. 

Now it’s time to look at the second part. Having compared the mortality rates of Facebook users and Facebook nonusers, they now investigated to what extent specific Facebook activities were associated with mortality.

To compare the dead and the living, they selected their samples in this manner:

To ensure age and gender covariate balance in our analyses, we compared all deceased individuals on Facebook to a stratified random sample of nondeceased individuals (SI Appendix, Fig. 5) from the full and voter populations described above. There were 179,345 people in our age- and gender-based probability sample of Facebook users born between 1945 and 1989, of whom 17,990 died between January 2012 and December 2013; 89,597 were also present in the California voter record, of whom 11,995 had died between January 2012 and December 2013.

Having identified their sample, they began by comparing two Facebook activities: initiating friendships and accepting friendships. They found a relation between accepting friendships and decreased mortality risk, but no such relation between initiating friendships and the same. Just to be clear about the difference: “(A) Initiated friendship: the subject sent a Facebook friendship request that was then accepted. (B) Accepted friendship: the subject received and accepted a friendship request.” They used a Cox proportional hazard model to estimate the relative risk;  they report the details in their appendix.

It appears from their findings that larger numbers of accepted friendships correlate with lower relative mortality; no such correlation exists for initiated friendships. However, it seems a bit arbitrary to examine these activities in terms of quantity; for instance, they could have examined time lapses between invitation and acceptance, or relation of invited friends to existing friends. I don’t mean that these comparisons would have been better; there’s just no reason to suppose that the number of friend requests or acceptances is particularly important, especially on Facebook.

Onward. Next, they compared text-based activity and photo-based activity; they found that “mortality risk declines with increased photos, whereas it actually increases with increased statuses.” Here again I question these correlations. Aren’t healthier people more likely to post photos of themselves and their families (if they’re posting on Facebook to begin with)? Relatively speaking, statuses might appeal more to people who are ill. If you’re unwell, you might use other methods to let people know how you are. You might not want to post pictures.

The authors see photos as indicative of face-to-face interactions; they write, “These results are suggestive that offline social activities—and not online activities—are driving the relationship between overall Facebook activity and decreased mortality risk.” Well, possibly, but it may be that those “offline activities” are already triggered by health or illness.

Next, they consider activities directed at specific individuals: They compared the sending of posts (with tags) and messages with the receiving of photos with tags. Here it’s a little more complicated, but the receiving of photos with tags correlates with lower mortality risk than the sending of posts and messages. Yet this may have to do with one’s existing health status: it could be that those who are healthier tend to receive photos with tags (after all, they made it out to the party or other event), whereas those who are ill may rely on posts and messages.

Finally, they examine how specific Facebook activities relate to rates of mortality for specific causes of death. Then they discuss the results overall. I will take this up next time (probably not today).

My thoughts so far: There may well be an association between certain Facebook activities and mortality, but one’s health status could influence one’s Facebook activity at least as much as vice versa. There’s no reason to believe that the dead people would have prolonged their life if they had engaged in a different Facebook practice.

To shed some light on this, you’d need some information about the subjects’ health. There’s a lot of grey area between “dead” and “alive.”

Update: I learned from a commenter that this is called a case control study. I now wonder whether this is appropriate for a study of Facebook and mortality, given the many possible causes of death. Where case control studies examine the relation between exposure and an “outcome,” I have difficulty seeing death here as an “outcome.” One death may be nothing like another; the causes may be completely unrelated to each other. In addition, while we’re all “at risk” for death, the control subjects might be at negligible risk for these particular deaths; we just don’t know. I’d think the degrees of risk would matter. I welcome any comments on this issue.

Note: I revised this piece substantially after posting it; I later made a few cuts for the sake of conciseness and flow.

Facebook and Mortality: How Little We Know

Almost every day I check to see what the Science of Us (New York Magazine) writers have posted. Although the quality varies, it’s usually interesting. Some of it I find enlightening, some of it makes me shake my head, and some evokes a combination of reactions. They’re relatively skeptical in their descriptions of psychological research, but sometimes their skepticism seems “token” and safe. It may depend on how much time they have to research and write a given piece.

So after reading Melissa Dahl’s (nominally skeptical) piece about the just-published study in PNAS about Facebook activity and mortality, I vowed to take a close look at the study and give my own take on it. Dahl found it “deeply sad” that the study suggested that a lower mortality risk was associated with friend requests accepted, not friend requests initiated. This seems to suggest (to her) that if you’re trying to reach out to people, you may not be making yourself healthier.

But I found it hard to accept any of this. Why are people associating Facebook “friending” with anything substantial? How is it possible to isolate one Facebook activity from another and measure its relation to mortality, when there are so many other factors at work? Isn’t this a classic case of forking paths–where you can look for all kinds of associations and are bound to find something?

Instead of just criticizing in a void, as I sometimes do, I vowed to read the actual study from start to finish. I suspect it has some serious problems, but let’s see. I want to preface this by acknowledging the good intentions of the authors. One of them, James Fowler, told Dahl, “The whole reason that I’m in this game is because I want to figure out how to use social networks to make people healthier.” In criticizing the study, I do not disparage its underlying intent. Moreover, Fowler himself expressed some caution about the study’s conclusions.

With all of that in mind, here we go. I may have to do this in several parts.

Here’s the citation: William R. Hobbs, Moira Burke, Nicholas A. Christakis, and James H. Fowler, “Online Social Integration is Associated with Reduced Mortality Risk,PNAS, Early Edition, published ahead of print on October 31, 2016, doi: 10.1073/pnas.1605554113. 

In this study, the researchers do two things (boldface added): “Using public California vital records, we compare 12 million Facebook users to nonusers. More importantly, we also look within Facebook users to explore how online social interactions—reflecting both online and offline social activity—are associated with longevity.”

Here’s how the abstract summarizes the results:

The results show that receiving requests to connect as friends online is associated with reduced mortality but initiating friendships is not. Additionally, online behaviors that indicate face-to-face social activity (like posting photos) are associated with reduced mortality, but online-only behaviors (like sending messages) have a nonlinear relationship, where moderate use is associated with the lowest mortality. These results suggest that online social integration is linked to lower risk for a wide variety of critical health problems.

All right, let’s see how they arrived at this. Some of it seems intuitively right (moderate online behaviors, combined with offline behaviors, would be more conducive to health than online-only behavior generally). But hunches aren’t the point here; I want to see how the study fits together.

First, they compared the age- and gender-matched mortality rates of the full population of Facebook users (in California, that is) to the mortality rate in the California voter record. It seemed that the Facebook rate was 63% of the California voter rate, but they recognized that this figure could be off because of the difficulty of matching Facebook users to vital records. So they tried again, this time focusing on the “voter” subpopulation of Facebook users. This time, the Facebook mortality rate was about 88% of the rate of Facebook nonusers (within the voting population).

They then disaggregated this by cause of mortality. They found no difference between the populations for mortality rates due to sexually transmitted diseases, several types of cancer, unintentional injuries, drug overdoses, and suicides. For certain other causes (infections, diabetes, mental illness or dementia, ischemic heart disease, stroke, other cardiovascular diseases, liver disease, and homicide), they found a significantly lower rate in Facebook users.

They acknowledged possible problems with these figures: “It is important not to read too much into the comparison between Facebook users and nonusers because many factors may confound the apparent association between being a Facebook user and experiencing lower mortality. This is an observational result, and we have few socioeconomic controls because we do not have much information about nonusers.”

I give them credit for this. I agree heartily and suspect that economic factors play heavily into the results. Facebook users may have more resources in general, including health insurance. (I suspect that among nonusers, those who abstain from Facebook out of preference or principle are a minority, and that others simply can’t afford certain computer activities. I might be wrong about this.)

In addition, as Fowler said to Dahl, it’s important not to confuse correlation with causation. The mortality rate for Facebook users could be lower because they’re generally healthier when they begin using it; the very ill may be too overwhelmed or incapacitated to use it much.

Now comes the second and more complex part of the study: the analysis of specific Facebook activities and their relation to mortality. I’ll leave that for next time. For now, I don’t think there’s too much we can learn from the first part; the Facebook mortality rate could be lower than the nonuser rate for numerous reasons; there’s no reason to suppose that Facebook helps you live longer. The authors acknowledge this.

Note: I made some minor edits to this piece after posting it.

Formal and Informal Research

I have been thinking a lot about formal and informal research: how both have a place, but how they shouldn’t be confused with each other. One of my longstanding objections to “action research” is that it confuses the informal and formal.

Andrew Gelman discusses this problem (from a statistician’s perspective) in an illuminating interview with Maryna Raskin on her blog Life After Baby. It’s well worth reading; Gelman explains, among other things, the concept of “forking paths,” and acknowledges the place of informal experimentation in daily life (for instance, when trying to help one’s children get to sleep). Here’s what I commented:

[Beginning of comment]

Yes, good interview. This part is important too [regarding formal and informal experimentation]:

So, sure, if the two alternatives are: (a) Try nothing until you have definitive proof, or (b) Try lots of things and see what works for you, then I’d go with option b. But, again, be open about your evidence, or lack thereof. If power pose is worth a shot, then I think people might just as well try contractive anti-power-poses as well. And then if the recommendation is to just try different things and see what works for you, that’s fine but then don’t claim you have scientific evidence one particular intervention when you don’t.

One of the biggest problems is that people take intuitive/experiential findings and then try to present them as “science.” This is especially prevalent in “action research” (in education, for instance), where, with the sanction of education departments, school districts, etc., teachers try new things in the classroom and then write up the results as “research” (which often gets published.

It’s great to try new things in the classroom. It’s often good (and possibly great) to write up your findings for the benefit of others. But there’s no need to call it science or “action research” (or the preferred phrase in education, “data-driven inquiry,” which really just means that you’re looking into what you see before you, but which sounds official and definitive). Good education research exists, but it’s rather rare; in the meantime, there’s plenty of room for informal investigation, as long as it’s presented as such.

[End of comment]

Not everything has to be research. There’s plenty of wisdom derived from experience, insight, and good thinking. But because research is glamorized and deputized in the press and numerous professions, because the phrase “research has shown” can put an end to conversation, it’s important to distinguish clearly between formal and informal (and good and bad). There are also different kinds of research for different fields; each one has its rigors and rules. Granted, research norms can also change; but overall, good research delineates clearly between the known and unknown and articulates appropriate uncertainty.

Update: See Dan Kahan’s paper on a related topic. I will write about this paper in a future post. Thanks to Andrew Gelman for bringing it up on his blog.