Sensible efforts to curb harmful comments on social media can backfire

A study of 2.5 million pro-eating disorder Instagram posts suggests efforts to reduce harmful communication by social media users should be done very carefully as it can backfire badly (Chancellor, et al., 2016).

The study titled #thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities looked at changes in user behaviour both before and after Instagram took action to reduce the spread of pro-eating disorder posts in 2012.

While Instagram stopped short of banning the use of identified pro-eating disorder hashtags, they did prevent people from searching on those hashtags and instead published warning messages and links to credible eating disorder help-lines and websites.

Instead of reducing the use of those hashtags and the damaging information posted on them, users simply moderated the terms, using new hashtags such as #thinspoooooo and #thyghapp.

Of the 17 moderated terms each spawned an average of 40 alternatives, including 107 variations of “#thighgap.”

The number of likes and comments on these new hashtags increased by 30 per cent in some cases.

Even worse, the new terms appeared to encourage the pro-eating disorder posters to become more brazen, with a reported increase in the amount of discussions of self-harm, self-loathing and other more negative topics.

As this is an observational study it is not possible to definitively determine the cause of the growth of these hashtags (particularly as this was during an overall growth period for the platform), however it stands to reason that by seeking to banish the problem, they merely drove it underground, where it potentially became even more harmful and harder to monitor.

Discussion

Banning books has never worked in the past. Likewise social media platforms must realise that banning or preventing people from sharing information must be done extremely carefully, even if it is done for good societal and heath reasons. Where people have a reason to find a workaround, they will find one. Our species is an ingenious lot, even when we are trying to harm ourselves.

Publishers are all too aware of growth of harmful online communities and the negative consequences they can cause, including making eating disorders look like a sensible option. As the authors of this study point out it is important that people reading posts on a hashtag like "#imugly" get exposed to offers of help, rather than just information that seems to normalise dangerous behaviour or confirm their biases.

Perhaps a better solution is to maintain the original hashtags but encourage mental health experts to post on them regularly, offering help and providing individual advice, rather than trying to address a wicked problem with a blunt software update to solve a wicked problem that is even more wicked thanks to the networking abilities of social media.

References

  • Chancellor, S. et al., 2016. # thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities. San Francisco, Association for Computing Machinery, pp. 1199-1211.