The backfire effect: numerous failures to replicate

It is good practice in psychology to assume that every finding is bullshit unless proven otherwise. Of course, not all of psychology is bullshit (replication rates differ by field: social psychology 26%, cognitive psychology 53%). Examples of findings in psychology that I would consider not bullshit (i.e well-established): Stroop effect, spacing effectMüller-Lyer illusiontask-evoked pupillary response. In light of numerous failures to replicate however it seems that the so-called “backfire effect”, a favorite hobbyhorse of so-called “rationalists“, does not fall into that rather small category of research findings. This is not a meta-analysis

Original paper: When Corrections Fail: The Persistence of Political Misperceptions, Nyhan & Reifler 2010

  • Study 1:
    • Sample: 130 students
    • Topics: WMDs in Iraq
    • Found backfire effect
  • Study 2:
    • Sample: 130 students
    • Topics: Iraq WMD, tax cuts, stem cell research
    • Found backfire effect for all topics

Second paper: Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information Nyhan & Reifler 2015

  • Sample size: 1000 for wave 1, 822 for wave 2, from YouGov/Polimetrix PollingPoint Panel and the E70 Rewards and Western Wats panels
  • Topic: flu vaccine safety
  • Found backfire effect

Failure to replicate: Electoral Consequences of Political Rumors: Motivated Reasoning, Candidate Rumors, and Vote Choice during the 2008 U.S. Presidential Election Weeks & Garrett 2014

  • Sample size: 600 random digit telephone survey
  • Topics: Various political rumors (e.g Obama is a Muslim)
  • No backfire effect found. Rumor rebuttals were uniformly effective.

Failure to replicate: The limitations of the backfire effect, Haglin 2017

  • Sample size: 474 MTurks
  • Topic: flu vaccine safety
  • Exact same procedure used as in Nyhan & Reifler 2015
  • No backfire effect found

Failure to replicate: The Elusive Backfire Effect: Mass Attitudes’ Steadfast Factual Adherence, Wood & Porter 2017

  • Sample size: 10,100 MTurks
  • Topics: 52 issues tested for potential backfire (e.g. Spiraling Gun Violence, Drugs Drive Prison Growth, Hedge fund mangers pay less tax than workers, Discrimination sole cause of gender wage gap, Mexican Immigrants disproportionately criminal, US taxes highest in world, etc.)
  • No backfire effect found for any topic

Conclusion:

We find that backfire is stubbornly difficult to induce, and is thus unlikely to be a characteristic of the public’s relationship to factual information. Overwhelmingly, when presented with factual information that corrects politicians—even when the politician is an ally—the average subject accedes to the correction and distances himself from the inaccurate claim.

In light of the recent failures to replicate (with much larger and more diversified samples than the original studies), it seems to reasonable to conclude that the effect either is very weak to nonexistent or only appears under very restricted conditions. Such is the sad state of affairs in the social sciences.

This is not a meta-analysis.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s