How Political Bias Changes Our Understanding of Data

Data literacy is the ability to understand the results and process of collecting, analyzing, interpreting, and reporting data. This skill is becoming increasingly relevant with internet-driven improvements to data accessibility (such as online journals and databases) and sharing (for example, on social media). While this surge in accessibility has caused a shift in the way we interact with data — as of 2016, 62% of adults in the U.S. accessed news via social media — it can also be used, whether intentionally or unintentionally, to disperse misinformation. In this era of “fake news” and information overload, educators are working to address gaps in data literacy to equip students with the critical thinking tools necessary to interpret and communicate information. However, public conflict over societal issues – like gun violence, global warming, and COVID-19 vaccination – continues to persist, despite widely accessible and compelling scientific evidence. This raises the question: is improving data literacy enough to combat misinformation?

A recent article by researchers at Yale, Ohio State, Cornell, and the University of Oregon tested two approaches that could potentially answer this question. The “Science Comprehension Thesis” (SCT) identifies the source of these controversies as defects in the public’s knowledge and reasoning capacities (i.e. insufficient data literacy). The second, the “Identity-protective Cognition Thesis” (ICT), asserts that cultural conflict can disable one’s ability to make sense of decision-relevant science. The authors tested political and non-political data literacy in a group of 1,111 American adults who pre-rated their political affiliation and beliefs.

To determine a baseline for data literacy (defined as “numeracy” in this study), the authors provided participants with data from a fictional skin-care study and asked them to interpret the results to determine whether the skin cream being tested was effective at improving a rash. You can test your data literacy by trying this exercise:

This type of data presentation is called a contingency table, and can be solved by dividing the number of participants who saw improvements to the rash after using the cream (223 people) by the total number of those who used the cream (223 + 75). This calculation (223/298) reports that using the skin cream was effective in 75% of people who used it. In comparison, the number of patients who did not use the cream but saw improvements to the rash (107) divided by the total number of those who did not use the cream (107 + 121 = 128) yields an improvement rate of 84%. Based on the results of this study, those who did not use the cream saw a higher rate of improvement to their rash (84%) than those who did use the cream (75%). Therefore, the data reveals that the cream does not work.

This task was specifically designed to be a little bit difficult – it’s not just comparing the absolute numbers of patients who experienced positive or negative outcomes, but rather required the participants to realize that they needed to calculate ratios in a contingency table and compare those ratios. As you might expect, there was a wide range of correct responses, but self-described liberals and conservatives did not perform significantly differently on these types of questions.

However, the authors then placed the same data in a political context, framing it about gun control. Instead of a fictional study about skin cream, they now asked whether enacting a ban on concealed handguns in public increased or decreased crime. They also balanced it between liberal and conservative contexts by switching whether the “right answer” was an increase or decrease in crime:

“Liberal Condition”

“Conservative Condition”

Despite these being the exact same numbers, results were wildly different. In the skin cream version of the task, regardless of a participant’s political leaning, as their data literacy score increases, the better they are at correctly interpreting the data, no matter whether the correct answer was that the cream increased or decreased skin rash. However, in the political version, both liberals and conservatives were more likely to correctly interpret the data when the data already lined up with their preconceived notion of how it should be. Liberals got the right answer more often in the “liberal condition,” while conservatives performed better in the “conservative condition.” When faced with data that was incongruent with their pre-formed political views, both groups were significantly less likely to correctly interpret the data. And this is still true – possibly even more true – when they have high data literacy: when the authors divided people into high and low numeracy groups, the high numeracy groups were actually more biased than the low numeracy groups.

Overall these results support the Identity-preserving Cognition Thesis (ICT), meaning that certain political conflicts disable the capacities that individuals have to make sense of decision-relevant science: when policy-relevant facts become identified as symbols of membership in and loyalty to affinity groups that figure in important ways in individuals’ lives, they will be motivated to engage with and “cherry-pick” empirical scientific evidence and other information in a manner that more reliably connects to their beliefs, regardless of which political beliefs they hold.

Subjects were more likely to correctly identify the result most supported by the data when doing so affirmed the position they were already politically predisposed to accept—that the ban decreased crime, in the case of more liberal subjects; or that the ban increased crime, in the case of more conservative ones—than when the correct interpretation of the data threatened or disappointed their predispositions.

Under the Science Comprehension Thesis (SCT), we would expect that polarization among high-numeracy participants would be lower, but in fact, they saw that numeracy actually magnified polarization. More data-literate participants were actually slightly more biased than less data-literate participants. They interpret this to mean that accurately discerning the identity-affirming outcome depended on a high degree of data literacy.

Proponents of the Science Comprehension Thesis usually push for improving science education and critical thinking skills, and while that is probably still important, these results show that it is not enough to combat polarization and misinformation. The authors say that only removing the source of the motivation to process scientific evidence in an identity-protective manner can do that. The conditions that generate symbolic associations between positions on data and facts, on the one hand, and cultural identities, on the other, must be neutralized in order to assure that citizens make appropriate use of their capacity for science comprehension. And in order to do this, they advocate that identifying strategies to protect science communication from antagonistic cultural meanings —and decontaminating it when such protective measures fail—will be critical.

~~~

Written by Elizabeth Burnette

Edited by Yuki Hebner and Rebeka Popovic

Illustrated by Lisa Vial

~~~

Become a Patron!

References

1. Gottfried, J. & Shearer, E. (2016). News Use Across Social Media Platforms 2016. Pew Research Center.
2. West, J.D. & Bergstrom, C.T. (2021). Misinformation In and About Science. Proceedings of the National Academy of Sciences, 118(15). DOI: 10.1073/pnas.1912444117
3. Kahan, D.M., Peters, E., Dawson, E.C., & Slovic, P. (2017). Motivated Numeracy and Enlightened Self-Government. Behavioural Public Policy, 1(1): 54-86. DOI: 10.1017/bpp.2016.2