Putting truth to the test
Putting truth to the test in the “post-truth” era, Boston College psychologists conducted experiments that show when Americans decide whether a claim of fact should qualify as true or false, they consider the intentions of the information source, the team reported recently in Nature’s Scientific Reports.
That confidence is based on what individuals think the source is trying to do – in this case either informing or deceiving their audience.
“Even when people know precisely how accurate or inaccurate a claim of fact is, whether they consider that claim to be true or false hinges on the intentions they attribute to the claim’s information source,” said Professor of Psychology and Neuroscience Liane Young, an author of the report. “In other words, the intentions of information sources sway people’s judgments about what information should qualify as true.”
Professor of Psychology and Neuroscience Liane Young and doctoral student Isaac Handley-Miner. Their project involved working with nearly 1,200 participants and examining about 16,200 responses. (Lee Pellegrini)
Lead author Isaac Handley-Miner, a PhD student and researcher in Young’s Morality Lab, said the so-called post-truth era has revealed vigorous disagreement over the truth of claims of fact — even for claims that are easy to verify.
“That disagreement has alarmed our society,” said Handley-Miner. “After all, it’s often assumed that the labels ‘true’ and ‘false’ should correspond to the objective accuracy of a claim. But is objective accuracy actually the only criterion people consider when deciding what should qualify as true or false? Or, even when people know how objectively accurate a given claim of fact is, might they be sensitive to features of the social context—such as the intentions of the information source? We set out to test whether the intentions of information sources affect whether people consider a claim of fact to be true or false even when they have access to the ground truth.”
The researchers showed participants a series of claims accompanied by the ground truth relevant to those claims, according to the report. In one experiment, the claims concerned politicized topics such as climate change, abortion, and gun violence. In another experiment, these claims concerned non-politicized topics such as the average lifespan of a car and the price of a pair of headphones. The researchers asked participants in both experiments to decide whether they would consider each claim of fact to be true or false.
When presented with a claim of fact, study participants were presented with one of two scenarios about the source of the information they were assessing: the information source either wanted to deceive or inform them. To do this, the researchers swapped out the news outlet that allegedly published the claim. For example, one participant might be told that a claim about climate change came from Fox News, while another participant might be told that the same claim about climate change came from MSNBC, Handley-Miner said.
In the experiment with claims about non-politicized topics, the researchers told the participants whether the information source was trying to be informative or deceptive, he said.
“We presented participants with claims of fact and ensured that participants knew precisely how accurate or inaccurate those claims were,” Handley-Miner said. “Across participants, we varied whether the source of those claims intended to inform or deceive their audience. Participants reported whether they would consider the claims to be true or false given the supplied ground truth. We then evaluated whether participants were more likely to classify claims as true when the information source was trying to inform versus deceive their audience.”
The researchers worked with 1,181 participants and examined approximately 16,200 responses fielded during their experiments.
Although participants knew precisely how accurate the claims were, participants classified claims as false more often when they judged the information source to be intending to deceive them.
Similarly, they classified claims as true more often when they judged the information source to be intending to provide an approximate account rather than a precise one, according to the study. For instance, what if someone knows for certain that 114 people attended an event, but one source reports 109 people attended, and another source reports that 100 attended? An individual is likely to view the latter number as true because it’s assumed the source is providing an estimate, Young said.
“It’s often assumed that the labels ‘true’ and ‘false’ should correspond to the objective accuracy of a claim. But is objective accuracy actually the only criterion people consider when deciding what should qualify as true or false?”
The findings suggest that, even if people have access to the same set of facts, they might disagree about the truth of claims if they attribute discrepant intentions to information sources.
The results demonstrated that people are not merely sensitive to the objective accuracy of claims of fact when classifying them as true or false. While this study focused on the intent of the information source, Young and Handley-Miner say intent is probably not the only other feature people use to evaluate truth.
In future work, the researchers hope to develop an expanded understanding about how people think about truth. Moreover, given the rise in popularity of Artificial Intelligence models, such as ChatGPT, the researchers may investigate whether state-of-the-art AI models “think” about truth similarly to humans, or whether these models merely attend to objective accuracy when evaluating truth.
The project team included a multidisciplinary group of researchers funded, in part, by the Schiller Institute for Integrated Science and Society’s Grants for Exploratory Collaborative Scholarship (SI-GECS) program.
In addition to Young and Handley-Miner, co-authors of the report included doctoral candidate Michael Pope, Boston College Associate Professor of Philosophy Richard Atkins, Associate Professor of Communication Mo Jones-Jang, and Associate Professor of Philosophy Daniel McKaughan. Dartmouth College’s Jonathan Phillips was also part of the team.
In addition to SI-GECS funding, the research was supported by the John Templeton Foundation and the National Defense Science and Engineering Graduate Fellowship.