Confirmation Bias
Are Numbers feeding our Confirmation Bias?
Statistics are not as simple as they seem, and yet are used callously across the internet to create ‘alternative facts.’ How can you read statistical headlines better? The first step is accepting their complexity.
By Anindita Sen and Biswajyoti Banerjee
Can we trust statistical data fed to us by the media?
Do we understand numbers well enough to say that we have statistical literacy?
While historically statistics have had a significant impact on our worldview and human way of life, it has also inadvertently perpetuated biases.
In the modern world where truth is fragmented, statistics is often the go-to tool not only to generalise but to create stereotypes that allow one particular narrative to prevail over others. While it gives you valuable insights and lets you create context, it leaves room for misinterpretation by cherry-picking data sets.
A classic example is how toothpaste advertorials often come with a tagline that says the brand has been recommended by x number of dentists. What the audience doesn’t realise is that these claims are usually based on studies funded by the manufacturer that allow the dentist to choose more than one brand for recommendation. Something that is never explicitly stated anywhere.
Remember the sensational piece of ‘news’ that did the rounds with headlines like, “Census 2021 in charts: Christianity now minority religion in England and Wales.” An assumption from reading this headline alone might be that observers of Christianity are being overtaken in numbers by those who observe other minority religions. However, a closer look at the article tells us that the fastest growing cohort in the UK are in fact people who told surveyors that they had ‘no religion’ as a preference. This shows us how there is more to the article than meets the eye. In other words, the headline has been curated to provoke intrigue rather than to accurately inform the reader.
Facts vs. Opinions
The use of statistical data and graphs to sensationalise media claims is not a new thing. What is new, is how today these claims get further skewed and influence bias. Such articles are picked up and shared by people within echo chambers, only to reinforce their worldview and ideology.
Individuals tend to be swayed by strong opinions. The danger arises when those opinions are backed by statistics that the media has edited without adequate attention to their own or their audiences' statistical literacy. These then become speculation, disguised as fact, and feed our confirmation bias.
Professor Sanghamitra Bandyopadhyay, Director of the Indian Statistical Institute in Kolkata, believes that in a place like South Asia we have to expect the absence of statistical literacy and the possibility that data is going to be misread. Even data that is correctly presented may visually appear to convey something that is not actually the case to a layperson. Forming a viewpoint on the basis of statistics involves analysing the data very carefully.
“The way forward is to be conscious of how we all approach numbers, graphs and charts; making an effort to be as clear as possible about what is being implied both as authors and readers.” says Professor Bandyopadhyay.
News often oversimplifies complex statistical information to make attention-grabbing headlines, which makes the messaging accessible to a wider audience, but also runs the risk of oversimplifying complex issues.
Statistical news often misleads us due to a lack of context. An article that confers unemployment rates may fail to give us the whole picture. Today’s unemployment can be a fallout of past education policies or lack of skill development, not only economic recession or profit-hungry employers.
Searching for truth
Today, many researches are syndicated by corporations, federations and trade bodies, and are not commissioned by credible authorities. We often read statistics without knowing where they come from. Media organisations and big brands commission research to prove their claims of quality, superiority and leadership.
According to a study by a team of Paris-based researchers called, ‘Facts, alternative facts, and fact checking in times of post-truth politics,’ fact checking may not be such a simple solution once the seed of an idea has been planted. They examined whether fact-checking could correct some of the false “alternative facts” propagated by Marine Le Pen in France’s 2017 presidential election, such as inaccurate and misleading claims about the number of men vs. women among refugee populations and about unemployment statistics for migrants.
“Zhuravskaya and her colleagues found that fact-checking helped correct knowledge of the relevant statistics, but that it did not necessarily change people’s minds about Le Pen’s arguments. As they put it, “success in correcting factual knowledge does not translate into an impact on voting intentions.” Even armed with the facts, audiences may continue to believe arguments based on incorrect information or to act according to those prior beliefs.”
Keeping our own confirmation bias in mind, let us then explore the guidelines set by the BBC for its own media professionals. These are their five most important questions to ask while engaging with statistics:
“WHO has produced the statistics? How reliable is the source?
WHY have the statistics been produced and why now? What question is the statistic an answer to? Does the source have a vested interest or hidden agenda?
HOW have the statistics been compiled? What exactly has been counted? Are the underlying assumptions clear?
WHAT does the statistic really show? Does the study really answer what it set out to test? What are the producers of the statistics not telling you? Avoid automatically taking statistics at face value.
WHERE can you find the underlying data and is it available?”
These questions can be a good guide and starting point not just for journalists and media professionals but also for us as the audience, reader, or layperson trying to build our own statistical literacy.
In the current media landscape, it is essential to find helpful and reputable sources of information. For example, iseek.com is a specialised search engine that allows you to look across thousands of pre-approved sources, including universities, governments, and nonprofits, on a wide variety of topics. Or, one can browse uis.unesco.org, a portal created by UNESCO Institute for Statistics to cross check information.