Bias, assumptions and emotions: why we think what we think
While highly efficient, the way we process information to make decisions is far from perfect. Finding out how this process works – and how we can use this awareness to make better decisions – is at the heart of cognitive science. At The University of Melbourne in Australia, Professor Andrew (Andy) Perfors is studying the nature of bias and other features of cognition, and even applying these findings to machine learning.
TALK LIKE A COGNITIVE SCIENTIST
Artificial intelligence (AI) — computer systems able to perform tasks normally requiring human intelligence
Bias — a tendency to believe or support a particular perspective
Cognitive science — the study of thought, learning and mental organisation
Machine learning — a type of artificial intelligence in which computers learn from past data in order to predict outcomes
Misinformation — false or inaccurate information created and spread without an intention to harm others
Reason — to think logically, based on facts and evidence
Every day we are bombarded with information, ranging from things we read online, to conversations with family and friends, to events we see and hear in the world around us. Our brains have the unenviable job of taking all this information in, deciding which bits are important, and using these bits to inform our opinions and behaviour. However, as Professor Andrew Perfors at The University of Melbourne knows all too well, this process is subject to bias – when certain pieces of information are favoured over others – which can lead to us making decisions based on dubious assumptions or poor reasoning.
The nature of bias
“Biases aren’t intrinsically bad,” says Andy. “In fact, they’re often very useful. They are what allow us to make lots of decisions quickly, without having all the data, which humans are actually very good at.” For instance, we all have a bias to pay attention to (and like) things we see frequently. This is useful for learning language, understanding social norms and recognising neighbours. However, this bias is also exploited by advertisers, who know that people who see a brand more frequently are more likely to buy it.
Biases also contribute to us choosing to accept some types of information and ignore or reject others, which can lead to people believing misinformation. “It’s very hard to know what’s true when you don’t have much direct evidence,” says Andy. “While we all know the sky is blue because we can see it, we don’t have direct evidence on things like how vaccines work or how the economy is doing.”
Truth and sense
Because we often do not have direct evidence, we have to rely on two other things: our trust in the source of information, and whether it makes sense to us. “However, the issue with trust in the modern world is that people are very good at lying, and we also usually don’t know the person stating the information personally,” says Andy. Even if we are told something by a trusted friend or relative, they might have heard the information from a politician or journalist whose trustworthiness is difficult to gauge. “Because trust is difficult, we often also rely on whether the information makes sense to us,” says Andy. “Unfortunately, in our minds, whether something makes sense usually equates to whether it’s easy to process or whether it fits with our prior experience.” This is dangerous, because it means we are more likely to believe ‘simpler’ explanations over nuanced ones, and we fall back on our preconceptions rather than open up to new ideas. “We usually don’t have the time or expertise to scrutinise information properly, so it’s more ‘efficient’ to believe the simpler thing,” says Andy.
Assumptions
While we all like to think our decisions are rational, this is actually an impossible pursuit, as we can never have all the data needed for total objectivity. “There’s no such thing as ‘perfect’ rationality,” says Andy. “Even the most advanced machine needs to make assumptions about what information is most relevant when making a decision. The real question is to ask ourselves why we are making the assumptions we do and whether they’re beneficial, correct or neither.”
The problem with assumptions is that we often do not notice we are making them until we take a moment to step back and reflect. “While it’s really hard, it’s important to recognise that we might sometimes make incorrect assumptions and need to change our mind,” explains Andy. “Building trust in your own reasoning – for instance, by seeking out alternative viewpoints when given a certain perspective and making a decision based on more evidence – is the best way to ensure your decisions are as good as they can be.”
Emotions vs logic
Decisions based on emotions and those based on logic are often viewed as two ends of a spectrum, but Andy says they are actually part of the same system. “Facts are useless to us on their own. Emotions tell us what to do with the information we get,” he says. “For instance, if something makes me feel scared, I know it’s time to run or fight; if I feel ashamed, I know I’ve probably done something wrong that I should try to make right.” However, while these reactions are useful in the majority of cases, they can be abused. “Emotions are blunt tools and can be manipulated,” says Andy. “Someone might tell me something they know will make me scared because they want me to stop being sceptical and have an emotional response that benefits them, rather than me.”
Taking time to reflect on the information you are being given and your emotional response to it can help you avoid manipulation. “If something is making you feel angry or fearful, especially if it is about a group or person you don’t know, stop and ask yourself why you are being told this information,” says Andy. “What is the goal of whoever is spreading this information? Does it actually make sense? Is it possible the information has been distorted or key parts have been left out?” Misinformation often works in exactly this way, seeking to sow hatred and make people fear and hate each other, relying on half-truths to stimulate an emotional response in people.
Reference
https://doi.org/10.33424/FUTURUM369
Emotion and logic are often viewed as two ends of a spectrum, but emotions tell us what to do with fact.
© EtiAmmos/shutterstock.com
“Building trust in your own reasoning – for instance, by seeking out alternative viewpoints when given a certain perspective and making a decision based on more evidence – is the best way to ensure your decisions are as good as they can be,” says Andy.
© fizkes/shutterstock.com
“Machines don’t have basic ideas like causation or sensation or emotion, whereas these are instinctive concepts for people,” explains Andy.
© Alexander Supertramp / shutterstock.com
© Gorodenkoff/shutterstock.com
Thinking machines
The rise of artificial intelligence (AI) has been heralded by some as the start of the age of ‘machines that think like us’, but Andy thinks we still have a way to go. “You’ve probably heard of AI like ChatGPT and DALL-E that produce impressive results, but the thing to remember is they are trained on huge amounts of data – trillions of words or billions of images – far more than any human could ever see in their life,” he says. “These models need that much data to make good decisions, but humans don’t. This indicates we learn in a very different way to machines.”
These machines are basically learning by association, analysing which sort of things typically ‘go together’ within the datasets they use and replicating these associations. “There isn’t any actual knowledge beneath this ability,” explains Andy. “Machines don’t have basic ideas like causation or sensation or emotion, whereas these are instinctive concepts for people.” This is why machines make mistakes that humans never would: made-up facts, logically inconsistent strings of sentences, ‘lumpy’ faces or extra fingers in AI art, and so on. “AI will continue to get better, but my guess is that it will think differently to people for a long time,” says Andy. Though there is a long path ahead, his research into why we make the decisions we do is helping to pave the way for such machines in the future, by applying these cognitive processes in artificial systems.
Experiments and models
Testing the concepts behind cognitive psychology involves carefully planned experiments and the support of models. “We use models as a tool to transform what might be quite a vague concept, like ‘people’s beliefs are influenced by social media’, into something precise and testable,” says Andy. “A good theory needs to specify as much as possible: what sorts of information sources we’re examining, what we mean by influence, how we consider social groups, how we measure beliefs, and so on.”
As an example, Andy’s team – led by research fellow Keith Ransom – recently used a mixture of experimentation and modelling to examine the more specific question of whether people are more likely to change their minds if presented with many different people on social media arguing for a viewpoint, as opposed to just one person arguing many times. “While considering this question and deeper theories about how we process information, we made a model that makes very precise predictions about how these beliefs should change in a range of scenarios,” says Andy. “Then, we designed an experiment that tests this model using real scenarios.” For this experiment, participants were presented with fake social media posts created by the researchers, either from multiple ‘accounts’ or from one ‘account’. By being asked about their beliefs before and after, the researchers found a strong match with what the model predicted: that participants exposed to the same viewpoint from multiple people were more likely to shift their beliefs. Once the model has been tested using various experimental conditions, it can then be used to extrapolate predictions about how people might behave in similar scenarios, without the need to experimentally test that specific scenario.
Professor Andrew (Andy) Perfors
Director, Complex Human Data Hub, Melbourne School of Psychological Sciences, The University of Melbourne, Australia
Fields of research: Psychology, Cognitive Science
Research project: Using experimental and modelling techniques to investigate where biases come from, and using these findings to inform the development of machine systems that reason like people
Funders: Australian Research Council (ARC), Australian Defence Science & Technology Group (DSTG)
ABOUT COGNITIVE SCIENCE
Cognitive science involves the study of the human mind, how we learn, how we process information, and how we make decisions. Andy explains more about what drew him to the field and how he thinks it will develop in the future.
“There are so many open questions in cognitive science. I began my undergraduate studies thinking I wanted to do physics, which I really enjoyed, but I realised all the basic questions had been solved, and the remaining questions seemed too complicated to make realistic progress. Cognitive science is the ‘sweet spot’ for me, as we have the tools and knowledge to study it scientifically, but there’s still so much we don’t know.
“I’ve found the kind of people who end up in this field are awesome. They’re smart, interested in important questions, value people as well as reason, and are a lot of fun. Ultimately, good science has a lot to do with playing and exploring, and I’m able to do that with my research and the people around me.
“There’s a lot still to learn in cognitive science. Looking ahead, I hope we’ll be more able to get data from the real world rather than just behaviour in the lab, but it’s challenging to measure people’s real-life environments and behaviour accurately while also respecting their privacy and autonomy. Historically, participants in cognitive science experiments have tended to be fairly well-off and well-educated, usually white and English speaking, and often male and heterosexual, which is obviously a tiny proportion of humanity! I hope the next generation blows this open and we learn a lot more about everyone and how our backgrounds shape how we think.
“There are plenty of ways to learn more about the discipline. Universities will often have public talks from real scientists on the cutting edge of psychology. You can find cognitive science reading lists and recommendations online. Follow your nose on what you find most interesting. For experience in computer programming or data science, there are plenty of free YouTube introductions, and open-source programming software like R can be used for everything from data analysis to making webpages or art. Play and try things, and explore what captures your interest.”
Pathway from school to cognitive science
While Andy says that psychology is useful, he especially advises taking subjects such as computer science, data science, statistics and linear algebra. Being able to program experiments and analyse results are important skills for modern-day cognitive scientists. He also suggests investigating philosophy, anthropology and linguistics, which all have important lessons about what it means to be human.
Explore careers in cognitive science
• PsychTalks is the Melbourne School of Psychological Sciences forum for ideas and discussion. It provides public talks, podcasts and a newsletter.
• The University of Melbourne runs a number of school outreach programmes, including a computer science summer school, science events for girls and Indigenous students, and work experience opportunities.
• According to Payscale, graduates of cognitive science degrees in Australia earn an average of AUS$129,000 per year.
Meet Andy
I thought I wanted to be a physicist when I started university. Before that, I wanted to be an astronaut, and my career has also involved random things like the Peace Corps. Like many, my path to my current role has been anything but direct!
Rather than specific eureka moments, science is more like slowly building an awesome temple. The best scientists are not lone geniuses but smart, thoughtful people surrounded by other smart, thoughtful people. Just like laying bricks, the scientific process can, at times, feel tedious or repetitive, but when you step back and look at your section of the temple, and how it fits with the sections built by others, you appreciate the importance of collaborative work.
Any advice I received that told me to ‘be someone I wasn’t’ was terrible. I was told that I should be more feminine and conciliatory to get a job, to not come out as transgender as academia was hostile to trans people, to study things I wasn’t interested in, and that I shouldn’t move to Australia. I realised that if I had to follow any of this advice to succeed, I didn’t want that ‘success’. And, as it turns out, people appreciate authenticity and bravery.
No obstacle is as insurmountable as it first appears. That’s one of the best things about age: now I can look back and see all the times something went wrong but I survived, and I know this time I will survive as well. I also keep an eye on my values. I love my career, but I love my family a lot more. Perspective is important!
It’s nice to get awards and papers, but the things I’m most proud of are more intangible. I’m proud that I’m a good teacher and have made a difference to a lot of students. I am very proud that I have made my choices in line with my values and interests rather than external pressures.
I hope to keep doing what I’m doing and take more of a leadership role in the field. I want to help ensure that our academic system embodies important values and nurtures the next generation through that system.
Andy’s top tip
Almost nobody knows what they’re doing when they start their career. Don’t worry about following a direct path, pay attention to who you are and what makes you happy, rather than what you ‘should’ do. Nobody knows you better than you. The very best scientists approach their work as play and enjoy what they do.
Do you have a question for Andy?
Write it in the comments box below and Andy will get back to you. (Remember, researchers are very busy people, so you may have to wait a few days.)