Are there ‘rules’ for conveying emotion through art?
While art and science are often separated in academia, there is a lot to be learnt by considering them together. Dr Dirk Bernhardt-Walther and Dr Claudia Damiano, at the University of Toronto in Canada, and Dr Pinaki Gayen, at Visva Bharati University in India, are combining their expertise in cognitive neuroscience and visual art to investigate patterns in how abstract art conveys specific emotions.
Talk like a visual cognition researcher
Abstract art — a loosely-defined field of art that does not attempt to represent physical reality, but rather aims to convey emotions and effects through shapes, colours and other properties
Angularity — the degree to which something has sharp points or corners
Cognitive — relating to conscious brain activity, such as thinking or remembering
Contour — lines that typically form an outline of a shape
Neuroscience — the study of the structure and function of the brain and nervous system
Orientation — the position and direction of something (e.g., a line)
Quantify — to measure the number or amount of something
To what extent can art be used to communicate specific emotions? While we all know that art evokes emotion, it is less well-understood whether there are themes and patterns in this effect. For instance, do certain colours or symbols always lead to a certain emotion in the viewer? Or are their effects unpredictable?
To investigate these questions, a team of researchers from across the world combined their skills and expertise to perform some intriguing experiments. “We explored how people use associations with colours and line properties to communicate emotions,” says Dr Dirk Bernhardt-Walther, Associate Professor at the University of Toronto’s Department of Psychology. “In particular, we sought to measure which visual features are associated with each emotion.”
Culture and biology
“For thousands of years, art has been used to communicate the experiences and emotions of daily life,” says Dr Claudia Damiano, previously a Postdoctoral Researcher at GestaltReVision Lab, KU Leuven, in Belgium, and now Research Associate at Toronto’s Department of Psychology. “Through intuition and practice, artists know how to visually evoke certain emotions through their work.”
These associations between emotions and visual cues are probably partly cultural and partly biological in origin. “Some associations between colours, lines and emotions are grounded in the real world – such as a person’s face turning red when angry,” says Dr Pinaki Gayen, Assistant Professor of the Department of Design at Visva Bharati University in India. “Others, however, are cultural. For example, a ‘blue’ person is sad in English, but drunk in German!”
The experiment
The team recruited 46 arts students from the Ontario College of Art and Design University and 45 STEM students from the University of Toronto to take part in the experiment. The students were asked to draw six emotions in abstract form: anger, disgust, fear, joy, sadness and wonder. “First, participants used only drawing pencils, to emphasise line properties; then, they used pastel crayons to emphasise the effects of colour,” says Claudia.
All drawings were scanned, and the distribution of colours was analysed via counting the number of pixels with a particular range of hues. “For the line drawings, we traced the contours with custom-made computer vision algorithms,” says Dirk. “We then measured the lengths of contours, their angularity, and the distribution of orientations.” These methods are a systematic way of quantifying something that might seem unquantifiable.
Guessing games
The next step involved seeing whether a computer could accurately predict the intended emotion from the drawings. “For each drawing, the computer compared it to a set of ‘reference’ drawings that was created by averaging all other drawings in each emotion category,” says Claudia. “We wanted to see whether there were enough consistent features in each category for the computer to be able to accurately categorise any one image.” The computer was instructed to make its guesses based on the properties the researchers had measured. For line drawings, the computer was able to correctly guess the emotion depicted in 25% of cases. When it came to colour drawings, 51% of predictions were correct for drawings by STEM students, but only 39% for drawings by arts students. If the computer had only guessed randomly, it would have had an accuracy of 17%.
The team then got people involved, recruiting 244 psychology students in Belgium to guess the emotions from the drawings. “The results were similar to the computational analysis,” says Dirk. “Emotions guessed from line drawings had a 32% accuracy, colour drawings by STEM students a 43% accuracy, and colour drawings by arts students a 40% accuracy.” In both cases, guessers were more accurate for drawings by STEM students. “Our best guess is that artists tend to aim to break cultural conventions, while non-artists more consistently adhere to them,” says Claudia.
Interestingly, people and the computer made similar mistakes, such as mixing up joy and wonder. “This suggests that people use the same features as the algorithm,” says Pinaki. “Therefore, people do use colours and line properties to make inferences about emotions from abstract artworks.”
From abstract to reality
Reference
https://doi.org/10.33424/FUTURUM428
B) One example of a set of colour reference drawings used in the computational emotion prediction procedure with colour drawings (top) and line drawings (bottom).
While the effects of art are difficult to measure, it is undoubtedly a core part of the human experience. “This research can help us uncover the emotional language of abstract art, which could lead to exciting collaborations and practical applications,” says Dirk. “For instance, it could aid communication efforts by utilising colours and shapes to evoke specific emotions, such as yellow to express joy, or red to inspire action against injustice.”
Other potential uses include for cinematographers and make-up artists to enhance the conveyance of emotions on screen and stage, or for interior designers and architects to create environments designed to evoke a specific emotion. “This could involve creating calming spaces in airports, fostering wonder in educational settings, or promoting joy in retirement communities,” suggests Pinaki.
Dr Dirk Bernhardt-Walther
Associate Professor, Department of Psychology, University of Toronto, Canada
Fields of research: Perception, Cognition, Cognitive Neuroscience, Visual Aesthetics
Dr Claudia Damiano
Research Associate, Department of Psychology, University of Toronto, Canada
Previously: Postdoctoral Researcher, GestaltReVision Lab, KU Leuven, Belgium
Fields of research: Visual Cognition, Perception, Visual Aesthetics
Dr Pinaki Gayen
Assistant Professor, Department of Design, Visva Bharati
University, India
Field of research: Visual Aesthetics
Research project: Using cognitive experiments to test how art and artists evoke specific emotions through lines and colour
Funders: Shastri Indo-Canadian Institute (SICI), Natural Sciences and Engineering Research Council of Canada (NSERC), Social Sciences and Humanities Research Council of Canada (SSHRC), Flemish Government
About cognitive neuroscience
For centuries, people have asked how it is possible that the physical matter of our minds can be the conduit for the full human experience. Cognitive neuroscience attempts to answer this question, through the study of the biological processes behind human cognition. The field combines biology and psychology to understand the roots of how people think and behave. Dirk explains more about the field:
“Cognitive neuroscience will undoubtedly remain an active area of research well into the future. The quest to try to understand how the brain’s neural activity results in thoughts and consciousness is still far from fully solved.
“Artists are often the pioneers of insights into how the human visual system works. Around 50,000 years ago, artists discovered how to elicit the impression of a pig by drawing its mere outlines on a cave wall in Indonesia. The Italian architect Filippo Brunelleschi discovered the rules of linear perspective long before scientists thought about it. Still today, artists inspire research on visual perception through their intuitive grasp on how to depict things – be they people, places, emotions, moods, and so on. Scientists and artists can form powerful teams to explore the visual processes underlying these depictions.
“Our knowledge about the brain is evolving quickly. It’s likely that some of what you learn about the brain in school will be outdated by the time you do research. However, the skills learnt from subjects such as mathematics and physics are everlasting and applicable to many different fields.
“Most researchers love to talk about their work. If you find an area of research that interests you, try to reach out to the scientists that study it to learn more!”
Pathway from school to cognitive neuroscience
Though biology and psychology may form the scientific basis for cognitive neuroscience, the team also recommends taking mathematics, physics, philosophy and computer programming to gain a strong foundation for a career in neuroscience. The team says that while our understanding of the brain quickly becomes outdated, the fundamental skills learnt through these other subjects are universally applicable through all scientific disciplines.
Explore careers in cognitive neuroscience
• The Collaborative Program in Neuroscience (CPIN) at the University of Toronto has various public outreach programmes, including NeuroSci 101, an annual lecture series for high school students.
• Follow societies such as The Schwartz Reisman Institute for Technology and Society and the Digital Society Institute, which give an insight into how cognitive neuroscience can be applied to a wide range of emerging fields to help improve society. Signing up to their newsletters is a good way of staying informed.
• The team recommends the ‘Art in Conversation’ seminar series from the University of St Andrews, which explores how art relates to other disciplines such as neuroscience.
• According to Talent.com, the average cognitive neuroscientist salary in Canada is around CA$90,000.
Meet Dirk
When I started out, the field of neuroscience did not exist as it does today. I was fascinated by the physical processes that underlie information processing, so studied physics and computer science initially. For my PhD, I switched to information processing in biological systems, and have been fascinated by how the brain interprets sensory information ever since.
My PhD in computational and neural systems was a formational time. I experienced collaborations between biology, engineering and social sciences with a playful attitude. This interdisciplinary spirit and playfulness still shape my approach to research to this day.
A single-minded approach to work has never worked for me. I thrive by having fun side projects, sometimes against my advisors’ advice. For instance, I have collaborated with people from a huge range of disciplines to develop algorithms for detecting animals in underwater videos, exploring how crooked teeth change the perception of faces, and monitoring moisture transport in laminated wood. This cross-disciplinary work often yields unexpected benefits, sometimes years later.
It is very gratifying to inspire young, budding minds. I have been proud to see four PhD students graduate and succeed with my support and guidance to help them become scientists in their own right.
I want to tackle seemingly fuzzy problems with rigorous science. When we struggle to clearly define a concept, we cannot understand its causes and mechanisms. This includes topics such as attention, the perception of complex real-world scenes, and the concept of beauty. Undertaking the intellectual work to study these is appealing to me.
Dirk’s top tip
Take all the mathematics classes you can! Maths helps me in my work all the time, even if it doesn’t take the limelight. Maths trains your brain to think clearly and methodically.
Meet Claudia
As a child, I was fascinated by human behaviour. I wanted to understand why people do what they do, say what they say, think what they think. That inspired me to pursue a career in psychology.
Collaborations with other researchers have shaped my career. Being able to discuss philosophical and scientific questions with brilliant and kind people is such a privilege. Learning that everyone experiences self-doubt about their research, career path and life decisions was also an epiphany for me.
When I started my PhD, I had a false impression that the journey had to be difficult to be meaningful. The idea was that it had to entail long hours and sleepless nights to be truly ‘earned’. In reality, conducting research is a job like any other – it’s a part of your life, not your whole life.
My proudest career achievement has been obtaining a Marie Skłodowska-Curie Postdoctoral Fellowship from the European Commission. In the future, I hope to continue doing research in psychology and cognitive science, and hopefully run my own lab one day.
Claudia’s top tip
Explore different topics until you find one that truly interests you! The rest – taking relevant classes, gaining research experience, applying to PhD programmes – will come naturally if driven by interest.
Meet Pinaki
In my early years, I used to watch my brother draw animals, people and landscapes. Sometimes, I would visit my artist neighbours, one of whom demonstrated to me how to paint with watercolour. That experience impacted my thinking profoundly. I kept thinking about how he had painted, and I dreamt again and again of being able to do the same.
At school, an art teacher taught me the basics of drawing and painting. I learnt about the great painters of the world – Picasso, van Gogh, Michelangelo, da Vinci, and many others. I started experimenting with my drawings and paintings to develop my own styles.
When I told my family of my intentions to enter art college, one relative suggested I should go for history or geography instead. Fortunately, I did not listen to him, and, later on, he understood that he was wrong!
I’ve been fortunate that many of my dreams to see landmark pieces of art have become reality. In recent years, I have visited the Louvre in France and seen Picasso’s Guernica in Spain.
I want to be one of the greatest artists of all time. This has always been my ambition, and I continue to work towards it.
Pinaki’s top tip
Dream big. Don’t compromise or limit the size of your dreams. The great artist Michelangelo said, “The greatest danger for most of us is not that our aim is too high and we miss it, but that it is too low and we reach it.”
Do you have a question for Dirk, Claudia or Pinaki?
Write it in the comments box below and Dirk, Claudia or Pinaki will get back to you. (Remember, researchers are very busy people, so you may have to wait a few days.)