A computer model that can predict the “memorability” of art

Is AI that predicts which paintings are memorable and famous, regardless of content, style, or context, a threat to creativity or tool for positive change?

What makes a painting memorable? Is it how it resonates with you personally? Does it relate to how the work makes you feel, its intrinsic beauty, the boldness of the colors, or the subject it depicts?

The answer, according to a new computer model that predicts the memorability of images, is none of the above. In a recent study, the system found that memory for artwork is predictable, regardless of the content, color, or cultural significance, and no matter who you are or whether you see the art online or in a gallery. It could even predict which paintings were most famous by analyzing images with no additional historical or cultural context.

If quantifiable features of a painting dictate its longevity in our minds, our assumptions about the subjectivity of art may need reassessing. It also raises questions of whether the ultimate “memorable” artwork can be engineered — something that would tempt both advertisers and artists clambering for attention in the dense online landscape.

But while this growing understanding about what makes paintings memorable adds a layer of intrigue to how we experience images, memorability is just one facet of art appreciation. And beyond the art world, the findings could play a role in the development of effective educational materials, and even new approaches to early diagnosis of Alzheimer’s disease.

Memory prediction

How can a computer model account for the fact that your memory of an image might be affected by when and where you see it?

Wilma Bainbridge, assistant professor of psychology at the University of Chicago, and her team devised a series of experiments to assess memorability of artworks found in the Art Institute of Chicago, both through an online activity and after a stroll around the gallery.

“Online, people were engaged in a memory experiment on a computer screen with controlled timing and key press responses,” said Bainbridge. “In person, people had a pretty normal visit to a museum and then indicated their memory later.”

Factors such as image size and placement in the gallery impacted memorability, and even though the tasks participants completed were different, the model, called ResMem, successfully predicted which images would be memorable in both cases.

ResMem is a neural network developed to predict people’s memory for images in a lab setting. “It is basically built off of a model similar to the human visual system, though not with anything like human memory built in!” Bainbridge explained. “It has artificial ‘neurons’ and it learns a mapping between an image and its memorability score through us showing it tens of thousands of examples. It can then apply these mappings to new images it’s never seen before.”

The model is available for anyone to use (provided the work is non-profit), and researchers are exploring what makes different images memorable in various situations.

What determines memorability?

The study showed that beauty, color, content, and other superficial features do not dictate memorability. It is not yet clear, however, what does make an image memorable. Despite the model’s success in predicting the results of human trials, it cannot explain what factors it is looking for.

Does this mystery frustrate the researchers? “It’s still an open question we are working on testing,” said Bainbridge. “I’m not frustrated at all not knowing the answer yet because it presents all sorts of exciting things to test!” 

The team does have some ideas, however. “We think memorability is tapping into something richer than just a combination of features you can measure about a painting. We find this for other types of images too – memorability isn’t well predicted by similar types of features for faces, or for real-world objects,” she said.

“We can’t remember everything that we see. So we think ‘memorability’ might be a measure tapping into how our brain prioritizes what should be remembered of the things we see,” Bainbridge continued. “We think this prioritization could be related to something like how easy an image is to process for the brain.”

A threat to the creative world?

Introducing this sort of quantifiable assessment into the art world could put creativity at risk, encouraging artists down an ever-narrowing funnel of creation until all the art in galleries becomes a homogenised stream of unforgettable blandness. 

“I think these results bring up a really interesting question about changes happening in the art landscape,” said Bainbridge. “It seems to me there is an increasing ‘Instagrammification’ of artwork and museums, and this sort of technology would be appealing to those applications.”

Memorability is interesting, and could add a layer to the work of artists and curators, but, as Bainbridge said, art is so much more: “Art is something more than just what images stick in our memories — for example, we found that memorability was not related to beauty or emotion. So, perhaps your goal is not always for a piece to be the most memorable, but something else, like to be the most emotionally evocative.”

The technology could appeal to those outside the art world, as well. “I do worry about the advertising or marketing directions, which is why we prevent for-profit use of ResMem – for example, making an advertisement that is unforgettable,” said Bainbridge. “Luckily, we’ve found in recent work that even if something is very memorable, it doesn’t necessarily influence your decisions, which mitigates my concerns.”

ResMem’s use in the wrong hands are counterbalanced by hope for its positive potential. “I really think the potential positive impacts for Alzheimer’s Disease diagnosis and education are right around the corner,” said Bainbridge. “We are actively working now on making systems that can identify which images are the best at diagnosing early dementia. And, we are also finding that ResMem can significantly predict kids’ memory as young as four years, meaning it could help play a role in the design of learning materials.”

Memorable teaching aids and early diagnosis of mental health conditions would take the technology on an impactful journey from art to application. But Bainbridge and her team aren’t done with the art world yet.

“Our next step is that we plan on running an art contest to challenge artists to create the most memorable piece, or the most forgettable piece. We are curious to see what artists pick up on in terms of what makes something memorable or forgettable. We then plan on holding an entirely memorable exhibit and an entirely forgettable exhibit and see how these paintings influence visitors’ memories.” 

Details of the competition for any artists — or scientists — up to the challenge will be on the lab website in the coming weeks, just don’t forget to check back.

Reference: Trent M. Davis, Wilma A. Bainbridge, Memory for artwork is predictable, PNAS (2023). DOI: 10.1073/pnas.2302389120

Feature image credit: Jessica Pamp on Unsplash