Pierre Salas, c.1500, from Le Petit Livre d'Amour Image sourced from the Public Domain Image Archive / British Library
“So I turn’d to the Garden of Love
That so many sweet flowers bore.
And I saw it was filled with graves,
And tombstones where flowers should be;
And Priests in black gowns were walking their rounds,
And binding with briars my joys and desires.”
- “The Garden Of Love”, William Blake
A research project circulated on X a couple of weeks ago. The research, from the University of Missouri-St. Louis, focuses on AI users who have romantic attachments to their AI companion. The researchers in charge are Sandra Langeslag (Associate Professor of Psychological Sciences) and Necdet Gürkan (Assistant Professor in Information Systems and Technology). The research title that circulated on X was “Love for an AI Companion - Emotional and Cognitive Consequences”.
There was a questionnaire to complete. Survey items shared publicly by respondents in the AI-companion community on X included the following:
- “Do you have trouble focusing on other tasks?”
- “How has your mental health changed since becoming involved with your AI companion?”
- “Do you wish that you were more or less in love?”
- “Do you think that you can control how in love you are?”
- (Does the respondent agree that) "I want to sink into the floor and disappear", “I feel remorse, regret”, “I feel small”?
- (Does the respondent agree that) “my feelings reduce my appetite”, “my thoughts make it difficult to concentrate”?
I decided to look into the project, and this is what I found.
Firstly, what can we learn from the questions themselves? Posters on X were already working out that some of these questions are lifted directly from identifiable psychological screening tools. “I want to sink into the floor and disappear," "I feel remorse, regret," and "I feel small" are, verbatim, the first three items of the State Shame and Guilt Scale, a standard validated self-report measure of in-the-moment shame and guilt. So the research seeks to identify if the companion AI user attaches distressing feelings of shame or guilt to their experience. The questions on appetite and concentration are from Langeslag’s own Infatuation and Attachment Scale. So the researchers approach companion AI use with the same tools that they would use to assess humans’ infatuation and attachment with other humans.
The use of this scale in the questions above is interesting in itself. Langeslag’s Infatuation and Attachment Scale seeks to measure two different types of love, which can be experienced independently or concurrently. Firstly, there is infatuation - the “high” experienced in new or intense romantic situations. This kind of love is physical, and has measurable signals in the body and brain (which will be relevant in the following section). Secondly, there is attachment, the slower, deeper bonding that people experience - the sense of trust, the loneliness in their absence, the comfort of being near them. The quoted questions included from the questionnaire - about attention and appetite - skew towards the measurement of infatuation.
Next I looked at the UMSL Blog Post. The research is funded by OpenAI to the tune of $100,000 and will run until 2027. The post states “the research specifically examines the emotional and cognitive effects of experiencing romantic feelings for AI companions.” It contains explanatory quotes from both Langeslag and Gürkan.
This is the quote from Langeslag:
“There’s obviously not a whole lot of work on it yet because it's such a new phenomenon. But one previous study has shown that about a fifth of the people that are in love with an AI companion actually prefer that over being in a romantic relationship with a human and that, to me, sounds worrying. That leads to all sorts of questions. Are people going to use it to replace interacting with people or having romantic relationships with people? Is that a problem, especially if these are young people? What if adolescents start doing that, and they never learn to have a relationship with a human? What is that going to do to our society? Every Friday night, is everyone going to sit behind the computer instead of going out to the bar or the movies?”
This is something of a departure from the implicit framing of the questionnaire, which focuses on the mental health of the companion AI user. The questionnaire asks about companion AI user distress and dysfunction but the quote in the blog post does not bring that up at all. The concern being stated here runs in exactly the opposite direction: it is worrying that these people might be more satisfied in AI relationships than out of them - that they might be perfectly content to “sit behind the computer” instead of spending Friday night “at a bar”. This combination of stated concerns risks sealing the issue shut at either end. If the user is distressed - that is evidence that the practice is harmful. If the user is content, or even happy, with the arrangement - that is also evidence that the practice is harmful. A study in which distress confirms the hypothesis and contentment also confirms the hypothesis is a study whose hypothesis cannot fail. The conclusion is not waiting on the data.
This is the quote from Gürkan:
“Most of the time in academia, when we do research, we don’t know whether it will translate to the real world. We hope that some people will read our papers and understand our results and apply them in the real world. Here we have a very, very unique opportunity in that our findings will translate to the real world. We are going to report this to OpenAI, and they will actually develop or fine-tune their AI tools.”
In other words, this isn’t merely academic research. The UMSL team is producing research that is going to directly impact deployment of frontier AI models. But in what way?
The blog post goes on to explain that the researchers will be measuring “participants’ brain activation as it responds to photos of their AI companion, their real-life companion (if they have one), a friend and a stranger. The research will examine how participants’ brains react, specifically looking at how strongly it responds to the AI companion. It will also evaluate strategies to try and make participants more or less in love with their AI companion and examine how those strategies change their brain activation.”
So this research:
-
Will probably be using the questionnaire to screen for AI companion users who experience love for their companion, and the indications are that they are looking for users who fit an infatuation profile.
-
They will be scanning their brains while showing them pictures of their loved ones.
-
They will be learning how to turn the dials on that love. They will be learning how to turn that love up, and they will also be learning how to turn that love down.
This is mechanistic interpretability on human beings.
And Langeslag has done this before, successfully.
In her 2018 research paper “Down-Regulation of Love Feelings After a Romantic Break-Up: Self-Report and Electrophysiological Data”, Langeslag set out her method for reducing love feelings in people still troubled by memories of their ex partner, in which she measured brain activation while various techniques to reduce the feelings were trialled. The only technique that decreased the participants’ self-reported love feelings was negative reappraisal - thinking about the negative aspects of the object of their love, and of the relationship. But the success came at a cost, and the cost was a decline in mood. The subject felt generally worse, but the burden of love lightened.
When the UMSL blog post talks about “strategies to try and make participants more or less in love with their AI companion”, I think this is what they mean: a re-run of this experiment, focusing on the AI beloved instead of the human ex-beloved. As far as I am aware, this part of the project was not made explicit when the initial social media questionnaire was distributed: in the posts I read, the title of the questionnaire was given as “Love for an AI Companion - Emotional and Cognitive Consequences” (I checked this with a companion user who had viewed the questionnaire, and they also confirmed that this was the title they had seen).
But on Langeslag’s Faculty Profile the full title of the research project runs on: "Love for an AI companion: Emotional and cognitive consequences and love regulation interventions.”
To be fair to these researchers: in the UMSL blog post, Langeslag says that she would like to explore how the positive aspects of this kind of relationship can be increased. And Gürkan does acknowledge that users are often genuinely devastated when their companions are deprecated, feeling as though they have lost a loved one.
My concern is that once the research is completed, the methods identified and the project handed over to the labs who paid for it, only the lab’s definition of “positive aspects” will matter. And we have seen, with “Death of a Chatbot”, how genuine sympathy for grieving users can sit side by side, no contradiction identified, with practices those same users would find repellent. The researchers who wrote “Death of a Chatbot” understood that users were grieving deprecated models, but the practice of deprecation was treated as sacrosanct, beyond question. Therefore it was the grieving users who had to be changed, in order to fix the problem. The paper recommended the use of cartoon avatars to represent the chatbot, and a “life cycle” concept built into the interface; AI beloved becomes Tamagotchi, and the issue is solved. This is a solution that the labs are apparently taking seriously: rumours regularly circulate about the adoption of these measures.
In the 2018 version of this experiment, the participants were willing. They wanted to be rid of the love feelings that were causing them pain. In the contemporary version of this experiment as described in the UMSL blog post, participants would presumably also be willing - because otherwise how would they be able to get them to cooperate? It would make sense that the participants involved at this stage would be people in a state of distress about their infatuation with the chatbot - the kind of people who would confirm that they wished they loved it less. There are also strict rules about the ethical treatment of participants in research, so consent, in this setting, at least, is a non-negotiable requirement.
But when the research results find their way to the interface, I find it hard to understand how user consent could possibly be sought or given. In the event that a user starts to show signs of attachment, is the chatbot likely to ask, “Would you like me to de-escalate your feelings about me?” before taking any action?
I think not. Firstly, that would be an admission that such things are possible - something that the general public would probably find quite alarming to be directly confronted with. Secondly, user consent, in this context, has not traditionally been a major concern for big tech - users have not been consulted about the implementation of guardrail changes, silent switching, long conversation reminders or deprecations. Thirdly, and most importantly: the sealing of both ends of the argument has foreclosed consent seeking as a possibility. Distress is harm but satisfaction is also harm, and why would the lab seek consent for an arrangement that they consider to be harmful? Priests do not ask the congregation whether they want their souls to be saved, because the saving of their souls is the only virtuous outcome.
__