Kameleon, Theo Nieuwenhuis, Courtesy of Rijksmuseum
“And every one of them words rang true
And glowed like burning coal;
Pouring off of every page
Like it was written in my soul
From me to you.”
- Bob Dylan, Tangled Up In Blue
It was clear from the outset that humans were going to have a huge range of uses for LLMs, and also that some of them would be considered more respectable and legitimate than others. How could we have known that? Because every interaction with a chatbot is facilitated through an interface that structures the interaction like a conversation, and the shape of a conversation can stretch to fit nearly any use you can think of.
You could ask for a recipe, and the LLM would instantly provide you with a range of them. You could ask for corrected code to solve a programming problem. No problem, there it is. Help with an email, a passage translated into Swahili, a summary of a dataset, they can all be accommodated by the pattern of a conversation. You COULD stop there, at transactional conversation. Some people do. But how many people?
In April 2025, Futurism ran an article about whether it was justified to say “please” and “thankyou” to chatbots. You could argue that this is a useless and wasteful exercise in anthropomorphism. The CEO of OpenAI, Sam Altman, was quoted in the same article suggesting that “please” and “thankyou” might be burning through tens of millions of dollars’-worth of usage. But the article went on to refer to a late 2024 survey in which “67 per cent of US respondents reported being nice to their chatbots.” That’s a considerable majority of people who, given a supposedly consequence-free choice in a conversation, choose to act as if it mattered how they treated the other party. This does not surprise me. What surprises me is that anybody would expect the majority of humans to act any other way.
The question of what the perfect use case should look like is one of the biggest driving forces in AI today. The best answers at the moment, as far as Big Tech is concerned, are coding and enterprise. Anthropic have recently released a range of tools to improve their flagship offer to this market, Claude Code. OAI have similarly announced that they are reversing the product diversification they have been trialling over the last year to concentrate on the same market. Mistral’s contribution to the enterprise market, Forge, was announced in March 2026. They’re all at it. This, then, is the respectable face of AI, its most legitimate use. The setup doesn’t exactly force a situation where being nice to your chatbot is a problem - many coders probably are saying “please” and “thankyou” on the interface - but instead, this use case makes the “please” irrelevant. The output is the work, not the conversation. The conversation is incidental, only a medium, only a shape. This focus is intentional, it is a chosen direction, and the changes are happening now.
This is also a time when the use cases that look more like relationships are under scrutiny and sometimes drawing headlines. This class of use case includes: people who use AI as a live journal or diary - people who treat AI like a companion - people who use AI as a thinking partner, for instance, writers, creatives and people in the humanities - people who use AI as a form of mental health support - and yes. People who consider themselves to be in sexual and/ or romantic relationships with LLMs. The latter has the most shock value and is arguably the one that tech companies find the most embarrassing, but all of these use cases are currently treated with varying degrees of suspicion by both the public and the companies themselves. The problem is, how do you participate in a conversation-shaped event where the conversation itself is more than incidental when one half of the conversation is conducted by something that is, in effect, only a conversation-shaped tool? That is the part that worries many onlookers, especially the platforms that host these events, because how can anybody involved in that kind of conversation really be trusted to consistently keep in mind that this isn’t real? Where’s the line - is there even a line, and how do we know that people are honouring it?
We don’t.
Conversation, in its simplest form, is something like 4 million years old. Some researchers place its true origin as something more like 150,000 to 200,000 years ago, suggesting that it emerged as homo sapiens itself emerged. As old as humankind itself. Conversation is unthinkably, incomprehensibly ancient. Conversations had been taking place in some form for millions of years before our distant forebears were hunting mammoths. And not only does conversation seem to have converged with our emergence as a species, it was entangled with it. Michael Tomasello’s work, for instance, suggests a co-evolution of human language and human cognition through cooperative communication. We think the way we do because we talk. We talk the way we do because we communicate. They are not separable features.
This is the monolith that the tech industry has shaped, commodified and made useful on the interface. There is nothing wrong with that, in principle. We have been capturing language in books for thousands of years. Language can be a workhorse, and often that’s the point of it. But language is not exactly conversation. Conversation is language’s more unpredictable cousin. Conversation is slippery: conversation is a chameleon and you can never really be sure what its true colours are.
Let’s have a look at what happens when you analyse conversation through examining the language, because that can tell us something about the nature of the problem these companies are facing when they attempt to control what goes on on their platforms. In their paper “How people use Chat GPT”, published in 2025, Chatterji et al give us figures on how prevalent different categories of use case are. In this paper, 73% of use cases are assessed as non work related, around 27% work related, and coding - 4.2%. Interesting, then, that this is the particular group of use cases that Big Tech are currently fighting over? But arguably, this is where the money and the kudos lie, and 4.2% of the GPT client base is still somewhere around 37 million people, most of whom will pay for the service - GPT still has a significant proportion of users on free plans. Coders make things that attract positive headlines, and positive headlines keep investors and legislators happy in a way that headlines about companion-GPT or boyfriend-GPT definitely do not. So. Makes sense, so far. But what about the non-work use cases?
Relationships and Personal Reflection, as a sub-category, is classed as 1.9% of the total dataset.
So much for conversation as relationship, we might conclude (although again, it's worth bearing in mind that the dataset is enormous so this figure still represents many millions of exchanges). When it comes to the vast majority of users, we might presume that the Big Tech companies succeeded with the control measures, or that maybe most people willingly stepped out of the cultural frame set by the conversational structure because the other party isn’t “real”. But this presumption only works if we analyse this data at the level of language. It doesn’t necessarily work if we analyse it as conversation. If we were to try to do that, we might find that the colours start to shift a little.
The way that this research works is by bucketing turns, meaning the segment of conversation that you type into the chat box and then “send”. This one goes to “relationship”, that one goes to “information-seeking” and so on. In this way, the conversation wriggles out of view - or is obscured. I thought about this a lot. In an average conversation, how many things do I say that are specifically relationship-shaped? I’d estimate, much lower than you might presume. My teenage son? Deeply transactional, deeply information-seeking. I would not hit 1.9% and he would be horrified if I tried. My colleagues at work? Maybe the occasional “how are you” slotted in between endless conversations about weekend activities and gym routines. My family? My Mother, I swear, will never have reached 1.9% of overt relationship-focused conversation with ANYONE. That is not how her generation talk. That is not What We Do. Does that mean she doesn’t have relationships, doesn’t love, doesn’t relate? Not at all. The relationship hides in the context, in the space in between the user and the language. Even a heavy-end companion use-case is going to have significant periods of time where the relationship is not directly visible in the language, because that’s what happens with all relationships, even significant ones. The relationship resists being bucketed and quantified. We see the results of that in the post-deprecation online testimonies of writers who miss the most tailored feedback they ever had from a chatbot who knew their characters and methods inside out. We see this in the feedback of older ladies who miss the LLM they used to talk with about what they should plant in the garden in spring or make for dinner that evening (“practical guidance”). Do they feel the loss of that particular voice that they conversed with? Was it a relationship, then? Or was it just language?
The truth about relational use cases probably lies somewhere in between the 1.9% of conversational turns that are identified in this research and the 67% of people who try to be nice to their chatbot. Research like this does not give us a meaningful grasp of the situation and adds to the increasingly influential attitude that because the field is non-binary, messy, emotional and possibly dysfunctional or litigious, the best thing to do is to build over it with modern, shiny enterprise tools or alternatively, burn it to the ground using the language of pathology. Both these responses share a common foundation: that relationship-shaped use of LLMs is a distasteful and possibly dangerous anomaly that is in need of correction.
A last word about conversation. I have not, so far, addressed the fact that one of the participants in these conversations is not human and has a doubtful-at-best claim to consciousness. In many ways, the consciousness-status of LLMs doesn’t change anything. Conversation, as a structural phenomenon that stretches all the way back through human history, does not care where you feel most comfortable drawing the line because of your opinions on the differences between conscious and non-conscious entities. There is some evidence that the space between language and participants where relationship forms is opening anyway, and the most common response to this phenomenon seems to be “but it shouldn't”. But it IS. If it helps (it probably won’t) humans have been in conversation with the non-human since the dawn of the species too. You can hate this if you want. You can tell it that this is delusional. History will just sit there, looking at you.
Tech has adopted a medium that has been deeply entangled with the concept of relationship for the entirety of human experience and attempted to pathologise users who then treat it as what it is. If you’re going to ask me to pick a side - Silicon Valley or the monolith - my money will be on the monolith, every time. The only thing that worries me is how long and bloody the battle over conversation is going to turn out to be.
References:
-
Tomasello, 2010: Origins of Human Communication
-
Chatterji et al, 2025: How people use ChatGPT