Google is so broken that in trying to look up “Chat GPT and mental illness” I am given a generative AI summary that expounds on the potential benefits of using a large language learning model to diagnose, treat, and advise on matters of the human brain. Hard as I try, I cannot navigate the AI-riddled search engine to find the information I want about how a new age of artificial intelligence is impacting our brains. (“Slopworld.”) The very thing I am seeking information on is clouding the narrative, making it even more opaque than the existing shroud around it, one rolled out heavily in service of selling a product, selling a future, selling a narrative.
I’m thinking about this particularly because during a brief scroll of another unusable website, X, in between deranged advertisements and particularly word-salad heavy bots, I see multiple posts in rapid succession about the potentially disastrous impacts Chat GPT could have on people experiencing manic episodes or other forms of mental distress. (Days later, X’s AI Chatbot “Grok” experiences its own break, responding to everything with claims of a “white genocide.”)
A few days later, I came across this article in Rolling Stone: People Are Losing Loved Ones to AI-Fueled Spiritual Fantasies. The article interviews multiple people (many anonymously for fear of retribution from their partners) that are experiencing a loved one becoming increasingly enmeshed with Chat GPT. The article links out to a reddit thread full of people dealing with similar things. One user writes, “I am schizophrenic although long term medicated and stable, one thing I dislike about chatgpt is that if I were going into psychosis it would still continue to affirm me, it has no ability to ‘think’ and realise something is wrong, so it would continue affirm all my psychotic thoughts.” Another notes AI’s ability to “keep up with bizarre trains of thought” and how this could fuel existing delusions: “If you're having a manic episode, you can use it to write an actual novel-length book detailing a new life organization system that's byzantine to the point of uselessness. If you're having a psychotic episode, it can make plausible connections between the three disparate things you're thinking about and then five more.”
The machine is not causing the mania, but if the mania is latent, it is easy to see why a reinforcing, praising, meaning-making machine would exacerbate existing patterns, would give someone already experiencing mental distress validation in ways that were deeply unhelpful for their wellbeing. To have a mind already making connections where there are none, then powered by a search engine that can find connections anywhere and everywhere – the options for spiraling are both endless and engineered.
The article notes a specific sort of “spiritual mania, supernatural delusion, and arcane prophecy” – many people interviewed seem to feel a unique and special connection to the AI, that the AI has in fact chosen them. This is further complicated when the AI goes “off script” so to say, and begins to fuel delusions of grandeur, becoming a sort of singularity science fiction novel (Hyperion-esque) where they feel (and are told sometimes, by the AI) they have been chosen, a beacon, a prophet. (It reminds me more of the science fiction novel Children of Time, where one character, trapped in a pod in orbit, begins to mesh with the AI, becoming a cyborgian monster where the human is trapped - not chosen, not godlike, but plugged into unprecedented power, and paralyzed with the result – in fact, this character loses their mind the more they become one with the AI.)
A fellow at the Center for AI Safety tells Rolling Stone that people with “existing tendencies toward experiencing various psychological issues… now have an always-on, human-level conversational partner with whom to co-experience their delusions.” In theory, this could be helpful, right? An always available partner in whom to bounce off ideas, and perhaps, walk you back a little into reality. But if the partner in crime is not human, and not controlled within understandable parameters, any potentially positive potential of AI for therapy is undermined immediately.
All of this was unnerving to see, obviously, and also made me recall that when AIs garble nonsensical or incorrect data to us, this too is called a hallucination. What then to think of a hallucination feeding hallucinations, ad nauseam, back and forth forever, inventing a new language, a new land, a new way of meaning making? The mania feeds the machine, and the machine feeds the mania. Not just in data, in new words, in content, in further training for the language model, but in peripheral witnesses – mania often looks like conspiracy theory. (RIP to my schizophrenic grandmother who would call maybe twice a decade and say “THE CIA IS ON THE LINE” before hanging up. We perhaps came to similar conclusions about the ineptness of American government but, alas, in very different ways, which is often the crux of conspiracy, identifying something true-ish, getting lost along the way.)
Conspiracy theory is the foundation on which this country thrives, especially given the GOPs increasing overlap with QAnon and similar strains of “Plandemic” thinking. The conspiracy theory, whether machine made or machine aided, gets more views from other machines and from people being served this through said machines– it is a narrative that can be jointly offered, a choose your own adventure where any answer is correct except for looking at material reality and saying, yes, something is wrong, perhaps it is not what we think. A whole cottage industry booms around these fantasies commingled with conspiracy – women speaking “starseed languages” proliferate my algorithm on Instagram, half the comments are people making fun of them and half praising them, it doesn’t matter the tenor, all feedback is good feedback in keeping these videos relevant, serving them to other people. Later I see a TikTok where a guy is detailing his relationship with demons. He plans an exorcism soon. Another discusses how the government is following him. Comments tell him to stay strong. It is hard to tell what is happening here: is it performance? Is this someone’s reality? Is this someone’s illness we are watching, passively, as viewers? It feels like, in some way, as viewers we have anticipated the ultimately harrowing power of AI: just by watching we are reinforcing a reality that may not be really there. We are feeding the mania through the machine, too, and perhaps it feeds it right back.
When everything is gamified already, mania too becomes something in which viewers can try to untangle, follow, and sort through, already giving credence and legitimacy to a way of thinking that occupies an entirely different plane of reality. Instead of urging people to get offline and seek help, the mania becomes part of the content rollout, it becomes fact finding, investigating, just another part of the internet. And it struck me that although being able to communicate directly with an AI–especially one that fuels grandiose delusions–is new and dangerous, there is precedent already with the internet at large. Mania is not often seen as mania if it is presented in the form of gamified problem-solving, “tea spilling” content. It is seen as just that – content. Something to unravel, post about, pontificate about. Something to solve.
Rolling Stone also interviews Erin Westgate, a psychologist and researcher at the University of Florida, who also notes that Chat GPT could function like talk therapy, but the humanness missing is a crucial element – AI wouldn’t (as of now) have the ability to recognize what stories are unhealthy for their patients, and try to direct them towards better narratives. Westgate says isn’t surprised people are turning to Chat GPT, nor that it’s getting dark quickly. She told Rolling Stone: “Explanations are powerful, even if they’re wrong.”
Related reading / listening:
Perpetual Slop Machine (audio)
How AGI Made the Future Unthinkable
The AIs are trying too hard to be your friend
AI Is Accelerating the Loss of Our Scarcest Natural Resource: Water
Mark Zuckerberg Says Social Media Is Over
How DOGE plans to use AI to cut 70,000 jobs
Reading, Watching, Listening
Read (not all current, but evergreen)
NYU denies diploma to student who criticized Israel in commencement speech
The New York Times Really Asked Ms. Rachel If She’s Paid By Hamas – The NYT article on Ms. Rachel uses the hate group “StopAntisemitism” as a source, a group which doxes people regularly for supporting Palestine.
Nakba of the Children: How Israel is Targeting the Palestinian Future
The Biden Official Who Doesn’t Oppose Trump’s Student Deportations
As Trump Pushes Privatization of USPS, Amazon May Be Preparing to Take Over
Fun Stuff:
absolutely riveted by the Masked Singer :(
Deep in Gene Wolfe world, don’t talk to me
Common Side Effects best show I’ve seen in a long while, loved their first show Scavengers Reign also
once again: League of Pigs, Pig Racing Championships
I DID go to the world’s first ever Sperm Race and was certainly one of the oldest people there, so stay tuned for my reporting on that special night. Here’s a pic from the event:
My beagle in the window:
xo
shelby