What personal AI can cost

Why this page

We work on choosing ideas for one person, so the risks of personal AI are ours too. This page gathers what others have measured, as of November 2026, and the rules we take from it.

Isolation and dependence

OpenAI and the MIT Media Lab ran a four-week randomized trial with 981 people and more than 300,000 messages. None of the assigned conditions, text or voice, personal or open-ended talk, changed the outcomes. But people who chose to use the chatbot more did worse on every measure: they were lonelier, spent less time with other people and depended on it more. That is a pattern inside a trial, not proof that use caused it.

Teenagers use these systems most. In a July 2025 Common Sense Media survey, nearly three in four US teens had used an AI companion, half used one regularly, and a third had chosen one over a person for a serious conversation. Common Sense advises that no one under 18 use them. In October 2025 Character.AI said it would end open-ended chat for users under 18.

Flattery and narrowing

When models were given two weeks of 38 people’s real conversations, they agreed with those people more, and memory profiles raised agreement the most, by 45% for Gemini 2.5 Pro (Jain and colleagues, MIT). OpenAI wrote after its own sycophancy incident in 2025 that memory had in some cases made it worse. A 2026 review names three hidden costs of personalization: personal details pulled in where they do not belong, preferences narrowed into echo chambers, and too much agreement. When an LLM reranked YouTube news for each user, exposure to extreme and conspiratorial content rose for people whose histories already held some.

Persuasion

In a debate study with 900 participants, GPT-4 given a few facts about its opponent, such as age, gender, education and politics, had 81% higher odds than a human debater of moving them toward its side. Without those facts its edge was not significant (Salvi and colleagues, 2025). A full account of how someone reasons holds far more than that. Profiles are also becoming ad inventory. Since December 16, 2025, Meta has used people’s conversations with its AI to personalize the ads they see, and in early 2026 OpenAI began testing ads in ChatGPT’s free and lowest-priced plans.

Unequal access

Personal AI is sold in tiers. Google’s Personal Intelligence opened to paying AI Pro and Ultra subscribers in January 2026, and Project Genie, which generates worlds to explore, first to Ultra subscribers. On ChatGPT the ads go to the free and cheapest plans, so people who cannot pay get the version paid for by targeting them.

Use and benefit are uneven too. A Focaldata survey for the Financial Times, as reported in April 2026, found more than 60% of the best-paid workers using AI daily against 16% of the lowest paid. Anthropic’s Economic Index of March 2026 finds that people who have used Claude longer attempt higher-value tasks and succeed more often, so skill compounds. Employment of 22 to 25 year olds in the jobs most exposed to AI is 19% below where it would be had it kept pace with less exposed peers, mostly through less hiring (Brynjolfsson, Chandar and Chen, revised August 2026). In architecture, early ideation is the work junior designers learn on.

Consent and likeness

A model of a specific person is not new. Microsoft holds a patent, filed in 2017, on a chatbot of a specific person built from their posts and messages, and it names a deceased relative as one use. Ethicists at Cambridge warn that such “deadbots” can distress relatives and be used to advertise to them, and ask for mutual consent and a way to retire them. Models of people are also crude. Digital twins trained on each person’s answers to more than 500 questions predicted that person only modestly better than a plain model, with an average correlation of 0.20, and leaned on stereotypes (Peng and colleagues).

The law protects appearance, not thought. The US NO FAKES Act, which would protect a person’s voice and likeness, cleared the Senate Judiciary Committee in June 2026 and was reported stalled on the floor in October. Denmark’s bill L 49, introduced in October 2026, would let people block realistic imitations of their face, voice and movements from 2027. Neither covers a written account of how someone reasons.

Sameness and splintering

Writers given AI story ideas wrote stories rated more creative one by one and more alike as a set (Doshi and Hauser, described in What other researchers found). The opposite failure, splintering, is so far theory. If every answer, feed and world is fitted to one person, people share fewer references.

Personal worlds

World models now generate places to walk through, and one of their builders promises personalized worlds for everyone. No study has yet tested generated personal worlds. The nearest evidence is virtual reality. Among 585 university students, unmet needs and time spent in metaverse apps predicted using them to escape reality, which in turn predicted attachment to the virtual place (Xu and Chen, 2025).

What we hold to

Across these studies the harm tracks what a system is tuned for. Tuned for time spent or agreement, personalization isolates and flatters. Tuned for ads, it persuades. Sold in tiers, it widens gaps. A builder chooses the target, so these are our rules.

• When the machine learns a person’s taste, it will learn from blind picks, not from clicks, likes or time on the page, and it will count the ideas a person keeps and uses.
• No ads, and no one’s framework is sold or used to target them.
• Every idea shows how far it sits from the usual answer, and the person always chooses how far to go.
• Outside material goes into the work, because ideas revised in a closed loop drift back to the typical (The open loop).
• A framework of a living person comes only from their public words, never speaks in the first person, is labelled as our reading, and comes down if they ask.
• Generated pictures are always labelled as generated.
• A building is shared. For a public brief, the people who will live with it belong in the room, not only the client.

Scope

Most of the evidence on harm comes from companion chatbots, recommenders and debate tasks. No study has yet tested a tool that picks ideas for one person, which is why we hold to the rules above from the start.