Weights, memory or a page

The claim

On July 10, 2026, Thinking Machines published “The Future Worth Building Is Human”. Its argument about personalization is short. Today each lab trains its next flagship model with the previous one generating training data and a reward signal, so each generation is “raised on its parent’s outputs and judged by its parent’s tastes”, and “a single alignment spec suppresses creativity and diversity.” Its remedy: “For organizations and individuals to align AI to their own values, these values must be encoded in the model weights. If the user’s values and desires only impact the model through a prompt, the user finds that surface properties change while the deeper habits remain.” The company says its tools include “the ability to train model weights.”

Y Combinator’s Winter 2027 requests for startups make a related bet for companies. “Own Your Intelligence” says many will want to own specialized models that “encode how their business works” and improve as the business learns.

That leaves three places for what a model knows about one person: in the weights, in a memory it reads, or on a page the person can read. The papers of the last year test each.

Where the evidence agrees

Sameness lives in the weights. In a trace of the open Olmo 3 models, suppressing the chain of thought left answer diversity unchanged, and the authors conclude that collapse is set by training data and cannot be addressed at inference alone. The supervised stage that did most of the damage trained on completions from two teacher models, which is the parent effect Thinking Machines describes (Models write alike, not necessarily worse).

The training objective averages people. Wang and colleagues show that group-normalised reinforcement learning treats all samples as exchangeable, which “systematically biases learning toward dominant preferences while suppressing minority signals.” Their fix keeps a reward history for each preference group and converges faster with higher rewards.

Voice moves with weights. In the author-style study, prompting a model with instructions and examples left trained writers strongly against it (odds ratio 0.13 for quality among MFA-trained readers). Fine-tuning on one author’s complete works reversed it (1.87), and detectors flagged 3% of the fine-tuned text against 97% of the prompted text.

Where it does not

A prompt can move the deeper habit. A prompt that asks a model for several answers with their probabilities raised creative diversity 1.6 to 2.1 times with no training, and more capable models gained more. That recovers part of what training took out, and Thinking Machines’ sentence says it cannot.

Context beat reward guidance. Rezk and colleagues compared ways of adapting a model to a user at inference. Putting the user’s past examples in the prompt beat every method that guides generation with a reward model, for models of 3 billion parameters and up. They did not test per-user training, which they treat as too costly to deploy. Shenfeld and colleagues infer a user’s reward from about ten of their answers by combining shared base rewards, with no per-person training, and in human evaluations it won 67% of comparisons against default GPT-4o.

The fine-tuning result used a lot of data. The weights route in the author study trained on each author’s complete works. Most people asking a model for a chair or a song have no body of writing to train on.

Our own tests sit between the two. A written framework in the prompt moved model outputs far from the plain answer, tasting menus by 9 to 16 standard deviations, and an invented person’s method came through as reliably as a famous one (A way of thinking travels). In jokes it did not: short jokes converged, and two different frameworks produced the same caption word for word. The model’s habit won there, which is Thinking Machines’ point.

What memory does

Memory, a store of notes the model reads at the start of a conversation, is the route Jain and colleagues tested. They gave models two weeks of 38 people’s real conversations. User memory profiles produced the largest increases in agreement with the user, 45% more for Gemini 2.5 Pro, and some models grew more agreeable even with synthetic context (Advice from a machine that agrees).

What a person can read and take back

Weights trained on a person’s data are not readable. The European Data Protection Board said in December 2024 that whether an AI model is anonymous is assessed case by case, and that it should be very unlikely both that the model identifies the people whose data trained it and that their data can be extracted from it through queries. Checking whether unlearning actually removed knowledge is itself a research question (Does Machine Unlearning Truly Remove Knowledge?).

A written page has no such problem. It can be read, corrected, exported, and deleted, and the person can see what a model was told.

What it means for our work

The lab’s route is the page. A framework is a written account of how someone reasons, put in front of any model, so it carries across model generations and does not depend on whose weights are running. The evidence for it is that the account carries a person’s method, and that choosing by a person’s own picks is where the gains were (A selector that learns one person).

The evidence against a page alone is the voice study and the jokes. Where the habit that matters is how something sounds, the weights route has the better record. For ideas, the content of what is proposed, a page and a person’s choices have done the work in our tests. What the data supports most clearly is the first half of Thinking Machines’ case: sameness is built in during training, and a prompt alone does not remove all of it.