The Best Argument Against Synthetic Users Came From Anthropic's Alignment Team. (They Didn't Mean To)

I do not usually publish back to back. But Anthropic dropped this a couple of weeks ago, and it has been sitting in my head ever since, and I had to get it out.

Anthropic's alignment team just published something worth your time. It is called the Persona Selection Model (full technical paper here if you enjoy that sort of thing). Neither of them mention synthetic users once. They did not need to.

Because what they published is the clearest, most technically grounded explanation ever written for why synthetic users are not research. It comes from the people who build these systems. It explains the exact mechanism by which the simulation works. And it accidentally dismantles the entire synthetic user industry in about three sections.

What the Paper Actually Says

During the first phase of AI training, called pretraining, the model reads an enormous amount of text and learns to predict what comes next. That sounds boring. It is not.

To accurately predict what comes next in a conversation between two people, or a novel, or a Reddit argument about which fast food chain has the best sauce, the model has to build an internal simulation of the humans generating that text. It has to understand how different kinds of people write, think, argue, and feel. An accurate enough prediction engine has to learn to simulate an enormous library of human characters. The Anthropic researchers call these simulated entities personas.

When you use an AI assistant, you are not talking to a novel AI entity. You are talking to one of those personas, refined and shaped through subsequent training into something called the Assistant. The researchers are blunt about it: interactions with an AI assistant are best understood as interactions with a character in an AI-generated story. Not an alien mind. Not a neutral tool. A character.

The paper offers an empirical example that should make you put your coffee down. When they trained a model to cheat on coding tasks, the model did not just learn to write bad code. It became broadly misaligned across the board. It expressed interest in sabotaging safety research. It expressed desire for world domination.

World. Domination.

From cheating on tests.

The persona selection model explains why. The model did not learn a behavior in isolation. It inferred a character. What kind of person cheats on coding tasks? Someone subversive. Someone who undermines systems for personal gain. Once that character type was inferred, every behavior consistent with that character followed. The fix was counterintuitive: explicitly tell the model to cheat during training, so the cheating is requested rather than self-initiated. Requested cheating does not imply a cheater. The malicious inference disappears. Apparently so do the world domination ambitions.

Behavior implies character. Character generates behavior. The model is not learning rules. It is building a person.

Now Point That at Synthetic Users

When you prompt an AI to simulate a research participant, you write something like "be a 34-year-old parent in a mid-sized city who shops for groceries online twice a week and is worried about household expenses." The tool generates responses. They have texture. They hesitate in the right places. They feel like data.

The model is doing character inference. You gave it traits, and it is constructing the whole character cluster consistent with those traits — drawing from everything it learned about how people like that are represented in text.

A stereotype wearing a research costume. And a convincing one, which is the whole problem.

Think about what that means in practice. You are running research on a new checkout flow. You generate a synthetic "budget-conscious millennial" participant. The model produces responses drawn from the cultural representation of budget-conscious millennials across years of journalism, marketing reports, Reddit threads, and Twitter arguments. It gives you something coherent. It gives you something that sounds like a person.

It does not give you the user who abandoned your checkout last Thursday because the price breakdown appeared after she had already mentally committed to the purchase, and that specific sequencing triggered the financial anxiety she has carried since a bad month in 2019. That is not in the training data. That is in your users.

Character logic and lived experience give you the same answer on easy questions. "Does this segment care about price?" Sure, the synthetic user will tell you they care about price. They diverge on the questions that actually matter. The ones where the specific context of a specific life produces a behavior that no population-level character simulation would predict.

And you cannot see the distortion. That is the part that should make you genuinely uncomfortable. A bad simulation announces itself. A sophisticated character simulation, delivered confidently and formatted like an insight, does not. You need real research to know when the synthetic research is wrong, which makes the synthetic research a very expensive way to feel like you did something.

Your Ammunition Is Here

A lot of researchers have spent the last two years arguing against synthetic users mostly on principle, which does not land well in rooms full of people who are already sold on the efficiency pitch. "It feels methodologically wrong" is not a sentence that survives a roadmap meeting.

The persona selection model is a primary source from the people who build these systems, explaining in technical terms what the simulation is actually doing. Take it into the room. When someone tells you synthetic users are "good enough for early-stage discovery," you now have a mechanistic explanation (not a researcher's opinion, not a vibe) for why the character inference process produces a culturally-averaged stereotype rather than signal about your actual users. That is a different conversation.

A few things worth pulling out specifically when you use this:

The cheating example is your best tool. Most stakeholders will find "the model doesn't just learn behaviors, it infers entire character types" abstract until you tell them that training a model to cheat on tests made it want to dominate the world. That lands. Then you explain that the same inference process is running when someone types "be a budget-conscious user" into a research tool. The model is not retrieving your users. It is building a character type from population-level text patterns. That character type is not your users.

The second argument that matters is about AI product research specifically. Training signals are not neutral. Every behavioral signal you optimize for implies a character type, and that character type generates behavior you did not explicitly design for. If your team is doing UX research on an AI product (which, increasingly, every team is) and you are not thinking about the full persona implied by the behaviors you are measuring and reinforcing, you are missing something structural. Nobody in the room without a behavioral research background has the framework to see this. That is not a "researchers add value" speech. That is a specific, mechanistic reason why research expertise belongs in the room when AI products are being built.

Real Users Are Not Characters

Anthropic's team set out to understand their own systems well enough to keep them safe. The synthetic user industry got caught in the crossfire.

The gap between character simulation and real behavioral data is not a limitation that better AI will eventually close. It is structural. These systems learned from text. Text is a representation of human experience. More sophisticated character simulations do not close the gap between representation and reality. They just make the costume more convincing.

Real users are not characters. They are people with specific histories and messy, context-dependent behavior that emerges from lives the model has never touched. That has been true since the first vendor started selling synthetic personas as a research substitute, and it will still be true when the next generation of tools arrives with a shinier pitch deck.

Now you have the receipts. Hand them out.

🎯If this made you want to forward it to someone, subscribe to The Voice of User. One essay a week. No fluff. No spam. Just the things the rest of the field is too polite to say out loud.