5 min read

Question: Why Do UXR Teams Need Hundreds of AI Interviews a Week?

Question: Why Do UXR Teams Need Hundreds of AI Interviews a Week?
Photo by Cash Macanaya / Unsplash

Someone asked this on LinkedIn, and it is a better question than the people answering it seem to think:

"I keep hearing about teams running hundreds of interviews per week with an AI interviewer. Who are these teams? And more importantly, why do they need that many interviews?"

The honest answer to the second half is that they mostly do not need that many. Past the first twenty or thirty interviews on a given question, the marginal one tells you very little you did not already have. If insight were the only goal, the hundreds would be hard to justify.

The volume is not buying insight. It is buying currency.

I wrote a piece last week arguing that every org runs on a currency for evidence, a specific kind of artifact it will actually act on, and that research denominated in the wrong currency gets heard politely and changes nothing. A finding from eight interviews and the same finding from four hundred can be identical in what they claim, but only the second one is spendable in a room that runs on experimental lift, because the room reads N as a measure of how seriously it is allowed to take a thing. The N is the denomination.

So the teams running hundreds a week are mostly teams in orgs where qualitative work never had standing, now able to run enough of it to look like the currency the org already trusts. Which raises the actual question, the one the LinkedIn thread did not get to. If volume is how qual buys its way into those rooms, what is the machine doing when it runs the volume for you.

A Machine for Changing Money

Strip an AI-moderated research platform down to what it does and you get a machine that runs interviews and returns the result in a particular shape. Themes with counts on them. Prevalence figures. Sentiment split by segment. Quotes tagged and grouped. The output is arranged to be read at a glance, the way a dashboard is read, because a dashboard is the format experimentation orgs already trust.

Qualitative work used to enter an experimentation org as foreign currency. It was admitted into the room, politely, and then changed at a punishing rate, because the room did not know how to spend a transcript or an eight-person study. AI moderation hands the same qualitative work back already denominated in counts and prevalence and segments, which is to say already in the local money.

The function is conversion. The tool sits exactly on the gap the currency runs into, the distance between the evidence a method produces and the evidence a room will act on, and it shortens that distance to almost nothing.

That is also why the volume has to be what it is. Bridging the currency gap by hand was always possible in principle and almost never possible in practice, because minting qual at the denomination an experimentation room respects meant a budget and a timeline nobody had. The machine makes that denomination cheap. For the first time the bridge is affordable, and a thing that is affordable gets built.

The Tasks That Just Became Buyable

The currency piece had a section on the questions an org cannot buy with the money it has. Discovery in a growth org, the work of figuring out what to build rather than tuning what already exists. The long tail of small accounts in an enterprise. The actual reason a metric has been stuck for two quarters. Those questions did not go unanswered because they were unanswerable. They went unanswered because the methods that fit them produced evidence the room would not spend.

The bridge changes which of those questions are reachable. A growth team that wants the real reason users abandon onboarding used to get eight interviews and a shrug, because eight interviews could not survive a room that thinks in metrics. The same investigation, run across four hundred AI-moderated sessions, comes back as a ranked set of failure modes with prevalence attached, and now the room treats it as something it can build against. The question did not change. The human reasons behind the answer did not change. The answer simply arrived, this time, in money the room recognizes.

That is a real expansion, and it is worth saying plainly because it is easy to be too cool about it. An organization that could previously only act on what it could measure can now act on a wider range of things, including things that were always true about its users and were always getting ignored. The bridge does not just move qual faster. It widens the set of questions a team is allowed to settle.

A Bridge Has a Direction

Two honest things, because a bridge is worth using and also worth understanding.

The first is that this bridge runs one way. It converts qualitative work into the quantitative currency, and it is very good at that, which is why it feels like magic in an experimentation org. It does nothing in the other direction. It will not help a forecast land in a design critique, or get a beautifully observed story to mean anything to a room that only spends lift. The on-ramp and the off-ramp are fixed. So the bridge does not dissolve the currency problem. It serves one direction of travel, and it happens to be the direction the most powerful orgs already wanted to go.

The second is that the conversion has a toll. Some findings do not survive being denominated in counts. The single strange interview that quietly reframes the other thirty, the piece of texture that was the actual finding, the thing a researcher would have flagged and a prevalence table cannot hold. A bridge built for throughput is not built for cargo that is shaped wrong, and qualitative work produces cargo that is shaped wrong fairly often.

Neither of these is a reason not to use it. A bridge with a fixed direction and a toll is still a bridge, and the gap it crosses used to just sit there, uncrossed, with the research stranded on the far side of it. It is worth knowing what the bridge does not carry. It is not worth pretending the gap was better.

What Is Scarce Now

So, back to the teams running hundreds of interviews a week. They are not research-hungry in some way the rest of us are not. They have found a cheap way to mint qualitative work in a currency their org will spend, and they are using it, which is rational and mostly good.

But a cheap conversion does not make research cheap. It moves the scarce thing somewhere else.

What is scarce now, and more scarce precisely because it is one of the last scarce things standing, is the judgment about which currency a particular room actually needs, and whether the bridge carried a finding across intact or quietly flattened it on the way. The machine will convert anything you hand it. It does not know which conversion the question deserved, or whether the question should have been converted at all. That judgment did not get automated. It got more important, and also harder to point to, which is usually a bad combination, because the thing that is valuable and hard to point to is the thing an org quietly stops paying for, right up until it notices it has been making confident decisions on conversions nobody checked.

🎯 Subscribe for more of this. Low N, high conviction.