Some Thoughts on Why UXR Interviews Are an Audition

Yesterday I was at the Outliers conference, and I spent a good part of the day talking with other researchers about hiring. Every research conversation seems to end up there right now. I'm flying home now and want to write some of it down while it's fresh.

In May I wrote that pure qual is cooked, and that researchers should stop arguing with the market and read the job description. I still think that. But I kept coming back to the other half of it yesterday. Candidates were told to adapt, and the hiring process mostly hasn't. Some of the people it rejects should be rejected anyway. This post is about both.

Senior UXR job descriptions ask for experiments, SQL, and AI tools, and most interview loops test none of them

Put a senior UXR job description next to the interview loop for the same role. The job description asks for someone who can design an experiment, write SQL, read a regression, use AI research tools every day, and shape product strategy. The loop is a recruiter screen, a hiring manager chat, a portfolio presentation, and a behavioral round about stakeholders.

Nobody hands you a dataset or an AI-generated synthesis and asks what it got wrong. You'll rarely be asked whether an experiment readout holds up. So the loop can't tell whether you have most of what the posting lists.

The loop mostly measures how well a candidate presents research they already finished

The portfolio presentation is the center of most loops, and it's a performance. You pick your best project, cut the parts that went sideways, and rehearse until the story is clean. The panel knows this, you know they know, and everyone scores it anyway.

What the panel ends up scoring is how well you tell a story about research. That's a skill, and it matters in the job. It's also a different skill from scoping a vague question, spotting weak evidence, or knowing when five sessions are enough.

The behavioral round has the same problem. "Tell me about a time you pushed back on a stakeholder" rewards the candidate with a prepared answer, and there's a whole coaching industry that teaches prepared answers.

This is the part that gets me. Researchers spend their careers telling PMs not to trust what users say about their own behavior. Watch what people do, we tell them. Then we hire researchers almost entirely on what they say about their own past behavior, polished and rehearsed. If a PM brought us a study designed like that, we'd reject the findings.

The bar you're auditioning for comes from the team's history, so it changes from company to company

In March I wrote that most UXR teams are never designed and just accumulate. A team usually gets its first researcher after something ships and fails, and that person is often hired into a role that six stakeholders each described differently, based on what they needed last quarter. The shape of the team tells you what was going wrong eighteen months ago more than what the team is supposed to do.

That history is the bar you're auditioning for, and nobody writes it down. One team might have had a researcher whose findings got ignored, so now it wants a strong presenter. Another might have a head of product who only trusts numbers, so it wants someone close to a data scientist. A third might have inherited a repository nobody maintains and want someone to fix it. All three job descriptions read the same, because they started from the same template with "AI" added.

The panel is the accumulated team, and it scores candidates against the people already on it, because that's the picture of research it has. So a candidate who matches one company's history gets an offer. The same candidate, with the same portfolio, fails the next company's loop in round one.

That's why a rejection tells you so little. Often it means you didn't match a history you had no way to see. It says less about your ability than it feels like it does.

Some rejected candidates could not do the job, especially boom-era bootcamp hires and PhDs who never adjusted to industry

Now the other side. Sometimes it's a skill issue, and two groups come up a lot when researchers talk about hiring. Both learned a version of research that was rewarded somewhere else, and both can get further in a performative loop than they should.

The first group came in during the boom. UXR postings went from an index of 100 in Q1 2018 to 566 in Q1 2022 on Indeed's data. Companies hired almost anyone who could run an interview, and bootcamps and certificate programs filled the demand one cohort at a time. Many graduates learned one script: recruit five people, run a usability test, make a persona, present a deck.

In 2021 that script was enough, because demand was high and nobody checked much else. Some of those people kept learning and are now excellent. Others have four years of research titles and one year of experience repeated four times. The script also produces a good portfolio deck, and the deck is what the loop scores.

The second group came from academia and assumed the training would transfer. Some of it does. A PhD teaches study design and careful handling of evidence, and both matter in industry.

But in a PhD the question comes from curiosity and the deadline is years away. In industry the question comes from a decision, the deadline is two weeks, and the output is something a PM can act on Monday. A forty-page report delivered in week six has failed, however rigorous it is.

The degree also covers less than people assume. A psychology PhD doesn't mean you can write a survey a product team trusts, run a SQL query, or follow a data scientist's model. And academics spend years practicing the job talk, a rehearsed presentation of finished work, which is almost exactly what the portfolio round tests. So the loop rewards the part of the PhD that transfers least.

A PhD is good evidence that someone can do research. It is weak evidence that they can do this research.

Both groups need the same things: scoping a question to a decision, checking evidence before trusting it, and enough quant to read what the data team sends over. That's also what a better loop would test.

A better UXR loop writes the bar down first and tests it in one working session

Here's what I'd do instead. None of it is complicated, and it doesn't take longer than a normal loop.

First, write the bar down before you post the role. Name the decisions this person will inform in their first six months and the two or three skills those decisions need. If the team can't agree on that list, the problem is the team, and no candidate will fix it.

Second, replace the portfolio presentation with a ninety-minute working session. Give the candidate a vague stakeholder question, a handful of transcripts, the AI summary of those transcripts, and a small dataset. Ask them to scope the question, check the summary, and say what they'd tell the PM by Friday. The panel works through it with them and sees how they think while they're doing the job.

Third, keep the portfolio, but as a conversation. Ask what went wrong, what they'd cut, and what the research failed to change. People who did the work can answer those questions easily. People who rehearsed a story have a harder time.

Fourth, score everyone against the written bar with the same rubric, and tell rejected candidates which part they missed. One sentence of feedback helps a candidate more than a template, and it forces the panel to say what it was measuring.

If you're the candidate, ask early: what decisions will this person inform in the first six months? If nobody can answer, you've learned what the loop will be like.

These are thoughts from some discussions from the conference so I'm sure I've missed things. If you hire researchers and do something that works better than this, I'd like to hear about it.

🎯 If this resonated, subscribe to The Voice of User.

📖 If you want to build the skills that working session tests for, scoping a question, checking what the AI produced, and getting to a decision fast, that's my book: AI-Powered UX Research: How to Run Research at the Speed Your Team Actually Needs.