What we do when we do UX research: on the philosophy of a profession

This is a long post, and a slower one than usual. Most of what I write here is about how to run research or how to build a research team. This one is a collection of somewhat random thoughts on the philosophy of UX research that I've been turning over lately. They are about what UX research is underneath the methods, what kind of knowledge it produces, and what it is for.

The question I keep coming back to is simple to ask. When we say research found something, what kind of knowledge did we produce? And what gives us the right to make claims about other people's needs and motives?

I don't think the profession has spent much time on that question. We have spent a lot of time on methods.

We borrow from psychology, anthropology, sociology, cognitive science, statistics, and human-computer interaction. We take their methods and their ways of analyzing data. We mostly leave behind the arguments those fields had about what their methods can prove, and those arguments are what gave the methods their meaning.

So we run interviews without asking what an interview can tell us. We run surveys without checking whether the questions measure the thing we named. We read behavioral data as if clicks told us what people meant. Then we produce themes, personas, segments, and recommendations, and we treat each one as if it were obvious what kind of claim it is.

None of this makes UX research illegitimate, and I'm not arguing that every researcher needs a philosophy degree. My worry is narrower. We are very comfortable talking about how to do research, and much less comfortable talking about what it means to know something. I touched on a version of this when I wrote about what makes someone an expert in UX research, and the longer I think about it, the less it looks like an academic problem.

I don't have a neat answer, and the sections below don't add up to a theory. They are me thinking out loud, one question at a time.

Does UX research need a philosophy at all?

I'll start with an admission that cuts against my own argument. UX research doesn't need a philosophy to function. A team can run hundreds of studies, shape roadmaps, find usability problems, and move business numbers without anyone mentioning epistemology, the branch of philosophy that asks how we know what we know.

Companies don't hire researchers to settle philosophical disputes. They hire them to reduce uncertainty, understand customers, compare options, and help people decide. By that standard, UX research can succeed for a long time without a clear view of what it knows. Some of that success may even come from its flexibility.

Look at how easily we move between traditions. In one study we act like positivists, people who treat attitudes and behaviors as stable things we can measure and count. In the next we study how people make meaning out of what happens to them. Elsewhere we run experiments to establish cause, or do ethnography to see behavior in its social setting. When we combine them, we call it mixed methods.

Using many methods is fine. Human experience is complicated, and no single method covers all of it. But combining methods is not the same as reconciling what each one assumes.

Take two findings. One describes how a person experiences a product from the inside. The other is an experiment's estimate of how much a change moved a metric. These are not two techniques for producing the same kind of knowledge. They answer different questions, and they need different kinds of proof.

Yet we put both on the same slide, often under the heading Key Insights, and nobody marks the difference. When the decision is which button to redesign, that probably doesn't matter. It matters a lot once we start making claims about motivation, identity, or why people do what they do.

The distinction I keep coming back to is this one. A practice can work well without being able to justify the authority it claims, and I think UX research is in exactly that position.

Are we producing knowledge, or the appearance of knowledge?

This is the part of the argument I find hardest to dismiss. Picture an ordinary study. We interview twelve people, ask about their experience, find patterns, and write up a few themes. Then we build a story about what users need, why they behave the way they do, and what the product team should do next.

Assume the study was done well. Recruiting was right, the interviews were careful, and the analysis was systematic and documented. We still have to ask what we established.

We might have learned something about the world. We might have produced a defensible reading of twelve people's experiences, or some plausible hypotheses about a larger group. Or we might have built a coherent account that lets an organization make a decision. Those are four different achievements, and in practice we treat them as one.

The steps are familiar. An observation becomes a pattern, the pattern becomes a theme, the theme becomes an insight, and the insight becomes a recommendation. Eventually the recommendation becomes part of what the organization believes it knows about its customers.

Each step changes the original evidence. Some of the changes are justified. Others need extra assumptions, and some are leaps that nobody ever writes down.

What interests me is how easily the uncertainty disappears. A researcher starts with a tentative reading, one of several explanations that fit what people said. By the time it reaches the executive deck, the "might" has fallen off, and the slide says what users want.

I call this epistemic inflation, by which I mean a claim slowly gaining more authority than its evidence supports. It is usually not deliberate. Most researchers know the limits of their findings.

Organizations reward clarity and confidence, though. Nuance is hard to present, ambiguity sounds indecisive, and caveats weaken a recommendation. So we simplify, and in making research easier to act on, we make it look more certain than it is.

That leads somewhere uncomfortable. Some of what we call research knowledge may be better described as interpretation that is useful to the organization, presented with the authority of evidence. That doesn't make it false. It means we should be more careful about what we say it proves.

I've written about the practical side of this before, in The readout is an event, knowledge is a state. The fix I keep coming back to is in Build a memory, not an archive: a finding that carries its own confidence level and the date it was last checked is harder to inflate, because the inflation shows.

Is interpretation the opposite of knowledge?

I want to be careful here, because this argument is easy to push too far. It would be tempting to conclude that research involves interpretation and therefore can't produce reliable knowledge. I think that would be a mistake.

All empirical work involves interpretation. Scientific measurement depends on definitions, instruments, and decisions about analysis. Statistical models depend on choices about which variables to include and how they relate. Even an experiment needs interpretation before anyone can say what it means outside the conditions where it ran.

Interpretation doesn't disqualify knowledge. It raises the question of what justifies it, and that is where philosophy helps, because different traditions answer that question differently.

Phenomenology, the tradition associated with Edmund Husserl and Maurice Merleau-Ponty, studies experience as people live it. From that view, a person's experience is more than a set of behaviors to measure. It depends on their body, their situation, and what things mean to them.

Interpretivism, a related position in the social sciences, holds that meaning is not an object we extract from participants. Meaning can come out of the conversation itself, shaped by language, context, and the relationship between researcher and participant.

Roy Bhaskar's critical realism takes a third position. It holds that the world exists whether or not we describe it well, and that every description we have is partial and can turn out to be wrong.

These positions lead to different practices and different standards of evidence, and that is the point. Before an interview, we need to know what kind of knowledge an interview can produce. Before an experiment, we need to know which causal claims the design allows. When we build a segmentation, we need to separate a useful statistical grouping from an account of how people see themselves.

Using several traditions is fine. The trouble starts when we treat their outputs as interchangeable because they all carry the label research. No amount of care in running a method makes up for confusion about what the method can establish. (I've written about the practical side of rigor in Making minimum viable rigor work for real teams.)

Is UX research a discipline at all?

I've also started to think we ask the wrong question when we look for one philosophy of UX research. UX research may not be a discipline in the academic sense. It may be a professional practice that sits where several disciplines meet and exists to serve an institutional purpose.

Psychology has long traditions of theory about thinking, perception, and emotion. Anthropology has decades of argument about culture, meaning, and how to represent other people. Sociology has competing accounts of institutions, agency, and power. Each field has its own history, its own disagreements, and its own rules for judging knowledge, and UX research draws on all of them.

So what do UX researchers share? I don't think it is a theory of human behavior, a single view of knowledge, or one methodology. What we share is practical. We use empirical inquiry to inform decisions about products, services, and technology. That is a purpose, and a purpose is not a philosophy. I made a related argument in UXR is not one profession, and it never was.

I think we should accept that. Nothing requires a practice that studies human experience to adopt one account of knowledge, and adopting one would probably narrow what we can see.

Accepting many approaches comes with a duty, though. Pluralism, the position that several approaches can each be valid, can't mean picking whatever method is convenient and treating every result as equally authoritative. It means knowing why an approach fits the question, what kind of claim it can support, and how its conclusions relate to evidence from other approaches.

UX research doesn't need one philosophy. It needs a worked-out philosophy of its own pluralism.

Does influence mean the research was right?

There is a possibility I find even more uncomfortable. UX research may not mainly produce knowledge for organizations. Its main job inside an organization may be producing justification for action.

Think about how research usually enters product work. The organization faces uncertainty, a team is weighing several directions, and stakeholders disagree about what customers want. Someone commissions research to settle it. The researcher collects evidence, interprets it, and recommends. Ideally the knowledge improves the decision.

Research also gives the organization legitimacy, meaning reasons it can point to later when it explains a decision. Sometimes those reasons are well supported. Sometimes they are weaker than the language used to present them.

And sometimes the research is commissioned after the team has already picked a direction. Everyone expects the findings to supply confidence, and nobody expects them to change the decision.

This isn't unique to UX research. Organizations use expertise, metrics, and models to legitimize action all the time. But it complicates the claim that research exists mainly to find out what is true.

Michel Foucault argued that knowledge and power develop together, so the institutions that pay for knowledge also shape which knowledge counts. You don't have to think research is manipulative to see this at work. Organizations decide which questions get asked, which kinds of evidence get valued, which findings get attention, and which conclusions become actionable.

So one researcher can produce an accurate account of something the organization doesn't care to fix. Another can produce a weaker but more commercially convenient interpretation that ends up at the center of the roadmap. If that happens, influence is not a reliable sign of quality. A study that changed a decision did not prove anything by changing it.

Yet influence is one of the main ways we judge whether a researcher is good at the job. It shows up in performance reviews as impact, and in every argument about research ROI. That contradiction deserves much more attention than it gets.

Useful to whom, and for what?

John Dewey's pragmatism looks like a natural philosophy for UX research. For Dewey, inquiry starts from a problem and works toward better action, rather than piling up facts for their own sake. That describes in-house research well. We rarely study something because it is interesting. We study it because there is a decision, a problem, or an opportunity.

But I don't want pragmatism to be the whole foundation, because usefulness hides a question. Useful to whom, and for what?

Take a hypothetical. Research finds a psychological weakness that a product can exploit to raise engagement. The research may be sound, the finding may be useful, and the change may move the numbers. None of that shows the change is a good idea.

Or suppose a company learns that customers struggle to cancel a subscription. That knowledge could produce a clearer cancellation flow. The same knowledge could show exactly where people get confused, so the company can keep them confused. The understanding of behavior is the same, and the ends are opposite.

Philosophy keeps four things apart, and organizational research tends to merge them. They are whether a claim is true, whether it is justified, whether it is useful, and whether acting on it is right.

A claim can be true and useless. A decision can work out even though the explanation behind it was wrong. A conclusion can be well justified and still turn out false. An intervention can be effective and still be something you shouldn't ship.

We talk about actionable insights as if being actionable were a mark of good knowledge. It isn't. Actionability describes the relationship between a finding and something someone could do. It tells you very little about whether the finding is true or whether doing the thing is justified.

To be fair to Dewey, pragmatism is richer than "true means whatever works for the business." His idea of inquiry involves testing ideas against their consequences and revising judgments as experience comes in. The narrow version is the one companies run on.

When usefulness is defined only by the organization's goals, research becomes a tool for optimizing those goals instead of examining them. I think that is one of the central philosophical problems of UX research.

Can the researcher stand outside the system?

We often describe researchers as standing between the organization and its users. The researcher listens to users, interprets their experience, and carries their view into decisions. That role has value, but I think we have romanticized it.

Researchers are not neutral observers outside the institution. We are employees or consultants, with careers, theoretical preferences, habits of method, personal values, and relationships with the people who fund the work.

We decide which questions to ask and whom to recruit. We decide which observations matter, how to read ambiguous answers, which findings to emphasize, and how to present them to people who weren't in the room. Every one of those decisions shapes what ends up counting as knowledge.

This is where reflexivity comes in. Reflexivity means examining how your own position, assumptions, relationships, and circumstances shape the knowledge you produce. In practice we often shrink it to a sentence about personal bias at the top of a report. In the interpretive and critical traditions, it is a working practice, and the sentence at the top is the least of it.

Even the full version has limits. Knowing your assumptions doesn't correct them. Naming institutional pressure doesn't remove it, and declaring your position doesn't make your interpretation valid. Reflexivity is a way of examining your work. It does not certify the result.

I'd go further. In organizational research, I think the most useful reflexive question is about the institution, not about me. How does the company I work for shape what I am allowed, encouraged, and rewarded to know? That question moves the problem from individual bias to the economics of who pays for which knowledge.

Do we understand users, or only the models we make of them?

UX research often claims to represent the user. We interview people, watch what they do, and analyze what they say. Then we turn that into themes, personas, journey maps, mental models, and behavioral segments. These representations can be useful. They are still not the people.

A persona is not a person. A behavioral segment is not an identity. A statistical cluster is not a natural category just because an algorithm found it. Each of these is an abstraction built for a purpose, and it is only as good as its fit to that purpose.

Ian Hacking, a philosopher of science, described what he called looping effects. When institutions classify people, the classification changes how those people are treated and how they see themselves, and their behavior changes in response. The category then has to be revised, because the people it described have moved.

Technology products run a version of the same loop. Researchers find a behavioral pattern, and the company uses it to segment customers. The product is personalized by segment, and people respond to the product built around them. Their new behavior becomes the next round of evidence. At that point the system is not only observing preferences. It is helping to form them.

So research does not always study something that holds still. Sometimes it takes part in changing the thing it claims to understand. In products driven by algorithms, where classifications and predictions keep shaping which choices people even see, this is hard to avoid.

That leaves a question I can't answer well yet. What does it mean to know a user's preferences when the product helped form them? I don't think our vocabulary of needs, behaviors, and insights was built for it.

Why does research sound more certain than it is?

Organizations expect research to reduce uncertainty, and it often does. Reducing uncertainty is not the same as removing it, though. A good study can rule out explanations, surface problems nobody knew about, estimate effects, and raise confidence in some conclusions. It can also open new questions.

Sometimes the most valuable result of a study is learning that the original question was badly framed. Sometimes it is learning that the evidence can't tell two explanations apart, or that the question can't be answered with enough confidence on the time and budget available. Those results are hard to present in a meeting that expects a recommendation.

The pressure to be actionable turns into pressure to be certain, and the pressure to be certain pushes researchers to overstate what they know. That matters because looking certain is much easier than being certain.

A compelling story can make weak evidence persuasive. A sophisticated model can hide shaky assumptions. A well-designed deck can make a complicated human behavior look explained. I wrote about one version of this, old findings presented with full confidence, in the case for research expiration dates.

The problem is usually not dishonesty. It is that the conventions of the job reward certainty more than accuracy. The deck format, the word insight, and the recommendation slide at the end all pull in the same direction, and a researcher who follows them can inflate the evidence without ever saying anything false.

How would we know if we were wrong?

Most fields that make claims about the world have some way to find out when they got it wrong. A weather forecaster finds out tomorrow. A drug trial has follow-up. Psychology found out the hard way. In 2015, the Open Science Collaboration repeated 100 published psychology studies, and 36 percent of the repeats produced a statistically significant result, compared with 97 percent of the originals.

UX research rarely gets a check like that. A study recommends something, the team ships it, and the metric moves or it doesn't. Teams rarely go back to ask whether the finding behind the decision was right. If the launch went well, the research gets some of the credit. If it went badly, the blame usually goes to execution or timing, and the finding itself is never tested.

Think about what that means over a career. A researcher can work for ten years without learning which of their findings held. They get feedback on whether stakeholders liked the readout and whether a decision followed. They rarely get feedback on whether they were right.

Without that feedback, the field can't calibrate. Calibration means your confidence matches your record. If you say you are 70 percent sure about a hundred things, you should turn out right about seventy times. Forecasters can calibrate because the weather answers back. Researchers mostly can't, because the product changes, the market moves, and nobody connects the outcome to the claim.

So the field falls back on the only signal it has, which is how confident a finding sounds. That brings us back to epistemic inflation. It survives partly because nothing ever deflates it.

Karl Popper, the philosopher of science, argued that a claim is scientific only if some observation could prove it wrong. Most findings in a research deck pass that test in principle. In practice, nobody makes the observation. That is why, when I wrote about building a memory instead of an archive, I defined a claim as one sentence about users that evidence could prove wrong. The point of the definition is that someone can go and check.

I don't think every study needs a replication. Most product decisions aren't worth that. But once a year, a team could take its most-cited findings and ask which ones were tested by what happened after launch, and how many held. Even that small check would tell the field something it doesn't know about itself.

Who is UX research for, the company or the user?

Two earlier sections circled this question without asking it. The cancellation example showed that the same knowledge can help users or work against them. The section on the researcher inside the system showed who pays for that knowledge. Put them together, and you get the question underneath the whole profession.

Researchers say they speak for users. You see it in job descriptions, in conference talks, and in the name of this blog. The company pays the salary, sets the questions, and decides what to do with the answers. Most of the time the two loyalties point the same way, because a product that serves its users usually serves the business too.

The hard cases are the ones where they split. Take a few familiar ones. A subscription that is hard to cancel keeps revenue. A notification sent late at night raises engagement. A pricing page that nudges confused people toward the bigger plan raises revenue per customer. In each case, the research that finds the friction is the same research. What changes is whose interest the findings end up serving.

I don't think the answer is for researchers to become activists inside their companies. That rarely lasts, and it isn't the job. I also don't think "the user's advocate" means much if it never costs anything. The minimum, I think, is that research writes down what users would want, even when the company decides to do something else.

The decision still belongs to the company. In the post on who decides what counts as evidence, I suggested that every decision should record which evidence it rested on, or which risk it accepted. This is the same idea applied to users. If a team ships against what users would want, the record should say so, with a name on it.

That is a smaller role than the one we describe on stage. It is also one a researcher can keep for a whole career, and I think that is what speaking for users can mean inside a company.

What might this ask of us?

So what do we do when we do UX research? After all these questions, my best answer is this. We make claims about other people on behalf of an organization. We use methods borrowed from fields whose arguments we mostly left behind. And we do it under steady pressure to sound more certain than we are, now with tools that make sounding certain free.

Said that way, the profession doesn't need more methods. It needs more care with its claims. I don't think that means more philosophy in research reviews, and nobody needs Husserl in sprint planning. What would help is a few habits that put the philosophy into ordinary work.

The first habit is to say what kind of claim a finding is. "Six of twelve participants described the setup as confusing" is an observation. "Users abandon setup because it is confusing" is an interpretation. "Simplifying setup will raise activation" is a prediction. Each needs different evidence, and they should not share a slide without a label.

The second habit is to say how sure we are, and to keep saying it as the finding travels. If the confidence drops off between the readout and the executive summary, the summary is wrong, even when every sentence in it came from the study.

The third habit is to judge research by whether it held up, and not only by whether it moved a decision. That means going back after launch to see which findings turned out to be right. It also means rewarding the researcher who said "we can't tell yet" when that was the right answer.

The fourth habit is to know where every claim came from, especially when a machine helped make it. A theme a model found and a quote a model pulled get checked against the transcripts before anyone repeats them, and a synthetic user's answer never gets called a finding.

None of this needs a philosophy degree. It needs a profession that is as precise about what it knows as it already is about how it runs a study. The claims and confidence levels in the Frame are my attempt to build those habits into an organization's memory, so the "might" stays attached to the finding after the researcher leaves the room.

🎯 If this post made you squint at your last readout, subscribe. The next one is less philosophical, I promise.

📖 If you want the practical side first, that's my book: AI-Powered UX Research, the operating manual for running research at the speed your team needs. No Husserl required.