The Same Study Is Not Worth the Same Everywhere: Evidence Has an Exchange Rate
The same study does not have the same value everywhere. Run it at one company and it anchors a roadmap decision. Run it at the next, same method, same sample, same quality of insight, and it gets a polite nod and then a slow death in a repository nobody opens.
For a while I called that a maturity gap, the comfortable idea being that some orgs are simply better at research than others. The pattern was too clean for that, though. It was the same study, sometimes the same researcher, landing hard in one building and sliding straight off the next, and "they are more mature" does not explain why the work itself did not change at all while its reception completely did.
Then I called it culture, which is the word you reach for once you have given up on explaining something properly. Culture is real and also useless, because it does not tell you what to do on Monday.
What I think is actually going on is narrower and more mechanical than either of those. Every org runs on a currency for evidence. Not "data" in the loose, everyone-loves-data sense, but a specific kind of artifact that the org will act on, defend in a review, and move real budget behind. Other kinds of evidence can come into the room, and they will get heard, politely, and then they leave again without changing anything, because they were never the currency that room actually spends.
The currency has a center of gravity that shifts by where you are standing. Growth-stage consumer tech leans on experimental lift; if a change did not move a metric in a controlled test, it more or less did not happen. Design-led orgs lean on a good story about a specific user, told by someone whose taste the room already trusts. Enterprise leans on named-account testimony, which is to say what a real, logo-bearing customer said in a real meeting that someone senior was in. Regulated industries lean on documented research that can survive an audit. None of these are wrong. They are denominations, and each one got set by years of the org being rewarded for trusting that kind of evidence and burned for trusting something else.
What Wins the Room
That is the tidy version, and like most tidy versions it is slightly false. A company has a center of gravity, but no decision actually gets made at the center of gravity. Decisions get made in rooms, and a room is full of people, and the people do not all spend the same currency.
Product spends experimental lift. Sales spends whatever a large named account said on a call last Thursday. Design spends the story. Finance spends the forecast. Legal spends documentation that will survive contact with a regulator. And the executive chairing the meeting is often spending none of those directly, only whatever version of events is least likely to end up attached to their name if the decision ages badly. A real product decision is four or five of these currencies in one room at once, and your research does not get to choose which one it is judged in.
Picture a meeting about whether to kill a feature. Your research is solid and it says users are quietly confused by the thing and churning because of it. In the room, Product has an experiment showing the feature lifts a secondary metric. Sales has two large accounts who asked for it by name and will be loud if it disappears. Finance built part of next year's forecast on its revenue. Your finding is true, and everyone present will even agree that it is true. It also has no denomination any of them can spend, because it is not an experiment, not a named account, not a line in a forecast. So the decision gets settled in the collision of the currencies that do have standing, and the confusion you carefully documented gets acknowledged, set down gently, and left out of the actual math.
This is the distinction that took me longest to see. Almost every org will tell you it listens to users, looks at the data, and values research, and almost every one of them is telling the truth. But getting admitted into the room is not the same thing as getting to end the argument, and the gap between those two is where the real currency lives. You find out what it is the first time two pieces of evidence point in opposite directions and somebody has to decide which one loses.
Quant Over Qual Is a Local Ordinance
Once you can see the currency, the method hierarchy in any given room stops looking like a statement about rigor and starts looking like an exchange rate.
The belief that quantitative work simply outranks qualitative work gets carried from job to job as if it were a fact about rigor. It is mostly a fact about where you happen to be standing. In a room that spends experimental lift, quant is the local money and qual is foreign currency you have to change at a bad rate, with a fee, while someone watches you skeptically. Carry the same two methods into a design critique and the ordering flips: the deeply observed user story is the currency, and the statistically clean survey is the thing that reads as thin and a little bloodless. Carry them into an account review and neither pure number nor pure story wins outright, because a single sentence from a VP at a large customer will beat both.
So a researcher who carries "quant beats qual" between rooms as though it were a law of nature is going to be wrong, badly, a good fraction of the time. The value was never really sitting in the method. It was sitting in how cleanly that method's output converts into the currency of the room it landed in.
The Tasks Your Currency Can't Buy
Some questions cannot be answered in the local currency at all, and those are very often the most important questions on the table.
A growth org doing real discovery, the work of figuring out what to build rather than how to tune what already exists, cannot run that on experiments, because you cannot A/B test something that does not exist yet. So the most consequential early research arrives in exactly the denomination that org is worst at reading. The same trap catches everyone eventually. A design-led org with a conversion number stuck for two quarters wants a story about why, but a story cannot tell you whether the fix worked. An enterprise org wants to understand the long tail of small accounts and has no instrument pointed at them, because the long tail never shows up to the advisory board.
When the task needs one currency and the room only spends another, the right method gets discounted without anyone deciding to discount it. Nobody runs an evaluation and finds the method weak. It just shows up in a denomination the room cannot count, and so it reads as soft, or anecdotal, or, worst of all, interesting.
Mistaking Your Currency for the Truth
Rooms have to have a currency. You cannot re-litigate the entire nature of evidence every morning before standup. The damage comes from something subtler, which is that most of them cannot see their currency as a currency at all.
The growth org does not think of itself as having chosen to value experimental evidence over other kinds. It thinks of itself as rigorous, and of the qual people as anecdotal. The currency has been quietly promoted into a definition of truth. And once that promotion happens, the org loses the ability to notice when its currency is the wrong instrument, because noticing would mean seeing the currency as one option among several, and it has stopped being able to see it as an option at all.
I have watched a growth org run something like a thousand clean, well-instrumented experiments over a couple of years and remain genuinely unable to tell you what its users were actually like, what they wanted, what quietly confused them, what they would never think to say in a survey. Experiments are very good at improving a thing and close to useless at describing the person using it. The org had a beautiful, rigorous, fully instrumented map of a territory nobody had walked. It did not experience this as a gap, because the currency kept paying out, and a currency that keeps paying out feels exactly like being right.
Translation Is the Job
A good senior researcher does not actually spend most of their energy choosing the right method. They spend it translating findings into whatever currency the room in front of them happens to spend.
The translating is the visible part. Underneath it sits a question that has nothing to do with method and everything to do with the room: which evidence the people in front of you can act on without having to defend the decision upward when someone questions it later. That is the currency that will decide things. On a bad day it is not the one your study is written in, and knowing that before you walk in is most of the job.
The same finding then goes into a growth review as a number, "this specific confusion tracks with a four-point drop in the activation proxy," and into a design critique as a person, "this is the user we will call Maria, and here is the exact thirty seconds where the flow lost her," and into an account review as testimony, "three of your top ten accounts raised this without being prompted." One study. One underlying thing that is true. Three currencies, because three rooms, and the same finding handed to the wrong room in the wrong currency does not get marked down, it gets no mark at all. It simply fails to land and everyone moves on.
This translation work is most of what seniority in research actually is, and it is nearly invisible. It does not photograph well in a portfolio. It is not a clean line on a brag document. But it is most of the distance between a researcher whose work changes decisions and a researcher whose work is correct, well-crafted, and completely ignored.
Translation has a limit, though, and the limit is the uncomfortable part. Sometimes the question genuinely cannot be denominated in any currency the room spends, and no amount of skillful reframing fixes that. The discovery question never pays out in experiment-dollars. And at that point the only honest move left is the one almost nobody makes, which is to stop translating and put the currency itself on the table. To say, in the room, that the question they are about to decide cannot be answered in the kind of evidence this org has agreed to trust, and that they are reaching for an instrument that does not measure the thing they claim to care about.
I am not going to pretend that is a cheap thing to do, or that I have always done it. Telling a room that its definition of evidence has a hole in it tends to go about as well as you would expect. But the researchers who actually change what an org becomes, as opposed to the ones who are simply very good and very busy, seem to mostly be the ones willing to make the currency visible. The rest of us get quite skilled, over the years, at getting paid in a currency we have agreed not to ask too many questions about.
🎯 Subscribe for more of this. Consider it the one denomination of evidence I get to set the exchange rate on.