Sooner or later somebody is going to pitch you synthetic panels. The demo looks great. You type in a question, pick a demographic, and out comes a tidy paragraph of what a 27-year-old woman in Pune supposedly thinks about your product. No recruiting, no incentives, no waiting three days for responses to trickle in. It feels like the future.
We have looked at these tools closely, and we do not use them for anything that matters. Not because they are fake in some obvious way, but because of what they actually are underneath. A synthetic respondent is a language model predicting the most probable thing a person in a category might say. That sounds close to research. It is the opposite of it.
A model gives you the average. Research lives in the exceptions.
Think about why you run a study in the first place. You are not paying good money to confirm what everyone already assumes. You are paying to be surprised. The insight you actually needed was usually the one nobody at the table saw coming, the quiet objection, the weird use case, the segment that behaves nothing like the rest.
A model, by design, smooths all of that out. It was trained to produce the likely answer, which means it drifts toward the middle every single time. Ask it about spice levels for a snack and it will tell you people prefer things mild, because on average, across the whole internet, that is roughly true. Then you put the mild version in front of forty real people in Hyderabad and half of them call it boring. The model was not lying. It just handed you the average of a country that does not eat like an average.
“The thing you were paying to learn is exactly the thing a model is built to erase.”
It cannot react to something it has never seen
Real research puts a real stimulus in front of a real person. New packaging they have to squint at. A price they have to actually swallow. A feature that sounds clever in a slide and confusing in their hands. The value is in the reaction to the specific thing you built, which by definition did not exist when the model was trained.
So a synthetic panel cannot tell you how your customer responds to your product. It can only tell you how the internet, on average, talks about products that sound a bit like yours. Those are not the same sentence, and the gap between them is where launches go wrong.
The validation is circular
Here is the part that gets glossed over in the pitch. How do you check whether the synthetic answers are correct? You compare them against real data. But if you already had the real data, you would not have needed the synthetic version. And if you do not have it, you are trusting a system whose only proof of accuracy is that it agrees with assumptions you fed it in the first place. It is a mirror that tells you your face looks familiar.
Confident and wrong is worse than honest and unsure
People are messy respondents. They contradict themselves, they misremember, sometimes they tell you what they think you want to hear. Good research is built to catch that. But at least a real person will occasionally shrug and say they do not know, or hesitate in a way that tells you the question does not land.
A model never does that. It produces something fluent and plausible for every prompt, including the ones it has no business answering. Plausible and wrong is the most expensive kind of wrong, because it reads like signal. You will build a roadmap on it before you notice it was invented.
Where synthetic tools are genuinely useful
We are not against the technology. It is good at the work that happens before the research, not instead of it:
- Pressure-testing your survey wording so you catch a leading question before real people see it.
- Generating a first draft of hypotheses to go and check, not conclusions to act on.
- Rough brainstorming when you are still figuring out what to even ask.
All of that saves time. None of it replaces the moment a real person tells you something you did not expect. That moment is the entire point.
The short version
Research is a bet against your own assumptions. A system trained to reproduce the average of everything ever written cannot take that bet, because it has no way of knowing anything you and the rest of the world do not already know. When there is real money riding on the answer, ask a real person. Preferably one you paid, verified, and actually listened to.