A respondent told me last week that she loved the packaging. She said it twice. She also said it while looking at the top right corner of her own screen, at the little window where she could see her own face, the way we all do now.
She did not love the packaging. She loved not being rude to a stranger on a Thursday afternoon while her kid was doing remote school in the next room.
I have run qualitative research entirely over video for a full year now. Something like sixty sessions. And the thing I want to say out loud is that the data is politer than it used to be, and politer is not better.
A video call is a performance space with a mirror in it, and people do not tell the truth in front of a mirror.
Three specific distortions, and I would bet money every researcher reading this has hit all three.
First, the silence problem. In a room, I can let a pause sit for eight seconds and the respondent will fill it with the real answer, which is almost always the second thing they say, not the first. On a call, a pause reads as a frozen connection. Somebody says can you hear me. The pause is gone and so is the second answer.
Second, the audience problem. In a focus group facility the respondent knows there is glass and clients behind it, and that knowledge is at least honest. On a call, the respondent sees a grid of muted rectangles with names like Marketing Director and a black tile that might be recording. Nobody criticizes a product in front of the people who made it.
Third, the body problem. I used to learn more from how someone picked up the box than from anything they said about the box. Hesitation, the turn to find the opening, the little grimace. Over video I get a face and a thumbnail of a hand. The most informative half of the session is off camera.
None of this means remote research is worthless. It means we have been running a year of studies with a systematic bias toward agreeableness and reporting the results as if the bias were not there. I have signed off on decks I would like back.
The data is politer than it used to be, and politer is not better.
What I do now: ask for the second answer explicitly, turn off self view and ask them to as well, get clients out of the room and into a recording, and treat any unanimous positive as a red flag rather than a finding.
Barnaby walks across the keyboard during roughly one session in four. It is the only unfiltered feedback anyone gives me all week.