In 1996, Donald Redelmeier and Daniel Kahneman recorded the pain of 154 colonoscopy patients in real time during the procedure, then asked them afterward how bad the whole thing had been. Their paper in Pain found that the remembered total tracked two moments, the worst one and the last three minutes, while the length of the procedure barely registered. Twenty minutes of discomfort and five minutes of discomfort could be remembered the same way if they peaked and ended the same way.
That finding is the best argument I know for running a diary study before you trust what customers tell you in an interview. An interview is a conversation with someone’s memory. A diary study is a record of what happened while it was happening. For most of the questions product teams care about (how often, in what order, what went wrong on Tuesday), those are different data sets, and the memory version is the one we usually build from.
What an interview actually measures
Product people tend to treat interview answers as observations. They are reconstructions. When you ask “walk me through the last time you submitted an expense report,” the customer assembles a story from fragments, smooths the boring parts, and leans hard on the peak and the ending, exactly as Redelmeier’s patients did.
Frequency is where reconstruction fails most visibly. A 2021 study of 1,585 iPhone users by Kaitlyn Burnell and colleagues compared what students said they spent on four social apps with what iOS Screen Time had logged. People overestimated every platform, by roughly 14 minutes a day on Facebook and 27 to 28 minutes a day on Instagram and Snapchat. Heavier users overestimated more. And here is the part that should bother anyone who builds a roadmap from self-reports: the self-reported numbers showed small links to poorer well-being, while the logged numbers did not. The memory data produced a finding the behavior data couldn’t support.
None of this means interviews are useless. They are the right tool for motivation, language, and the meaning people attach to a problem, which is the territory I covered in why customer answers drift from the truth. They are the wrong tool for anything that depends on accurate counts, sequences, or the texture of an ordinary day. Ordinary days are exactly what memory compresses away.
The diary study, stripped of ceremony
Kim Flaherty at Nielsen Norman Group defines a diary study as “a qualitative user research method used to collect insights about user behaviors, activities, and experiences over time and in context.” In practice it means a small group of real customers logs short entries about a behavior as it happens, over days or weeks, and you read those entries while the study is still running.
The method is older than UX research. Psychologists Mihaly Csikszentmihalyi and Reed Larson spent years refining the experience sampling method, in which participants carried pagers and filled out a short booklet entry each time the pager went off at a random moment. Their 1987 paper on its validity and reliability is the reason the approach earned scientific credibility: in-the-moment reports held up as measurement in a way that retrospective ones often didn’t.
Kahneman’s group later built a cheaper middle ground. Their Day Reconstruction Method, published in Science in 2004, had 909 working women reconstruct the previous day episode by episode and rate each one. Commuting came out as the least enjoyable activity of the day, below housework and work itself. Ask most people in the abstract what they dislike about their week and commuting rarely leads the list. Ask them to walk through yesterday in sequence and it surfaces on its own. That is the whole case for structured, near-time logging in one result.
Choosing how people log
NN/g groups logging into three patterns, and the choice matters more than the tool you use to collect entries.
Event-based logging
Participants write an entry every time the behavior occurs: each time they submit an invoice, each time they abandon a cart, each time they open the reporting screen and close it again. This gives you the most complete picture of a specific behavior, and it works best when the behavior happens a handful of times a week. If it happens forty times a day, nobody will log it honestly past day two.
Interval-based logging
Participants write at fixed times, such as end of day or end of week. This handles high-frequency behavior and costs participants less effort. The trap is that the interval itself biases the data. An end-of-day entry written at 5 p.m. is a small act of reconstruction, and the morning gets compressed the same way Redelmeier’s patients compressed their procedures.
Signal-based logging
You prompt participants at varied times, by text or push notification, and they report what they are doing or what just happened. This is Csikszentmihalyi’s pager, updated. It catches moments people would never think to log, which is useful when you suspect the problem lives in the dull middle of a workflow rather than at its dramatic edges.
Most studies blend two of these. A common pairing is event-based entries for the behavior you are studying plus one short end-of-week reflection, so you get both the raw record and the participant’s own interpretation, and you can compare them.
Sizing it: length, headcount, money
Length should follow the rhythm of the behavior, not a calendar default. NN/g’s examples are useful anchors: about one week for a voice assistant people use several times a day, about three weeks for a meal-delivery app used once or twice a week, and about two weeks for a phone purchase because that is roughly how long the decision takes. If the behavior is monthly (closing the books, running payroll, renewing a contract), you either run long or you recruit people who are about to hit that moment.
For headcount, NN/g suggests roughly 5 to 12 participants for a small discovery project and 12 to 30 for a large one, with extra recruits to cover dropouts. Plan on dropouts. Logging is tedious, and enthusiasm in week one is a poor predictor of week three.
Pay more than you would for a one-hour interview. NN/g’s guidance is about $40 per hour of total time commitment for general US participants, paid in installments at milestones for anything longer than a week. Two details in their incentive advice are easy to miss and matter a lot:
- Set a minimum number of entries so participants know what counts as finishing.
- If more entries earn more money, set a maximum too. Otherwise you are paying people to invent interactions, and some will.
Two short conversations bookend the logging. A kickoff call explains what to log, how, and why it matters. A debrief interview after logging ends lets you ask about vague entries and gaps. The debrief is where the diary and the memory finally meet, and the differences between them are often the finding.
What the first week of entries tells you
Running network operations at a large telecom, I sat through a post-rollout review for a new mobile dispatch app for field technicians. Every interview said the same thing: the app was fine, a bit slow, nothing serious. Then someone asked a handful of technicians to jot a line in the app’s notes field at the end of each job for two weeks, nothing fancy, just what had happened.
By the fourth day the entries had a pattern the interviews never mentioned. Technicians working outside the cities were finishing jobs in dead zones, writing closure details on paper, and re-keying everything at the end of the day from a parking lot with signal. Nobody raised it in an interview because by the time they sat down to talk, it had become normal. It wasn’t a peak and it wasn’t an ending. It was the dull middle, and memory had filed it as “a bit slow.”
That is the most common shape of a diary study finding: not a dramatic surprise, but a workaround so routine that the customer has stopped seeing it. It’s the same blind spot contextual inquiry attacks by sitting beside people while they work. The diary covers the days you can’t be in the room.
Read entries daily while the study runs. You can’t ask a useful follow-up question three weeks after the fact, and a short reply on day four (“you mentioned re-entering this; what were you re-entering it into?”) turns a throwaway line into evidence.
A diary study you could start next week
Say you run product for a small-business expense tool and receipts are arriving an average of nine days late. Interviews blame “being busy.” Here is a design that would test that story against behavior.
- Decision first. Write the decision this study informs, for example whether to build automatic receipt capture from email or invest in mobile photo capture. If no answer would change that decision, don’t run the study. (Fitting the study to the decision is its own discipline, covered in scoping research to the decision.)
- Recruit 10 to 12 active users, split across company sizes, and aim for people who expect at least three business purchases in the next two weeks.
- Event-based entries for every business purchase: where it happened, what form the receipt took (paper, email, app, none), and what they did with it in the next ten minutes. Add a signal-based prompt every second afternoon: “Any receipts sitting somewhere right now? Where?”
- Run two weeks, with a short Friday reflection entry.
- Read daily, reply within 24 hours, and log each emerging pattern with the entries that support it.
- Debrief each participant for 30 minutes, opening with their own entries rather than general questions.
If the entries show paper receipts piling up in wallets and car consoles, mobile capture wins. If they show receipts buried in email that nobody forwards, the case flips. “Being busy” can’t tell you which product to build. Two weeks of entries usually can.
Where diary studies go wrong
Asking for essays. Entries should take under two minutes. A form with three short fields and an optional photo outperforms an open text box that asks people to “describe your experience.”
Studying a behavior that doesn’t happen often enough. If participants hit the behavior twice in three weeks, you will get two entries and eighteen apologies. Screen recruits on expected frequency, not just fit.
Letting the incentive shape the data. Uncapped per-entry pay produces padded logs. Capped pay with a clear minimum produces honest ones.
Treating entries as a survey. Ten people’s logs are not a sample from which to estimate percentages. They are a dense qualitative record. Use them to find what is happening and why, then use your product analytics to size how often it happens across the base.
Substituting simulated participants. Some teams now ask a language model to “keep a diary as a typical user.” A model can’t log a Tuesday it never had, and the problems with synthetic users are worse here than anywhere, because the entire value of the method is real events in real time.
When to skip the diary entirely
A diary study is the wrong choice when the behavior is a single moment you can observe in a usability session, when your analytics already capture the sequence you care about, or when you need an answer in four days. It is also wasteful when you are still at the stage of figuring out who the customer is; interviews are faster and broader for that.
Use it when the question involves time: how a habit forms, why something gets abandoned in week two, what happens between the moments your product can see. Those are the questions where memory is least reliable and where, as the colonoscopy patients showed, people will sincerely tell you something that didn’t happen the way they remember it.