How Cognitive Interviewing Works

In a cognitive interview, the respondents are shown a mock-up of a report card that contains either real data (with fake plan names) or fictional data that is as realistic and internally consistent as possible. The interviewer then encourages respondents to share their thought processes as they look at it. Here are three ways to do this:

Observation: Watch the respondents to see what they look at and how they navigate through the document. Where do they begin reading? Where do they go next? How long do they spend on each topic? Experienced interviewers recommend that you give respondents a colored highlighter to mark any confusing words, sentences, or sections. After they go through all of the materials, you can probe further into the things they highlighted.
Listening: In "think-aloud" interviews, you ask the respondents to share every thought and opinion as they go through the document. Ask them to react to specific sections or pieces of the report (such as charts), to tell you when they find something confusing, to explain what they've seen in their own words, or to point out unfamiliar terms or concepts.
Direct questioning: You can also ask direct questions to learn how the respondents interpret what they're looking at. For example, given the data in the report, which plan would they say performed best? Which would they choose for themselves and why?

In the debriefing period after they have reviewed the materials, the interviewer asks open-ended questions to probe deeper into certain comments or observations. Interviewees may also offer an overall assessment and any suggestions for improving what they saw. Be sure to note anything they didn't like or found unclear, but don't get defensive in response to critical comments. Remember that your goal is to collect opinions and learn as much as possible about the reasoning behind them. After the testing is over, you can decide which of the responses you will use to refine your report.

Return to Document