How to Conduct a Survey

• Choosing participants: Select survey respondents at random from the intended audience. If at all possible, identify a comparison group that doesn't get the information so that you can see how much of the change in knowledge, attitude, and/or behavior is a result of your information versus a result of other factors in the marketplace. This is a variation on a control group; in a real experiment, you would randomly assign people to either a group that gets the information or the control group, that would not. But random assignment isn't feasible in the context of report cards, so a comparison group is an acceptable alternative. For example:
• Researchers used comparison groups to evaluate the outcomes of CAHPS® reports in Oregon and Kansas.
• An evaluation in Minnesota relied on a de facto "control group" by taking advantage of differences in the information distributed to two groups. 
One of the easiest ways to create a comparison group is to collect baseline data, i.e., responses to key questions collected before the information was disseminated. This is often referred to as a "pre/post" survey. You do not have to contact the same people before and after the distribution period. But be sure to survey a representative sample each time so that their responses are comparable.
• Telephone and/or mail: Surveys are usually conducted over the telephone, but you can also design self-administered surveys (usually sent by mail, sometimes with a follow-up call to non-responders). Both approaches have their pluses and minuses. For example, mailed surveys are more private and they encourage people to think through their answers. But some people discard written surveys or simply find them too hard to read. Telephone surveys are more expensive, but they tend to get a better response rate and are more effective with people with limited reading skills or vision problems. For more on this topic, see Fowler FJ, et. al. Comparing Telephone and Mail Responses to the CAHPS® Survey Instrument. Medical Care, Supplement. 1999 March;37(3):41.
• Profiling the respondent: Include questions that will allow you to identify differences in the responses of subpopulations in your audience. For example, you may want to segment the responses by the respondents' characteristics, such as attitudes towards health and lifestyle issues, age, or gender.
• Showing the subject: If possible, include pictures or even copies of your report with the survey so that the respondents are clear about the subject of the questionnaire. When information has not been provided on paper (e.g., reports on the Web or information delivered through presentations), it can be harder to get useful survey responses because you cannot be sure that everyone is referring to the same thing.

Return to Document