At Magnolia Consulting, we often use online surveys as a data collection tool in our evaluation studies. Online surveys are an efficient and effective way to gather information from participants to answer evaluation questions. While survey response rates can be impacted by many different factors, it’s important to aim for a high response rate. A high survey response rate gives credibility to survey data, whereas a low response rate may pose several challenges, such as (a) decreasing the statistical power of the data, (b) undermining the ability to generalize the results to a larger population, and (c) indicating a nonresponse bias within the sample, meaning that survey respondents differ in meaningful ways from nonrespondents (Nulty, 2008). With this in mind, we make every effort to secure a high response rate when we use a survey for data collection. Here are seven tips we recommend for increasing online survey response rates based on our experience:

    • Add sender email address(es) to approved senders list.  A common problem when sending an online survey occurs when the survey is incorrectly flagged as unwanted spam. Emails received by survey participants may go directly to their spam mailbox or may “bounce” if the sender’s email address is not on an approved list of senders. To proactively ensure this won’t happen to you, contact participants prior to survey administration to request they add any necessary emails to their approved list as well as inform them of the upcoming survey invitation. If issues persist, email the participant using a different email address that has fewer restrictions.
    • Send a pre-survey letter or notification. Prior to sending the actual survey, have a respected authority figure (such as a school principal, or district-level administrator), send a survey notification letter to participants. This letter should inform participants why the survey is important to the school or district, which will help contextualize the survey in the local information needs of the participants. Additionally, it will inform the participants that a survey is coming from an independent evaluator, so they can keep an eye out for the survey invitation.
    • Use an engaging and informative survey invitation. When sending a survey via email, the initial survey invitation is your first chance to grab the attention of your participants. First, use a short, engaging email title to ensure your invitation is not buried in their inbox. Next, the body of the email should provide all relevant information that participants will need to know about the survey, including the following: (a) a clear survey purpose and why participant feedback is important, (b) an accurate estimate of the time needed to complete the survey, (c) a date by when the survey should be completed, (d) information on incentives to complete the survey (if applicable), (e) a statement about survey confidentiality and/or anonymity, (f) and a contact name and email address for any questions about the survey.
    • Ensure your survey is easy to complete.  As you are creating your survey, keep certain readability factors in mind: Is the survey clear and easy to read? Is it free of jargon? Is it too long and time-consuming? You don’t want survey participants to lose interest in completing a survey because it’s difficult to read or too lengthy. One suggestion for improving survey quality is confirming that survey questions are as straightforward and simple as possible, as complex questions can carry a high cognitive load. Another recommendation to make the survey easier for participants is to ensure that survey questions are spread across several pages rather than one large survey all on one page. To do this, one can break pages into segments of questions that relate to one another. Finally, if possible, ensure your survey is optimized for both computers and mobile devices such as cell phones and tablets.
    • Ensure participants have protected time to complete the survey. When we send surveys to teachers, we often ask principals to give teachers time during scheduled school events, such as meetings, to complete the survey. This way, teachers aren’t left to complete the survey on their own time outside of the school day.
    • Send reminders and follow up with nonresponders.  Track survey responses closely and monitor response rates on a daily or weekly basis. This is especially important if survey responses are time sensitive such as when the survey measures change before and after an intervention. It is essential to follow up with nonresponders by sending at least two reminder emails. In some cases, you can contact the participants via phone or mail if you have that information. As a last effort, you may also enlist the help of the respected authority figure mentioned in tip two.
    • Show appreciation for time and effort.  With everyone’s busy schedules, it is important to remind participants how valuable their feedback is to the overall goals of the evaluation. To show appreciation for your participants’ time and effort, offer any assistance (e.g., emailing a paper copy) to make survey completion less burdensome for them. Incentives can be used to show appreciation for completing the survey at both the teacher level (for example, a chance to win an Amazon gift card, or using donorschoose.org to fund classroom supplies), and the school level (for example, a gift card to put on a pizza party or a contribution to a school fundraiser). Be sure to check with the school or district to learn whether there are any restrictions on teachers or administrators receiving incentives. Finally, always thank your survey participants for their time and efforts toward completing the survey.

References:

Nulty, D. (2008). The adequacy of response rates to online and paper surveys: what can be done? Assessment & evaluation in higher education, 33(3), 301–314.