Increase Online Survey Response Rates with These Seven Tips

At Magnolia Consulting, we often use online surveys as a data collection tool in our evaluation studies. Online surveys are an efficient and effective way to gather information from participants to answer evaluation questions. While survey response rates can be impacted by many different factors, it’s important to aim for a high response rate. A high survey response rate gives credibility to survey data, whereas a low response rate may pose several challenges, such as (a) decreasing the statistical power of the data, (b) undermining the ability to generalize the results to a larger population, and (c) indicating a nonresponse bias within the sample, meaning that survey respondents differ in meaningful ways from nonrespondents (Nulty, 2008). With this in mind, we make every effort to secure a high response rate when we use a survey for data collection. Here are seven tips we recommend for increasing online survey response rates based on our experience:

    • Add sender email address(es) to approved senders list.  A common problem when sending an online survey occurs when the survey is incorrectly flagged as unwanted spam. Emails received by survey participants may go directly to their spam mailbox or may “bounce” if the sender’s email address is not on an approved list of senders. To proactively ensure this won’t happen to you, contact participants prior to survey administration to request they add any necessary emails to their approved list as well as inform them of the upcoming survey invitation. If issues persist, email the participant using a different email address that has fewer restrictions.
    • Send a pre-survey letter or notification. Prior to sending the actual survey, have a respected authority figure (such as a school principal, or district-level administrator), send a survey notification letter to participants. This letter should inform participants why the survey is important to the school or district, which will help contextualize the survey in the local information needs of the participants. Additionally, it will inform the participants that a survey is coming from an independent evaluator, so they can keep an eye out for the survey invitation.
    • Use an engaging and informative survey invitation. When sending a survey via email, the initial survey invitation is your first chance to grab the attention of your participants. First, use a short, engaging email title to ensure your invitation is not buried in their inbox. Next, the body of the email should provide all relevant information that participants will need to know about the survey, including the following: (a) a clear survey purpose and why participant feedback is important, (b) an accurate estimate of the time needed to complete the survey, (c) a date by when the survey should be completed, (d) information on incentives to complete the survey (if applicable), (e) a statement about survey confidentiality and/or anonymity, (f) and a contact name and email address for any questions about the survey.
    • Ensure your survey is easy to complete.  As you are creating your survey, keep certain readability factors in mind: Is the survey clear and easy to read? Is it free of jargon? Is it too long and time-consuming? You don’t want survey participants to lose interest in completing a survey because it’s difficult to read or too lengthy. One suggestion for improving survey quality is confirming that survey questions are as straightforward and simple as possible, as complex questions can carry a high cognitive load. Another recommendation to make the survey easier for participants is to ensure that survey questions are spread across several pages rather than one large survey all on one page. To do this, one can break pages into segments of questions that relate to one another. Finally, if possible, ensure your survey is optimized for both computers and mobile devices such as cell phones and tablets.
    • Ensure participants have protected time to complete the survey. When we send surveys to teachers, we often ask principals to give teachers time during scheduled school events, such as meetings, to complete the survey. This way, teachers aren’t left to complete the survey on their own time outside of the school day.
    • Send reminders and follow up with nonresponders.  Track survey responses closely and monitor response rates on a daily or weekly basis. This is especially important if survey responses are time sensitive such as when the survey measures change before and after an intervention. It is essential to follow up with nonresponders by sending at least two reminder emails. In some cases, you can contact the participants via phone or mail if you have that information. As a last effort, you may also enlist the help of the respected authority figure mentioned in tip two.
    • Show appreciation for time and effort.  With everyone’s busy schedules, it is important to remind participants how valuable their feedback is to the overall goals of the evaluation. To show appreciation for your participants’ time and effort, offer any assistance (e.g., emailing a paper copy) to make survey completion less burdensome for them. Incentives can be used to show appreciation for completing the survey at both the teacher level (for example, a chance to win an Amazon gift card, or using donorschoose.org to fund classroom supplies), and the school level (for example, a gift card to put on a pizza party or a contribution to a school fundraiser). Be sure to check with the school or district to learn whether there are any restrictions on teachers or administrators receiving incentives. Finally, always thank your survey participants for their time and efforts toward completing the survey.

References:

Nulty, D. (2008). The adequacy of response rates to online and paper surveys: what can be done? Assessment & evaluation in higher education, 33(3), 301–314.

Crafting a Powerful Message: Steps to Develop an Impactful Infographic

In late February we visited Columbia, South Carolina to deliver our Introduction to Infographics and Strategies for Use in Evaluation workshop at Children’s Trust of South Carolina. Children’s Trust is a statewide nonprofit focused on strengthening families and leading communities to prevent child abuse and neglect. Children’s Trust uses infographics to share statewide data with policy makers and legislators in a visually compelling and succinct way. The workshop audience included participants from several different agencies who provided unique perspectives on data visualization and different ways to “tell the story” using an infographic.

We used this workshop as an opportunity to debut a new resource for creating infographics, a worksheet titled, “Crafting a Powerful Message.” The goal of this worksheet is to serve as a guide in developing an infographic’s audience, purpose, and message. The worksheet provides several examples, as well as a framework to assist you when creating your own infographic. Clearly and intentionally defining these three elements is a critical first step in developing an infographic, and makes the process of creating an infographic much easier! To read more about these first three steps in our 10 Steps to Creating an Infographic, check out a blog post from Stephanie Wilkerson here. Crafting a Powerful Message and several other resources for creating infographics can be found on our website here. We hope you find these resources useful as you begin creating your own infographics.

AEA Summer Institute: Introduction to Infographics

This month, we presented Introduction to Infographics at the American Evaluation Association’s 2017 Summer Institute in Atlanta, Georgia. As one of 35 professional development workshops geared toward evaluation professionals, this workshop focused on how to use infographics to communicate evaluation findings in an effective and engaging way. In our experience, lengthy written reports often go unread and unused. As an alternative, infographics can be a powerful visual tool to communicate evaluation findings in an easily accessible way. This workshop was an opportunity to share what we’ve learned about infographics along the way with other evaluators who are interested in using infographics to disseminate information.

As part of our workshop, participants were introduced to our Checklist for Reviewing Infographics as a tool for guiding the infographic design and review process. We demonstrated practical, easy-to-use resources and tools for creating an infographic, including an overview of software and websites, icon collections, and stock photo websites. Finally, our workshop participants went through our 10 Steps to Creating an Infographic and took a stab at creating their own infographic. We were impressed by our participants’ willingness to dive into the process, and many were on track to developing inspiring and creative infographics.

Based on the feedback we received, we know our workshop participants learned a lot, and so did we! Specifically, we are now working to integrate more resources for ensuring that infographics are accessible to those with visual disabilities. We also plan to translate our resources into Spanish in order to reach a wider audience. We hope that workshop participants will join our Magnolia Infographics LinkedIn community as a way to continue to share questions, ideas, and examples of infographics.

We are thrilled that we were able to share our knowledge with other evaluators, and we look forward to future collaboration about how infographics can be used in evaluation. Check out some photos of the Summer Institute workshop in action, and feel free to visit our Tools and Resources page for more information on the various tools mentioned here.