An Evaluator’s Guide to Performing Successful Site Visit Observations

For external evaluators, site visits (e.g. visiting schools) provide an opportunity to experience the activity, program, or product first-hand while also offering a chance to connect in-person with participants from a study. As these observations provide an important addition to report findings, Magnolia Consulting follows several steps for successfully navigating site visit observations. Based on our experience, here are five key guidelines for successful site visit observations:

    • Conduct an in-person study orientation. Conducting an in-person study orientation before site visits offers the opportunity to meet and interact with participants, which helps to establish a trusting relationship. Having a positive rapport with participants is important as it allows for open communication and can help reduce any participant concerns associated with site visit observations. During the study orientation, inform participants of details about the purpose of the site visits, expectations for participation, and how site visits will be scheduled to minimize disruption in the classroom. During the orientation, explain how the data will be used and how evaluators will maintain participant confidentiality. If the budget does not allow for an in-person orientation, webinars with video access are also a helpful tool to introduce yourself to the participants.
    • Create an observation protocol. Using an observation checklist or protocol helps to reduce bias associated with observations and assures that preestablished guidelines are followed. These protocols typically focus on the quality and extent to which an activity, program, or product is implemented and/or aligned to best instructional practices in the field (i.e., reading instruction, STEM). If multiple evaluators perform observations across participants or sites, the measure should be checked across observers for agreement and accuracy.
    • Enlist a site coordinator. If budget allows, plan for having an on-site coordinator who knows the site’s participants and inner workings. This individual can communicate with the evaluation team’s program coordinator on details throughout the study such as scheduling observations across multiple participants at the site. Furthermore, the site coordinator can communicate details with participants before the observation and answer any questions that arise. In addition to coordinating site visit observations, the site coordinator might be responsible for other helpful tasks such as managing consent forms and ensuring assessments are distributed and returned in an organized manner.
    • Be flexible with scheduling. Allowing flexibility and accommodating a site’s scheduling needs supports understanding and recognition of “real world” complications. If possible, create a schedule that follows the site’s established routines. This may require conducting observations across several days rather than consolidating multiple sessions into a shorter timeframe. If the evaluation budget is limited and requires observations to be performed within a brief period, work with the site coordinator to determine another mindful, yet suitable schedule. If appropriate, send participants a direct email one week prior to the visit restating the purpose of the observation and confirming the schedule. This can help avoid an observation from being unannounced.
    • Follow-up with sites. Email site coordinators and participants a day or two after conducting site visit observations to express gratitude for their time and willingness to be observed. This follow-up also allows for any questions pertaining to the site visit or study.

Essential Tools for Recruiting Sites for Studies: Finding the Needle in a Haystack


At Magnolia Consulting, one of our specialities is designing and implementing curriculum efficacy or effectiveness studies, and we are often tasked with recruiting sites (i.e., schools or school districts) to participate. Finding potential sites that both fit the requirements for a given study and are able to participate can feel like finding a needle in a haystack. Based on our experience, it is possible to find these sites, but it can be challenging and requires a well-thought-out plan of action. We have found that an organized, collaborative, and personal approach to recruitment is fundamental to success. The following list includes several key elements for effectively navigating the recruitment process:

  • Start early! Allow ample time for the recruitment process, as it can be quite time consuming to identify sites and to fully bring them onboard. In terms of study implementation, it is easier to confirm sites early rather than at the last minute. If possible, start recruitment in early spring before testing or before summer break when contacts may be out of the office.
  • Create clear study documents. As each study is unique in terms of site selection criteria and benefits to participating sites, it is important to develop clear study descriptions for potential sites. Consider visually appealing ways to present information (e.g., a one-page infographic or handout about the study), as well as various methods of dissemination to a wide audience (e.g., website links or mailing lists). Ensure team members within your company review these documents and are able to clearly explain the study details to potential sites.
  • Develop a list of potential applicants. Before contacting sites, consider developing a tool to track potential sites, which may include details on site demographics and student enrollment information. Sources for this list may include information from a national database (e.g., National Center for Education Statistics) or a curriculum provider’s mailing list of users and may be limited by specific requirements of the study, such as certain areas of the country, size or locale (urban, suburban, rural) of the site, use of specific programs, or access to technology.
  • Create email and phone protocols. Utilizing email and phone communication protocols for initial contacts and any follow-up communications provides an outline for professional, consistent messaging across multiple interactions and staff members. Being approachable, positive, and grateful in all correspondence sets the stage for a potential longer-term connection.
  • Track all efforts. Tracking every interaction with potential sites is essential. For example, in an Excel spreadsheet or Google Sheet, it is possible to track all dates and methods of contact, name/phone/email of the contact, site selection criteria met, and key points from the communication, such as next steps. Tracking efforts streamlines the process, promotes greater understanding of recruitment efforts among team members, and supports the study team in making the final decision on which sites to include in the study.
  • Follow up! Consistent, timely follow-up with contacts in a way that balances persistence with consideration of busy school and district schedules is key. It is also generally good practice to communicate whether a site is selected to participate in the study and to show appreciation for time invested.
  • Confirm final sites. As study sites are selected, continue to communicate regularly with them regarding next steps. For example, ensure that all district and school approval processes are followed and request that sites sign a Memorandum of Understanding (MOU), which outlines the roles and responsibilities of all study parties. Once MOUs are signed, move forward with next steps regarding various study start-up tasks.