Options for Evaluation
This section discusses two evaluation methods for local programs to consider: standard and basic. The standard method of evaluation is the preferred method as it is more comprehensive and allows for adjustments during the program. However, it is understood that not everyone will have the resources for standard evaluation. Therefore, the basic evaluation will also be explained.
- Standard evaluation:
- Evaluation is done before, during, and after the program.
- Basic evaluation:
- Evaluation is done before and after the program.
Standard evaluation can be conducted by the program implementers or with the help of a professional evaluator. Standard evaluation is ideal since it provides information throughout the life of the program and can be used to examine the effectiveness of specific strategies. However, for some programs the ideal must be balanced with the time and the resources available. Basic evaluation may be the choice that fits the circumstances.
This information is shaped around standard evaluation conducted by the program implementer, since it provides a more complete picture than basic evaluation and does not have the costs associated with engaging an evaluation specialist. However, the worksheets and evaluation tools can be used for either of these evaluation plans. Below are descriptions of these options.
Standard Evaluation
- Data collection is done before, during and after the program (Steps 1, 4 and 5)
- Provides more information for improving the program along the way than basic evaluation
- Provides a better understanding of what worked in the program than basic evaluation
- Requires more time and resources than basic evaluation
Similar to basic evaluation, standard evaluation includes the information collected before and after the program. However, standard evaluation also requires monitoring program activities while they are underway. In addition, programs may also choose to add questions to the parent survey or gather other information to learn more about specific issues relevant locally. For example, if speed around the school is believed to be a problem and a public education campaign to remind drivers about appropriate driving speed is planned, then gathering speed data before, during and after the program would make sense.
The diagram to the right shows how the timing of the data collection relates to the six steps for evaluation.
Standard Evaluation with the Assistance of a Professional Evaluator
- Ideal for large, complex programs.
- Allows for a more rigorous research design for understanding what worked.
- Requires more resources than standard or basic evaluation conducted by the implementers.
A professional evaluator offers the opportunity for a more rigorous research design which often provides a more valid and reliable picture of what is happening with the program, why it is happening, and the relationship between the evaluation results and program activities. While involving an evaluator is not necessary or feasible for every program, for some, such as large, multi-component programs, it may be most appropriate. See Working with an Evaluation Specialist for information on how to find and best use an evaluation specialist. More complex evaluation designs require additional considerations of what and how information will be collected. See Evaluation Designs for more information.
Basic Evaluation
- Data collection is done before and after the program (Steps 1 and 5)
- Provides the minimum amount of evaluation a program should conduct
- Requires the least amount of time and resources
Basic evaluation includes collecting baseline information using a student travel tally and parent survey and using these tools again when the program is over. This will enable program implementers to gain insights on attitudes, behaviors, and conditions before a program begins and what changes have taken place after the program. For example, changes in student travel tallies will reveal any increase or decrease in the number of students walking or bicycling to school, and the responses to the parent survey will reveal any changes in parent attitudes toward walking and bicycling to school.
The diagram to the right shows how the timing of the data collection relates to the six steps for evaluation.
Basic Evaluation Tips
When time and resources are very limited, some basic steps can help.
Local programs often have many responsibilities, just one of which is monitoring the progress and effects of their Safe Routes to School (SRTS) program. If time and resources are very limited, collecting data before and after the program can provide information to help guide program planning, understand the progress and identify next actions. This basic evaluation calls for data collection during Steps 1 and 5 of the standard evaluation process.
Note: Using the SRTS student travel tally and parent survey developed by National Center for Safe Routes to School enables programs to use online tools to enter data, generate reports and summarize results.
Before the Program
Collect Baseline Information
Before starting any SRTS activities:
- Use a student travel tally and parent survey to identify current student walking and bicycling rates and parent attitudes toward children walking or bicycling to school.
- Compile the information. Baseline information from the tools in this section can be entered into a data system to summarize information and create basic reports.
- Ask the school principal to describe:
- the main walking and bicycling routes
- any safety concerns
- any known pedestrian or bicyclist crashes in recent past
- any rules relating to walking/bicycling to school
- Assess the main walking and bicycling routes. Walk the main routes that students take or would take when walking of bicycling to school looking for any safety concerns and other potential barriers.
Use Results to Inform SRTS Program Plan
Use information to determine strategies and goals. Correct unsafe conditions before conducting encouragement activities.
After the Program
Collect Information Again at the End of the Program
Collect the student travel tally and parent survey information again. Enter the data into the Web-based tools. These tools can generate reports that compare findings. If engineering improvements were made, reassess the walking and bicycling routes affected with the audit checklist.
Compare Results Collected Before and After the Program to Identify Changes
Did walking and bicycling increase? Did parents’ attitudes change? Did safety improvements occur? Did parents recognize these improvements?
To plan and conduct a more thorough evaluation that could provide deeper insights on a program’s achievements and results, see standard evaluation.
Who Will Be Involved in the Evaluation?
One person cannot do all the planning for evaluation alone. The group responsible for planning and conducting the Safe Routes to School (SRTS) program will also most likely be responsible for evaluation. The following stakeholders all can play important roles.
- Implementers:
- Those involved in running the SRTS program.
- Partners:
- Those who support the program with resources, such as finances or time
- Participants:
- Those served or affected by the program, including students, parents/caregivers or neighbors.
- Decision-makers:
- Those in a position to do or decide something about the program.
- Professional evaluators:
- Those whose assistance is required if a complex research design or data analyses is planned.
- SRTS program leader:
- The person who oversees the evaluation process and convenes the stakeholders.
Sharing Information as it is Collected
Since each stage of evaluation provides important information that can strengthen or improve a program, the results need to be put to use as soon as possible at each stage:
- Before the program:
- Helps inform the program objectives and activities so the findings can be shared with those who can get the program started.
- During the program:
- Identifies what is or is not working while the program is being conducted. These results should be shared with those who can make mid-way changes to the program to improve it.
- After the program:
- Highlights the changes since the program began. These results need to be shared with those that can re-fund the program or make other decisions about whether to expand or change the program.