E. Working with an Evaluation Specialist
Some programs will have the resources and interest in conducting a more comprehensive, complex evaluation and will seek the assistance of a specialist in order to do so. The role of an evaluation specialist and tips for creating a successful product are described here.
The Role of an Evaluation Specialist
If a program plans to use an evaluation specialist, the specialist should be included from the very beginning. The specialist can help identify what can be measured and what questions an evaluation will be able to answer. The evaluation specialist can anticipate potential future problems that may arise when gathering or analyzing particular types of data. For example, the evaluator will recognize the potential effect of seasonal differences in the number of walkers, or the impact political changes might have on enforcement activities. Beyond recognizing the potential problems, however, the evaluator will also know how to deal with them.
A specialist may perform the following tasks:
- Design the evaluation
- Identify and train data collectors
- Collect the data
- Provide interim feedback during the program
- Analyze data and present the findings
- Provide input on recommendations
The evaluator can determine survey tools, train data collectors and decide how to analyze results. Program implementers need to stay in communication with the evaluator to make sure that the processes and products align with expectations.
Finding an Evaluation Specialist
For Safe Routes to School programs, a local college or university will most likely be the best source of evaluators or leads on where to find someone. When seeking an evaluator, consider the following list of desirable characteristics:
- Explains evaluation and related processes in understandable terms.
- Demonstrates previous evaluation experience, particularly in use of observations, surveys, and analysis of existing data.
- Writes technical information in a clear, logical manner and uses graphics to help explain findings.
- Knows where to access needed data, such as pedestrian and bicycle injury and fatality rates.
- Has experience working with non-professional data collectors.
- Statistical analyses may be appropriate, depending on what is going to be measured. Describe what is known about the planned program activities and ask the evaluator what he or she would recommend. Ask for a description of situations which would not require statistical analyses. If the answer is “Statistical analyses are always required,” that may indicate a problem.
Adapted from the National Highway Transportation Administration’s Art of Appropriate Evaluation.