Selecting an Evaluator
Types of Evaluators<ref>[http://www.wkkf.org/Pubs/Tools/Evaluation/Pub770.pdf Evaluation Handbook , W.K. Kellogg Foundation</ref>
In general, there are three types of evaluators: external evaluators, internal evaluators, and internal evaluators with an external consultant.You must determine what type of evaluator would be most beneficial to your project.
External Evaluator: External evaluators are contracted from an outside agency or organization to conduct the evaluation.These evaluators often are found at universities, colleges, hospitals, consulting firms, or within the home institution of the project. Because external evaluators maintain their positions with their organizations, they generally have access to more resources than internal evaluators (i.e., computer equipment, support staff, library materials, etc.). In addition, they may have broader evaluation expertise than internal evaluators, particularly if they specialize in program evaluation or have conducted extensive research on your target population. External evaluators may also bring a different perspective to the evaluation because they are not directly affiliated with your project. However, this lack of affiliation can be a drawback. External evaluators are not staff members; they may be detached from the daily operations of the project, and thus have limited knowledge of the project’s needs and goals, as well as limited access to project activities.
Internal Evaluator: A second option is to assign the responsibility for evaluation to a person already on staff or to hire an evaluator to join your project. This internal evaluator could serve as both an evaluator and a staff member with other responsibilities. Because an internal evaluator works within the project, he or she may be more familiar with the project and its staff and community members, have access to organizational resources, and have more opportunities for informal feedback with project stakeholders. However, an internal evaluator may lack the outside perspective and technical skills of an external evaluator.
When hiring an internal evaluator, keep in mind that university degrees in evaluation are not common; many people now working as evaluators have previously held managerial/administrative roles or conducted applied social research. Consider hiring those who do not label themselves as professional evaluators, but who have conducted evaluation tasks for similar projects.
Internal Evaluator with an External Consultant: A final option combines the qualities of both evaluator types. An internal staff person conducts the evaluation, and an external consultant assists with the technical aspects of the evaluation and helps gather specialized information.
With this combination, the evaluation can provide an external viewpoint without losing the benefit of the internal evaluator’s first-hand knowledge of the project.
The Evaluator’s Role
Whether you decide on an external or internal evaluator or some combination of both, it is important to think through the evaluator’s role. As the goals and practices of the field of program evaluation have diversified, so too have evaluators’ roles and relationships with the programs they evaluate. (At the same time, it is important to note that the idea of multiple evaluator roles is a controversial one.Those operating within the traditional program evaluation tenets still view an evaluator’s role as narrowly confined to judging the merit or worth of a program.)
From our view, the primary goals of evaluation are that stakeholders are engaged, active participants in the process and that the evaluation process and findings will be meaningful and useful to those ultimately responsible for improving and assessing the program. In the end, this means that there is no one way to do evaluation. Given that premise, the critical skills of an effective evaluator include the ability to listen, negotiate, bring together multiple perspectives, analyze the specific situation, and assist in developing a design with the evaluation team that will lead to the most useful and important information and final products.
With your staff and stakeholders, think through all of the potential evaluator roles and relationships and determine which configuration makes the most sense given your particular situation, the purpose of the evaluation, and the questions you are attempting to address.
One important role to think through is the relationship between the evaluator and primary stakeholders or the evaluation team. Questions to consider include: Should this relationship be distant or highly interactive? How much control should the evaluator have over the evaluation process as compared to the stakeholders/evaluation team? How actively involved should key staff and stakeholders be in the evaluation process?
Depending on the primary purpose of the evaluation and with whom the evaluator is working most closely (funders vs. program staff vs. program participants or community members), an evaluator might be considered a consultant for program improvement, a team member with evaluation expertise, a collaborator, an evaluation facilitator, an advocate for a cause, or a synthesizer. If the evaluation purpose is to determine the worth or merit of a program, you might look for an evaluator with methodological expertise and experience. If the evaluation is focused on facilitating program improvements, you might look for someone who has a good understanding of the program and is reflective. If the primary goal of the evaluation is to design new programs based on what works, an effective evaluator would need to be a strong team player with analytical skills.
Experience tells us, however, that the most important overall characteristics to look for in an evaluator are the ability to remain flexible and to problem solve.
1. Evaluation Handbook , W.K. Kellogg Foundation (PDF)