How to Hire     The Right Evaluator               For YourTobacco Prevention and Control Program                   Jamey W...
How to Hire the Right Evaluator                      How to Hire the Right Evaluator              for Your Tobacco Prevent...
How to Hire the Right Evaluator    The focus in defining the scope of the evaluation is to identify the issues intended to...
How to Hire the Right Evaluator    5. Proposed timeline of evaluation activities.    6. Proposed deliverables – typically ...
How to Hire the Right Evaluator    change your programs objectives. It is crucial that the evaluator understand that evalu...
How to Hire the Right Evaluator   6. How relevant and useful are the proposed deliverables?       The most common evaluati...
How to Hire the Right Evaluator     The actual interview might begin with the evaluators presenting their proposed evaluat...
How to Hire the Right Evaluator     Following the interview process, reviewers should individually rate the candidates on ...
How to Hire the Right Evaluatorfuture plans for the program will help set the context for any recommendations made by the ...
How to Hire the Right EvaluatorEvaluator Rating Guide                      Candidate Name:________________________________...
How to Hire the Right Evaluator                       Evaluator Conflict of Interest Questionnaire              Name of Pr...
Upcoming SlideShare
Loading in …5



Published on

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. How to Hire The Right Evaluator For YourTobacco Prevention and Control Program Jamey Wise, MS Director of Evaluation, Florida Tobacco Pilot Program Bureau of Epidemiology Florida Department of Health February 2000
  2. 2. How to Hire the Right Evaluator How to Hire the Right Evaluator for Your Tobacco Prevention and Control Program1Introduction Program evaluation is an important aspect of every tobacco prevention and control program. The essentialcomponents of successful programs can be identified and replicated through well-designed, systematic evalua-tion. Evaluation also identifies less effective program elements that need to be modified. Ultimately, evaluationresults provide stakeholders − including the intended audience, community leaders and policy makers − withevidence of the impacts of tobacco prevention and control activities, and help them make decisions about howto improve these activities to achieve the desired outcomes. Many programs find hiring an external evaluator to be an effective means of obtaining a useful evaluation.An external evaluator must be chosen carefully, however, because program personnel can easily become dis-satisfied with an inappropriate or inadequate evaluation and may come to question the usefulness of the resultsobtained. Well-planned and well-conducted evaluations are invaluable in determining where a tobacco prevention andcontrol program succeeds and how it can be improved. The following four guidelines can assist you in selectingand working with an appropriate external evaluator. 1. Define the evaluation. 2. Select an evaluator. 3. Work closely with the evaluator. 4. Use the results.Guideline 1: Define the evaluation. Defining the evaluation includes three primary steps: a) defining the focus of the evaluation; b) defining the evaluation activities; and c) developing a request for evaluation proposals. Evaluation Focus The focus of the evaluation is what you want to have evaluated. You need a clear understanding of whatyou want the evaluation to accomplish. Do you want an assessment of how well you planned and implementedyour services? Do you want an assessment of the results of your services? Keep in mind that the evaluation should be decision-focused. At each step of the design process ask your-self how the data collected will inform programmatic decisions. Evaluation can assist you in making many typesof decisions, including: 1. information decisions: options for changes in collection and use of program performance information. 2. management decisions: options for changes in program activities; and 3. policy decisions: options for changes in program resources or objectives.1 This guide is based on an anonymously authored document, “Hiring the Right Evaluator for Your Program,” developed for the Corporation forNational Service, to which I contributed advice but did not author. I would also like to express my appreciation to the people who providedhelpful comments: Edward Trapido, Norman Weatherby, and Sharon Luebbers. 1
  3. 3. How to Hire the Right Evaluator The focus in defining the scope of the evaluation is to identify the issues intended to be the focus of theevaluation. Once the focus is established, then the key evaluation questions follow: ♦ How well are we managing our program? ♦ How well did we plan and implement our program? ♦ What are the outcomes achieved by our program? An evaluation can address any or all of these questions. These questions will eventually serve as the basisfor the evaluator to elaborate more detailed questions in an evaluation plan. Once you know what questionsyou want answered, then you are ready to define the evaluation activities. Evaluation Activities Evaluation activities are the scope of work expected from the evaluator. These tasks should relate directlyto the conduct of the evaluation. The evaluator may be expected to: ♦ develop the evaluation plan; ♦ develop evaluation instruments; Using an Evaluation Committee ♦ select the sampling procedures and draw The question of what should be evaluated is often best an- swered by committee, and an Evaluation Committee can be a the evaluation sample; very valuable work group for all aspects of the evaluation. ♦ collect evaluation data; An evaluation committee is useful for overseeing the entire ♦ analyze evaluation data; evaluation process, from initial planning through implementa- tion and crafting of the final report. Persons representing the ♦ compose the evaluation report; and various aspects of your program, including Partnership and ♦ present the evaluation results. SWAT Members, supervisors and staff, partner agency staff, and other key people involved in programmatic planning best Requesting Evaluation Proposals constitute this committee. The committees size will depend upon the size of your Once the evaluation tasks have been program. A large program might support an evaluation com-determined, you are ready to develop a request for mittee of five to seven persons, chaired by the program direc-evaluation proposals (RFP)2. The RFP is designed tor. Smaller programs, of course, may require no more thanto solicit written evaluation proposals from which three people; some, only a single staff member. In other cases,you can select the best candidate. The RFP must the evaluation committee might need to include the entire pro-follow your agency’s purchasing guidelines, and gram staff.should include a statement of the evaluation focus, The evaluation committee has primary responsibility for allscope of work, and any other requirements. An aspects of the evaluation and receives regular reports from theoverall description of your tobacco prevention and external evaluator. Committee members must be actively in-control program, an estimate of the available volved in the evaluation process to increase their understand- ing of it and to allow them to feel ownership of, and make useevaluation funds, and all required meetings (e.g. of, evaluation results.planning meeting, regular updates, interim reportpresentation, final debrief) should also be included in this document. The RFP should require evaluators to include several main sections in their evaluation proposal: 1. Statement of the purpose of the evaluation. (This section ensures that the evaluator understands the intended focus of the evaluation being sought.) 2. Statement of the program goals and objectives that are relevant to the focus of the evaluation. (This section ensures that the evaluator understands the program design.) 3. Statement of proposed evaluation questions. These should be developed from the information provided under Guideline 2. 4. Proposed methodology for addressing the evaluation questions.2 In Florida, the service procurement document may take the form of a less formal “invitation to negotiate” or more formal “request for proposals”(or some other form). Check with your procurement office to determine which approach you should use. The steps outlined here work well under anumber of procurement methods. 2
  4. 4. How to Hire the Right Evaluator 5. Proposed timeline of evaluation activities. 6. Proposed deliverables – typically a) evaluation instruments; b) data set; c) monthly reports; d) interim report (if needed); and e) final report. 7. Proposed budget. Once the RFP has been completed, it should be distributed to a variety of local organizations. Suitable can-didates with good credentials in program evaluation can usually be found in social science research organiza-tions, institutions of higher education, and independent consultants. Try using your existing network to identifypersons who have conducted program evaluations for other organizations in your community or for other to-bacco prevention and control programs. Helpful Hint: Under some circumstances it is not necessary that an RFP or other formal competitive process be used. Check with your budget or contract office for advice on the appropriate procedure for your program. If you do not conduct an RFP, the review and selection guidelines outlined her are still applicable – even for only one application.Guideline 2: Select an evaluator. There are three basic steps in selecting the evaluator. a) reviewing and ranking the applications; b) interviewing the candidates; and c) negotiating and writing a contract.An evaluator is not actually selected until a contract is executed. Steps a and b are used to assess and rankcandidates. During these phases of the selection process, the evaluators are also deciding whether to selectyou. If you and the evaluator both agree that a good match is in the offing, then steps a and b become thebackground work for step c, developing the contract. Each of these steps is described below. Reviewing and Ranking the Applicants Depending on your available resources and the number of available evaluators in your area, you will proba-bly receive several applications or proposals. I recommend a two-step review process involving a review of theproposals and then formal interviews with the top candidates. You would then use both the results of the pro-posal review and of the interviews to hire the right evaluator for your program. The first step in selecting from among the applicants is to winnow down the proposals to the top three ap-plicants through a thorough review of the proposals. The following review criteria will allow to you assess theproposals based on the requirements established in the RFP. A sample evaluation review form is provided atthe end of this guide which includes these seven criteria and six additional criteria for interviewing the appli-cants. 1. How well does the evaluator understand the focus of the evaluation? If your intent is to assess how well services were planned and implemented, does the evaluator clearly state this focus in the proposal? If you want an assessment of the results or impact of the services, does the evaluator clearly state this focus in the proposal? Does the evaluator propose that the evalua- tion will be decision-focused − that is, that the evaluation findings will help you make decisions about your program, including the need for additional information, what activities might be changed, or new objectives that might be implemented? Importantly, the applicants statement about the focus of the evaluation will be an indication of whether the evaluator understands the distinction between research and evaluation. Research is con- ducted primarily to expand our knowledge of a topic or program. Evaluation will help you decide what information to use and how to use it, how you might improve service activities, or how you might 3
  5. 5. How to Hire the Right Evaluator change your programs objectives. It is crucial that the evaluator understand that evaluation must help you make decisions about how to improve your program.Program Evaluation or Research? Evaluation is closely related to, but distinguishable from social research. Evaluation utilizes many of the samemethods used in social research. According to Bill Trochim, evaluation takes place within a political and organ-izational context and therefore it requires additional skills, including group process skills, management ability,political savvy, and sensitivity to multiple stakeholders. A common definition of program evaluation is: the systematic assessment of the worth or merit ofsome human service program in order to improve it. (Adapted from Michael Scriven) In contrast, a common definition of research is: any systematic activity designed to develop or con-tribute to generalizeable knowledge. (Federal Regulations Regarding Human Subjects Protection)Research emphasizes acquiring and assessing information. The primary purpose of research is to generate newknowledge. Evaluation emphasizes assessing worth or merit in order to improve. The primary purpose ofevaluation is to determine the value of program strategies and improve the ones that can be improved.2. How well does the evaluator understand the program goals and objectives? A good evaluation will be based on the goals and objectives you have established for your program. Does the evaluator communicate in the proposal a clear understanding of your programs goals and objectives? Does the evaluator understand your programs priorities? For example, if you emphasize retailer edu- cation over school-based instructional programs, the evaluation proposal should note these priorities.3. How well do the proposed evaluation questions address the program goals and objectives? Evaluation questions are based on program goals and objectives. The most direct evaluation questions are simply objectives written as questions. For example, To what extent have comprehensive tobacco use prevention education programs been implemented in grades 9-12? (implementation assessment). Or, How have community leaders attitudes toward tobacco changed during the past 12 months? (out- come assessment).4. How appropriate is the proposed methodology for the evaluation questions? This can be a difficult issue to assess. Selecting evaluation methodologies can be a lot like selecting foods; not only are the needs of the people (program) important, but also their individual tastes. How- ever, the evaluator should clearly explain in the proposal why the methods being proposed are the most appropriate. In general, you will readily see the logic behind using surveys, records reviews, pre/post tests, or other approaches for answering questions about your program. Being intentionally redundant, whatever approach is used should yield information that you can use to make information, management, or policy decisions about your program. If you are not familiar or comfortable with assessing the proposed methodology, then I recommend that you enlist the services of an experienced evaluator to assist you with the review of proposals. An evaluator on the review team will serve two primary purposes: 1) assessing the appropriateness of the methodology; and 2) making recommendations for improving the evaluation methodology, if improve- ments are needed.5. How reasonable is the timeline of evaluation activities? The reasonableness of timelines depends on several factors, including your program budget period, when you need the evaluation results, and the evaluation methods used. Dont rush the evaluation un- necessarily, but do allow yourself enough time after the evaluation is completed to use the results for your program planning. 4
  6. 6. How to Hire the Right Evaluator 6. How relevant and useful are the proposed deliverables? The most common evaluation deliverable is a written report. I recommend that you look for several types of reports, each of which is used for a different purpose. Obviously, the most important reports are the final reports which present the evaluation results and the conclusions and recommendations of the evaluator. To ensure that the evaluation activities are being conducted as planned and within time- frames and budget, you should get monthly updates (these generally are brief reports). If the method- ology includes a pretest and posttest, then I recommend that you ask for a report of the pretest results as well as a report of the analysis of both the pretest and posttest. If the evaluation uses procedures that build on one another (for example, conducting a focus group and using the results to develop a survey) then a report on each component will be useful. For evaluations being conducted over a long time period, a comprehensive interim report might be useful. Helpful Hint: The larger the scope of the evaluation, or the more complex the methodology, the more there will be to report; it is commonly easier to digest several shorter and more focused reports than one large, detailed report. 7. How reasonable is the proposed budget? Does the candidate believe your evaluation can be conducted for the available funds? Candidates must indicate that their proposed evaluation approach can be carried out for the funds you indicated would be available (or less!). You might find that a proposed evaluation plan is excellent, but unfeasible under your anticipated budget. Interviewing the Applicants The second step in selecting from among the applicants is to conduct formal interviews with the top threecandidates. The interviews should be used to clarify any concerns or questions raised during the proposal re-view and to gauge evaluators ability to work with your program. Request that each candidate interviewed provide you with previous evaluation reports they have writtenand with references of people or organizations for whom they have previously conducted evaluation projects. 8. How useful are the candidate’s previous evaluation reports? Look for evaluation reports for which the candidate served as lead author or as project direc- tor/manager (typically the second author in a report from a research institute). Assess the reports for clarity, organization, readability, and potential usefulness for decision makers. Pay particular attention to how well the reports would help a program improve its information collection, activities, or objec- tives. Candidates providing technical, poorly written, disorganized, difficult-to-understand, or lengthy evaluation reports will likely compose similar reports for your evaluation. 9. Does the candidate have good references? References should be contacted to obtain their experiences with and opinions about the candidates. Some questions you might ask the references include: a. How well did the evaluation approach used by the evaluator address the needs and desires of your organization? b. Was the evaluation conducted in a timely fashion? c. Was the evaluation conducted within your budget? d. What was the most useful aspect of the evaluation report? What changes, if any, did you make as a result of the evaluators recommendations? e. Would you hire the evaluator to conduct another evaluation for you? 5
  7. 7. How to Hire the Right Evaluator The actual interview might begin with the evaluators presenting their proposed evaluation plan. Such pres-entations provide you with much invaluable information about the candidates. You will learn about the evaluat-ors ability to organize information, communicate ideas, and respond to questions. Invite your staff, supervi-sors, and program partners to the presentation to get their input. Remember: they also will be using the re-sults. 10. How effectively does the evaluator communicate with you and other stakeholders? If the discussion becomes very technical and a candidate is unable to present information in an easily understood manner, it is unlikely that this candidate will meet your needs. A candidate unable to com- municate effectively at this time will probably not overcome the problem during the evaluation. Effec- tive communication is a key for success, and the interview gives committee members a good idea of how effectively a candidate can communicate. In addition to whatever questions might arise during the presentation, the following questions might alsobe useful. 11. What is the candidates prior evaluation experience? Experience is an important factor to consider. A candidate probably will not have performed exactly the same evaluation that you require, but many similarities between previous programs and your own can be found. A candidate that has performed evaluations of substance abuse prevention programs, for ex- ample, would have experiences that translate directly to tobacco prevention and control programs. De- pending on the emphases of your program, evaluation experience with law enforcement programs, school or community based educational programs, marketing, or community building might be relevant experience. The candidates prior experience will be your main opportunity to discover and weigh that persons strengths and weaknesses. 12. Will the candidates existing professional commitments interfere with the planned evaluation? Good program evaluators are usually in high demand. A candidate who is engaged in several projects, however, may be unable to devote sufficient time to your program evaluation. However, you should expect that the quality of work you receive is comparable to any other project in which an evaluator is engaged. Ask the candidate to describe current and expected professional commitments. If the commitments seem excessive, ask how the candidate plans to conduct your program evaluation along with these other tasks. If the candidate indicates that other persons will be used to assist with the evaluation, determine which tasks win be performed by whom. Also determine if these other persons are capable of performing the tasks assigned to them, including requesting copies of resumes or vitas from the candidates associates. Using a team of trained and experienced persons to perform an evaluation is common, but the team leader (i.e., the candidate) should be involved in all tasks that the you believe require this persons direct involvement. Finally, it is important to ensure that the evaluator you select does not have a conflict of interest. An evaluation is truly independent when the program evaluator has no stake in the program’s success or failure. For example, if the evaluator were a member of the management team, then he or she would obviously have a clear interest in the program results – both in terms of prestige and continued em- ployment. Such an interest in the program outcomes can affect the decisions of event the most honor- able evaluators. A conflict of interest questionnaire is included at the end of this guide that you can ask all evaluators to complete and include with their proposal. 13. What is your general reaction to the candidate? During interviews, be alert to the candidates ability to communicate in a straightforward manner, and be alert to your own expectations of how effectively you and your colleagues can work with this per- son. A clash of working styles can certainly be a problem, and the chemistry between a candidate and your programs stakeholders should not be ignored. 6
  8. 8. How to Hire the Right Evaluator Following the interview process, reviewers should individually rate the candidates on all of the issues previ-ously described. Candidates might be rated on a five-point scale, ranging from "Definitely hire as our evaluator"to "Definitely do not hire as our evaluator." "No opinion" should be the midpoint. (See Attachment 1 for a sam-ple form for rating candidates.) Reviewers scores can be combined in a number of ways, from simply adding upthe total points for each criterion, to averaging the points for each criterion, to asking the reviewers to discusstheir individual ratings and then reach consensus on how a candidate should be rated on each criterion. Aftercombining reviewers individual scores, the candidates should be ranked so that the project can be offered tothe candidate most acceptable to the reviewers. Negotiating and Writing a Contract The desired relationship between the evaluation committee and the external evaluator is one of partnershipand should be reflected as such in the contract. The contract should state, in a single paragraph if possible, theevaluators general responsibilities. If you have the evaluator produce a work plan, it can be incorporated intoyour agreement. The contract should list the contract deliverables and provide a timetable for each one. Con-sider including language describing how changes in the scope of work or work plan will be handled. Manyevaluation contracts also specify who owns the data gathered during the evaluation, as well as who has theright to publish the results of the evaluation study. Finally, indicate how the evaluator will bill for services ren-dered and include a schedule of payments. It is common to withhold between twenty and thirty percent of theevaluators fee until all deliverables have been submitted and reviewed to ensure that the deliverables respondto the contracted scope of work. The contract should also detail the programs responsibilities. These responsibilities might include providingthe external evaluator with timely and appropriate guidance and reviewing and approving evaluation instru-ments and documents in a timely and constructive manner. Your responsibilities might also include providingthe evaluator with program appropriate program records and other information, and assisting the evaluator insolving problems that arise during the evaluation.Guideline 3: Work closely with the evaluator. Successful evaluations are the result of good partnerships. Your involvement − and that of otherprogram stakeholders − is essential to the success of the evaluation. Your involvement in the evaluation proc-ess should not be limited to periodic meetings with the evaluator. The greater your participation in all of thevarious aspects of the evaluation, from planning to final report, the more complete the evaluation will be, andthe greater your understanding of the evaluation results and recommendations will be. You should continually monitor the evaluator and the evaluation. Review all major work elements, includingsampling plans, instrumentation, data collection plans, etc. You and your staff and partners can be of consider-able service to the evaluator by facilitating involvement with host sites, service recipients, and others fromwhom the evaluator wishes to collect information. Solicit feedback from your evaluator as the study progresses. Informal insights gained during the study pe-riod can be valuable. Finally, manage the process actively to make sure you get what you need. A committeethat is intermittently involved in the evaluation process may discover that the study has gone in an inappropri-ate direction after considerable time, money, and effort have been expended. As your evaluation nears completion, you and the evaluator should agree on a format for the evaluation re-port. The evaluation report should address evaluation questions directly and briefly and should be understand-able to the target audience. Any report, of course, must provide useful and direct guidance for program deci-sion makers. You and the evaluator should also agree at this time on the evaluators role in the release of theevaluations results. You might request that the evaluator be available to meet with decision makers, conductinterviews with news media, and make public presentations of the results. Be involved in the development of the final report. This is not to say that you decide what the evaluator re-ports; the point of an external evaluation is to get an independent assessment your program. However, yourknowledge of the target audience will be valuable in framing the report, and your knowledge of policy makers 7
  9. 9. How to Hire the Right Evaluatorfuture plans for the program will help set the context for any recommendations made by the evaluators. Youshould carefully review a draft of the final report and provide detailed, written feedback to the evaluator to usein developing the final product. A cautionary note: If the changes suggested to the evaluator are significant and would have the effect ofchanging the findings, recommendations, or overall focus of the report, they should be discussed at a meetingwith the evaluator. If the evaluator does not believe that the suggested changes are consistent with the data,the evaluator has the right (and an ethical obligation) to be disassociated from the report. In such an instance,the evaluator may make the requested changes and assign authorship of the report to the program, or mayhand over all work to the program to complete the report. Such situations should be avoided, however, as theyusually place the integrity and public acceptance of the report in jeopardy.Guideline 4: Use the evaluation results. Once your study is complete, its value will be very limited unless the results are used to improve your pro-gram. Use the insights gained to review your program design and modify, or replicate, program elements asappropriate. Use your evaluator as a resource in this activity. Your evaluators assistance in interpreting theresults for their program management implications can be very valuable. Address the most compelling findingsfirst. Dont get overwhelmed trying to do everything at once. Make note of findings that suggest further areasof investigation. Future evaluations may shed light on unclear data. Use your study not only to improve operations, but to market your program. Potential funders and servicepartners will be more willing to become involved with your program if you can show them evidence of effec-tiveness and a willingness to make improvements based on evaluation data. Report the results of your evaluation not only to your staff and other program stakeholders, but also to fi-nancial contributors, key community leaders, and organizational partners. Contact your local newspaper aboutthe possibility of doing a feature article on your program. Have program participants − especially youth mem-bers − develop exhibitions including the evaluation results for use in community presentations and other activi-ties. Present your results at professional conferences and other "issue networks" (groups with a common inter-est).Conclusion A program evaluation properly designed to provide interested individuals with information that they will usewill likely be a successful evaluation. However, while utilization of the results of the evaluation is clearly theprimary goal of an evaluation, important benefits may also be gained from the evaluation process itself. Theprocess of undergoing an evaluation can, for example, build shared meaning and understanding, support andenhance the program (by building evaluation-based data collection and analysis into the program design), andsupport human and organizational development by training staff in new skills. A well conceived evaluation willprovide benefits no matter the results. A well-conceived evaluation results from a good working partnerships between program stakeholders andevaluators. An effective functional partnership is founded on agreement on the objectives of the evaluation, anunderstanding of the responsibilities and authority of each partner, and mutual respect for the contributionsthat each partner provides to the evaluation. If the steps identified here are followed, the evaluation processshould be enjoyable and productive for both the program and the external evaluator. 8
  10. 10. How to Hire the Right EvaluatorEvaluator Rating Guide Candidate Name:_______________________________________ 2 1 0 -1 -21. How well does the evaluator under- Clearly understands the No Opinion Has no understanding of stand the focus of the evaluation? focus, including the dis- the evaluation focus or tinction between evalua- the distinction between tion and research evaluation and research 2 1 0 -1 -22. How well does the evaluator under- Has an excellent under- No Opinion Has no understanding of stand the program goals and objec- standing of program program goals and ob- tives? goals and objectives jectives 2 1 0 -1 -23. How well do the proposed evaluation Evaluation questions fully No Opinion Evaluation questions to questions address the program goals address program goals not address program and objectives? and objectives goals and objectives 2 1 0 -1 -24. How appropriate is the proposed Appropriate: can clearly No Opinion Not appropriate: cannot methodology for the evaluation see that the results ob- see how results obtained questions? tained will be useful might help program 2 1 0 -1 -25. How reasonable is the timeline of The evaluation can be No Opinion The evaluation cannot be evaluation activities? conducted well within the completed within the proposed time frame proposed time frame 2 1 0 -1 -26. How relevant and useful are the Very useful for program No Opinion Not useful for program proposed deliverables? improvement improvement 2 1 0 -1 -27. How reasonable is the proposed Evaluation is very likely No Opinion Evaluation cant be con- budget? to be completed within ducted with available the available resources resources 2 1 0 -1 -28. How useful are the candidate’s pre- Previous reports were No Opinion Previous reports were not vious evaluation reports? understood and used understood or used 2 1 0 -1 -29. Does the candidate have good refer- Received excellent No Opinion Received poor references ences? references 2 1 0 -1 -210. How effectively does the evaluator Communicates effectively No Opinion Communicates effectively communicate with you and other with most stakeholders with few stakeholders stakeholders? 2 1 0 -1 -211. What is the candidates prior evalua- Has extensive experience No Opinion Has limited evaluation tion experience? evaluating similar pro- experience grams 2 1 0 -1 -212. Will the candidates existing profes- Commitments will not No Opinion Commitments will inter- sional commitments interfere with interfere with evaluation fere with evaluation the planned evaluation? 2 1 0 -1 -213. What is your general reaction to the Definitely hire as our No Opinion Definitely DO NOT hire candidate? evaluator has our evaluator 9
  11. 11. How to Hire the Right Evaluator Evaluator Conflict of Interest Questionnaire Name of Program:_______________________________________ Yes No1. Do you, your immediate family, or business partners have financial or other interests in the program to be evaluated?2. Have you been employed by the program within the last 24 months?3. Do you plan to obtain a financial interest, e.g. stock, in the program?4. Do you plan to seek or accept future employment the program?5. Are there any other conditions that may cause a conflict of interest?If you answered Yes to any of the above questions, please provide a written explanation ofyour answer.I declare all of the above questions are answered truthfully and to the best of my knowledge.signature date 10