505ferdon evaluationproposal


Published on

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

505ferdon evaluationproposal

  1. 1. Evaluation of Determining Instructional Purposes Training Program A Proposal Submitted to Far West Laboratory by Building Evaluation Solutions Together, Inc.IntroductionFar West Laboratory for Educational and Research Development (FWL) issued a request forproposal for an evaluation of its Determining Instructional Purposes (DIP) training program.This document is a proposal from Building Evaluation Solutions Together, Inc. (BEST) of Boise,Idaho submitted in response to the FWL request for proposal. The purpose is to present theBEST evaluation design, development, and implementation, which will provide FWL withcomprehensive, valid and reliable data on which to base marketing decisions related to DIPtraining materials.Description of Program Being EvaluatedThe program to be evaluated is the Determining Instructional Purposes (DIP) training programfor school administrators and graduate students, which is comprised of three self-contained units:Setting Goals, Analyzing Problems and Deriving Objectives. In each unit, trainees read materialsin each of four-to-six modules, and practice skills in individual or small group activities, withfeedback provided. Many of the practice activities place trainees in planning teams in whichskills are applied to the operations of a hypothetical school district. Training time for the threeunits varies, with an estimated range from 10 – 18 hours per unit, and is overseen by acoordinator who has been part of a prior training group. Training materials are intended for usein short-term workshops or individual sessions spread over a period of time. Materials are inprint form with unit length between 155 and 259 pages. Units are available separately ($8.95) oras a set ($24.95) and a Coordinator Handbook ($4.50) includes instructions for all three units.Materials have been developed but not yet disseminated and the purpose of the evaluation is todetermine the marketability of the product and value to potential purchasers.Evaluation MethodThe primary purpose of this evaluation is to provide information and recommendations regardingdissemination of the units. To do so, the BEST evaluators will gather information about thecontent of DIP materials, their use in training sessions, and their perceived effectiveness. TheBEST team of evaluators will gather information from three sources: program developers/staff, asubject matter expert, and potential users of the training materials. Observations, interviews,surveys, and review of documents will be used to collect data.Program developers and staff will be asked to 1) provide background information regarding theprocess of developing DIP training materials, 2) identify perceived strengths and weaknesses oftraining materials, 3) communicate marketing considerations: production schedule, targetaudience/region, cost comparison of digital vs. print, past practice regarding production andpromotion of new materials, and 4) share any additional information they feel is valuable.

  2. 2. Information from program developers and staff will be gathered through interviews with teamleaders, informal meetings with project staff, and review of documents.For the purpose of determining accuracy and comprehensiveness of the content, DIP trainingmaterials will be evaluated a subject matter expert. The Subject Matter Expert will be providedwith a full set of training materials and, upon review, will use a rating instrument to evaluate thecontent of DIP materials in the following categories: accuracy, completeness, relevance,timeliness, and clarity. The Subject Matter Expert will participate in a video conference with theBEST evaluator and FWL program director.For the purpose of determining usefulness and usability, two groups representing potential userswill pilot the material: school administrators and graduate students in educational administrationprograms. Administrators will be divided into three groups, with each group participating in a 3-4 day training session for one of the three DIP units. BEST teams, comprised of ResearchAssistants and Interns and overseen by the lead evaluator, will collect observational data astrainees work through each unit. All participants will complete an exit survey (content, context,impressions, usability) and a random sample of trainees will participate in a focus groupdiscussion. This process will be repeated with groups of graduate students. Participants in thetraining workshops will receive an honorarium along with a complementary set of trainingmaterials.Task ScheduleBegin Date: May 3, 2010End Date: July 22, 2010 TASK Responsible # of Days PartyDiscuss Proposal and revise Evaluation Plan as needed. FWL and 1 BESTCollect background information from FWL program director BEST 5and staff; document review.Revise Evaluation Plan and submit to FWL BEST 3FWL feedback on revisions to BEST FWL 3Final Evaluation Plan revisions BEST 1BEST Review of DIP Training Materials BEST 10Develop/Adapt data collection instruments BEST 5Recruit school administrators and graduate students to take part BEST 20in DIP training workshops.Subject Matter Expert review of DIP Training Materials, SME 10completion of rating instrument and report.

  3. 3. TASK Responsible # of Days PartyVideo conference with Subject Matter Expert FWL, BEST, 1 SME12 K-12 Administrators participate in DIP Training sessions BEST 3-4and complete an exit survey. FWL coordinator12 Graduate Students participate in DIP Training sessions and BEST 3-4complete an exit survey. FWL coordinatorFocus Group discussion with 5-6 trainees selected at random. BEST 1Data Analysis by BEST Research Assistants and Interns BEST 5Preparation of Final Report (Draft) BEST 5FWL feedback on Final Report FWL 5Final Report submitted to FWL BEST 2

  4. 4. Project PersonnelDr. Daniel L. Stufflebeam, Distinguished University Professor at Western Michigan University,serves as the BEST lead evaluator. With over 40 years of experience in the field, and a leader inthe development of efficient and effective evaluation models, Dr. Stufflebeam served as Directorof the Evaluation Center at University of Western Michigan from 1973 until 2002. He holds aB.A. in Music Education, and M.S. in Counseling and Psychology, and Dr. Stufflebeam’sdoctoral and post-doctoral work is in Measurement, Statistics, and Experimental Design.Currently on sabbatical from his duties at Western Michigan University, Dr. Stufflebeam leadsthe BEST team of evaluation professionals in a grass roots campaign to provide up-and-comingprofessionals in the field of evaluation with expert models and supervised field experiences inthe BEST Evaluator Mentoring Program. In his leisure time, Dr. Stufflebeam directs the BoiseCommunity Band.Subject Matter Expert: Dr. Esperto Elegante is currently Chair of Curriculum, Instruction, andFoundation Studies in the Boise State University College of Education. A former 7th gradescience teacher, principal and superintendent, Dr. Elegante’s area of expertise is professionaldevelopment in K12 education.Research Assistants: Recent graduates of Evaluator Certification Programs undergo a rigorousselection process and those admitted receive additional training as part of the BEST EvaluatorMentoring Program. With oversight by BEST leadership, Research Assistants help to gatherinformation, collect data, and administer surveys, utilizing current technologies throughout theprocess of gathering and analyzing data. Provided with a small stipend, Research Assistants gainvaluable experience with minimal cost to potential clients.Interns: Students currently enrolled in graduate programs related to research, education, andevaluation work under the supervision of Research Assistants and BEST evaluators. Chargedwith data entry and verification of data accuracy, hands-on experience benefits interns and theirservice is provided at no cost to clients.

  5. 5. BudgetFees for the evaluation of the Determining Instructional Purposes Training Programare not to exceed $50,000.Professional Salaries (daily) # Rate TotalDr. Daniel Stufflebeam 60 $450.00 $27,000.00Dr. Esperto Elegante 11 $400.00 $4,400.003 Research Assistants 35 $50.00 $1,750.003 Interns 35 $0.00 $0.00Project AdministrativeAssistant 45 $120.00 $5,400.00 $38,550.00HonorariumsStudy Participants -Administrators 12 $350.00 $4,200.00Study Participants - GraduateStudents 12 $100.00 $1,200.00 $5,400.00TravelOne 6-day round trip to FWLsite 6 $350.00 $2,100.00Estimated mileage 500 $0.30 $150.00 $2,250.00OperationsTelephone (per month) 3 $100.00 $300.00Postage $120.00Rent and utilities (per month) 3 1,000 $3,000.00 $3,420.00MaterialsDIP Training materials, sets ofthree unites (28 sets) to beprovided, at no cost, by FWL 28 $0.00 $0.00DIP Training materials,Coordinators Handbook (28)to be provided, at no cost, byFWL 28 $0.00 $0.00Supplies and photocoping $350.00 $350.00 TOTAL: $49,970.00