The document discusses different models for evaluating educational technology: 1. Kirkpatrick's Four-Level Model measures reaction, learning, behavior change, and results. It is the best known model. 2. Bates' ACTIONS Model evaluates access, costs, teaching/learning fit, interactivity, organization, novelty, and speed of different technologies. 3. The EITIMI Model considers elements, instructional design, and impact in a three-step evaluation of technologies.

1. TECH4102 Evaluation in Educational Technology Dr. Alaa Sadik Department of Instructional & Learning Technologies www.alaasadik.net [email_address]

2. TECH4102 Evaluation in Educational Technology Week 4 Models of Evaluation in Educational Technology

3. Before Evaluation… What makes quality teaching and learning? What are the variables that affect technology-based learning and need to be evaluated? What instruments can be used to evaluate the quality of technology-based learning and provide designers with information for improvement? Try to answer the following questions with your team, then share the findings with us.

4. Models of Evaluation A Model is a hypothetical description of a complex entity or process. In educational technology, an evaluation model is used to provide a framework for selecting, evaluating and improving the learning process.

5. Models of Evaluation Models of Evaluation in Educational Technology. Kirkpatrick’s Four-Level Model (1975) Bates’ ACTIONS Model (1995) EITIMI Model (1999)

6. Models of Evaluation Kirkpatrick’s Four-Level Model (1975) A four-step model used to evaluate technology-based training as well as traditional forms of delivery. Reaction Learning Behavior Results

7. Kirkpatrick’s Model: The best known model

8. Kirkpatrick’s Model 1. Reaction It measures how well students liked the program and technology, and includes questions about: The relevance of the objectives. The ability of the course to maintain interest. The appropriateness of interactive exercises. The ease of navigation. The perceived value.

9. Kirkpatrick’s Model 2. Learning It measures students’ learning results and whether students actually learn the knowledge, skills, and attitudes the program was supposed to teach, or not.

10. Kirkpatrick’s Model 3. Behavior/Transfer in the Workplace Attempts to answer whether or not students’ behaviors actually change as a result of new learning technology, and conducted three to six months after the program.

11. Kirkpatrick’s Model 4. Results Measures the success of the program in terms that managers and decision-makers can understand. Examples are, increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment.

12. Models of Evaluation Bates’ ACTIONS Model (1995) A generic framework developed at the British Open University to help in analyzing and selecting the appropriate technology. Access How accessible is a particular technology for learners? How flexible is it for a particular target group?

13. Models of Evaluation Costs What is the cost structure of each technology? What is the unit cost per student? What are the opportunity costs vs. other technology choices? Teaching and Learning What instructional approaches will best meet these needs? Can the content be adapted to the technology? What skills does the technology develop?

14. Models of Evaluation Interactivity and User-friendliness What kind of interaction does this technology enable? How easy is it to use and learn? Organizational Issues What are the organizational requirements to provide stability and support? What are the barriers to using this technology successfully within the institution? What changes need to be made within the organization to incorporate the technology?

15. Models of Evaluation Novelty How new is the technology? What are its technical capabilities? Speed How quickly can courses be created and distributed with this technology? How quickly can materials be changed?

16. Models of Evaluation EITIMI Model (1999) A three-step model considers the elements of the learning process and used for selection and evaluation of technologies to enhance the quality of teaching and learning.

17. Models of Evaluation EITIMI Model Elements

18. EITIMI Model: A Medical Course Example

19. Discussion & Conclusion Lap application: Apply one of the models above to evaluate an educational technology project.