TECH4102 Evaluation in Educational Technology Dr. Alaa Sadik Department of Instructional & Learning Technologies www.alaasadik.net [email_address]

TECH4102 Evaluation in Educational Technology Week 4 Models of Evaluation in Educational Technology

Before Evaluation… What makes quality teaching and learning? What are the variables that affect technology-based learning and need to be evaluated? What instruments can be used to evaluate the quality of technology-based learning and provide designers with information for improvement? Try to answer the following questions with your team, then share the findings with us.

What makes quality teaching and learning?

What are the variables that affect technology-based learning and need to be evaluated?

What instruments can be used to evaluate the quality of technology-based learning and provide designers with information for improvement?

Models of Evaluation A Model is a hypothetical description of a complex entity or process. In educational technology, an evaluation model is used to provide a framework for selecting, evaluating and improving the learning process.

A Model is a hypothetical description of a complex entity or process.

In educational technology, an evaluation model is used to provide a framework for selecting, evaluating and improving the learning process.

Models of Evaluation Models of Evaluation in Educational Technology. Kirkpatrick’s Four-Level Model (1975) Bates’ ACTIONS Model (1995) EITIMI Model (1999)

Models of Evaluation in Educational Technology.

Kirkpatrick’s Four-Level Model (1975)

Bates’ ACTIONS Model (1995)

EITIMI Model (1999)

Models of Evaluation Kirkpatrick’s Four-Level Model (1975) A four-step model used to evaluate technology-based training as well as traditional forms of delivery. Reaction Learning Behavior Results

Kirkpatrick’s Four-Level Model (1975)

A four-step model used to evaluate technology-based training as well as traditional forms of delivery.

Reaction

Learning

Behavior

Results

Kirkpatrick’s Model: The best known model

Kirkpatrick’s Model 1. Reaction It measures how well students liked the program and technology, and includes questions about: The relevance of the objectives. The ability of the course to maintain interest. The appropriateness of interactive exercises. The ease of navigation. The perceived value.

1. Reaction

It measures how well students liked the program and technology, and includes questions about:

The relevance of the objectives.

The ability of the course to maintain interest.

The appropriateness of interactive exercises.

The ease of navigation.

The perceived value.

Kirkpatrick’s Model 2. Learning It measures students’ learning results and whether students actually learn the knowledge, skills, and attitudes the program was supposed to teach, or not.

2. Learning

It measures students’ learning results and whether students actually learn the knowledge, skills, and attitudes the program was supposed to teach, or not.

Kirkpatrick’s Model 3. Behavior/Transfer in the Workplace Attempts to answer whether or not students’ behaviors actually change as a result of new learning technology, and conducted three to six months after the program.

3. Behavior/Transfer in the Workplace

Attempts to answer whether or not students’ behaviors actually change as a result of new learning technology, and conducted three to six months after the program.

Kirkpatrick’s Model 4. Results Measures the success of the program in terms that managers and decision-makers can understand. Examples are, increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment.

4. Results

Measures the success of the program in terms that managers and decision-makers can understand.

Examples are, increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment.

Models of Evaluation Bates’ ACTIONS Model (1995) A generic framework developed at the British Open University to help in analyzing and selecting the appropriate technology. Access How accessible is a particular technology for learners? How flexible is it for a particular target group?

Bates’ ACTIONS Model (1995)

A generic framework developed at the British Open University to help in analyzing and selecting the appropriate technology.

Access

How accessible is a particular technology for learners?

How flexible is it for a particular target group?

Models of Evaluation Costs What is the cost structure of each technology? What is the unit cost per student? What are the opportunity costs vs. other technology choices? Teaching and Learning What instructional approaches will best meet these needs? Can the content be adapted to the technology? What skills does the technology develop?

Costs

What is the cost structure of each technology?

What is the unit cost per student?

What are the opportunity costs vs. other technology choices?

Teaching and Learning

What instructional approaches will best meet these needs?

Can the content be adapted to the technology?

What skills does the technology develop?

Models of Evaluation Interactivity and User-friendliness What kind of interaction does this technology enable? How easy is it to use and learn? Organizational Issues What are the organizational requirements to provide stability and support? What are the barriers to using this technology successfully within the institution? What changes need to be made within the organization to incorporate the technology?

Interactivity and User-friendliness

What kind of interaction does this technology enable?

How easy is it to use and learn?

Organizational Issues

What are the organizational requirements to provide stability and support?

What are the barriers to using this technology successfully within the institution?

What changes need to be made within the organization to incorporate the technology?

Models of Evaluation Novelty How new is the technology? What are its technical capabilities? Speed How quickly can courses be created and distributed with this technology? How quickly can materials be changed?

Novelty

How new is the technology?

What are its technical capabilities?

Speed

How quickly can courses be created and distributed with this technology?

How quickly can materials be changed?

Models of Evaluation EITIMI Model (1999) A three-step model considers the elements of the learning process and used for selection and evaluation of technologies to enhance the quality of teaching and learning.

EITIMI Model (1999)

A three-step model considers the elements of the learning process and used for selection and evaluation of technologies to enhance the quality of teaching and learning.

Models of Evaluation EITIMI Model Elements

EITIMI Model Elements

EITIMI Model: A Medical Course Example

EITIMI Model: A Medical Course Example

Discussion & Conclusion Lap application: Apply one of the models above to evaluate an educational technology project.