Comparative & Non-Comparative Evaluation in Educational Technology Dr. Alaa Sadik Department of Instructional & Learning Technologies www.alaasadik.net [email_address] April 2009

Comparative and Non-Comparative Evaluation in Educational Technology Review of the literature shows that evaluation research in educational technology falls into four major categories: Developmental studies Cost-benefit analysis studies Perception and performance studies Non-comparative studies

Review of the literature shows that evaluation research in educational technology falls into four major categories:

Developmental studies

Cost-benefit analysis studies

Perception and performance studies

Non-comparative studies

Comparative and Non-Comparative Evaluation in Educational Technology Developmental studies The most common type of developmental studies is conducted to investigate the strengths and weakness of the design of technology (e.g., on-line environments).

Developmental studies

The most common type of developmental studies is conducted to investigate the strengths and weakness of the design of technology (e.g., on-line environments).

Comparative and Non-Comparative Evaluation in Educational Technology Developmental studies The purpose of these studies is to suggest models to be followed in further development research.

Developmental studies

The purpose of these studies is to suggest models to be followed in further development research.

Comparative and Non-Comparative Evaluation in Educational Technology Developmental studies Examples The following examples compare the potential and benefits of on-line learning to traditional method.

Developmental studies

Examples

The following examples compare the potential and benefits of on-line learning to traditional method.

Comparative and Non-Comparative Evaluation in Educational Technology Developmental studies Example A: “ The results revealed that most students favored the user-friendly design of the user interface and a straightforward navigation system. However, the major weakness was found in the design of discussion boards that require reading, jumping forward and back ” .

Developmental studies

Example A:

“ The results revealed that most students favored the user-friendly design of the user interface and a straightforward navigation system. However, the major weakness was found in the design of discussion boards that require reading, jumping forward and back ” .

Comparative and Non-Comparative Evaluation in Educational Technology Developmental studies Example B: “ Some students did not understand the concept of threading and posting to existing messages, and as a result, new threads were frequently incorrectly posted. This often made the direction of the discussion confusing ” .

Developmental studies

Example B:

“ Some students did not understand the concept of threading and posting to existing messages, and as a result, new threads were frequently incorrectly posted. This often made the direction of the discussion confusing ” .

Comparative and Non-Comparative Evaluation in Educational Technology Developmental studies Example C: “ Students’ feedback and behavior in discussions emphasized the importance of instructor’s participation, the need to reduce the number of topics per forum and the need to prepare students to learn and interact on-line ”.

Developmental studies

Example C:

“ Students’ feedback and behavior in discussions emphasized the importance of instructor’s participation, the need to reduce the number of topics per forum and the need to prepare students to learn and interact on-line ”.

Comparative and Non-Comparative Evaluation in Educational Technology Cost-benefit analysis of technology The aim of these studies is to investigate the design elements that should be considered in costing the technology.

Cost-benefit analysis of technology

The aim of these studies is to investigate the design elements that should be considered in costing the technology.

Comparative and Non-Comparative Evaluation in Educational Technology Cost-benefit analysis of technology Examples The following studies were conducted to analyze and compare the cost and evaluate the effectiveness of training courses provided in different formats (online, broadcasting, printed materials).

Cost-benefit analysis of technology

Examples

The following studies were conducted to analyze and compare the cost and evaluate the effectiveness of training courses provided in different formats (online, broadcasting, printed materials).

Comparative and Non-Comparative Evaluation in Educational Technology Cost-benefit analysis of technology Example A: “ The results of cost-benefit analysis showed that although Web-based education seems to have higher fixed costs than the traditional campus, the cost per student was offset by lower variable ”.

Cost-benefit analysis of technology

Example A:

“ The results of cost-benefit analysis showed that although Web-based education seems to have higher fixed costs than the traditional campus, the cost per student was offset by lower variable ”.

Comparative and Non-Comparative Evaluation in Educational Technology Cost-benefit analysis of technology Example B: “ Compared to printed materials, there is a need to consider three variables in developing and delivering on-line courses. These variables are the number of students per course, multimedia objects in the content and the live presence of the instructor during delivery ”.

Cost-benefit analysis of technology

Example B:

“ Compared to printed materials, there is a need to consider three variables in developing and delivering on-line courses. These variables are the number of students per course, multimedia objects in the content and the live presence of the instructor during delivery ”.

Comparative and Non-Comparative Evaluation in Educational Technology Cost-benefit analysis of technology Example C: “ Benefits include student/instructor satisfaction, increased access, flexibility and ease of use as well as other ‘value added’ benefits (e.g. reduce traffic and pollution and the potential of new market) ”.

Cost-benefit analysis of technology

Example C:

“ Benefits include student/instructor satisfaction, increased access, flexibility and ease of use as well as other ‘value added’ benefits (e.g. reduce traffic and pollution and the potential of new market) ”.

Comparative and Non-Comparative Evaluation in Educational Technology Learners’ perception and performance A common area of evaluation aims to investigate the effectiveness of technology in students’ perception and learning outcomes, in comparison with traditional method.

Learners’ perception and performance

A common area of evaluation aims to investigate the effectiveness of technology in students’ perception and learning outcomes, in comparison with traditional method.

Comparative and Non-Comparative Evaluation in Educational Technology Learners’ perception and performance Examples The following examples investigate students’ perception toward online learning compared to traditional method.

Learners’ perception and performance

Examples

The following examples investigate students’ perception toward online learning compared to traditional method.

Comparative and Non-Comparative Evaluation in Educational Technology Learners’ perception and performance Example A “ On-line learning could be as effective as traditional instruction, or better ”

Learners’ perception and performance

Example A

“ On-line learning could be as effective as traditional instruction, or better ”

Comparative and Non-Comparative Evaluation in Educational Technology Learners’ perception and performance Example B “ The on-line course allowed students to be engaged in synchronous (chat) and asynchronous (e-mail and forums) interaction to communicate with the instructor and with each other ”.

Learners’ perception and performance

Example B

“ The on-line course allowed students to be engaged in synchronous (chat) and asynchronous (e-mail and forums) interaction to communicate with the instructor and with each other ”.

Comparative and Non-Comparative Evaluation in Educational Technology Learners’ perception and performance Example C “ the performance of on-line students was similar to that of those in the on-campus course ”.

Learners’ perception and performance

Example C

“ the performance of on-line students was similar to that of those in the on-campus course ”.

Comparative and Non-Comparative Evaluation in Educational Technology Non-comparative studies Non-comparative studies examine factors that may be related to successful learning and students’ satisfaction or how students with different learning styles perform in a technology-based environment.

Non-comparative studies

Non-comparative studies examine factors that may be related to successful learning and students’ satisfaction or how students with different learning styles perform in a technology-based environment.

Comparative and Non-Comparative Evaluation in Educational Technology Non-comparative studies Examples The purpose of the following study was to identify relationships among student learning styles , learning strategies , and achievement

Non-comparative studies

Examples

The purpose of the following study was to identify relationships among student learning styles , learning strategies , and achievement

Comparative and Non-Comparative Evaluation in Educational Technology Non-comparative studies Example A “ The results indicated that learning styles and student characteristics did not have an effect on students’ achievement in online courses ”.

Non-comparative studies

Example A

“ The results indicated that learning styles and student characteristics did not have an effect on students’ achievement in online courses ”.

Comparative and Non-Comparative Evaluation in Educational Technology Non-comparative studies Example B “ students’ performances improved and were supported by the enhanced perception of the relevance of the practice and feedback , because students provided their own structure in the course and created their own interpersonal dialogue ”.

Non-comparative studies

Example B

“ students’ performances improved and were supported by the enhanced perception of the relevance of the practice and feedback , because students provided their own structure in the course and created their own interpersonal dialogue ”.

Comparative and Non-Comparative Evaluation in Educational Technology Application Summarizing a comparative study and non-comparative study and publishing them to blogger.com. Follow the guidelines provided by the tutor.

Application

Summarizing a comparative study and non-comparative study and publishing them to blogger.com. Follow the guidelines provided by the tutor.