Evaluation of learning resources


Published on

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Evaluation of learning resources

  1. 1. Damian Gordon
  2. 2.    There is a plethora of learning materials of all sorts now available on-line. The range of ways in with these materials are produced and consumed means that there cannot be one single way to evaluate these materials. However there is a major issue with quality control with regard to this material. Continued:
  3. 3.    A great deal of material is developed without any evidence of it ever having being tested on learners. A logical way to evaluate the quality of learning materials is to try it out on learners and get their feedback (both formative and summative). It perhaps make most sense to submit this learner feedback in conjunction with the learner materials to verify that the materials have been tested by learners.
  4. 4.  There are two main evaluation tools for learning materials ◦ MERLOT ◦ LORI  Two slightly less used tools are ◦ MicroSoft Evaluators Guide ◦ National Council for Educational Technology for evaluating CD-DROM
  5. 5.  MERLOT (www.merlot.org) is a repository containing educational resources classified into seven broad subject categories: Arts; Business; Education; Humanities; Mathematics and Statistics; Science and Technology; Social Sciences.
  6. 6.  There are three general categories of evaluation standards used within MERLOT: 1. Quality of Content 2. Potential Effectiveness as a Teaching-Learning Tool 3. Ease of Use
  7. 7.   Quality of Content There are two general elements to quality of content: 1. Does the software present valid (correct) concepts, models, and skills? 2. Does the software present educationally significant concepts, models, and skills for the discipline? Continued:
  8. 8.   Quality of Content To evaluate the educational significance of the content, reviews use the following guidelines: Continued:
  9. 9.  Quality of Content ◦ Content is core curriculum within the discipline. ◦ Core curriculum topics are typically covered to some degree in the introductory classes within the discipline and/or "Everyone teaches it" and/or it is identified as a core area by the discipline's professional organizations ◦ Content is difficult to teach and learn. ◦ Content is a pre-requisite for understanding more advanced material in the discipline
  10. 10.   Potential Effectiveness as a Teaching-Learning Tool Evaluating POTENTIAL effectiveness is asking the Peer Reviewer to judge, based on his expertise as a teacher, whether the learning material is likely to improve teaching and learning given the ways the faculty and students could use the tool. Continued:
  11. 11.   Potential Effectiveness as a TeachingLearning Tool What stage(s) in the learning process/cycle could the materials be used? ◦ Explanation or description of the topic/stating the problem ◦ Demonstration of the curriculum/exploration of the problem ◦ Practice using the curriculum/analysis of the outcomes from solving the problem ◦ Applying the curriculum to "new" problems/application of the outcomes to other problems Continued:
  12. 12.    Potential Effectiveness as a Teaching-Learning Tool What is(are) the learning objective(s)? What should students be able to do after successfully learning with the materials? What are the characteristics of the target learner(s) Continued:
  13. 13.   Potential Effectiveness as a TeachingLearning Tool There are other general elements to effectiveness as a teaching-learning tool that MERLOT asks reviewers to consider: ◦ Does the interactive/media-rich presentation of material improve faculty and students' abilities to teach and learn the materials? ◦ Can the use of the software be readily integrated into current curriculum and pedagogy within the discipline? ◦ Can the software be used in a variety of ways to achieve teaching and learning goals? ◦ Are the teaching-learning goals easy to identify? ◦ Can good learning assignments for using the software application be written easily?
  14. 14.  Ease of Use  The basic question underlying the ease of use standard is: how easy it is for teachers and students to use the software for the first time?  Elements that affect ease of use include: Continued:
  15. 15.  Ease of Use ◦ Are the labels, buttons, menus, text, and general layout of the computer interface consistent and visually distinct? ◦ Does the user get trapped in the material? ◦ Can the user get lost easily in the material? ◦ Does the module provide feedback about the system status and the user's responses? ◦ Does the module provide appropriate flexibility in its use? ◦ Does the learning material require a lot of documentation, technical support, and/or instruction for most students to successfully use the software? ◦ Does the material present information in ways that are familiar for students? ◦ Does the material present information in ways that would be attractive to students?
  16. 16.   eLera is a distributed group that researches and evaluates e-learning. Their specific interests include learning objects, eportfolios, and learning design specifications, and related topics. eLera provides tools and information for learning object evaluation and research, maintains a database of learning object reviews, and supports communication and collaboration among researchers, evaluators and users of online learning resources.
  17. 17.  The goals of eLera are to: ◦ improve the quality of online learning resources through better design and evaluation ◦ develop effective pedagogical models that incorporate learning objects ◦ help students, teachers, professors, instructional designers and others to select pedagogical models and digital resources that meet their requirements
  18. 18.   The Learning Object Review Instrument (LORI) is used to evaluate the quality of e-learning resources. LORI is an online form consisting of rubrics, rating scales and comment fields. The current version of LORI available from eLera is version 1.5. http://www.elera.net/eLera/Home/Articles/L ORI%201.5.pdf
  19. 19. Nine Dimensions of LORI 1. 2. 3. 4. 5. 6. 7. 8. 9. Content Quality Learning Goal Alignment Feedback and Adaptation Motivation Presentation Design Interaction Usability Accessibility Reusability Standards Compliance
  20. 20. Nine Dimensions of LORI 1. Content Quality: Veracity, accuracy, balanced presentation of ideas, and appropriate level of detail
  21. 21. Nine Dimensions of LORI 2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics
  22. 22. Nine Dimensions of LORI 3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modelling
  23. 23. Nine Dimensions of LORI 4. Motivation: Ability to motivate and interest an identified population of learners
  24. 24. Nine Dimensions of LORI 5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing
  25. 25. Nine Dimensions of LORI 6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features
  26. 26. Nine Dimensions of LORI 7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners
  27. 27. Nine Dimensions of LORI 8. Reusability: Ability to use in varying learning contexts and with learners from differing backgrounds
  28. 28. Nine Dimensions of LORI 9. Standards Compliance: Adherence to international standards and specifications
  29. 29.  diagram here is showing individual reviews merging to a panel review and then published on web.
  30. 30.             INSTRUCTIONAL QUALITY The purpose of the package is well defined The package achieves its defined purpose Presentation of content is clear and logical The level of difficulty is appropriate to the target audience Graphics/colour/sound are used for appropriate instructional reasons Use of package is motivational The package effectively stimulates student creativity Feedback on student responses is effectively employed The learner controls the rate and sequence of presentation and review Instruction is integrated with previous student experience Learning is generalisable to an appropriate range of situations             CONTENT The content is accurate. The content has educational value The content is free of race, ethnic, sex, and other stereotypes TECHNICAL QUALITY The user support materials are comprehensive The user support materials are effective Informative displays are effective Intended users can easily and independently operate the program Teachers can easily employ the package The program appropriately uses relevant computer capabilities The program is reliable in normal use
  31. 31.                    1. Which computer system will the disc run on? 2. Will your computer system do justice to the illustrations? 3. Is the operation by keyboard or mouse, or both? 4. Can we have the disc for a trial period? 5. Is the language and spelling on the disc Queen’s English or American English? 6. How much bias is there in the content of the disc? 7. Is printing out easy and intuitive? 8. Can the selected material readily be down-loaded to disc? 9. Can subsections of the disc be searched? 10. Is the software to control the CD-ROM on the disc itself or is it supplied on a separate floppy disc? 11. Does the software manage memory resources well? 12. What search procedures are available? 13. What is the language level on the disc? 14. Is the user interface tolerant of typing and spelling errors? 15. Can you select exactly what you want to print out or save to disc? 16. Are there any supporting features? 17. Can the illustrations be printed out? 18. Can images be readily transferred? 19. Is there a sound capability to accompany the pictures?