E-learning Research Article Presentation

3,132 views
2,996 views

Published on

UAA
EDAE 637

Published in: Education, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,132
On SlideShare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

E-learning Research Article Presentation

  1. 1. e-Learning Research Review Presentation Presented by Liberty Joy Open Book Testing in Online Learning Environments
  2. 2. Full Citation <ul><li>Rakes, G. (2008). Open book testing in online learning environments. Journal of Interactive Online Learning, 7 ( 1), 1-9. </li></ul>
  3. 3. What did the book say? Online course and program offerings have penetrated the mainstream of colleges and universities across the United States One continuing concern associated with online courses is assessment of student performance A variety of performance assessments, including examinations, should be employed to assure quality in online instruction One option for online assessment is the use of open book tests
  4. 4. Open Book testing <ul><li>Promotes more realistic learning opportunities </li></ul><ul><li>Emphasizes higher order thinking skills </li></ul><ul><li>Relates more closely to real-world work environments. </li></ul><ul><li>Eliminates the need for total rote memorization </li></ul><ul><li>Allows for use of reference materials </li></ul><ul><li>Better measures students’ ability to organize , use and apply information </li></ul>
  5. 5. Review of Literature Most of the related research available concerning open book testing is based on traditional, face-to-face classroom settings Past studies maintain different views on open book testing as well as student performance on such tests Students tend to not prepare adequately for open book tests; not investing as much study time or effort into preparation compared to preparation for traditional, closed book, supervised examinations Open book examinations reduce anxiety levels in students Tendency to “cram” at the last minute and dependency on rote memorization is reduced Comprehensive student preparation and more consistent learning during a course
  6. 6. Research Motivation Lack of preparation on open book examinations may be related to the fact that anxiety is reduced , to the point students believe little or no preparation is required Mild anxiety can enhance learning and performance on assessments by focusing learners’ attention Lack of readiness may also be due to a lack of understanding differences between preparation for traditional, proctored tests and preparation for open book tests Students may not understand what strategies are needed to succeed on open book examinations, spending too much time with books and notes during the test period, which limits their ability to successfully complete the test
  7. 7. Evaluate the impact of training in open book test-taking strategies on student performance in online, timed, open book, unproctored test environments <ul><li>Students provided with Web-based training in open book testing strategies would score significantly higher than students who are not provided with the training </li></ul><ul><li>There would be no difference in test scores when training materials were not administered immediately before the test </li></ul>How to survive open book test day intent hypotheses
  8. 8. 1) Is there a difference in midterm examination scores for students who were provided training in open book testing strategies immediately before the midterm and midterm scores for students who were not provided any training? 2) Is there a difference in final examination scores for students who were provided training in open book testing strategies only before the midterm examination and final exam scores for students who were not provided any training? Research Questions
  9. 9. Convenience sample - 122 masters-level graduate students enrolled in eight sections of the same online introductory instructional technology course in a teacher education program accredited by the National Council for the Accreditation of Teacher Education (NCATE) at a public, mid-southern university Control group - 71 students who took online, unproctored, open book examinations in Blackboard No training regarding open book test taking strategies The experimental group - 51 students enrolled in same course Provided Web-based training regarding open book test taking strategies before the midterm examination only Study Sample
  10. 10. Method <ul><li>100 point multiple choice midterm and final examinations based on standardized tests provided by the textbook publisher for the current edition </li></ul><ul><li>Exams administered through the Blackboard course management system </li></ul><ul><li>Imposed 90 minute time limit </li></ul><ul><li>Exams contained both recall and application-level questions (1/3 questions on midterm and ¼ on final) </li></ul><ul><li>A required practice test to experience the Blackboard test environment </li></ul><ul><li>Web based training materials covering open-book testing strategies were developed based on the review of literature and provided to the experimental group prior to the midterm examination: http://www.utm.edu/staff//grakes/750/tutorial </li></ul>
  11. 11. <ul><li>Information regarding open book test-taking strategies was divided into three major categories in addition to an overview: </li></ul><ul><li>Organize Your Information </li></ul><ul><li>Manage Your Time </li></ul><ul><li>Prepare the Environment </li></ul><ul><li>Experimental group required to review the open book testing tutorial 24-72 hours prior to the midterm examination and successfully complete a 21-question quiz with 100% score (students allowed to take the quiz as many times as necessary) </li></ul>Web-Based Tutorial
  12. 12. Results to First Research Question Is there a difference in midterm examination scores for students who were provided training in open book testing strategies immediately before the midterm and midterm scores for students who were not provided any training?
  13. 13. When the tutorial was required before beginning the examination, the experimental group ( n =51) scored significantly higher ( M =88.4; SD =8.3) than the control group ( n =71) that did not complete the tutorial ( M =84.8; SD =9.3). The Cohen’s d effect size of the difference in means was moderate
  14. 14. Results to Second Research Question Is there a difference in final examination scores for students who were provided training in open book testing strategies only before the midterm examination and final exam scores for students who were not provided any training?
  15. 15. When the tutorial was completed immediately before the midterm examination but was not completed before the final examination, the experimental group’s ( n =51) mean final examination score ( M =87.0; SD =8.6) was higher than the control group’s ( n =71) mean final examination score ( M =86.4; SD =5.4), but the difference was not significant.
  16. 16. <ul><li>Open book test performance may be adversely affected due to not understanding proper preparation </li></ul><ul><li>Training in open book testing strategies may improve the inclination to study less for open book examinations </li></ul><ul><li>Preparation immediately before examination may result in significantly better scores </li></ul><ul><li>Lack of significance effect on final exam scores may indicate the study sample may not have enough experience with open book testing for one training experience to make a lasting change in test taking </li></ul>Discussion of Results
  17. 17. Author’s Recommendations for Future Research <ul><li>Effective assessment strategies as online learning environments increase </li></ul><ul><li>Possibility for open book testing in online courses using a larger sample </li></ul><ul><li>Longitudinal comparison of traditional and online assessments </li></ul><ul><li>Studies evaluating online learning and assessment in graduate level students </li></ul><ul><li>Exploration of the use of open book testing for application-level knowledge </li></ul><ul><li>Types of assessments that are optimal for specific course objectives </li></ul><ul><li>The effects of open book testing on academic honesty in online courses </li></ul>
  18. 18. Article Analysis
  19. 19. Importance of the research problem? E-learning communities and programs are rapidly growing and becoming an integral component of education, therefore adequate evaluation/assessment/grading methods must be examined. This research study presented an introductory view of one method of learner evaluation and resources to encourage positive learning practices. As the author mentioned, learner assessment in online communities is an area that would benefit from further research and is in fact a worthy and important area to be examined .
  20. 20. How well defined is the purpose? The intent of this study is clearly delineated: Evaluate the impact of training in open book test-taking strategies on student performance in online, timed, open book, unproctored test environments.
  21. 21. How well-developed and appropriate is the literature review? The literature review is comprehensive encompassing both studies in support and against open-book testing. There is an obvious absence of citations of studies performed in e-learning environments, in part due to the fact that online learning is in its infancy. How sound are the methodological approaches? The study employs ANOVA testing to analyze the difference of variance between the two independent sample groups. The effect size was analyzed through the partial eta-squared formula. The significance level was set p <0.05. The methodology appears to be appropriate for the testing. How well-organized, well-written, and readable is the article? The article is presented in a clearly outlined and organized manner. The details are presented in a straightforward, easily readable fashion.
  22. 22. <ul><li>Thorough, well written article with empirical data </li></ul><ul><li>Vast opportunity for future study as field grows and evolves </li></ul><ul><li>Delivers informative view of e-learning assessment options </li></ul>Concluding Thoughts
  23. 23. Thank You for joining me!

×