1) Dominique Turnbow and Talitha Matlin created an online tutorial to teach students how to evaluate health websites after previously providing in-person workshops. 2) The tutorial was designed using Articulate Presenter and included practice activities and assessments to evaluate student learning. 3) Based on student assessments, 95% could recall evaluation criteria and 75% could create search strategies, and students found the interactive activities and examples helpful for learning. 4) Future improvements could include adding more examples, practice exercises throughout, and audio/video to accommodate different learning styles.

1. Beyond walls: moving from in-person to online instruction to teach evaluation skills Dominique Turnbow, UC San Diego Talitha Matlin, San Jose State University March 2009

2. Background UC San Diego & Student Health Services Student Health Advocates (SHAs) Library Partnership Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

3. In-Person Workshop Outcomes: Identify and apply criteria to critically evaluate websites Logistics Format – Think/Pair/Share Factors that led to the change Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

4. Tutorial Design Software Articulate Presenter (PowerPoint plug-in) Quia Time – about 70 hours to design tutorial Included practice activities for students to self-assess their learning Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

5. Tutorial The features of the tutorial we would like to point out today: Tutorial layout Organization of content Help MLGSCA Technology Symposium, March 6, 2009 Turnbow & Matlin

6. Assessment Students required to complete assessment exercises at the end of the tutorial to receive credit Librarian provided individual feedback to students via e-mail based on their responses to assessment questions Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

7. Assessment Questions asked students to: Recall criteria (and definitions) used to critically evaluate online health information Evaluate a web site about teen suicide using each criterion Create a search strategy they would use to find statistics about smoking and incidence of lung cancer Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

8. Assessment How did students do? 95% of students were able to correctly recall evaluative criteria and definitions 75% were able to create a sound search strategy to find health statistics after the tutorial Evaluate a web site about using each criterion Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

9. Results – Evaluation (based on the results of 13 out of 40 responses) Time spent on tutorial 11-35 minutes (reported by Quia, which only tracks time once the “quiz” is started) 11-60 minutes (reported by students) Useful topics covered in tutorial How to find health statistics – 40% Criteria to evaluate online health information – 33% Tools for citing resources – 27% Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

10. Results – Evaluation (based on the results of 13 out of 40 responses) Attachments downloaded by students Criteria for evaluating web sites – 48% RefWorks Quick Start Guide – 38% Received the attachments from TA – 10% Was the tutorial worth your time? Yes – 100% No – 0% Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

11. Results - Evaluation (based on the results of 13 out of 40 responses) Students liked: “it was very interactive (rather than just a typical read-only tutorial)” “I was able to look at and compare real [web site] examples for myself” “activities were helpful – being able to look over the sites and then answer the quiz questions was a good way of ensuring that I was looking at the appropriate aspects of each site” “the quizzes” (i.e. practice activities) Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

12. Results - Evaluation Improvements recommended by students: “more examples of bad web sites” “narration” “music” “more quizzes/website activities” “make it available to other health-field bound students” Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

13. In-Person vs. Online Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

14. Future Improvements Incorporate “practice exercises” and formative assessment if possible e.g. “Practice exercises” at the end of any section in the Bioengineering 1 tutorial Add audio narration and videos of search strategies to accommodate additional learning styles e.g. “movies” in Sections 3 and 4 of the Bioengineering 1 tutorial Add more examples, including “non-examples” Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

15. Questions? Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

16. Contact Us Dominique TurnbowUndergraduate Services Librarian, UCSDdturnbow@ucsd.edu Talitha MatlinSan José State University (student)tmatlin@gmail.com Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009

17. Links Student Health Services Tutorialhttp://biomed.ucsd.edu/dturnbow/shs/player.html Practice Exercise (Quia)http://www.quia.com/quiz/1380519.html Assessment (Quia)http://www.quia.com/quiz/1380694.html Bioengineering 1 Tutorial(includes examples of the improvements for the next version of the SHS Tutorial – e.g. videos, practice exercises within the Articulate module, etc. )http://biomed.ucsd.edu/dturnbow/beng1_tutorial/player.html Turnbow & Matlin MLGSCA Technology Symposium, March 6, 2009