Your SlideShare is downloading. ×
#oersymposium2014 S5 P1 Paul Kawachi
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

#oersymposium2014 S5 P1 Paul Kawachi

203
views

Published on

2nd Regional Symposium on Open Educational Resources: …

2nd Regional Symposium on Open Educational Resources:
Beyond Advocacy, Research and Policy
24 – 27 June 2014

Sub-theme 5: Quality

The TIPS Quality Assurance Framework for Creating Open Educational Resources: Validation
Kawachi Paul

Published in: Education, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
203
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Open Education Network http://www.open- ed.net The TIPS Quality Assurance Framework for creating OER : Validation -------------------------------- ------ ------------------------------------ O E R OPEN EDUCATIONAL RESOURCESopen educational resources Commonwealth Educational Media Centre for Asia
  • 2. Open Education Network http://www.open- ed.net The TIPS Quality Assurance Framework for creating OER : Validation -------------------------------- ------ ------------------------------------ Paul Kawachi FRSA kawachi @open-ed.net http://www.open-ed.net /oer-quality/ validation.ppt
  • 3. Open Education Network http://www.open- ed.net overview TIPS Framework http://cemca.org.in/ckfinder/userfiles/files/ OERQ_TIPS_978-81-88770-07-6.pdf Version 1.0
  • 4. Open Education Network http://www.open- ed.net overview TIPS Frameworkfour layers of ideas T I P S
  • 5. Open Education Network http://www.open- ed.net overview TIPS Framework
  • 6. Open Education Network http://www.open- ed.net overview TIPS Framework from among 205 QA criteria the most comprehensive compilation to date 65 items indicated by 60+ global experts http://www.open-ed.net/oer-quality/criteria.pdf
  • 7. Open Education Network http://www.open- ed.net overview for teachers in class
  • 8. Open Education Network http://www.open- ed.net overview TIPS Framework 65 items are here subjected to delphi-style referral for content validation
  • 9. Open Education Network http://www.open- ed.net definitions : an open educational resource (OER) is defined as a digital self-contained unit of self-assessable teaching with an explicit measurable learning objective, having an open-licence clearly attached to allow adapting, and generally being free-of-cost to reuse UNESCO - CoL 2011 ; Kawachi 2013 introduction
  • 10. Open Education Network http://www.open- ed.net • wave analysis to confirm coverage - Leslie 1972 • content validity ratio analysis - Lawshe 1975 • multiple surveys and online discussions methods
  • 11. Open Education Network http://www.open- ed.net wave analysis three waves of responses from individual OER experts
  • 12. Open Education Network http://www.open- ed.net the three waves each constructed using the survey responses from ten individual OER experts are visually similar increasing confidence that a sufficient number of data are collected, and CVR analysis can proceed wave analysis
  • 13. Open Education Network http://www.open- ed.net wave analysis similar to fourth wave in gold from anon OER groups
  • 14. Open Education Network http://www.open- ed.net wave analysis fairly similar to waves 5 & 6 from individual teachers
  • 15. Open Education Network http://www.open- ed.net age age distribution for each set of responses
  • 16. Open Education Network http://www.open- ed.net age distribution : there was no clear normal distribution there are tendencies for the - Set-1 OER experts to be older - Set-2 OER group participants to be younger and - Set-3 teachers also to be relatively young age
  • 17. Open Education Network http://www.open- ed.net geographic distribution of all responses geographic CC BY SA blank world map image by anon from WikiMedia http://commons.wikimedia.org/wiki/File:World_map_blank_black_lines_4500px_monochrome.png
  • 18. Open Education Network http://www.open- ed.net survey population characteristics : gender Set-1 : individual OER experts Set-2 : group OER experts Set-3 : individual teachers N 35 14 21 male 21 7 0 38 female 13 7 11 31
  • 19. Open Education Network http://www.open- ed.net content validity ratio : survey asks content experts to rank each criterion as validation • Essential • Not-essential but useful • Not necessary
  • 20. Open Education Network http://www.open- ed.net content validity ratio : min av CVR for a criterion to be retained at p<0.05 validation
  • 21. Open Education Network http://www.open- ed.net content validity ratio : av CVRE for all items = CV Index for instrument ~ 0.80 for validity at p<0.05 validation Set-1 Set-2 Set-3 retaining 24 criteria gives overall CVI = 0.80 0.79 0.79
  • 22. Open Education Network http://www.open- ed.net content validity ratio : validation Set-1 Set-2 Set-3 ( 9th May 2014 ) N=32 N=13 N=19 retaining 21 criteria : overall CVI = 0.80 0.79 0.79 ( 5th June 2014 ) N=35 N=14 N=21 retaining 38 criteria : overall CVI = 0.84 0.81 0.79 ( 27th May 2014 ) N=35 N=13 N=19 retaining 24 criteria : overall CVI = 0.84 0.79 0.79
  • 23. Open Education Network http://www.open- ed.net content validity ratio : 5th June 2014 these 38 criteria retained are all those described in the shorter version TIPS Framework validated at CVI>0.80 at p<0.05 validation http://www.open-ed.net/oer-quality/tips.pdf
  • 24. Open Education Network http://www.open- ed.net content validity ratio : moreover the original 65 criteria of version 1.0 TIPS Framework has CVRE+U = 0.94 by the 35 OER experts indicating original was valid validation
  • 25. Open Education Network http://www.open- ed.net suggestions : six criteria indicated at highest CVRE by OER experts validation • Don't use difficult or complex language, and do check the readability to ensure it is appropriate to age / level • Make sure that the knowledge and skills you want the student to learn are up-to-date, accurate and reliable. Consider asking a subject-matter expert for advice • Be sure the open licence is clearly visible • Ensure your OER is easy to access and engage • Present your material in a clear, concise, and coherent way, taking care with sound quality • Use open formats for delivery of OER to enable maximum reuse and re-mix.
  • 26. Open Education Network http://www.open- ed.net Quality Assurance Guidelines for Open Educational Resources version 2.0 TIPS Framework revised criteria with some highlighted as more important and rubric added for Quality Improvement results version 2.0 38 criteria
  • 27. Open Education Network http://www.open- ed.net literature references : CoL (2011). UNESCO-CoL Guidelines for Open Educational Resources (OER) in Higher Education : OER Workshop, 25 May, Dar-es-Salaam. Retrieved April 25, 2014, from http://oerworkshop.weebly.com/uploads/4/1/3/4/4134458/2011.04.22.oer_guidelines_for_higher_education.v2.pdf Kawachi, P. (2013a). Quality assurance guidelines for open educational resources : TIPS framework, version 1.0. New Delhi, India : Commonwealth Educational Media Centre for Asia (CEMCA). Retrieved March 7, 2014, from http://cemca.org.in/ckfinder/userfiles/files/OERQ_TIPS_978-81-88770-07-6.pdf Kawachi, P. (2013b). Quality not static. OER Quality, blog posting 30th January. Retrieved March 7, 2014, from http://oerquality.wordpress.com/2013/01/30/quality-not-static/ Lawshe, C.H. (1975). A quantitative approach to content validity. Personnel Psychology, 28 (4), 563-575. Retrieved February 21, 2014, from http://www.bwgriffin.com/gsu/courses/edur9131/content/Lawshe_content_valdity.pdf Leslie, L.L. (1972). Are high response rates essential to valid surveys ? Social Science Research, 1, 323-334. references
  • 28. Open Education Network http://www.open- ed.net acknowledgements : the author thanks Sanjaya Mishra and the Commonwealth Educational Media Centre for Asia (CEMCA), New Delhi, and the Commonwealth of Learning (CoL), Vancouver, for their support the author also thanks Ramesh Sharma for technical assistance in reaching the survey population thank you
  • 29. Open Education Network http://www.open- ed.net You can download these slides freely from the website http://www.open-ed.net /oer-quality/ validation.ppt ______________________________________________ or by email to me at kawachi @open-ed.net