Quality standards and user-based mechanisms for Open Educational Resources - convergence or contradiction? Experiences from the OpenScienceResources project OnlineEduca Berlin, December 2010 Prof. Dr. Jan M. Pawlowski Kati I. Clements
Licensing: Creative Commons You are free: to Share — to copy, distribute and transmit the work to Remix — to adapt the work Under the following conditions: Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). Noncommercial. You may not use this work for commercial purposes. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one. http://creativecommons.org/licenses/by-nc-sa/3.0/ http://www.slideshare.net/jan.pawlowski
JYU: Global Information Systems Projects OpenScout: OER for Management NORDLET: Nordic Baltic Network for Learning, Education and Training COSMOS, Open Science Resources: Exchange of Scientific Content ASPECT: Open Content and standards for schools iCOPER: New standards for educational technologies LaProf: Language learning in ICT and agriculture Focus areas Global Information Systems Supporting globally distributed workgroups Open Educational Resources Reference Modeling E-Learning Supporting international education settings Cultural adaptation Standardization & Quality Management Mobile & Ambient Learning Innovative tools and solutions
Contents Barriers of OER use Quality Trust Case Study Results: What does quality mean for users? Mixed approach of quality standards and user-based quality mechanisms The users’ view
Open Educational Resources…
Challenges in quality assurance Huge amount of available contents No “wisdom of the crowd” effect Potential problems in OER repositories Scientifically incorrect content Broken links IPR violations Inadequate recommendations: Context – content – preferences Voluntary / non-profit contributors How to assure the quality in OER repositories in a sustainable solution which can still support itself after projects end?
Instruments to assure quality Quality standards which are agreed on by a formal standardization body such as ISO or CEN ISO 9000, EFQM, or ISO/IEC 19796-2 for organizations Quality marks for resources Certifications qualifications for individuals User oriented mechanisms Meeting users’ quality needs Recommendation mechanisms Rankings Peer-reviewing What do the users consider useful?
Twosurveys on teachers Second survey (n=66): Teachers from Lithuania, Portugal, Finland, Belgium, Romania Austria, Sweden, Greece, United Kingdom, Bulgaria, Turkey and one teacher from Ethiopia. All teachers were from the domains of IT, Maths and Science First survey (n=80): Teachers from Lithuania, Portugal, Finland, Belgium and Romania
European teachers find resources…
Barriers of re-use
Preliminary findings Users do not trust materials (in contrast to wikipedia) Users are willing to share and re-use Users base their decisions on trust Formal standards can create trust in organizations, User based mechanisms can create trust in resources. How do users perceive and trust different mechanisms?
Quality of resources means to the users… Good use of multimedia (animations, simulations): 83% Scientifically correct: 80% Fit their lessons or curriculum: 79% Technical interoperability: 68% Created in an organization with a good reputation(e.g., CERN, Harvard, NASA): 55% Own quality strategy: 17%
Which resources do you trust? “I trust organizations with good reputation” (85%) “Quality for me means that the resources come from an organization with good reputation” (55%)
Users’ willingness to contributetowardsquality of resources
OSR Approach Utilize existing certificates, standards, qualifications to minimize operational efforts Trust as the key concept Integrate different levels Organizations Resources Individuals Engage users in quality assurance Create a sustainable model
An example: COSMOS Portal
First degree trusted network Topic / ContextA Topic / ContextB Second degree trusted network Collaboration network construction Efficient network organization is the key to success Tools are needed to facilitate the process Open Issues:
The right network?
Key findings Quality in OER communities should be assured by a combination of mechanisms Quality standards to ensure reliability User based mechanisms to achieve sustainability Trust in quality is the key factor and the connecting key concept! Main mechanisms Recommendation systems Ratings Trusted networks of people and colleagues Standards are fully compatible with user-based mechanisms Main requirements of standards can be fulfilled with user centered approaches Only the perception of standards is a contradiction, in reality the combination is not only useful but a must!
Further findings Trust is a key quality instrument Teacher's awareness on quality approaches is still limited Teachers are willing to contribute by ranking, commenting and recommending or even becoming accredited reviewers Creating trust for different users needs different approaches: e.g. some trust organizations with good reputation, others trust technologies or their personal friends…)
Contact us… Prof. Dr. Jan M. Pawlowski email@example.com Kati Clements, Lead Researcher firstname.lastname@example.org GLIS on the web… http://users.jyu.fi/~japawlow