Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using Open Badges as a Certification Solution for Evaluators


Published on

At American Evaluation Association conference in 2014. In the presentation we discussed our proposed framework for using micro credentials to certify professional evaluators. This framework was later published in the American Journal of Evaluation.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Using Open Badges as a Certification Solution for Evaluators

  1. 1. Using Open Badges as a Certification Solution for Evaluators Randall S. Davies Daniel L. Randall Richard E. West BYU Image by photosteve101. Used under CC BY License.
  2. 2. The Issue of Evaluator Certification In 1999 the topic of certification was fervently debated Not everyone feels the AEA needs an evaluator certification Others feel a certification process would be valuable, and that it may become required for practitioners working with government organizations (Smith et al., 2011) AEA is no closer to having an evaluator certification than we were a decade ago. Many seem to have concluded that an evaluator certification system may be overly complicated and controversial (Morris, 2011) AEA
  3. 3. Challenges to Certification Credentials, Certifications, and Licenses An Evaluator Certification would evaluate an individual’s Knowledge, Skills, and Experience, verifying that he or she has attained a certain level of expertise. Any solution AEA might adopt must be – Transparent (Clear Expectations) – Rigorous BUT Flexible and Efficient – Sustainable AEA
  4. 4. Working Examples of an Evaluator Credentialing Designation Canadian Evaluation Society’s (CES) Credentialed Evaluator (CE) designation This is a designation is not a certification of evaluation proficiency or a license to practice evaluation. It simply verifies “the holder has provided evidence of the education and experience required by the CES to be a competent evaluator” (, np). AEA
  5. 5. The CE designation is voluntary. Candidates must have at least two years of evaluation-related work experience within the last ten years, with letters of reference from clients. They must pass a test requiring a series of narratives (each 150 words or less) adequately addressing at least 70% of the specified evaluator competencies in each of five domains. AEA CES Credentialed Evaluator Designation
  6. 6. CES Credentialed Evaluator Designation, Cont. Members of a credentialing board consider the application and award the designation. In addition, credentialed evaluators are required to complete ongoing training (40 hours every three years) in order to maintain their status. This application and renewal process is implemented though an online system designed to manage applications and candidate accounts, with a fee to cover administration costs. AEA
  7. 7. Issues & Concerns AEA Must Consider Training Concerns: – Degrees vs training, Experience without a degree (capacity building). What combination of training, Skill, and Experience is needed. Assessment Concerns: – Certification Exams vs Credentials, Validity of assessment. Administration Issues: – Costs involved in establishing and maintaining a system AEA
  8. 8. Potential for an Open Badges Solution Transparency: – Badges communicate specifics, Meta-data, Flexibility: – Modification is relatively easy, Skill subsets can be defined, badges can be added or revised individually. Rigor: – Individuals Authorized to Award Badges, Allows for levels of proficiency to be identified (basic – advanced) Management Sustainability: – Top down design, Automation of much of the process possible, Division of labor model AEA
  9. 9. What are “Open Badges”? Images from
  10. 10. Merit Badges and Digital Badges Boy Scout Merit Badges Digital Badge - Acknowledge accomplishment - Display skills gained - Motivation Same Benefits as physical badges - Typically not sharable - Gamification -
  11. 11. Khan Academy Badges (not Open!) Retrieved from 11/30/13.
  12. 12. Duolingo (not Open Badges) Screenshot from the Duolingo app on an Android Tablet. 1/24/14. Encourages daily practice Completed badge Progress indicators Competition with others is optional Previously earned badge not at full strength
  13. 13. Open Badges Open Badges Same Affordances as Digital Badges, Plus: - Uses Open Badge Infrastructure (OBI) - Display badges via web - Metadata (Criteria and Evidence links)
  14. 14. Open Badge Metadata
  15. 15. Mozilla Backpack
  16. 16. Backpack Collection Multiple collections can be created. Collections can remain private or can be made public and shared.
  17. 17. Mozilla’s Vision of Credentialing Learn and gain skills in formal and informal settings Collect and Display Badges New Opportunities
  18. 18. Applications Gamification: breaking education into achievable levels where progress is recognized and rewarded Duolingo, Khan Academy, many others Recognizing expertise within a community Coding communities, translation communities Marrying informal and formal learning Concordia, WGU Providing more authentic credentials, tying evidence to criteria to credential IPT Ed Tec
  19. 19. Strong criteria • A badge’s value is based on the rigor of the criteria for earning it Easy submission • Make it simple for people who deserve a badge to get it Web-based evidence • Portfolios, links, Google documents, etc. Allows for evidence to be viewed with a badge Skilled assessments • It is essential to trust the person making the assessment Open and shareable badge collections • What’s the point if you can’t share your badges in multiple ways? Key Principles of Badging
  20. 20. iMovie Personal Tech Technology Integration One Possibility for an AEA Badge System AEA
  21. 21. Mozilla’s Webmaker badges plan used the “constellation” concept in which smaller badges are earned and added together to receive a higher level badge. Badges image adapted from the Mozilla Open Badges Project. Retrieved from on 11/9/12. Early Badge System by Mozilla
  22. 22. Google Sites Personal Tech Choice 1 Choice 2 Choice 3 IPT EdTec Badge System Student selected Internet Communication Technology Student selected Multimedia Technology Student selected Personal Technologies Additional Concepts Mobile Learn Internet Safety Copy right iMovie Lower level badges are not issued for these projects Project level badge not issued for these additional concepts Educational Technology Course Level Badge Project Level Badge Lower Level Badges
  23. 23. iMovie Personal Tech First Iteration Theory Skill Experience Technology Integration Certified Evaluator Level Badge 2nd Level Badge 3rd Level Badges Possible AEA Badge System
  24. 24. Anatomy of a CE Badge Pips act as stars: 1 = Basic 2 = Intermediate 3 = Advanced Outer ribbon signifies this badge is the “Certified Evaluator” (CE) badge Each badge indicates what category the badge is from (theory, experience, skill, or green for the CE badge). Center of badge is reserved for images to individualize badges
  25. 25. Pips and the CE Badge 7 - 9 pips earned from 2nd level Basic Intermediate Advanced 4 - 6 pips earned from 2nd level At least 1 pip from each category
  26. 26. Randall, D. L., Harrison, J. B., West, R.E. (2013). Giving credit where credit is due: Designing Open Badges for a technology integration course. TechTrends, 57(6), 88-95. doi:10.1007/s11528-013-0706-5 View our presentation at the Open Education Conference by clicking here. To Learn More…
  27. 27. Randall S. Davies – Daniel L. Randall – Richard E. West – Contact us with Questions Thank You