The document summarizes research on implementing open digital badges in three preservice teacher education programs. It describes challenges faced such as students feeling overwhelmed by the constant introduction of new badges. Lessons learned included setting scheduled due dates and feedback deadlines to reduce procrastination. Research found that while student learning outcomes were similar, perceptions of learning were higher for students earning badges. Overall, badges were seen as motivating and useful for displaying skills to potential employers if criteria and evaluation processes were clearly defined.

1. Boise
State
Timothy Newby
Purdue University
Daniel L. Randall
Richard E. West
Brigham Young University
BYU Purdue
Implementing Open Badges in Three
Preservice Teacher Education Programs:
Challenges, Lessons, and Opportunities
Chris Haskell
Boise State University
AECT 2015

2. Merit Badges and Digital Badges
Merit (Physical) Badges
Digital Badges
- Acknowledge accomplishment
- Display skills gained
- Motivation
Same Benefits as physical badges -
Typically not sharable -
Gamification -

3. Khan Academy Badges (not Open!)
Retrieved from https://www.khanacademy.org/badgeson 11/30/13.

4. Open Badges
Open Badges
Same Affordances as Digital Badges, Plus:
- Uses Open Badge Infrastructure (OBI)
- Display badges via web
- Metadata (Criteria and Evidence links)

5. Open Badge Metadata

6. Mozilla Backpack

7. Backpack Collection
Multiple collections can
be created.
Collections can remain
private or can be made
public and shared.

8. Mozilla’s Vision of Credentialing
Learn and gain skills in formal
and informal settings
Collect and Display Badges
New Opportunities & Lifelong
Learning

9. Value of Badges in Credentialing
CC BY-SA Class Hack http://classhack.com/post/39932478440/indianajones
A badge is only as good as:
The criteria (rigor/weight)
attached to it.
The process used to evaluate
the learner’s work.
It’s value to students and/or
stakeholders.

10. Badge Inflation
CC BY-SA Class Hack http://classhack.com/post/50915858999/carpetbadging
Mass awarding of
badges with little or
no assessment of
work.
Or criteria is so easy
and short everyone
earns the badge.
Often happens as a
part of gamification.
“Carpet Badging”

11. Purdue: Tim Newby

12. Evolving with badges…
• Background from Fall 2014
– EDCI 270 (Intro to Ed Tech)
– Passport

13. EDCI 270 Badges

14. Fall 2014 Lessons Learned
• Designer’s viewpoint
– Passport
• Ease of design and integration
– Instructional (scenario-based) badges
– Timely feedback challenges
• Student’s viewpoint
– Overwhelmed
• New badges constantly introduced
• Poor skills with scheduling their time on badges
– Difficulty with concept of mastery

15. Spring 2015 – 1st Evolution
2014 Lessons learned
• Overwhelmed
– New badges constantly
introduced
– Poor skills with scheduling their
time on badges
• Difficulty with concept of
mastery
2015 updates
• Badges set and scheduled
– Feedback deadlines
– Bulk due dates
• Discussions about benefits of
feedback and mastery
learning

16. Spring 2015 – Research
• Research study
– Traditional vs. Badge sections
• Pre-post course survey on perceived effectiveness
• Comparison of project grades
• Focus group analysis
• Research questions
1. Are the badges perceived to be as effective as the
traditional projects for learning the key course concepts
and skills?
2. Is the overall quality of the course projects similar for
both the traditional and the badge sections?

17. Pre-Post Course Participant Survey
Pre-post course ratings of perceived abilities to:
– Describe digital literacy and explain how it impacts 21st
Century skills.
– Access, retrieve, annotate, and properly cite research articles.
– Plan and develop an individualized learning module that
effectively integrates technology.
– Select and integrate appropriate technology tools to create an
effective learning experience.
– Identify and describe key issues (e.g., privacy, security, equity)
that have been impacted by the expanding integration of
technology.
Results:
Both traditional and badge section participants showed significant increases in their
perceived abilities.

18. Post Course Participant Survey
Between group/section post course ratings of perceived
abilities to:
– Describe digital literacy and explain how it impacts 21st
Century skills.
– Access, retrieve, annotate, and properly cite research articles.
– Plan and develop an individualized learning module that
effectively integrates technology.
– Select and integrate appropriate technology tools to create an
effective learning experience.
– Identify and describe key issues (e.g., privacy, security, equity)
that have been impacted by the expanding integration of
technology.
Results:
Cross section comparison on the post survey reported significantly higher perception scores
for those in the badge section.

19. Project Grades
• Digital Literacy
• Information Literacy
• Web 2.0
• Video Production
• Individualized Instruction
• Web Portfolio
Results:
Cross group comparison, no significant differences in grades for all projects except Video
Production.
Traditional section participants scored significantly higher for the Video Production Project.

20. Focus groups
• Bunched due dates – procrastination and then
panic
• Badge point values not consistent with demands
of the badge
• Difficult to offer correct amount of needed
support within badges
• Feedback needs to be timely and directly
relevant to content

21. Spring 2015 – Lessons Learned
• Overwhelming feelings better, but not gone
– Desire for due dates – conflict with mastery learning
• Timely, content related feedback essential
• Number of badge challenges is overwhelming –
just too many

22. Fall 2015 – 2nd Evolution
• Badge reconfiguration
– More comprehensive, less basic badges

23. EDCI 270 Badges

24. Fall 2015 – 2nd Evolution
• Badge reconfiguration
– More comprehensive, less basic badges
• Mastery learning vs. due dates
– Set “final feedback” due date, then one final revision
prior to specific date
• Feedback rubrics

25. Boise State: Chris Haskell

26. Follow the link below to watch Chris
describe what he’s done at Boise
State.
http://youtu.be/1bCZkhdrn0k

27. Chris Haskell & Lisa Dawley designed 3DGamelab

28. 3DGLFully GAME-BASED LMS
• QUESTS
• Experience Points
• Badges, Achievements,
and Awards
• Student choice/paced
• Mastery Approach

30. BRANCHING
Curriculum | EDTECH202

31. HIERARCHY
Reward Structures
BADGES represent
collections of quests which
describe learner experience.
ACHIEVEMENTS
describe system behaviors
AWARDS show/reward
additional learning, and serve
to create BRANCHED
structures in the path.

33. SPECIALIZED CURRICULUM

34. Mastermind – 43%
Achiever – 23%
Conqueror – 20%
Socializer – 10%
Seeker – 3%
Survivor – 0%
Daredevil – 0%

35. MASTERY
QUEST submission
details are the same for
each student.
EVIDENCE is given
through the system.
APPROVALof quests
happens when ALL conditions
are met. AWARDS can be
given to recognize additional
value.

36. Haskell, C. & Krebs, W. (2014) Patterns and journeys in quest-based learning. Virtual
World Best Practices in Education.

37. RATE of Progress – 3 patterns
ROCKETS do everything as soon as possible. They are consumers of
curriculum.
STEADIESprogress in a linear fashion, working on a few things at a time.
JUST IN TIMERS JITers do ,most things at the last minute.
Winning condition

38. JUST ENOUGH Some were satisfied with a lower grade than an A, and
stopped.
MOVED ON Some decided the course was not for them, and left early.
OVERACHIEVER But most people earned an ‘A;, and some were over
achievers and kept questing even after they had the A locked down. To get an A we
needed 2,000 points. Some made it to 3,000! I had to change the scale of the graph
Winning condition
COMPLETION– 3 patterns

43. QBL/Badges Whitepaper Attractive Quest/Badge Design
Links to Badge Research

44. BYU: Dan Randall & Rick West

45. Marrying Formal and Informal Learning Through Open Badges
Grateful
acknowledgement
of the team!

46. Google
Sites
Personal
Tech
Choice
1
Choice
2
Choice
3
Student selected Internet
Communication Technology
Student selected
Multimedia Technology
Student selected
Personal Technologies
Additional
Concepts
Mobile
Learn
Internet
Safety
Copy
right
iMovie
Lower level badges are not
issued for these projects
Project level badge not
issued for these
additional concepts
Educational
Technology
Course
Level Badge
Project
Level Badge
Lower
Level Badges

47. Badge Creating Team
Nicole Westenskow,
English teaching
iBooks Author,
Storybird
IPT grad student, B.S.
in Psychology
Ubersense
IPT grad student
Initial IPTEdTec
website
Danielle Martin
Health Education
Fitness Tracking,
Virtual Chemlab
Janelle Frossard,
English teaching
Social networking/bookmarking, Google
Earth, iMovie, Moviemaking, Blogger,
Google Sites
Jerika Newitt
Physics Teaching
5 Logger Pro badges,
Physion
Emily Goates
Science Teaching
Botany Research
Assistant
Plant Diversity TA
Biodigital Human
Ben Mitchell
Social Science
teaching
Computer lab
assistant, History TA
EdPuzzle, Classcraft
Chauncey Rogers
History teaching
Interactive Timelines

48. Student Perception of Badges:
An Evaluation
• Selected four former preservice teachers
– Used extreme (or deviant) case sampling (Patton, 1990) to
capture full spectrum of student perceptions.
– Selection was based on post-course survey data (2013-
15).
• Asked students
– “Are badges working?”
– “What future potential do they have for you?”

49. What’s Working
• Learner self-confidence increased (having
earned a badge).
• Natural inclination to earn higher-level badges.
• Students appreciated having more than a report
card to prove proficiency.
• Badges stand out on a resumé.
• Badges serve as unique talking points for school
interviews.

50. Self Confidence ▲
“People who earned badges
were masters of technology.”
—Jessica

51. Higher Achievement
“I’ve even thought about going back to
Dr. West [now that the class is over] and
seeing if I can upgrade my badges to the
next level.” —J.

52. Physical Proof of Competency
“So rather than me saying I'm
good at Prezi or I can do a poll!
Someone else can attest that I
can do those things in a
classroom setting.”
—Johanna

53. Resumé Booster
“My professor was reviewing
my resumé, and he was like,
what are badges? Why is this
on your resumé? And then I
explained it and he was like,
that's really cool!” —Johanna
Technical Skills
– Languages:
• English, Spanish & Portuguese.
– Earned IP&TEDTEC Badges:
• Audacity, Prezi, Poll Everywhere,
iMovie, Internet communications
– Adobe CS4–CS6 Suites:
• Photoshop, Illustrator, InDesign

54. Unique Job Interview Talking Point
“You need to have a
distinguishing
talking point in your
job interview.
Badges can do that.
You can say, here’s
all the cool stuff
that I have. You
want me.”
—Dalen

55. 1. Any better than traditional
rubrics?
2. A badge website, in addition
to normal class assignments,
caused confusion.
3. Most learners selected the
easiest badges to fulfill course
requirements.
4. People (principals) outside of
the class didn’t know/appreciate
badges and their rigor.
What’s Not Working

56. Please, Make It A Thing! 
• “Hopefully—eventually—people will know what
[Badges] are. As for right now, maybe not, but I
can explain it until it catches on. [Dr. West]
talked about badges like they were going to be a
thing. And I really hope it is, so when I go to
apply for a job, they'll be like, oh, you have
badges; that's cool!” —Joanna

57. We’ll Wait ‘Till It’s a Thing…
• “Badges just aren't there yet. If you could get it to the point
where it was something that administrators were asking for, or
if people knew about badges in the professional community,
then I could see them being useful.” —Lindsey
• “There's no place on a teaching application that says, hey,
what badges do you have? So until then, it's going to be hard
for me to be, like, yes, I really do need badges.” —Dalen
• “On the first day, I remember our instructor saying that we're
hoping that this becomes something in the future. So since it
wasn't really established yet, I was like, I don't really care.” —
Jessica

58. Badges = Professional Development?
• “If there were breakout session at conferences where badges
were taught and graded, I'd do it.” —Lindsey
• “If it counted toward my PD hours, I would definitely be more
motivated to earn badges.” —Joanna
• “That would be great! Some teachers at PD meetings don’t
even know how to use computers…[implying a waste of his
time.]” —Dalen
• “Traditional PD is just a bunch of people chatting and no one
actually learns anything. So if I had a choice to earn a badge or
attend a professional development meeting, I'd much rather
earn a badge; I might be able to actually use it in my
classroom.” —Jessica

59. New Website
• Roles: Earner, Reviewer, Issuer, Administrator
• Better management of badges, reviewers, and
reviewing assignments
• Easier to browse and search for badges
• Capability to have multiple issuers
• Reviewer/Earner Profiles
• Better tracking of analytics

65. New Directions
• Challenge: Providing more credits without
hindering graduation
• Badges are the criteria; classes are labs designed to
support
• Multiple options (face to face/online) for earning
• Challenge: Make badges respectable
• Could like-minded institutions collaborate to endorse
badges, lending credibility?
• Challenge: Scale beyond preservice
• Partner with DOEs, schools, and professional
development?

66. Davies, R., Randall, D., & West, R. E. (2015). Using Open Badges to Certify
Practicing Evaluators. American Journal of Evaluation, 36(2), 151–163.
doi:10.1177/1098214014565505
Available Now!
Recent Articles
West, R. E., & Randall, D. L. (in-press). The Case for Rigor in Badges. In L.
Muilenburg & Z. Berge (Eds.), Digital Badges in Education: Trends,
Issues, and Cases. Routledge.
Coming Soon!

67. Questions?
Please Feel Free to Contact Us
Thank You
Rick West
RickWest@gmail.com
Chris Haskell
chrishaskell@boisestate.edu
Dan Randall
Dan.Randall26@gmail.com
Tim Newby
newby@purdue.edu