July 26th 2014 6:57pm Today at OpenEd Jam we presented on what needs to be done to create an open assessment ecosystem for K-12 education. While other resources such as videos and games and even full-fledged lesson plans a thriving ecosystem is emerging. OpenEd participates in this (along with many other OERs such as Curriki, WatchKnowLearn, and OER Commons) and we feel we have done several things to make it easier to use such resources in daily teaching in a practical way. But the world of formative assessments in still quite closed and proprietary If you believe in formative assessments (frequent daily or weekly quizzes on subject matter to level set students abilities) there are not a lot of options besides your school or district paying a lot of money for an item bank, or spending a large amount of effort writing questions yourself. The only way way we see around this is creation of an open ecosystem for assessment content, just as has emerged around other types of educational content. We discuss the creation tools and interop standards that exist today that could potentially help start this shift. And identify what is stopping them from doing that. We then identify the requirements tools and content that would help begin this shift: free “modern” assessment item content free authoring tools free grading/analytics web-based (no client install) for easy access available hosted (no server install) all open source exchangeable, loadable content (reads/writes QTI or other standard) After briefly describing OpenEd’s overall mission and core resource library product we discuss why we need to build our own assessment tool to address these needs. We then presented what the OpenEd assesssment tool does including its core innovation of “resource backed” assessments. Finally we presented a call to action for educators and developers in helping to create this assessment ecosystem, whether or not in cooperation with or based on OpenEd.io.

1. Towards an Open Assessment Ecosystem
Adam Blum
adam@opened.io
@openedio
OpenEd Jam Conference 2014
San Antonio
July 26, 2014

2. What’s Going On In
Assessments
• Open educational resources are thriving
• OERs such as OpenEd, Gooru, WatchKnowLearn
make quality K-12 videos and games available for free
• But not formative assessments crucial for
mastery-based learning implied by CCSS
• Virtually no free formative assessment content
• SBAC and states recognize this but have been slow to
fill the gap
• Most districts only have one item bank
• Assessment item types have evolved via
CCSS/Smarter Balanced/ PARCC

3. What Exists Today
• standards for interoperable content
– QTI, SCORM, SIF
• some older free tools
–Hot Potatoes, eXe
• some expensive current tools
–QuestionMark, Respondus
– paid LMSes – expensive in costs AND complexity
• expensive item banks
–NWEA, McGraw Hill, HMH Riverside, Pearson
– not available to individual teachers

4. QTI 2.1 Conformant Authoring Tools
•Onyx editor
– 278 euros/seat, 3998 euros/institution
• Mocah QTI editor
– http://qti-work.lip6.fr/soumission/IMS-QTI-editeur/README.pdf
–Single Response, Multiple Response, Free Response
• Respondus?
–Not listed. Doesn’t load QTI
• Canvas
– Failed to load QTI samples
– No Technology Enhanced Items
– Otherwise pretty good – but embedded only in larger LMS

5. SCORM Quiz Creators
• Articulate Quizmaker
– $699
• WonderShare QuizCreator
– $129
•iSpring Quizmaker
– $397
• eXeLearning
– open source, no QTI
But SCORM doesn’t really standardize item
content

6. What Needs To Exist
• free “modern” assessment items
• free authoring tools
• free grading/analytics
• web-based (no client install) for easy access
• available hosted
• all open source
• exchangeable, loadable content

7. Choose Resources
Make/Take
Courses
Assess
Analyze
What OpenEd.io Does
OpenEd
Catalog
Teach, Assess and Analyze with the Largest K-12 Resource Catalog
Next Gen OER
Flipped Class LMS
Quizzing Tools
Mastery Charts

8. OpenEd.io’s Content Mission
• Largest K-12 educational repository
– resources aligned to all standards
– infuse all teaching with use of resources
– free for teachers and users
– paid for fees from paid content providers
• But where are the assessment items?
– If they are out there OpenEd would link to them –
free or paid
– But the free items aren’t out there
– Even paid items aren’t easily accessible by
individuals

9. OpenEd.io’s Assessment Tool
• free and open source
• web-based - hosted at www.opened.io
• modern item types
– Select Response, Multiple Response, Free Response
• loads QTI content
• offering multiple item banks from partners
–With teacher approachable payment model
• “resource-backed”
–Resources to students based on the questions they miss
Check out the Hangout demo:
https://plus.google.com/events/cp6jdts1vfjcino3c3ivgqjqo
44

10. What Educators Can Do To Help
•Create assessment item content
– free or paid, but easily available
– Creative Commons when possible
– Exchangeable and loadable in LMSes (QTI
– But hosted somewhere in some form
• Fullfledged assessments for all standards
– mix of DOK, Difficulty
– “bitesized” appropriate for formative (5-10 items)
•Guidelines for new items
OPENED PROVIDES CREDIT FOR SUBSCRIPTION TO
CONTRIBUTORS

11. What Developers Can Provide
• new Technology Enhanced Item types
–Not necessarily just the SBAC/PARCC ones
–Richer Technology Enhanced Items for specific disciplines: math, physics.
• better interop formats
–QTI is not good XML design
– SIF has virtually no adoption for assessments
– only 2 participants on IMS program groups
– assessment providers are engineering own formats
– how about JSON-based?
• hosted grading service
– broker assessment response to graders, software assistance for grading
OPENED WILL PROVIDE GRANTS FOR DEVELOPERS THAT
WANT TO TACKLE ANY OF THESE OPEN SOURCE

12. Tell Us What You’ve Done
• adam@opened.io
• @openedio
• @adamblum

13. Thank You

14. Assessment Interop Requirements
• assessment item content
• assessment assets
• item accommodations
• item metadata
–Standards alignment, perf statistics, usage/exposure
data, relationship to other items
• test forms (complete assessments)
• scoring/rubric
• reporting/results

15. Assessment Interop References
• http://www.imsglobal.org/cc/statuschart.cfm
• http://specification.sifinfo.org/Implementation/NA/3.1/
•
http://images.pearsonassessments.com/images/tmrs/tmrs_rg/AssessmentIn
teroperabilityStandards_FINAL_111710.pdf?WT.mc_id=TMRS_Next_Gener
ation_Assessment_Interoperability_Standards