Towards an Open Assessment Ecosystem
Adam Blum
adam@opened.io
@openedio
OpenEd Jam Conference 2014
San Antonio
July 26, 2014
What’s Going On In
Assessments
• Open educational resources are thriving
• OERs such as OpenEd, Gooru, WatchKnowLearn
make quality K-12 videos and games available for free
• But not formative assessments crucial for
mastery-based learning implied by CCSS
• Virtually no free formative assessment content
• SBAC and states recognize this but have been slow to
fill the gap
• Most districts only have one item bank
• Assessment item types have evolved via
CCSS/Smarter Balanced/ PARCC
What Exists Today
• standards for interoperable content
– QTI, SCORM, SIF
• some older free tools
–Hot Potatoes, eXe
• some expensive current tools
–QuestionMark, Respondus
– paid LMSes – expensive in costs AND complexity
• expensive item banks
–NWEA, McGraw Hill, HMH Riverside, Pearson
– not available to individual teachers
QTI 2.1 Conformant Authoring Tools
•Onyx editor
– 278 euros/seat, 3998 euros/institution
• Mocah QTI editor
– http://qti-work.lip6.fr/soumission/IMS-QTI-editeur/README.pdf
–Single Response, Multiple Response, Free Response
• Respondus?
–Not listed. Doesn’t load QTI
• Canvas
– Failed to load QTI samples
– No Technology Enhanced Items
– Otherwise pretty good – but embedded only in larger LMS
SCORM Quiz Creators
• Articulate Quizmaker
– $699
• WonderShare QuizCreator
– $129
•iSpring Quizmaker
– $397
• eXeLearning
– open source, no QTI
But SCORM doesn’t really standardize item
content
What Needs To Exist
• free “modern” assessment items
• free authoring tools
• free grading/analytics
• web-based (no client install) for easy access
• available hosted
• all open source
• exchangeable, loadable content
Choose Resources
Make/Take
Courses
Assess
Analyze
What OpenEd.io Does
OpenEd
Catalog
Teach, Assess and Analyze with the Largest K-12 Resource Catalog
Next Gen OER
Flipped Class LMS
Quizzing Tools
Mastery Charts
OpenEd.io’s Content Mission
• Largest K-12 educational repository
– resources aligned to all standards
– infuse all teaching with use of resources
– free for teachers and users
– paid for fees from paid content providers
• But where are the assessment items?
– If they are out there OpenEd would link to them –
free or paid
– But the free items aren’t out there
– Even paid items aren’t easily accessible by
individuals
OpenEd.io’s Assessment Tool
• free and open source
• web-based - hosted at www.opened.io
• modern item types
– Select Response, Multiple Response, Free Response
• loads QTI content
• offering multiple item banks from partners
–With teacher approachable payment model
• “resource-backed”
–Resources to students based on the questions they miss
Check out the Hangout demo:
https://plus.google.com/events/cp6jdts1vfjcino3c3ivgqjqo
44
What Educators Can Do To Help
•Create assessment item content
– free or paid, but easily available
– Creative Commons when possible
– Exchangeable and loadable in LMSes (QTI
– But hosted somewhere in some form
• Fullfledged assessments for all standards
– mix of DOK, Difficulty
– “bitesized” appropriate for formative (5-10 items)
•Guidelines for new items
OPENED PROVIDES CREDIT FOR SUBSCRIPTION TO
CONTRIBUTORS
What Developers Can Provide
• new Technology Enhanced Item types
–Not necessarily just the SBAC/PARCC ones
–Richer Technology Enhanced Items for specific disciplines: math, physics.
• better interop formats
–QTI is not good XML design
– SIF has virtually no adoption for assessments
– only 2 participants on IMS program groups
– assessment providers are engineering own formats
– how about JSON-based?
• hosted grading service
– broker assessment response to graders, software assistance for grading
OPENED WILL PROVIDE GRANTS FOR DEVELOPERS THAT
WANT TO TACKLE ANY OF THESE OPEN SOURCE
Tell Us What You’ve Done
• adam@opened.io
• @openedio
• @adamblum
Thank You
Assessment Interop Requirements
• assessment item content
• assessment assets
• item accommodations
• item metadata
–Standards alignment, perf statistics, usage/exposure
data, relationship to other items
• test forms (complete assessments)
• scoring/rubric
• reporting/results
Assessment Interop References
• http://www.imsglobal.org/cc/statuschart.cfm
• http://specification.sifinfo.org/Implementation/NA/3.1/
•
http://images.pearsonassessments.com/images/tmrs/tmrs_rg/AssessmentIn
teroperabilityStandards_FINAL_111710.pdf?WT.mc_id=TMRS_Next_Gener
ation_Assessment_Interoperability_Standards

Open Ed Jam 2014: Towards an Open Assessment Ecosystem

  • 1.
    Towards an OpenAssessment Ecosystem Adam Blum adam@opened.io @openedio OpenEd Jam Conference 2014 San Antonio July 26, 2014
  • 2.
    What’s Going OnIn Assessments • Open educational resources are thriving • OERs such as OpenEd, Gooru, WatchKnowLearn make quality K-12 videos and games available for free • But not formative assessments crucial for mastery-based learning implied by CCSS • Virtually no free formative assessment content • SBAC and states recognize this but have been slow to fill the gap • Most districts only have one item bank • Assessment item types have evolved via CCSS/Smarter Balanced/ PARCC
  • 3.
    What Exists Today •standards for interoperable content – QTI, SCORM, SIF • some older free tools –Hot Potatoes, eXe • some expensive current tools –QuestionMark, Respondus – paid LMSes – expensive in costs AND complexity • expensive item banks –NWEA, McGraw Hill, HMH Riverside, Pearson – not available to individual teachers
  • 4.
    QTI 2.1 ConformantAuthoring Tools •Onyx editor – 278 euros/seat, 3998 euros/institution • Mocah QTI editor – http://qti-work.lip6.fr/soumission/IMS-QTI-editeur/README.pdf –Single Response, Multiple Response, Free Response • Respondus? –Not listed. Doesn’t load QTI • Canvas – Failed to load QTI samples – No Technology Enhanced Items – Otherwise pretty good – but embedded only in larger LMS
  • 5.
    SCORM Quiz Creators •Articulate Quizmaker – $699 • WonderShare QuizCreator – $129 •iSpring Quizmaker – $397 • eXeLearning – open source, no QTI But SCORM doesn’t really standardize item content
  • 6.
    What Needs ToExist • free “modern” assessment items • free authoring tools • free grading/analytics • web-based (no client install) for easy access • available hosted • all open source • exchangeable, loadable content
  • 7.
    Choose Resources Make/Take Courses Assess Analyze What OpenEd.ioDoes OpenEd Catalog Teach, Assess and Analyze with the Largest K-12 Resource Catalog Next Gen OER Flipped Class LMS Quizzing Tools Mastery Charts
  • 8.
    OpenEd.io’s Content Mission •Largest K-12 educational repository – resources aligned to all standards – infuse all teaching with use of resources – free for teachers and users – paid for fees from paid content providers • But where are the assessment items? – If they are out there OpenEd would link to them – free or paid – But the free items aren’t out there – Even paid items aren’t easily accessible by individuals
  • 9.
    OpenEd.io’s Assessment Tool •free and open source • web-based - hosted at www.opened.io • modern item types – Select Response, Multiple Response, Free Response • loads QTI content • offering multiple item banks from partners –With teacher approachable payment model • “resource-backed” –Resources to students based on the questions they miss Check out the Hangout demo: https://plus.google.com/events/cp6jdts1vfjcino3c3ivgqjqo 44
  • 10.
    What Educators CanDo To Help •Create assessment item content – free or paid, but easily available – Creative Commons when possible – Exchangeable and loadable in LMSes (QTI – But hosted somewhere in some form • Fullfledged assessments for all standards – mix of DOK, Difficulty – “bitesized” appropriate for formative (5-10 items) •Guidelines for new items OPENED PROVIDES CREDIT FOR SUBSCRIPTION TO CONTRIBUTORS
  • 11.
    What Developers CanProvide • new Technology Enhanced Item types –Not necessarily just the SBAC/PARCC ones –Richer Technology Enhanced Items for specific disciplines: math, physics. • better interop formats –QTI is not good XML design – SIF has virtually no adoption for assessments – only 2 participants on IMS program groups – assessment providers are engineering own formats – how about JSON-based? • hosted grading service – broker assessment response to graders, software assistance for grading OPENED WILL PROVIDE GRANTS FOR DEVELOPERS THAT WANT TO TACKLE ANY OF THESE OPEN SOURCE
  • 12.
    Tell Us WhatYou’ve Done • adam@opened.io • @openedio • @adamblum
  • 13.
  • 14.
    Assessment Interop Requirements •assessment item content • assessment assets • item accommodations • item metadata –Standards alignment, perf statistics, usage/exposure data, relationship to other items • test forms (complete assessments) • scoring/rubric • reporting/results
  • 15.
    Assessment Interop References •http://www.imsglobal.org/cc/statuschart.cfm • http://specification.sifinfo.org/Implementation/NA/3.1/ • http://images.pearsonassessments.com/images/tmrs/tmrs_rg/AssessmentIn teroperabilityStandards_FINAL_111710.pdf?WT.mc_id=TMRS_Next_Gener ation_Assessment_Interoperability_Standards