Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using open source assessment andfeedback tools
Introduction•   JISCs Assessment & Feedback programme strand C:    Technology transfer    •   Open source    •   Embed use...
Questions?   Wilbert Kraan       w.g.kraan@ovod.net                             3
Rogō OSS                  Rogō         R        Open Source SoftwareDr Simon WilkinsonUniversity of Nottingham
Rogō OSSAim“ The Rogō OSS project aimed to take software  developed at the University of Nottingham and   package it up an...
Rogō OSSEnterprise-level e-Assessment•   7 paper types (inc formative, summative)•   15 question types•   Searchable – con...
Rogō OSSOpen Source• Released under GPL 3.0• Server: Linux, Apache, MySQL, PHP• Clients:   – PC or Mac – IE 8/9, Firefox, ...
Rogō OSSPartners           De Montfort University           University of Bedfordshire           University of East Anglia...
Rogō OSSChallenges• OSS – raising awareness• How best to operate?• Legacy code-base• Local idiosyncrasies/procedures
Rogō OSSTimelineBasic Housekeeping• Name change – TouchStoneRogō• Website• Blog• Mailing list• TRAC – ticketing system• I...
Rogō OSSTimelinePlanned Development• Upgrade script• QTI 1.2 import improved• LTI 1.0 support added
Rogō OSSTimelineFurther Development• Sub-directory support• UI styling – CSS rewrite• Language abstraction layerExternal C...
Rogō OSSLessons Learnt• Installation   –   Project start time   –   PHP versions   –   Sub-directory support   –   Bugs
Rogō OSSLessons Learnt• Testing  –   Manual (test plan in TRAC)  –   Selenium  –   Unit Testing  –   PHP scripts• Modular ...
Rogō OSSFuture – Community• Improved documentation  – UI, code and database• GitHub – access to code in development• Suppo...
Rogō OSSFuture – Design• OO rewrite• Modular design  – Questions
Uniqurate           Online        Demo WebinarSue Milne, University of Glasgow & ELandWeb Ltd         Paul Neve, Kingston ...
Project AimsThe Uniqurate project seeks to increase the number of open e-assessment content authors, and as a result, in...
QTI – what’s that? Question and Test Interoperability specification Version 2.1 is becoming final about now Standard fo...
Why use QTI? Interoperability:   Share assessment resources with colleagues in same or other    institutions     Effici...
Uniqurate – user-led design Uniqurate project:  JISC Assessment and Feedback programme Objective: increase usage of QTI ...
Uniqurate - Question components Constituent parts providing functionality between  question/exercise and student May map...
Uniqurate - Modes Friendly Mode  drag & drop components Intermediate Mode  edit HTML parts of existing files: ideal fo...
Uniqurate - output As QTI XML file As packaged QTI XML file As package containing QTI XML file and media:  images, styl...
Links... Uniqurate – web application – no installation, just go to  http://uniqurate.kingston.ac.uk/Uniqurate/startup QT...
Hands-on TimePlease type any questions or comments in the chat box...
Just in Case...The following slides are provided in case the live demo isnot available...The questions are quite mathemati...
Uniqurate Questions 1: Friendly Mode                                   Add more                                 components...
Uniqurate Questions 2: Intermediate Mode                  Click here to                  add MathML                 Only H...
Uniqurate Questions 2: Expert ModeFiles in package      QTI XML for this question
Packaging Tests (Uniqurate editor)
QTIDI          Online       Demo WebinarSue Milne, University of Glasgow & ELandWeb Ltd        Niall Barr, University of G...
Project Aims The project will provide a documented and packaged version of  standards-compliant assessment delivery tools...
ToolsLTI Connector – connect VLE with deliverysoftwareDelivery software:LTIQuizzes – deliver basic question typesQTIWorks ...
LTI Connector for VLEs                                  Our simple QTI 2.1                             Assessment Tool - L...
QTIWorks – QTIv2.1 delivery software Validates questions and tests – reports problems Supports all QTIv2.1 UK HE profile...
Installation? LTI – Moodle 2, newer versions of Blackboard and Sakai VLEs  are already LTI-enabled. LTIQuizzes – a singl...
Links... QTIWorks – online demo at  https://www2.ph.ed.ac.uk/qtiworks/ QTI Support Site (examples, tools, resources...) ...
Hands-on TimePlease type any questions or comments in the chat box...
Just in Case... The following slides are provided in case the live demo is not available... They are quite mathematical in...
QTI WorksQTI Works https://www2.ph.ed.ac.uk/qtiworks/Use the Demo section to try out questions and testsLog in to set up L...
QTI Works – Demos                        Try out andUpload questions    download example      Validate questionsor tests t...
QTI Works – VLE Setup                     Generate LTI dataUpload questions                       for setting upor tests t...
Uploading an Assessment Content of uploaded assessments –question or test – can  be changed (e.g. fix a bug) Date of las...
Setting Up Assessment Launch Data Saving a delivery with LTI enabled creates launch data Data is pasted into Moodle 2 Ex...
QTI in Moodle 2 – Question
QTI in Moodle 2 – Question with Solution
QTI in Moodle 2 – Maths Input
Test: Algebra 1 This test is the one used in the Maths T1 topic Algebra 1. It is also in the demo course.
Question in a TestThe indicator at top right shows the question status. No feedback isvisible during the test – students h...
Test feedback & reviewThese are the two parts of the feedback and review page; the scoresfor the questions are displayed a...
Upcoming SlideShare
Loading in …5
×

Using open source assessment and feedback tools

2,965 views

Published on

Slides from webinar in using open source assessment and feedback tools 15 Feb 2013

Published in: Education
  • Be the first to comment

Using open source assessment and feedback tools

  1. 1. Using open source assessment andfeedback tools
  2. 2. Introduction• JISCs Assessment & Feedback programme strand C: Technology transfer • Open source • Embed use in third party institutions  Lessons learned: http://tinyurl.com/afstrandc• The Tools:  Rogō  Full assessment suite  Uniqurate  Item and Test Editor, QTI compliant  QTIDI project:  QTIWorks • Item and Test renderer, QTI and LTI compliant  LTIConnector and LTIQuizzes • IMS Learning Tool Interoperability demonstrators 2
  3. 3. Questions? Wilbert Kraan  w.g.kraan@ovod.net 3
  4. 4. Rogō OSS Rogō R Open Source SoftwareDr Simon WilkinsonUniversity of Nottingham
  5. 5. Rogō OSSAim“ The Rogō OSS project aimed to take software developed at the University of Nottingham and package it up and create a vibrant open source development community. ”
  6. 6. Rogō OSSEnterprise-level e-Assessment• 7 paper types (inc formative, summative)• 15 question types• Searchable – content, keywords, Bloom’s Taxonomy• QTI 1.2 and LTI 1.0• Feedback – questions – learning objectives
  7. 7. Rogō OSSOpen Source• Released under GPL 3.0• Server: Linux, Apache, MySQL, PHP• Clients: – PC or Mac – IE 8/9, Firefox, Chrome – JavaScript and Flash• Languages: English, Polish
  8. 8. Rogō OSSPartners De Montfort University University of Bedfordshire University of East Anglia University of Oxford University of the West of Scotland
  9. 9. Rogō OSSChallenges• OSS – raising awareness• How best to operate?• Legacy code-base• Local idiosyncrasies/procedures
  10. 10. Rogō OSSTimelineBasic Housekeeping• Name change – TouchStoneRogō• Website• Blog• Mailing list• TRAC – ticketing system• Installation script
  11. 11. Rogō OSSTimelinePlanned Development• Upgrade script• QTI 1.2 import improved• LTI 1.0 support added
  12. 12. Rogō OSSTimelineFurther Development• Sub-directory support• UI styling – CSS rewrite• Language abstraction layerExternal Consultation• Testing procedures• Openness and sustainability
  13. 13. Rogō OSSLessons Learnt• Installation – Project start time – PHP versions – Sub-directory support – Bugs
  14. 14. Rogō OSSLessons Learnt• Testing – Manual (test plan in TRAC) – Selenium – Unit Testing – PHP scripts• Modular Design – Language – Authentication
  15. 15. Rogō OSSFuture – Community• Improved documentation – UI, code and database• GitHub – access to code in development• Support packages
  16. 16. Rogō OSSFuture – Design• OO rewrite• Modular design – Questions
  17. 17. Uniqurate Online Demo WebinarSue Milne, University of Glasgow & ELandWeb Ltd Paul Neve, Kingston University
  18. 18. Project AimsThe Uniqurate project seeks to increase the number of open e-assessment content authors, and as a result, increase the richness of assessment content available to studentsby delivering QTI authoring tools that can be used by a novice user embedding these tools into several “client” institutions
  19. 19. QTI – what’s that? Question and Test Interoperability specification Version 2.1 is becoming final about now Standard for representing questions and tests Many implementations worldwide:  UK, France, Netherlands, Germany, Spain  USA, Mexico  Korea Used by  Educational institutions for internal assessments  Examination authorities (Scotland, Netherlands, Germany, Korea)  Publishers for online content associated with textbooks
  20. 20. Why use QTI? Interoperability:  Share assessment resources with colleagues in same or other institutions  Efficiency, less reinvention of wheels  Economical – costly to produce good resources, so share and repurpose  Preserve materials to prevent loss when changing systems  Ensure content separate from software  All these contribute to SUSTAINABILITY Flexibility:  20 or so input types – and extendable, e.g. Maths  Several inputs in one question – engaging resources  Targeted, individualised feedback, related directly to input
  21. 21. Uniqurate – user-led design Uniqurate project: JISC Assessment and Feedback programme Objective: increase usage of QTI in HE by transferring technology from existing tools into "client" institutions Client partners describe what they need for teaching Uniqurate derived from user-experience driven Aqurate and multiple interaction power of Mathqurate Result – hopefully –a useful tool for authoring QTI content accessible to any user
  22. 22. Uniqurate - Question components Constituent parts providing functionality between question/exercise and student May map onto QTI elements – but may not! – hence the term question component  “What do you want the question to do?” Questions typically composed of several components  Several inputs in one question Identify components that provide optimal benefit
  23. 23. Uniqurate - Modes Friendly Mode  drag & drop components Intermediate Mode  edit HTML parts of existing files: ideal for contextualising Expert Mode  XML editor: customise questions when more control needed  also for packaging non-Uniqurate questions with media
  24. 24. Uniqurate - output As QTI XML file As packaged QTI XML file As package containing QTI XML file and media: images, stylesheets, etc.
  25. 25. Links... Uniqurate – web application – no installation, just go to http://uniqurate.kingston.ac.uk/Uniqurate/startup QTI Support Site (examples, tools, resources...) http://qti- support.gla.ac.uk Contacts...  Graham Alsop (Project Leader) Graham.Alsop@kingston.ac.uk  Paul Neve (Uniqurate Developer) P.Neve@kingston.ac.uk  Sue Milne (Consultant) sue.milne@e-learning-services.org.uk
  26. 26. Hands-on TimePlease type any questions or comments in the chat box...
  27. 27. Just in Case...The following slides are provided in case the live demo isnot available...The questions are quite mathematical in flavour, butauthoring resources for other disciplines works in just thesame way!
  28. 28. Uniqurate Questions 1: Friendly Mode Add more components if neededOne question can have several(different) inputs
  29. 29. Uniqurate Questions 2: Intermediate Mode Click here to add MathML Only HTML content is visible Use qv{variable-name}Click on MathML in LaTeX to display to edit it variables in MathML
  30. 30. Uniqurate Questions 2: Expert ModeFiles in package QTI XML for this question
  31. 31. Packaging Tests (Uniqurate editor)
  32. 32. QTIDI Online Demo WebinarSue Milne, University of Glasgow & ELandWeb Ltd Niall Barr, University of Glasgow
  33. 33. Project Aims The project will provide a documented and packaged version of standards-compliant assessment delivery tools  Learning Tools Interoperability (LTI) connection between popular VLEs and delivery software  QTIWorks rendering and responding engine The tools will be distributed to the project’s client partners  Kingston University,  Harper Adams University College  University of Strathclyde.
  34. 34. ToolsLTI Connector – connect VLE with deliverysoftwareDelivery software:LTIQuizzes – deliver basic question typesQTIWorks – all-singing, all-dancing delivery –includes facilities for Maths
  35. 35. LTI Connector for VLEs Our simple QTI 2.1 Assessment Tool - LTIQuizzes ConnectyLearning System thing (LTI) Our fancy QTI 2.1 Assessment Tool – QTI Works
  36. 36. QTIWorks – QTIv2.1 delivery software Validates questions and tests – reports problems Supports all QTIv2.1 UK HE profile Supports input and manipulation of Maths expressions Full randomisation, including Maths expressions Feedback may be targeted on analysis of input Connects to VLE through LTI Connector … available late summer 2012
  37. 37. Installation? LTI – Moodle 2, newer versions of Blackboard and Sakai VLEs are already LTI-enabled. LTIQuizzes – a single .war file needs to be copied to a Tomcat server, then an XML configuration file is edited to add the LTI details (server names and shared secrets). QTIWorks – online demo at https://www2.ph.ed.ac.uk/qtiworks/ , .war file expected soon for institutional installation on Tomcat server
  38. 38. Links... QTIWorks – online demo at https://www2.ph.ed.ac.uk/qtiworks/ QTI Support Site (examples, tools, resources...) http://qti- support.gla.ac.uk Contacts...  Niall Barr (Project Leader) niall.barr@glasgow.ac.uk  Sue Milne (Project Manager) sue.milne@e-learning- services.org.uk  David McKain (QTIWorks Developer) david.mckain@ed.ac.uk
  39. 39. Hands-on TimePlease type any questions or comments in the chat box...
  40. 40. Just in Case... The following slides are provided in case the live demo is not available... They are quite mathematical in flavour, but resources from other disciplines work just the same!
  41. 41. QTI WorksQTI Works https://www2.ph.ed.ac.uk/qtiworks/Use the Demo section to try out questions and testsLog in to set up LTI delivery data
  42. 42. QTI Works – Demos Try out andUpload questions download example Validate questionsor tests to check questions and tests and tests operation from collections
  43. 43. QTI Works – VLE Setup Generate LTI dataUpload questions for setting upor tests to use in questions and tests VLEs in VLEs
  44. 44. Uploading an Assessment Content of uploaded assessments –question or test – can be changed (e.g. fix a bug) Date of last change is displayed
  45. 45. Setting Up Assessment Launch Data Saving a delivery with LTI enabled creates launch data Data is pasted into Moodle 2 External Tool dialog or equivalent in other VLEs
  46. 46. QTI in Moodle 2 – Question
  47. 47. QTI in Moodle 2 – Question with Solution
  48. 48. QTI in Moodle 2 – Maths Input
  49. 49. Test: Algebra 1 This test is the one used in the Maths T1 topic Algebra 1. It is also in the demo course.
  50. 50. Question in a TestThe indicator at top right shows the question status. No feedback isvisible during the test – students have been using these randomisedquestions in formative mode for several weeks. Candidates return tothe list of questions to select another question.
  51. 51. Test feedback & reviewThese are the two parts of the feedback and review page; the scoresfor the questions are displayed and the candidate may return to seethe questions and their input.

×