Using open source assessment and feedback tools


Published on

Slides from webinar in using open source assessment and feedback tools 15 Feb 2013

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Critters, orderMountains, etc. in here too
  • Using open source assessment and feedback tools

    1. 1. Using open source assessment andfeedback tools
    2. 2. Introduction• JISCs Assessment & Feedback programme strand C: Technology transfer • Open source • Embed use in third party institutions  Lessons learned:• The Tools:  Rogō  Full assessment suite  Uniqurate  Item and Test Editor, QTI compliant  QTIDI project:  QTIWorks • Item and Test renderer, QTI and LTI compliant  LTIConnector and LTIQuizzes • IMS Learning Tool Interoperability demonstrators 2
    3. 3. Questions? Wilbert Kraan  3
    4. 4. Rogō OSS Rogō R Open Source SoftwareDr Simon WilkinsonUniversity of Nottingham
    5. 5. Rogō OSSAim“ The Rogō OSS project aimed to take software developed at the University of Nottingham and package it up and create a vibrant open source development community. ”
    6. 6. Rogō OSSEnterprise-level e-Assessment• 7 paper types (inc formative, summative)• 15 question types• Searchable – content, keywords, Bloom’s Taxonomy• QTI 1.2 and LTI 1.0• Feedback – questions – learning objectives
    7. 7. Rogō OSSOpen Source• Released under GPL 3.0• Server: Linux, Apache, MySQL, PHP• Clients: – PC or Mac – IE 8/9, Firefox, Chrome – JavaScript and Flash• Languages: English, Polish
    8. 8. Rogō OSSPartners De Montfort University University of Bedfordshire University of East Anglia University of Oxford University of the West of Scotland
    9. 9. Rogō OSSChallenges• OSS – raising awareness• How best to operate?• Legacy code-base• Local idiosyncrasies/procedures
    10. 10. Rogō OSSTimelineBasic Housekeeping• Name change – TouchStoneRogō• Website• Blog• Mailing list• TRAC – ticketing system• Installation script
    11. 11. Rogō OSSTimelinePlanned Development• Upgrade script• QTI 1.2 import improved• LTI 1.0 support added
    12. 12. Rogō OSSTimelineFurther Development• Sub-directory support• UI styling – CSS rewrite• Language abstraction layerExternal Consultation• Testing procedures• Openness and sustainability
    13. 13. Rogō OSSLessons Learnt• Installation – Project start time – PHP versions – Sub-directory support – Bugs
    14. 14. Rogō OSSLessons Learnt• Testing – Manual (test plan in TRAC) – Selenium – Unit Testing – PHP scripts• Modular Design – Language – Authentication
    15. 15. Rogō OSSFuture – Community• Improved documentation – UI, code and database• GitHub – access to code in development• Support packages
    16. 16. Rogō OSSFuture – Design• OO rewrite• Modular design – Questions
    17. 17. Uniqurate Online Demo WebinarSue Milne, University of Glasgow & ELandWeb Ltd Paul Neve, Kingston University
    18. 18. Project AimsThe Uniqurate project seeks to increase the number of open e-assessment content authors, and as a result, increase the richness of assessment content available to studentsby delivering QTI authoring tools that can be used by a novice user embedding these tools into several “client” institutions
    19. 19. QTI – what’s that? Question and Test Interoperability specification Version 2.1 is becoming final about now Standard for representing questions and tests Many implementations worldwide:  UK, France, Netherlands, Germany, Spain  USA, Mexico  Korea Used by  Educational institutions for internal assessments  Examination authorities (Scotland, Netherlands, Germany, Korea)  Publishers for online content associated with textbooks
    20. 20. Why use QTI? Interoperability:  Share assessment resources with colleagues in same or other institutions  Efficiency, less reinvention of wheels  Economical – costly to produce good resources, so share and repurpose  Preserve materials to prevent loss when changing systems  Ensure content separate from software  All these contribute to SUSTAINABILITY Flexibility:  20 or so input types – and extendable, e.g. Maths  Several inputs in one question – engaging resources  Targeted, individualised feedback, related directly to input
    21. 21. Uniqurate – user-led design Uniqurate project: JISC Assessment and Feedback programme Objective: increase usage of QTI in HE by transferring technology from existing tools into "client" institutions Client partners describe what they need for teaching Uniqurate derived from user-experience driven Aqurate and multiple interaction power of Mathqurate Result – hopefully –a useful tool for authoring QTI content accessible to any user
    22. 22. Uniqurate - Question components Constituent parts providing functionality between question/exercise and student May map onto QTI elements – but may not! – hence the term question component  “What do you want the question to do?” Questions typically composed of several components  Several inputs in one question Identify components that provide optimal benefit
    23. 23. Uniqurate - Modes Friendly Mode  drag & drop components Intermediate Mode  edit HTML parts of existing files: ideal for contextualising Expert Mode  XML editor: customise questions when more control needed  also for packaging non-Uniqurate questions with media
    24. 24. Uniqurate - output As QTI XML file As packaged QTI XML file As package containing QTI XML file and media: images, stylesheets, etc.
    25. 25. Links... Uniqurate – web application – no installation, just go to QTI Support Site (examples, tools, resources...) http://qti- Contacts...  Graham Alsop (Project Leader)  Paul Neve (Uniqurate Developer)  Sue Milne (Consultant)
    26. 26. Hands-on TimePlease type any questions or comments in the chat box...
    27. 27. Just in Case...The following slides are provided in case the live demo isnot available...The questions are quite mathematical in flavour, butauthoring resources for other disciplines works in just thesame way!
    28. 28. Uniqurate Questions 1: Friendly Mode Add more components if neededOne question can have several(different) inputs
    29. 29. Uniqurate Questions 2: Intermediate Mode Click here to add MathML Only HTML content is visible Use qv{variable-name}Click on MathML in LaTeX to display to edit it variables in MathML
    30. 30. Uniqurate Questions 2: Expert ModeFiles in package QTI XML for this question
    31. 31. Packaging Tests (Uniqurate editor)
    32. 32. QTIDI Online Demo WebinarSue Milne, University of Glasgow & ELandWeb Ltd Niall Barr, University of Glasgow
    33. 33. Project Aims The project will provide a documented and packaged version of standards-compliant assessment delivery tools  Learning Tools Interoperability (LTI) connection between popular VLEs and delivery software  QTIWorks rendering and responding engine The tools will be distributed to the project’s client partners  Kingston University,  Harper Adams University College  University of Strathclyde.
    34. 34. ToolsLTI Connector – connect VLE with deliverysoftwareDelivery software:LTIQuizzes – deliver basic question typesQTIWorks – all-singing, all-dancing delivery –includes facilities for Maths
    35. 35. LTI Connector for VLEs Our simple QTI 2.1 Assessment Tool - LTIQuizzes ConnectyLearning System thing (LTI) Our fancy QTI 2.1 Assessment Tool – QTI Works
    36. 36. QTIWorks – QTIv2.1 delivery software Validates questions and tests – reports problems Supports all QTIv2.1 UK HE profile Supports input and manipulation of Maths expressions Full randomisation, including Maths expressions Feedback may be targeted on analysis of input Connects to VLE through LTI Connector … available late summer 2012
    37. 37. Installation? LTI – Moodle 2, newer versions of Blackboard and Sakai VLEs are already LTI-enabled. LTIQuizzes – a single .war file needs to be copied to a Tomcat server, then an XML configuration file is edited to add the LTI details (server names and shared secrets). QTIWorks – online demo at , .war file expected soon for institutional installation on Tomcat server
    38. 38. Links... QTIWorks – online demo at QTI Support Site (examples, tools, resources...) http://qti- Contacts...  Niall Barr (Project Leader)  Sue Milne (Project Manager) sue.milne@e-learning-  David McKain (QTIWorks Developer)
    39. 39. Hands-on TimePlease type any questions or comments in the chat box...
    40. 40. Just in Case... The following slides are provided in case the live demo is not available... They are quite mathematical in flavour, but resources from other disciplines work just the same!
    41. 41. QTI WorksQTI Works the Demo section to try out questions and testsLog in to set up LTI delivery data
    42. 42. QTI Works – Demos Try out andUpload questions download example Validate questionsor tests to check questions and tests and tests operation from collections
    43. 43. QTI Works – VLE Setup Generate LTI dataUpload questions for setting upor tests to use in questions and tests VLEs in VLEs
    44. 44. Uploading an Assessment Content of uploaded assessments –question or test – can be changed (e.g. fix a bug) Date of last change is displayed
    45. 45. Setting Up Assessment Launch Data Saving a delivery with LTI enabled creates launch data Data is pasted into Moodle 2 External Tool dialog or equivalent in other VLEs
    46. 46. QTI in Moodle 2 – Question
    47. 47. QTI in Moodle 2 – Question with Solution
    48. 48. QTI in Moodle 2 – Maths Input
    49. 49. Test: Algebra 1 This test is the one used in the Maths T1 topic Algebra 1. It is also in the demo course.
    50. 50. Question in a TestThe indicator at top right shows the question status. No feedback isvisible during the test – students have been using these randomisedquestions in formative mode for several weeks. Candidates return tothe list of questions to select another question.
    51. 51. Test feedback & reviewThese are the two parts of the feedback and review page; the scoresfor the questions are displayed and the candidate may return to seethe questions and their input.