Questionmark
Ian Gardner
Blackboard London User Group Dec 2011
Background
• London BUG July 2010
A number of issues lead to Questionmark trial
Authoring
Delivery
Reporting
Scalability
What is Questionmark?
Purpose built eAssessment tool:
• Perception
– Authoring
– Scheduling
– Delivery
– Reporting
• ‘OnDemand’ and mobile services
Our experience
Pros Cons
Has proven stable and scalable Building block did not match with our
workflow/processes*
Mobile delivery options available Some issues with browser/OS consistency
Authoring/editing options fairly intuitive Not yet entirely browser based**
and include quality control measures
Vastly superior graphical reports available A new tool for tutors and administrators
through admin UI including assessment to learn (leading to some questions over
quality measures*** ownership)
Roadmap looks good including adaptive
testing****
Short presentation updating the group on the use of Questionmark.
- When presented at July 2010 meeting had decided to trial QM due to limitations of Blackboard including (from 2010 presentation):Authoring issues – interface, speed, multi versions for different students/time limits, pool manager and question manager not intuitive, MS Word copying issues. Author according to test not by question so sharing difficult.Delivery – crashes require manual reset, adding scores together, email correspondence, stress to student, staff time.BB crashed with the high numbers attempting.If problem with a test cannot change easily change a question.Reporting issues – GradeCentre crashes, limitations of reports, cannot do a good question by question analysis.Overall – scalability issues leading to a lack of confidence in using for large numbers.Reasons for choosing Questionmark included availability of building block.
Brief explanation of core Questionmark components, four areas with key improvements over Blackboard.Impressive list of clients (from HSBC to a number of universities) and, seemingly, still growing.
Some details of our experiences* However, LTI option looks like it might. Having a central admin or two do a lot of the work not how we have run other learning technologies.** Moving away from use of desktop application which we have had some issues with in terms of port/firewall restrictions. Version 6 of QM should no longer require the desktop application (due summer 2012).*** Reports include question quality indicators but also information such as IP address of participants. Have launched new ‘analytics’ strand with various new reports.**** Whilst Blackboard SP6 improved assessment considerably they still seem to be some way behind Questionmark. Adaptive testing in this context being the release of question depending on the success of a previous question.