Questionmark Briefing: Using Assessments to Mitigate Risk and Ensure Regulatory Compliance

1,341 views

Published on

Slides from Questionmark briefing that provides an overview of how surveys, quizzes and tests help organisations mitigate risk and maintain regulatory compliance.

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,341
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Customers use Questionmark technologies and services to implement and enhance efforts to the following key organizational objectives: In Education, to deploy assessments to: Improve learning outcomes Monitor and Increase student satisfaction Data for institutional effectiveness, accreditation In Business and Government – Learning, Training and HR professionals use Questionmark for… Tests, quizzes and surveys to ensure Regulatory Compliance Tests, quizzes and surveys to ensure Heath and Safety Verify the skills and knowledge Sales professionals and partners Verify the skills and knowledge technical staff Hire the right people Awarding bodies and test publishers use Questionmark for… Efficiently create and maintain valid, reliable tests Generate revenue from exams and practice tests
  • I have a few of these stock photo images if they are useful – a couple of them in high-res
  • Source: http://webarchive.nationalarchives.gov.uk/20130201171633/http://www.fsa.gov.uk/about/press/facts/fines
  • Source http://www.citizen.org/documents/20731.pdf
  • Source: http://www.fda.gov/downloads/ICECI/EnforcementActions/UCM346964.pdf
  • See http://blog.questionmark.com/to-your-health-to-err-is-human-but-assessments-can-help for source
  • Understanding Floor is 50%
  • Understanding
  • Ability
  • A number of questions could be derived from a single graph. Read the graph. Compare the numbers. Calculate probabilities to predict who will save the most/least next year.
  • A number of questions could be derived from a single graph. Read the graph. Compare the numbers. Calculate probabilities to predict who will save the most/least next year.
  • Combines adding with reading chart.
  • Translations can be conducted “on the fly” within a new browser-based tool in Perception. Or, for larger scale projects, the system enables the export/import of items and assessments for translations conducted via a third party application or vendor. Once translated, the items, topics and assessments remain connected to their base-language counterparts for efficient management of subsequent edits and reporting.
  • Supported Browsers for.. v5.4 OnDemand as of June 2012 release Internet Explorer 7 Internet Explorer 8 Internet Explorer 9 Firefox 3 Firefox 4 Firefox 5 Firefox 6 Firefox 7 Firefox 8 Firefox 9 Firefox 10 Firefox 11 Firefox 12 Firefox 13 Safari 4 Safari 5   Chrome 13+ Opera 11+ Android Browser 2+   Safari for iOS 4 & 5   Questionmark App for Android Questionmark App for iPhone Questionmark App for iPad
  • Supported Browsers for.. v5.4 OnDemand as of June 2012 release Internet Explorer 7 Internet Explorer 8 Internet Explorer 9 Firefox 3 Firefox 4 Firefox 5 Firefox 6 Firefox 7 Firefox 8 Firefox 9 Firefox 10 Firefox 11 Firefox 12 Firefox 13 Safari 4 Safari 5   Chrome 13+ Opera 11+ Android Browser 2+   Safari for iOS 4 & 5   Questionmark App for Android Questionmark App for iPhone Questionmark App for iPad
  • Questionmark’s Reporting and Analytics provides the tools and reporting capabilities you need to uncover the meaning hidden within your assessment results -- and to share data in a meaningful way with stakeholders. Getting results is the main reason many of us are using assessments. What are you looking to understand? Did the learning investment impact performance? Where are the skills and knowledge gaps in my organization? Are my questions and tests valid and reliable? What did participants think of the learning experience? What did participants learn from the training or course? How do I coach participants to improve performance? How did one group of test takers compare to another? Reports to cover here… Item Analysis Are my tests reliable (have report open) Test Analysis Have report open Survey alt Course Evaluation Explain types of reports available Coaching Documenting training (compare benchmark and others) Show the batch output (slide or just speak to it)
  • Item analysis Test analysis
  • On the right side is your Questionmark data – safe and protected in the repository, but accessible securely via the QMWISeR API You can access via QMWISeR, using the OData standard – which has been adopted by many common data analysis and visualization tools such as… Excel – Power pivot Access SharePoint web parts Tableau And more….
  • Customer:            SAIC Sales Person:        Dave Leon Amount of Sale:    $139,420 Type of Sale:        Annual Hosting Contract inclusive of ESSP + Training Usage:                 internal usage with employees to assess compliance and evaluations Key Factors:         flexibility of Hosting program, integration with Plateau, extensive authoring features
  • Sharepoint provides an excellent platform for proving streamlined, Single-sign-on access to Perception Assessments You can: If your organization uses Windows authentication, you can easily embed assessments or Perception.php (assessment list screens) in SP portal pages If your organization does NOT use Windows authentication, you may deploy the “SharePoint Connector”, which creates a “My Assessments” web part (basically a list and links to assessments scheduled for a participant), enabling one-click, SSO launching of scheduled assessments from within SharePoint
  • We introduced the SAP Connector a couple of years ago – which integrates Perception with SAP’s LMS – the SAP Learning Solution. How it works: The SAP Connector links the SAP Enterprise Learning environment to Questionmark Perception. Assessments in Questionmark Perception can be scheduled from the SAP Enterprise Learning environment. A learner will be able to launch the scheduled assessment from the SAP Learning Environment portal. The assessment is delivered by Perception which, upon completion of the assessment, passes results based to the SAP Enterprise Learning. We are pleased to announce that the Connector for v5 is already available – in fact it is built-in to the shipping software (though it still requires a license to activate) In fact, we are about to announce publicly that SAP has certified Perception for integration with SAP applications -- this means that Perception has undergone successful completion of extensive testing by SAP to ensure the soundness of the integration between the two platforms.
  • Key Message – A variety of ways to learn more about Perception and Questionmark QM Blog – Learn industry and product related information from a variety of Questionmark team members. From the CEO to the Analytics Manager to Consulting Services to Integration Specialists. RSS Feeds – Stay on top of the latest Perception news product or technical through email notifications or through the rss feeds. Training Course – 3 Day course, we can come to you, you can come to us. Full intense training course from Authoring to Reporting. Arrive a beginner and leave as an intermediate user of Perception. Learning Café – A library of Flash and PowerPoint tutorials of all the individual aspects of Perception Manuals and Best Practice Guides – Learn how to use Perception with in-depth manuals full of pictures and text.
  • Key Message – A variety of ways to learn more about Perception and Questionmark QM Blog – Learn industry and product related information from a variety of Questionmark team members. From the CEO to the Analytics Manager to Consulting Services to Integration Specialists. RSS Feeds – Stay on top of the latest Perception news product or technical through email notifications or through the rss feeds. Training Course – 3 Day course, we can come to you, you can come to us. Full intense training course from Authoring to Reporting. Arrive a beginner and leave as an intermediate user of Perception. Learning Café – A library of Flash and PowerPoint tutorials of all the individual aspects of Perception Manuals and Best Practice Guides – Learn how to use Perception with in-depth manuals full of pictures and text.
  • Key message – A variety of ways to complete an evaluation of Perception. Web Seminar – jump on a public web seminar that in an hour will go through Authoring, Scheduling, Delivery, Reporting. Learn all the basics of Perception. Download – A full working copy of Perception. Try it, like it, buy it and continue to use what you’ve already started. Everything seen on the demonstration can be done in the evaluation. Tutorial – Get step by step instructions to use the Authoring Manager. A hands on experience to create some questions and make an assessment.
  • Questionmark Briefing: Using Assessments to Mitigate Risk and Ensure Regulatory Compliance

    1. 1. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Questionmark Breakfast BriefingLondonJune 18th 2013
    2. 2. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.9:00 – 9:30 Breakfast and Networking9:30 – 10:45 • Welcomes and introductions• Brief Questionmark Overview• Using Assessments to Mitigate Risk and Ensure RegulatoryCompliance10:45 – 11:00 Networking Break11:00 – 12:30 Assessments Solutions enabled by QuestionmarkCase Study:Questionmark and London Fire BrigadeNed Kelly, Station Manager, LFBQ&A12:30 Close of BriefingAgendaSlide 2
    3. 3. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 3About Questionmark Founded in 1988 Offices in US, UK, Canada Technologies and services that enable organizations tomeasure knowledge, skills, and attitudes securely forcertification, regulatory compliance and improved learningoutcomes. Assessment Management System SaaS and On Premise Questionmark OnDemand Questionmark Perception Multilingual Available in English, Spanish, German, French, Dutch and Norwegian Participant facing software in 20 languages Questionmark Live SME Authoring available in 16 languages
    4. 4. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.John KleemanChairmanChe OsborneVP of SalesKerry WilliamsUK Sales ManagerIvan ForwardAssessmentSolutions ConsultantVivienneShaughnessyCustomer CareChloe MendoncaMarketingQuestionmark Staff at BriefingSlide 4
    5. 5. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Learning & HR inBusiness & Gov’t• Regulatory Compliance• Heath and Safety• Sales and Technical ChannelVerification• Employee SelectionUniversities andFurther Education• Improve learning outcomes• Increase student satisfaction• Data for institutionaleffectiveness, accreditationAwarding Bodies andTest Publishers• Efficiently create andmaintain valid, reliable tests• Generate revenue fromexams and practice testsWhy our customers use assessmentsSlide 5
    6. 6. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Assessment Solutions EnabledDiagnostic/PlacementtestsPre- and Post-course testsQuizzes &KnowledgeChecksSummativeassessmentCourseevaluationObservationalAssessmentsCertification& licensingexamsJob TaskAnalysis
    7. 7. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Classic Business Model:Where assessments and Questionmark fit inSlide 7ProductionFinanceChannelOfferCoreCapabilitiesSupplierNetworkProductDevelopmentCostStructureValuePropositionTarget CustomerRevenueStreamsServicesDevelopment
    8. 8. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Risks to ReputationSlide 8ProductionFinanceChannelOfferCoreCapabilitiesSupplierNetworkProductDevelopmentCostStructureValuePropositionTarget CustomerRevenueStreamsServicesDevelopment
    9. 9. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Risks to ReputationCan be mitigated via Learning & AssessmentSlide 9ProductionFinanceChannelOfferCoreCapabilitiesSupplierNetworkProductDevelopmentCostStructureValuePropositionTarget CustomerRevenueStreamsServicesDevelopmentExamples of Challenges• Compliance with laws (ABC laws)• Product/solution knowledge• Oversold and/or unsatisfied customersExamples of Challenges• Regulatory compliance• Processes & proceduresExamples of Challenges• Regulatory compliance• Processes & procedures• Health & safety
    10. 10. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Using Assessments to Mitigate Riskand Ensure Regulatory ComplianceJohn KleemanChairman, Questionmark
    11. 11. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.0501001502002503003502007 2008 2009 2010 2011 2012Financial services fines (UK FSA £m)Slide 11
    12. 12. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Pharma industry penalties (USA $m)Slide 12Source:citizen.org
    13. 13. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.US FDA formal “warning letters”Slide 13
    14. 14. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Increasing regulatory action Less tolerance from public Business and regulators international(e.g. FCPA, UK Bribery Act) Companies still expected to perform Can assessments make complianceeasier and reduce risk?Compliance is changingSlide 14
    15. 15. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Love to understand some of your context Feel free to interrupt with comments or questionsSmall groupSlide 15
    16. 16. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. 7 reasons why assessments mitigate compliance risk1. Regulators encourage them2. Demonstrate commitment3. Find problems early4. Reduce human error5. Document understanding after training6. Increase knowledge / reduce forgetting7. Reduce cost & time for compliance training Some practical advice on using assessmentsAgendaSlide 16
    17. 17. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Reason #1 : Regulators encourage themSlide 17
    18. 18. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. UK FSA “where staff understanding has not beentested, it is hard for firms to judge how well therelevant training has been absorbed” US FDIC: “Once personnel have been trained on aparticular subject, a compliance officer shouldperiodically assess employees on theirknowledge and comprehension of the subjectmatter”Financial services regulation
    19. 19. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. WHO good manufacturing principles “Continuing training should also begiven, and its practical effectivenessperiodically assessed.” European Commission (blood handling) “The contents of training programmes shallbe periodically assessed and thecompetence of personnel evaluatedregularly.”Health care regulatorsSlide 19
    20. 20. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Does your regulator Require you assess employees Encourage you to assess employees Doesn’t care about assessments?Comments?Slide 20
    21. 21. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Lower penalties for tobacco retailers if training in place Guidelines suggest “employees to take a written test prior to selling tobacco products” “All employees should receive refresher training, including a writtentest, at least yearly “US FDA – tobacco retailersSlide 21
    22. 22. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Reason #2 : Demonstrate commitmentSlide 22
    23. 23. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Because employees follow their leaders Charles Jennings (August 2012)“… common thread that runs through almost allhigh-profile compliance catastrophes. It is thatthe top-tier executives and middle managers inthe organisations simply don’t model thebehaviours that will lead to a culture ofcompliance”Why is it important to demonstrate toplevel commitment?
    24. 24. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. If you require all employees and managers to Take assessments every 6 or 12 months Remedial training and retake assessment if fail Assessments seen as fair Major stride in demonstrating commitmentHow do assessments demonstratecommitment?Slide 24
    25. 25. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. UK FSA 2009: “Aon Ltd should have ensured that appropriate members ofstaff – particularly those in the Aviation and Energy divisions –received focused training in relation to this area and weretested on their understanding of the relevant risks involved.Effective training and testing in this regard would haveemphasised to staff the importance of carrying out effectivedue diligence prior to authorising an Overseas Third Party forpayment.”Fine of UK£5.5m for not testing, partlydue to lack of commitmentSlide 25
    26. 26. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Reason #3 : Find problems earlySlide 26
    27. 27. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Surveys identify issues in culture and ethics Most employees want to help. From employee of bankinvolved in 2012 LIBOR scandal:“Always happy to help, leave it with me, Sir”. UK Ministry of Justice Bribery Act Guidelines“Staff surveys, questionnaires and feedback from training canalso provide an important source of information oneffectiveness and a means by which employees and otherassociated persons can inform continuing improvement of anti-bribery policies”.How surveys can find problems earlySlide 27
    28. 28. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Look at topic and assessment scores By department By geography Over time Look at wrong answers Look at test failures Assessments touch every employee: a unique toolHow tests/quizzes/exams can findproblems early
    29. 29. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Reason #4 : Reduce human errorSlide 29
    30. 30. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Is human error a problem in your business? What causes error?Your thoughts
    31. 31. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.22%23%29%15%6%5%Root causes of human error(data from MHRA 2011, graph by Questionmark)Process/procedure incorrect Procedural steps omitted Concentration errorTraining misunderstood Training missing Poor communication/rushing
    32. 32. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Assessments can fix Training misunderstood (15%) Training missing or out of date(6%) Procedural steps omitted(23%) Process or procedure incorrect(22%) Other causes of error Poor communication / rushing(5%) Concentration error (29%)AssessmentsSlide 3222%23%29%15%6%5%Process/procedure incorrectProcedural steps omittedConcentration errorTraining misunderstoodTraining missingPoor communication/rushing
    33. 33. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Reason #5 : Document understandingafter trainingSlide 33
    34. 34. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Because regulators require it common regulatory requirement, e.g. OECD recommends onethics/compliance programme:measures designed to ensure periodic communication, anddocumented training for all levels of the company Because makes business sense Trained employees more effective “If you can not measure it, you can not improve it.“Why document training for compliance?
    35. 35. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.TakeattendanceGet employeesignaturesTest to checkunderstandingCommon Methods of Documenting TrainingSlide 35
    36. 36. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How to do Trainer records attendance or takes photo of attendees Record online attendees Pros Easy to do Measures who turns up Cons Did they stay the whole time? Did they pay attention? Did they get it?Take attendanceSlide 36
    37. 37. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How to do Get employees to sign a form confirmingthat they attended training AND that theyunderstand the rules Pros Gets employee confirmation Employee committed to follow rules Cons People may sign but not take seriously People may sign without attending orunderstandingGet employee signaturesSlide 37
    38. 38. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Not where safety is involved according to 2005 UnitedStates Appeal Court:“merely having an individual sign a form acknowledging hisresponsibility to read the safety manual is insufficient to insurethat the detailed instructions contained therein have actuallybeen communicated.” Complete General Const. Co. v. OSHRC, 2005 U.S. App. LEXIS 5197 (6thCir.), March 29, 2005Does getting an employee to sign givelegal protection?Slide 38
    39. 39. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. SEC on FCPA compliance:Titan’s only related ‘policy’ is a statement in Titan’s Code ofEthics, which all Titan employees were required to signannually, stating employees must be familiar with and strictlyadhere to such provisions … Titan did not enforce that policy nordid it provide its employees with any information concerningthe FCPA or its purposes”. UK FSA example of poor practice:“… employees were merely required to sign a form to confirmthat they were aware of the firm’s AML procedures, but theirunderstanding was never tested.”What do regulators think?Slide 39
    40. 40. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How to do: After training, test employees People who fail test undergoremedial training Pros Genuinely check understanding Allows improvement of weaknesses Feedback to improve training Cons More effort You have to follow through & takeaction on failuresTest to check understandingSlide 40
    41. 41. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Reason #6 : Increase knowledge /reduce forgettingSlide 41
    42. 42. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Compliance TrainingSlide 42ReceiveinformationWorkingmemoryLong termmemory
    43. 43. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Cue• Question• Other stimulusSearch• Retrieve from longterm memoryAct• Act or respond Research showsretrieval practice helpsfuture retrieval Successful retrievalmodifies memorymaking it moreretrievable in futureTraining useful if can retrieveSlide 43
    44. 44. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. 120 students divided into groups randomly Given reading comprehension passages used in TOEFL Some students Study for 14 mins Some students Study for 7 mins Recall material for 7 mins Compared results after 5 mins and 1 weekRoediger & Karpicke experiment 2006Slide 44
    45. 45. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.0%10%20%30%40%50%60%70%80%90%After 5 mins After 1 weekRestudyQuizHow much learners retainedSlide 45
    46. 46. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 46Learning in the Actual WorldThey don’t get it All!
    47. 47. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Learning CurveSlide 47Ouch! TheForgettingCurveTimeKnowledge/SkillsLearningExperienceLearningExperienceLearningExperienceQuestionsQuestionsPurpose:• Content repetition• Memory retrieval practice• Strengthens memory recallPurpose:• Content repetition• Memory retrieval practice• Strengthens memory recall
    48. 48. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.In the Actual WorldThey Forget It!Slide 48
    49. 49. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slow the Forgetting CurveSlide 49TimeKnowledge/SkillsLearningExperienceLearningExperienceLearningExperienceQuestionsQuestionsQuestionsQuestionsPurpose:• Memory retrieval practice• Strengthens memory recall• Reduce Forgetting CurvePurpose:• Memory retrieval practice• Strengthens memory recall• Reduce Forgetting Curve
    50. 50. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Reason #7 : Reduce cost & time forcompliance trainingSlide 50
    51. 51. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Take testIf pass, skiptrainingOtherwisedo trainingTesting out of compliance trainingSlide 51
    52. 52. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Management Report ExampleA) Number of people 1,000B) Time for training if no testing (2 hours each) 2,000 hoursC) Time to take test (20 mins / test) 333 hoursD) 25% pass test and skip trainingE) 75% do compliance training 1,500 hoursF) Hours of training avoided (B – C - E) 177 hoursG) Average cost / hour $100H) Money saved (F * G) $17,700Example of how can save moneySlide 52
    53. 53. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Many regulators permit “testing out” to reducecompliance training ABA Bank Compliance magazine article 2009 interviewedsome US regulators OTS: Testing out acceptable OCC: Agree, providing testing program well structured. May besome exceptions FDIC : Agree Check with your regulatorWill the regulator allow?
    54. 54. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Do you test out of compliance training?Your thoughtsSlide 54
    55. 55. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Regulators encourage themDemonstrate commitmentFind problems earlyReduce human errorDocument understanding after trainingIncrease knowledge / reduce forgettingReduce cost & time for compliance trainingSummary on why to use assessments
    56. 56. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Medium and higher stakes Internal exams Post-course tests Diagnostic tests Observational/workplace assessments Lower stakes assessments Course evaluation surveys Employee attitude surveys Job task analysis surveys Practice tests Quizzes during learning 360 degree performance assessmentsCommon assessments in complianceSlide 56
    57. 57. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Good practice in using assessmentsSlide 57
    58. 58. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Create scenario questionsAssess application not just knowledge
    59. 59. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.EvaluationSynthesisAnalysisApplicationComprehensionKnowledge Questions betterif use application/ comprehensionNOT justknowledgeBlooms Taxonomy
    60. 60. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. What does a yellowtraffic light mean? Stop Go Caution If you are drivingtoward an intersectionand the light turns fromyellow to red, whatshould you do? Speed up and cross theintersection Stop suddenly Stop graduallyExampleSlide 60
    61. 61. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Use feedback at the topic levelIt is actionableSlide 61
    62. 62. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. What to do Divide assessment by topic areas Give individual feedback Topic scores Resources or advice to improve weak topics Analyse at department or organization level Topic feedback is actionable Direct participant to specific resources to improve learning Direct compliance to areas knowledge is weakHow topic feedback can help
    63. 63. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Feedback based topics of questionsSlide 63Corrective feedbackpoints participant toappropriate learningresources“Topics” ofquestionsanswered inassessmentScore ontopic
    64. 64. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Consider observational assessmentsMeasure skills and behaviourSlide 64
    65. 65. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Traditional assessment• Participant is presented withquestions for direct responseObservational assessment• Observer is presented withquestions to ‘rate’ participant tomeasure knowledge, skills andabilitiesObservational assessment vs. ‘traditional’quiz, test or examSlide 65
    66. 66. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.PreparationPerforming thetaskWrap UpObserver typically rates participant in 3 areasSmartphones/tablets make more practical
    67. 67. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Excellent book onassessment Criterion-Referenced TestDevelopment by Shrock and CoscarelliMore good practiceSlide 67
    68. 68. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Questionmark white paper www.questionmark.com/uk/whitepapersMore good practiceSlide 68
    69. 69. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Your questionsSlide 69
    70. 70. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.BreakSlide 70
    71. 71. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Assessment Management SystemSlide 71Collaborative, secureauthoring of surveys,quizzes, tests andexamsBlended, secure delivery tomany different types ofbrowsers and devicesEvaluateresults, informstakeholders withmeaningful reports
    72. 72. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.European Data CentreSlide 72ISO 2700 and 9001accreditedState-of-the-artdata centreSaaS /Rapid Deployment24/7 MonitoringGlobal 24/7support available99.9+% Uptime
    73. 73. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.How do we avoid this?
    74. 74. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Good tests should be part: Content  Easy to test for recall Cognitive process  Difficult to elicit appropriate cognitiveprocess in the test Context  performance environmentHow do we “go beyond recall?”
    75. 75. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Cognition Knowledge Skill Ability Do our tests elicit the right level of cognitive demand?What is cognition?
    76. 76. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Format Knowledge Skill AbilityMultiple Choice X XMatching X XExtended Matching X X XT-F X XComplex MC X XPictorial X X XProblem-solving XScenario XWhich type of question?Adapted from Haladyna, Developing and Validating Multiple-Choice Test Items
    77. 77. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Knowledge Matrix• Alternate Choice• Multiple True-False
    78. 78. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Multiple True-False
    79. 79. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Matching• Extended Matching
    80. 80. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Pull-down List• Extended Matching
    81. 81. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.DemonstrationMatching and Extended MatchingSlide 81
    82. 82. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Pictorial Item Set
    83. 83. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Pictorial / Video
    84. 84. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.DemonstrationSlide 84
    85. 85. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Problem-Solving Item Set
    86. 86. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Problem-Solving Item Set
    87. 87. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
    88. 88. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Scenario Item Set
    89. 89. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
    90. 90. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Simple corrective question feedbackSlide 90Correctivefeedbackgiven
    91. 91. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.DemonstrationScenario question with feedbackSlide 91
    92. 92. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Observer rates participants’performance, skills and abilitiesPreparationPerforming thetaskWrap UpObservational Assessment
    93. 93. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.DemonstrationObservational AssessmentSlide 93
    94. 94. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 94
    95. 95. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Distributed authoring anditem managementAuthoring ManagerEnterpriseManagerQuestionmark Live
    96. 96. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Authors, Admins and SMEs:Create Items & AssessmentsQuestionmark Live:Ideal for rapid item and assessment writingQuestionmark RepositoryContentItemsAssessments
    97. 97. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Interactive TranslationsUsing Questionmark TMSTranslations using 3rd PartyToolsTranslation Management SystemTranslation methodsSlide 97Export to XLIFF(XML)Translate using3rd Party ToolsImport fromXLIFF (XML)TranslationManagementSystemManages translation process….does not do translationSet Up Projectusing TMSOn-The-Fly
    98. 98. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 98
    99. 99. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Formative QuizzesCourse EvaluationsDiagnosticTestsLowStakes TestsMediumStakesExamsHighStakesExamsLevelsofMonitoringBlended DeliveryStakes of Assessment
    100. 100. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Support for Leading BrowsersSlide 100InternetExplorerFirefoxSafarifor MacChromeSafari for iOS 4& 5Android2+
    101. 101. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Browser Support Critical for ReliableParticipant ExperienceSlide 101
    102. 102. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Mobile Phone/Device Usage:Decade of Staggering Growth RateSlide 10250%58.20%8.40%32%57%84.70%11.80%90%% of homes with flushtoilets% of homes withelectricity% of Higher Educationcompleters% mobile phoneownership2011CensusofSouthAfrica2001 2011 2001 2011 2001 2011 2001 2011
    103. 103. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Responsive DesignSlide 1032. Device drivers used to format assessment for device/browserParticipantDevices4. On-the-fly auto-sizing of content and controls1. Senses device/Browser 3. Sends assessment to browser
    104. 104. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 104
    105. 105. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Did the learninginvestment impactperformance?Where are theskills andknowledge gaps inmy organization?Are my questionsand tests valid andreliable?What didparticipants thinkof the learningexperience?What didparticipants learnfrom the trainingor course?How do I coachparticipants toimproveperformance?How did one groupof test takerscompare toanother?Analyse and report on results
    106. 106. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.DemonstrationItem AnalysisSlide 106
    107. 107. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.OData API for Easy, Secure Access toAssessment ResultsSlide 107Data-analysis andvisualisation toolsAssessmentResults
    108. 108. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Data “Liberation”Meaningful DashboardsPush-Pull ReportingREST-basedResult ServicesBI Tools Dashboards(e.g. widgets)MicrosoftOfficeOData API for AnalyticsSlide 108
    109. 109. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.AuthoringDeliveryAnalyticsIntegrationSlide 109
    110. 110. Can I integrate Questionmark with various otherEnterprise Systems?
    111. 111. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Integrating with SharePointSlide 111Embedding assessment lists usingSharePoint Connectorfor OnDemandEmbedding assessment listsand assessments usingWindowsAuthentication
    112. 112. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.SAP ConnectorSlide 112
    113. 113. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.What questions do you have?Slide 113
    114. 114. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.• Content• Delivery• ResultsSecure• Millions of testsdelivered• 99.99% Uptime• Tests availablegloballyScalable• Author inBrowser• Multilingual• Mobile, BlendedDeliveryAgile• Accessible• Reliable• 24/7 SupportTrustableWhy Questionmark?
    115. 115. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.The Role of Assessments in Mitigating Risk forFinancial Services Organizations5 Steps to Better TestsAssessments through the Learning ProcessSecurity of Questionmark’s OnDemand ServiceAlignment, Impact and Measurement with the A-modelRecommended White PapersSlide 115https://www.questionmark.com/whitepapers
    116. 116. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 116Ways to Learn MoreQuestionmark Blog!• http://blog.questionmark.comWeb Seminars• http://www.questionmark.com/uk/seminarsWhite Papers• http://www.questionmark.com/whitepapersManuals, Knowledge Base and Best Practices Guides• http://www.questionmark.com/perception/help
    117. 117. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 117Ways to Learn MoreThis November 10th-12thQuestionmark Users will gather inBarcelona to:• Get vital info and training on the latest assessment technologies andbest practices• Learn about the Questionmark product roadmap• Discover real-world case studies by users in business, government andhigher education.• Be introduced to new solutions and features• Network with fellow assessment and learning professionals• Attend sessions explaining testing and assessment best practice sessions• Have the opportunity to influence future solutions• Partake in a one-on-one meetings with Questionmark technicianshttps://www.questionmark.com/uk/conference
    118. 118. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Ways to EvaluateSlide 118Try Questionmark OnDemand• 30 Day Trial – Software as a Service• https://www.questionmark.com/ondemandTry Questionmark Perception• Download – 30 day trial• https://www.questionmark.com/download
    119. 119. Copyright © 1995-2013Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmarkis a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Thank you for attending – Keep in Touch!Slide 119blog.questionmark.comwww.questionmark.com/newslettertwitter.com/questionmarklinkedin.com Look for Questionmark under “Groups” and joinyoutube.com/questionmarkvideoflickr.com/questionmarkpicturesfacebook.com Like us on Facebook

    ×