Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using the Qualtrics Research Suite as a Training LMS

261 views

Published on

While learning management systems (LMSes) are a particular technology class, plenty of other technologies have been harnessed and retrofitted for learning purposes. Alternate LMSes include social media platforms like Twitter and survey systems. This presentation summarizes the uses of an online research (survey) suite, Qualtrics, for large-scale compliance-based trainings at Kansas State University. Some affordances of Qualtrics as a training LMS include the following:
a multimedia approach (including mobile),
a file upload approach,
logic functions,
branching logic for versioning a training (based on profiles or behaviors),
a scoring feature for grading performance,
logic tools to set standards for pass/fail,
integration with Google Translate,
security features,
built-in data analytics tools,
a dashboard for versioning different reports for different data clients,
an API for recording training completions and performances in information systems,
easy data export for external analytics,
and unique “skins”.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Using the Qualtrics Research Suite as a Training LMS

  1. 1. Using the Qualtrics® Research Suite as a Training LMS SIDLIT 2017 | LMS PRECONFERENCE COLLEAGUE 2 COLLEAGUE AUG. 2, 2017
  2. 2. Presentation While learning management systems (LMSes) are a particular technology class, plenty of other technologies have been harnessed and retrofitted for learning purposes. Alternate LMSes include social media platforms like Twitter and survey systems. This presentation summarizes the uses of an online research (survey) suite, Qualtrics, for large-scale compliance-based trainings at Kansas State University. Some affordances of Qualtrics as a training LMS include the following: a multimedia approach (including mobile), a file upload approach, logic functions, branching logic for versioning a training (based on profiles or behaviors), 2
  3. 3. Presentation (cont.) a scoring feature for grading performance, logic tools to set standards for pass/fail, integration with Google Translate, security features, built-in data analytics tools, a dashboard for versioning different reports for different data clients, an API for recording training completions and performances in information systems, easy data export for external analytics, and unique “skins”. 3
  4. 4. Presentation Order Basic Needs for Automated Trainings on a Campus Qualtrics Research Suite Features How Qualtrics is a Training LMS Additional Desired Features 4
  5. 5. Basic Needs for Automated Trainings on a Campus 5
  6. 6. Automated Trainings An institution of higher education requires… ◦ a number of policy and safety compliance trainings for the entire campus as well as specialized trainings for specific populations (ability to roll out trainings to tens of thousands annually)…with a dynamic workforce and dynamic student population ◦ the ability to verify that the individual took the training and passed or failed (sufficient to the level of legal standards) (ability to respond to courts in lawsuits and to regulatory and oversight agencies) ◦ the ability to quickly update contents to a training to ensure that it is as accurate as possible for the particular topic (ability to ensure accurate training…to the minute, such as on issues of safety and security for lab workers, etc.) ◦ the ability to deliver trainings in an accessible way given the requirements from Section 508 and other federal accessibility requirements 6
  7. 7. Why Automated? Trainings have to be delivered to scale (to the full campus) ◦ Macro-level scale ◦ Micro-level scale ◦ All points in-between Trainings have to be delivered to recurring high quality for pedagogical and training purposes There have to be accurate records of the contents of the trainings, to ensure that they were designed to documented requirements The trainings may be delivered synchronously or asynchronously, face-to-face or at a distance, and in various other permutations… The most convenient approach is to have online contents that may be versioned in different ways for different uses. 7
  8. 8. Why Automated? (cont.) Trainings have to be delivered so that records are fully accurate and stand up to legal and regulatory agency scrutiny ◦ Legal requirements include protection of intellectual property, privacy protections, accessibility, and others Regulatory agencies require three main things: ◦ Delivery of the proper information and skills, without inaccuracies ◦ Delivery of the proper trainings to the proper individuals at the proper times ◦ Clear record-keeping of all trainings taken and the performance of the individuals (in a way that stands up to courts) The privacy of all involved has to be protected 8
  9. 9. About Qualtrics, Inc. and the Qualtrics Research Suite ABOUT THE COMPANY Based in Provo, Seattle, Dallas, and Washington, D.C.; Dublin; London; Munich; Melbourne, Canberra, and Sydney “Powers over 8,000 of the world’s leading brands and 99 of the top 100 business schools” Features a range of tools, with Qualtrics Research Suite as one ◦ Many be instantiated with a range of features (some of which are not in the K-State instance and some of which will not be included in this slideshow) ABOUT QUALTRICS RESEARCH SUITE Offers a cloud-based research suite (software as a service / SAAS) Survey tool typically used for qualitative, quantitative, and mixed methods research 9
  10. 10. Qualtrics Research Suite Features 10
  11. 11. Feature: Multimedia Integration Variety of question types (including visual ones like hotspot and heat map) Ability to integrate audio, video, and other forms of multimedia Ability to integrate external apps and websites through inline frames (iframes) Ability to integrate external application programming interfaces (APIs) and applications with Qualtrics API Ability to use scripts and codes to access external APIs (like Google Maps) Strengths for Automated Training: Ability to create a seamless multimedia-rich experience Ability to enrich ways of interacting within the training 11
  12. 12. Feature: A File Upload Approach Ability to request learners / trainees to take a photo and upload it…or create a digital file (of various types) and upload those Strengths for Automated Training: Broadens the range of possible assessment types Enables the use of smartphone and mobile device functionalities such as built-in cameras 12
  13. 13. Feature: Logic Functions Ability to set up notifications when particular events occur (email triggers) Ability to populate various Contact Lists based on particular preconditions (contact list triggers) Ability to set quotas for responses or particular occurrences (quotas) Ability to set up math logic for variable questions (like for story problems) and with the use of Embedded Data (to capture the dynamic data) Ability to use prior responses of the learner in a future customized question (piped text) 13
  14. 14. Feature: Logic Functions (cont.) Strengths for Automated Training: Ability for trainers to be aware of particular event occurrences Ability to collect information based on particular preconditions Ability to set ceilings on quotas Ability to customize the training experience Ability to add some non-deterministic elements (such as through math logic) 14
  15. 15. Feature: Branching Logic Ability to create branching logic based on particular conditions (like performance, like selection of a particular answer to a question, like identity-selection, or other feature) Ability to customize training based on performance (or learner profile) Ability to create different training experiences for people based on their level of understanding Strengths for Automated Training: Ability to make trainings unique to the individuals (based on their profiles, their performance, their selections, and other mixes of features) Ability to build one training for a wide range of learners with differing learner needs 15
  16. 16. Feature: Scoring Ability to set values for particular questions Ability to sum scores for a particular training (or segment of training) Ability to change scoring levels after data collection for accurate processing ◦ For example, if a Likert scale is applied, and the valuing is opposite of what is desired, the scoring may be reset to the correct values Strengths for Automated Training: Ability to add point values to each question…and to set granular conditionals for the earning of particular points within that question 16
  17. 17. Feature: Standards for Pass/Fail Ability to set a score threshold for pass / fail Ability to set scoring tiers for grouping or classification Ability to set defined granular conditions for passing (and / or) Strengths for Automated Training: Ability to set a threshold for “passing” and then sending learners off in different directions (based on performance) 17
  18. 18. Feature: Integration with Google Translate Ability to machine-translate an entire training into a number of different languages using Google Translate (with dozens of languages available and representable using UTF-8) ◦ But need vetting by native speakers to ensure the accuracy of the translations! Easy ability to update the translations manually Strengths for Automated Training: Ability to tailor the language of the training to trainee’s first or preferred languages 18
  19. 19. Feature: Security Tools Access protections: ◦ CAPTCHA (Completely Automated Public Turing test to tell Computers an Humans Apart) requirements (to prove respondent’s humanity against ‘bot) ◦ Password protections ◦ Going to a training site from a given URL (uniform resource locator) but disallowing other accesses ◦ Making a training not findable by web browser spiders and crawlers ◦ No ballot box stuffing based on Internet Protocol (IP) address (so a person may not continuously take a training by returning to a particular link from a particular computer on the network) ◦ Setting time limits on any training ◦ Enabling access to a training based on an Authenticator feature or email invite (with unique links) 19
  20. 20. Feature: Security Tools (cont.) Strengths for Automated Training: ◦ Striving to ensure that whomever is signed up to take the training is the one actually doing so ◦ Protecting training contents against unintended downstream other uses 20
  21. 21. Feature: Built-in Data Analytics Data visualizations from online trainings In-tool data analytics ◦ Cross-tabulation analysis (contingency table) ◦ Text analysis (rudimentary tool) Ability to study results data (such as pass/fail) as a dummy variable Strengths for Automated Training: Ability to learn about learner performances 21
  22. 22. Feature: Dashboard for Reportage Dashboard for reporting summary data about trainings to different data clients Dashboard enabling access to raw granular data (including single learner-level data, depending on privacy settings—but should never unmask learners) Ability to conduct item analysis on respective questions in a survey Strengths for Automated Training: Can enable others to access the summary performance data (but will need to turn off “View Personal Data” feature at the global level, so personal identifiers and data are not accidentally leaked 22
  23. 23. Feature: Dashboard for Reportage (cont.) Strengths for Automated Training (cont.): Ability to capture an overall sense of learner performance ◦ Can run a decision tree learning algorithm from the data to see if there are indicators of whether a learner will “pass” or “fail” Ability to capture the training efficacy Ability to study question effectiveness through an item analysis ◦ Can know when learners stop a training and drop out and at which part of the training 23
  24. 24. Feature: API for Grades Recording Ability to conduct a closed invite for a training…and then auto-capture learner performance for recording in external information systems ◦ Requires developer work through the Qualtrics API Strengths for Automated Training: Ability to handle grade recording to various information systems for college- or university-wide trainings and performance without the need for a human intermediary (human intermediaries may advertently or inadvertently introduce inaccuracies into the grade records) 24
  25. 25. Feature: Data Export Ease of data export for external analysis (such as in Excel and / or SPSS, or others) An integration with NVivo 11 Plus (integration) Strengths for Automated Training: Ability to leverage the automated online training data to greater effect Ability to create more data visualizations from the automated online training data 25
  26. 26. Feature: Unique “Skins” A variety of design looks-and-feels for laptops, desktops, and mobile devices Strengths for Automated Training: Branding trainings to particular learner groups based on look-and-feel Branding trainings to particular “use cases” 26
  27. 27. About Features A tool’s functions and features should not be considered in isolation but as part of a system. As with all technology systems, there are tradeoffs with different parameter settings. People who would design to complex systems would do well to ◦ Understand the full implications of going with one setting over another (even the hidden implications) ◦ Test and retest all programming of the training ◦ Test and retest all programming of the training whenever certain changes are made because a cloud-based service may occasionally have coding errors ◦ Keep open channels so that respondents can let a trainer know if something is not looking right 27
  28. 28. How Qualtrics is a Training LMS 28
  29. 29. Required Features of a Training LMS: Basics Learner enrollment ◦ Light learner tracking ◦ Light learner profiling Learner and user identity protections Delivery of digital contents ◦ Easy updatability ◦ Easy customizability ◦ Ability to deliver broadly ◦ Ability to target learners specifically Data protection and storage 29
  30. 30. Required Features of a Training LMS: Basics (cont.) Question creation and assessment ◦ Ability to provide real-time feedback Accurate scoring per question and “grade” totaling ◦ Setting thresholds for passing ◦ Setting paths for training re-study and retakes Accessibility features ◦ Accommodations for vision (visual acuity, color blindness, contrast, and others), hearing, symbolic processing (dyslexia, and others), mobility, and other challenges Mobile friendliness to enable broad device uses Learner aids: cognitive scaffolding, effective feedback, access to review, access to downloadable learning objects 30
  31. 31. Required Features of a Training LMS: Basics (cont.) Analysis of training efficacy / inefficacy (with downloadable data tables and effective data visualizations) Data analysis System robustness (to legal standards and legal defensibility) Automated and accurate data populating to university information systems (for recordkeeping) Seamless linking to other learning resources, through inline frames (iframes), links, uploads, and other integrations 31
  32. 32. Required Features of a Training LMS: Advanced Branching logic Customizations for learners ◦ Based on learner profiles ◦ Based on learner performance (prior or current) Select email contacts (through triggers)…to enable further human support or interventions Select contact list populating (through triggers) Built-in data analytics ◦ Cross-tabulation analysis ◦ Text analysis 32
  33. 33. Takeaways Technology tools may be designed for particular target purposes, but there are other applications that may be applied. It helps to decouple a tool from its original stated purposes and to look at functionality alone. A tool has to be tested from beginning-to-end to make sure everything functions as required. It helps to have developers look at API (application programming interface) functions, too, to add capabilities to a particular tool instance. 33
  34. 34. Additional Desired Features 34
  35. 35. Additional Desired Features in Qualtrics as a Training LMS Social profiles of learners ◦ Possible to build using {a} piped text feature Heightened learner awarenesses of each other (for learning communities) ◦ Possible with third-party tool integrations Ways to extract indicators that are correlated with learning ◦ Possible with decision tree learning in an external machine learning tool More item analysis and learner performance features Suggested ways to improve trainings Easier integration of off-campus individuals to collaborate on the design of trainings 35
  36. 36. Additional Desired Features in Qualtrics as a Training LMS (cont.) Ways to create learning communities Some third-party content integration for learning User-created content sharing (for training) Ability to create course sequences that are linked and auto-sequenced 36
  37. 37. Contact and Conclusion Dr. Shalin Hai-Jew ◦ iTAC ◦ 212 Hale / Farrell Library ◦ Kansas State University ◦ shalin@k-state.edu ◦ 785-532-5262 The presenter has no formal tie to Qualtrics, Inc. 37

×