The K-State Online Canvas LMS Data Portal and Five Years of Activated Third-Party Apps
•
2 likes•700 views
The presenter will introduce the K-State LMS data portal and introduce some available insights from there and focus on one particular facet of this big data--the third-party apps that K-State faculty, admin, and staff have activated and what that says about how we're using Canvas. Canvas LMS data portal for the Kansas State University instance A data dictionary: Version 1.16.2 (https://portal.inshosteddata.com/docs) Data extraction and processing What it can tell us: (un)available data and information Activated third-party tools in K-State Online Canvas LMS instance Some caveats What this says about what K-Staters (early adopters) are using Practical applications of this third-party app activation data Adding value to LMS data portal data
Recommended
More Related Content
What's hot
What's hot (20)
Similar to The K-State Online Canvas LMS Data Portal and Five Years of Activated Third-Party Apps
Similar to The K-State Online Canvas LMS Data Portal and Five Years of Activated Third-Party Apps (20)
More from Shalin Hai-Jew
More from Shalin Hai-Jew (20)
Recently uploaded
Recently uploaded (20)
The K-State Online Canvas LMS Data Portal and Five Years of Activated Third-Party Apps
- 1. The K-State Online Canvas LMS Data Portal and Five Years of Activated Third-Party Apps E V E N T : H A V E A B Y T E ! S E P T . 2 2 , 2 0 1 7 K - S T A T E U N I O N F O R U M H A L L K A N S A S S T A T E U N I V E R S I T Y
- 2. Presentation The presenter will introduce the K-State LMS data portal and introduce some available insights from there and focus on one particular facet of this big data--the third-party apps that K-State faculty, admin, and staff have activated and what that says about how we're using Canvas. 2
- 3. Overview Canvas LMS data portal for the Kansas State University instance ◦ A data dictionary: Version 1.16.2 (https://portal.inshosteddata.com/docs) ◦ Data extraction and processing ◦ What it can tell us: (un)available data and information Activated third-party tools in K-State Online Canvas LMS instance Some caveats What this says about what K-Staters (early adopters) are using Practical applications of this third-party app activation data Adding value to LMS data portal data And internal reports, too 3
- 4. Canvas LMS Data Portal for the Kansas State University Instance 4
- 5. K-State’s Learning Management System (LMS) K-State Online is based on the Canvas LMS. LMSes generally have the following capabilities: delivery of learning contents, intercommunications, collaboration, assessment creation and deployment, grading, collaborations, live web conferencing, persistent profile creation for presence, e-portfolio development, and secure links to various protected on-campus data systems. Canvas LMS is evolved on an agile and fast development cycle with updates every few weeks. This is a hosted solution, which means that the LMS functionality and data reside on Instructure servers. Canvas is known for its integrations with various social media and other platforms. In late 2015, Canvas rolled out its data portal, and K-State signed on to access its data in October 2016. ◦ The data tables range in size up to millions of rows of data. 5
- 6. 6
- 7. 7
- 8. Basic details The data is organized apparently in what is known as the “star schema,” with both “fact” and “dimension” tables (filename_fact, filename_dim). Generally, _fact tables contain foreign keys paired with the primary keys in dimension tables. Fact tables contain quantitative data about events applied to _dim content data. 79 data tables .gz files from SQL Uncompress Change to . csv ingest into Access to select data…and export to Excel for processing and visualizing (or) “union” combined files in SQL Express, query, and export to Excel for processing and visualizing 8
- 9. A brief inventory: Fact files assignment_fact assignment_group assignment_override assignment_override_user assignment_override_user_rollup communication_channel conversation_message_participant course_ui_navigation_item discussion_entry discussion_topic enrollment_fact external_tool_activation file grading_period group_fact group_membership module_completion_requirement 9
- 10. A brief inventory: Fact files (cont.) module module_item module_prerequisite module_progression_completion_requirement module_progression pseudonym_fact quiz_fact quiz_question_answer quiz_question quiz_question_group quiz_submission_fact quiz_submission_historical score_ submission_comment submission_comment_participant submission wiki wiki_page 10
- 11. A brief inventory: Dimension files account_dim assignment assignment_group assignment_group_rule assignment_override assignment_override_user assignment_rule_dim communication_channel conversation conversation_message course course_section course_ui_canvas_navigation course_ui_navigation_item discussion_entry_dim discussion_topic enrollment enrollment_rollup 11
- 12. A brief inventory: Dimension files (cont.) enrollment_term external_tool_activation file grading_period grading_period_group group_dim group_membership module_completion_requirement module module_item module_prerequisite module_progression_completion_requirement module_progression pseudonym_dim quiz_dim quiz_question_answer 12
- 13. A brief inventory: Dimension files (cont.) quiz_submission quiz_submission_historical role score submission_comment submission_comment_participant submission user wiki wiki_page 13
- 14. Canvas LMS Data Portal data The Canvas LMS data portal data is updated daily. Most of the data on the portal enables summary observations. Not all data generated in the Canvas LMS is available in the data portal. Outcomes information is not included, so running analyses by outcomes (Who passed? Who failed?) is not so easily captured. (This requires access to grade data from KSIS.) There are some challenges with both the foreign and primary keys. ◦ Querying this data as “flat files” has its limits. It’s much harder to query across data tables without clarity between the keys. Putting this up on SQL has its limits, too, even though it’s somewhat faster. The data portal includes personally identifiable information (PII) and outright names, and so the data has to be handled with care. 14
- 15. Canvas LMS Data Portal Data (cont.) According to a client recent web conference sponsored by Instructure, it does not look like a data dashboard is in the works yet. There are third-party efforts in higher education to create some programs to enable automated download, the assigning of keys, and so on, to enable some queries for local applications. ◦ Two presenters offered insights on an effort to create a Canvas DataViewer: ◦ Bill Jones, Director, Instructional Technology & Info Management, and ◦ Andrew Anders, LMS Coordinator, both of Eastern Michigan University 15
- 16. Activated Third-Party Tools in K-State Online Canvas LMS Instance 2013 (PARTIAL YEAR) – 2014 – 2015 – 2016 – 2017 (PARTIAL YEAR, 7 MONTHS) 16
- 17. Third-party apps Third-party apps in the Canvas LMS instance may be activated at the global level of the instance, which enables access to those apps by all users and all in-instance courses. Otherwise, third-party apps may be activated at course level by the instructor and / or admin of the course. ◦ The apps have to be activated in each course and cannot be activated across multiple courses simultaneously. 17
- 18. External Apps in K-State Instance of Canvas LMS Path: Settings (left menu) -> Apps (fourth tab over from the left) Can filter by “All,” “Not Installed,” “Installed” Listed in Alphabetical Order Then Have to Use the Dropdown for “More External Tools” 18
- 19. Some available summary activated app stats 295 unique integrated apps available as of August 2017 ◦ Enabled through Learning Tools Interoperability (LTI) standards (by the IMS Global Learning Consortium) Apps are a combination of free, mixed-cost, subscription-based access, and purchase-based access. The point is to integrate functionalities, resources, communities, and other elements in a seamless way from the LMS. At K-State, 3,753 activations of third-party (and some in-house) apps have been recorded during the lifespan of the LMS from late 2013 to late July 2017. ◦ There are popular apps which are widely activated, and then long tails of a few activations. ◦ In terms of how many unique apps K-Staters have activated, they have activated approximately 103 unique applications (both third-party created and locally created). 19
- 20. What app types are available? PROPRIETARY DIGITAL CONTENTS Proprietary digital contents (by third-party content providers for e-books, videos, virtual labs, and others) ◦ Online labs for experiential learning ◦ Recorded lectures Geographical map integrations Mass media contents / news sources OPEN EDUCATIONAL RESOURCES Open educational resources (OERs) Open-access educational resources Open-source digital learning object repository / referatory Coding / coding APIs 20
- 21. What app types are available? CUSTOMIZED LEARNING, LEARNING AIDS, TEACHING AIDS FOR LEARNING Artificial intelligence-informed learning tools Learning supports Dropout indicators for early warning (for teachers) EXPRESSION, DIGITAL CONTENT CREATION, RECORDINGS Gamified content creation Coding Mathematical expression ◦ Graphing calculator Visual expression ◦ Comic creation tools 21
- 22. What app types are available?(cont.) SOCIAL MEDIA Social media platforms ◦ Microblogging sites ◦ Video sharing (predominantly; open-source and proprietary) ◦ Video hosting and annotation ◦ Image sharing (predominantly) Folk tagging online DIGITAL CONTENT HOSTING SERVICES Digital content hosting sites (digital files, digital notebooks, and others) Online learning LMSes Portfolio building Third-party video media hosting (with access to desktop lecture capture software) 22
- 23. What app types are available?(cont.) MICRO-CREDENTIALING, BADGING Micro-credentialing Badging ASSESSMENT SUPPORTS Test-taking and quizzing ◦ Browser lockdown for high-value assessment- taking online ◦ Proctoring Scantron supports AI-informed assessment tool for customized learning 23
- 24. What app types are available?(cont.) REMOTE PROCTORING, SECURE REMOTE TESTING Remote proctoring Secure remote testing / assessment INTERCOMMUNICATIONS, COLLABORATIONS Radio frequency device registrations (such as clicker devices) 24
- 25. What app types are available?(cont.) WORK MANAGEMENT Time management application VIRTUAL MEETINGS Web conferencing, web meetings Recorded virtual meetings 25
- 26. What app types are available?(cont.) RESEARCH TOOLS, DATA TOOLS Collaborative online research (for learners) Course analytics (for instructors) NEWS MEDIA FEEDS, ACADEMIC PUBLISHING CONTENTS Mainstream media news News feeds 26
- 27. What app types are available?(cont.) MISCELLANY Bridging to general contents Single sign-on for online learning and others… 27
- 28. Some developer work A perusal of the activated apps shows some local developer work with some bridges to application programming interfaces (APIs). There are references to legacy accounts. For example, there are the following: “Scantron (dev),” “Scantron (hotfix),” “Scantron v3 (prod test),” etc., with multiple versions of the prior activations in multiple accounts (probably to enable developer access and testing and documentation) Another few were variations on “photo roster test.” There are “made at K-State” apps and tools. 28
- 29. Some local deployed apps University data reporting Submit ASL report All reports & survey results Several user names: Scott F. and Kakali B. Study area Senior Survey Results Alumni Survey Results ---- Survey Results ELP-KSIS Reporting Webassign Specific Twitter feed (#hashtagged) Aviation Reporting FGS MGS TEVAL pilot Hale Library Guide 29
- 30. Some local deployed apps (cont.) K-State Attendance NTSB FAA Student Learning Outcomes Canvas Help Canvas Guides 30
- 31. Other third-party apps (not mentioned in the Canvas apps page) Mathalicious Cerego OneDrive for Business iMathAS Test Google Charts CrowdMark WayPoint Outcomes Kool Learning Weebly Sapling Learning Word Press Canvas Commons 31
- 32. Badge Stack 32
- 33. Khan Academy 33
- 34. MyOpenMath 34
- 35. Programr 35
- 36. TEDEd 36
- 37. Wikipedia 37
- 38. YouTube 38
- 39. For free? Outright purchase? Subscription? The apps themselves are free, and most serve as bridges to online contents, services, and resources. ◦ Many of the apps enable access to wholly free services. ◦ Some apps offer some limited version of a for-pay service. ◦ Some apps bridge to wholly subscription-only services / resources. ◦ Some apps link to full-purchase resources. What are the proportions of the free vs. cost-based online services? ◦ Based on the app descriptions on Canvas…and on the Web, some early calculations were made. 39
- 40. Base rate of free to cost-based apps (from a rough count of all available apps) 40 Free or No Cost 12% Subscription or Purchase Required 88% An Estimate of Free vs. Cost-based Free or No Cost Subscription or Purchase Required
- 41. Base app types and counts 41 Categories of Third-Party Apps Available Proprietary digital contents 72 Open educational resources 23 Customized learning, learning aids, teaching aids for learning 30 Expression, digital content creation, recordings 28 Social media 9 Digital content hosting services 25 Micro-credentialing, badging 6 Assessment supports 32 Proctoring, secure remote testing 11 Intercommunications, collaborations 20 Work management 2 Virtual meetings 11 Research tools, data tools 10 News media feeds, academic publishing contents 4 Miscellany 12
- 42. General base types and percentages (from bottom-up coding) 42
- 43. 43
- 44. 2013 (partial year) 1 1 2 2 1 1 3 3 1 1 1 1 2 2 3 2 2 4 1 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 NumbersofActivations Third-Party Tools Third-Party Tool Activations in K-State Instance of Canvas (2013 Partial Year) 44
- 45. 2013 (in a pareto chart) 45
- 46. 2014 46 1 3 9 2 1 3 1 4 1 1 5 1 16 1 8 1 35 1 1 1 4 14 5 1 13 6 11 2 6 4 2 3 2 2 7 1 2 23 4 2 1 1 1 1 7 22 8 4 1 10 2 2 2 2 7 1 4 1 2 1 12 1 1 1 1 4 1 23 2 5 2 8 1 18 15 5 2 1 40 1 1 1 18 10 43 3 4 16 1 3 2 46 5 7 1 0 5 10 15 20 25 30 35 40 45 50 1Pearson-MyEdLab Bb(Blackboard)Collaborate Canvabadges CanvasInstructorQuickstart… CanvasStudentHelp CK-12 CodeEmbed Connect Dropbox DropboxforSandbox DropThought ElementaryPaper FlatWorldKnowledge Funbrain GitHub GoogleCharts i>Clicker InternetArchive KeithHarrisDropbox K-State LearnItLIve LTIMaps McGrawHillCampus MITOCW myBusinessCourse MyPsychLabAssignment… MyPsychLabStudentGrades NewYorkTimes OpenEdResourcesSearch PearsonMyLab PlaceKitten Programr PROJECT3 PublicResources QuizletFlashCards RollCall ScottFinkeldei SoftChalkCloud SpellingCity TEDEd Trello Twitter Vimeo WebmailRedirect Wiktionary WIRISeditor YouTubeForSchools Zoom NumbersofActivations Third-Party Tools Third-Party Tool Activations in K-State Instance of Canvas (2014)
- 47. 2014 (in a pareto chart) 47
- 49. 2015(in a pareto chart) 49
- 50. 2015 MyOMLab MyOMLab Versions Activated in 2015 50 MyOMLab All Assignments 11% MyOMLab Course Home 11% MyOMLab Homework 11% MyOMLab Multimedia Library 12% MyOMLab Pearson eText 12% MyOMLab Quizzes & Tests 12% MyOMLab Results 12% MyOMLab Study Plan 12% MyOpenMath 7% MyOMLab Versions Activated in 2015 MyOMLab All Assignments MyOMLab Course Home MyOMLab Homework MyOMLab Multimedia Library MyOMLab Pearson eText MyOMLab Quizzes & Tests MyOMLab Results MyOMLab Study Plan MyOpenMath
- 52. 2016 (in a pareto chart) 52
- 54. 2017 (partial year, in a pareto chart) 54
- 55. Named K-State resource activations in 2017 (partial year) 55 K-State Attendance 46% K-State Library Resources for Graduate Students 7% K-State Polytechnic Library 27% Library Resources 20% K-State Resources Activated in K-State Canvas LMS Instance (2017) K-State Attendance K-State Library Resources for Graduate Students K-State Polytechnic Library Library Resources
- 56. A cumulative count of newly activated third-party apps(2013 – 2017) 56 0 200 400 600 800 1000 1200 1400 2013 (partial) 2014 2015 2016 2017 (partial) CumulativeThird-PartyActivations Year-over-Year Counts Summary of Third-Party App Activations in K-State Instance of Canvas: 2013 - 2017
- 57. The year of the… (from the top dozen most frequently activated apps each year) 2013 (partial): social educational video, graphing, authoring tool, digital contents, microblogging, attendance tool, text chat 2014: … plus self-created contents, coding app, open-source learning contents, social imagery, mainline news media, Internet Archives 2015: … plus virtual labs, games for child learning, online collaborative grading platform, Scantron, plagiarism detection 2016: … slideshows from social sharing, quiz app mediation, K-State attendance, virtual notetaking 2017 (7 months in): … exam mediation, proctoring, Google apps, digital-to-paper assignments 57
- 58. Available third-party apps vs. activated third-party apps at K-State 58 Categories of Third-Party Apps Available At K-State Proprietary digital contents 72 25 Open educational resources 23 17 Customized learning, learning aids, teaching aids for learning 30 11 Expression, digital content creation, recordings 28 13 Social media 9 4 Digital content hosting services 25 5 Micro-credentialing, badging 6 2 Assessment supports 32 10 Proctoring, secure remote testing 11 1 Intercommunications, collaborations 20 3 Work management 2 1 Virtual meetings 11 4 Research tools, data tools 10 0 News media feeds, academic publishing contents 4 4 Miscellany 12 3
- 59. How does K-State’s third-party activations compare with the base categories? (basedon unique tool activations not #s of activations, expressed in a spider chart) 59 0 10 20 30 40 50 60 70 80 Proprietary digital contents Open educational resources Customized learning, learning aids, teaching aids for learning Expression, digital content creation, recordings Social media Digital content hosting services Micro-credentialing, badging Assessment supportsProctoring, secure remote testing Intercommunications, collaborations Work management Virtual meetings Research tools, data tools News media feeds, academic publishing contents Miscellany A Comparison between K-State's Third-Party Activations by Category with Available Apps: (2013 - 2017) Available At K-State
- 60. Apps used on Pages 60
- 61. Apps used in Modules 61
- 62. Apps used in Quizzes 62
- 63. Some Caveats 63
- 64. Multiple paths to integrate applications To understand this data better, though, it helps to note that there are other ways to integrate some of these tools within a Canvas course. ◦ Using an inline frame (iframe) to embed the online software as a service (SAAS) or the social media platform (like microblogging sites) or a third-party content provider portal… ◦ Using a uniform resource locator (URL) to link to online resources In other words, activating third-party apps is not an exclusive channel to the functionality. 64
- 65. Activation but not used? Decommissioned? ACTIVATION BUT NOT USED? Also, third-party apps may be activated but not used. ◦ For some of the tools, they have to be subscribed to and integrated with a learning sequence for them to be of actual use. Activation is just a first step. USED INITIALLY BUT LATER DE- COMMISSIONED? This data does not include end dates for when an activated app may have been de- commissioned. This data does include when an app was revised, but those findings were not included here. 65
- 66. Renaming of third-party apps Some users have also renamed apps for customizations, sometimes with some residua of a prior existing name and other times wholly unrecognizably named. Reliance on the names can be somewhat misleading. ◦ Names can “ambiguate” / confuse, and they can disambiguate / clarify. 66
- 67. What This says about What K-Staters (Early Adopters) are Using 67
- 68. Closer analysis of these data…might suggest… Any guesses?! Which departments (within which subject areas / domains) are activating third-party apps… Which news sources are more popular with K-State profs than others… Which publishers and third-party content providers are more popular than others… Which virtual labs are desirable for online / blended / F2F learning… Which user-generated video sources are preferred… Whether open-source digital learning objects are popularly used or not… What authoring tools and software programs are used to generate digital learning contents… Whether (and which) clickers are used in some of the larger classrooms… 68
- 69. Practical Applications of this Third-Party App Activation Data 69
- 70. How can this data be applied? K-Staters can raise awareness of these respective tools with their colleagues to see if there are ways to enhance the teaching and learning. Those who provide support for teaching and learning can offer support in this area. 70
- 71. Adding Value to LMS Data Portal Data 71
- 72. More about third-party apps on Canvas? While this example focused on a few related flat files, and the analytics were simple, it is totally possible to ramp up this work. One simple way to add value here would be not only focus mostly on the most frequently activated 3rd party apps but also look at the “long tail” of single activations. It may be helpful to ask instructors and other users why they activated particular tools and how they use them. It is possible to compare activations of types and specific tools over the years. With data from other comparable universities, it is possible to compare activation patterns. 72
- 73. Broadening askable questions Broaden the askable queries and questions from the data by… ◦ Engaging the front-end part of the LMS and working with faculty ◦ Using the PII and student information systems information (with IRB approval) ◦ Conducting analyses of student learning based on outcomes and machine learning (with decision trees) ◦ Capturing primary keys and foreign keys to be able to ask questions across the data tables (on a relational database) ◦ And others… 73
- 75. And pre-packaged reports data (admin access) Course Storage Grade Export LTI Report Last Enrollment Activity Last User Access MGP Grade Export Outcome Results Provisioning Public Courses Recently Deleted Courses SIS Export (student information systems) Student Competency Students Access Report Students with No Submissions Unpublished Courses Unused Courses User Access Tokens Zero Activity 75
- 76. 76
- 77. Downloadable reports configuration options May query by term May query by all terms May query by other methods of thin-slicing and ordering data 77
- 78. View course analytics (instructor access from course pages) Activity by Date Submissions Grades By Student (Page Views, Participations, Submissions, On Time, Late, Missing, Current Score) 78
- 79. View course statistics (instructor access from Settings) Totals Assignments Students File Storage 79
- 80. Conclusion and contact Dr. Shalin Hai-Jew ◦ iTAC, Kansas State University ◦ 212 Hale / Farrell Library ◦ 785-532-5262 ◦ shalin@k-state.edu Interested in a first skim of K-State LMS Data Portal data? ◦ “Wrangling Big Data in a Small Tech Ecosystem” (Oct. 2016, June 2017) ◦ A downloadable copy is on SlideShare: https://www.slideshare.net/ShalinHaiJew/wrangling-big-data-in-a-small-tech-ecosystem ◦ Rough Categorization of Third-Party Apps from Canvas (download): http://www.k- state.edu/ID/RoughCategorizationofThirdPartyAppsfromCanvas.xlsx or http://www.k- state.edu/ID/RoughCategorizationofThirdPartyAppsfromCanvas.pdf ◦ This is based on a subjective assessment of the often-multifaceted tool’s main functions based on short descriptions in Canvas and website descriptions if the initial description is insufficient. Slideshow on Adobe Spark: https://spark.adobe.com/page/PQRxrknjH2nxp/ The presenter does not have any formal ties with any of the mentioned technologies. 80