Learning from the Learning Management System
•Download as PPTX, PDF•
2 likes•255 views
Presented at the 2017 Faculty Summer Institute A Learning Management System (LMS) is a mission-critical resource for institutions of Higher Education, particularly for institutional goals around academic fulfillment, administration, and most importantly, student success. But how are faculty and students actually leveraging the LMS to achieve these academic goals? What motivates faculty to use available tools, and how does this impact student learning and achievement? In this session, you will learn about a research project conducted by Northern Illinois University about faculty motivation behind LMS usage and adoption. The audience will also be exposed to other research studies conducted by an industry-leader partner about the impact popular LMS tools have on student success. Presenter(s) and audience will discuss these research findings and reflect on the use of an LMS, regardless of which solution your institution may use, and how it can improve the teaching and learning environment for everyone.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Learning from the Learning Management System
Similar to Learning from the Learning Management System (20)
More from Stephanie Richter
More from Stephanie Richter (16)
Recently uploaded
Recently uploaded (20)
Learning from the Learning Management System
- 1. FSI 2017 Learning from the Learning Management System May 23, 2017 Stephanie Richter, Northern Illinois University Edgar Gonzalez, Blackboard Inc. Slides available at facdev.niu.edu/fsi2017research
- 2. Presenters Stephanie Richter Assistant Director srichter@niu.edu @slrichter Edgar Gonzalez Customer Success Advocate edgar.gonzalez@blackboard.com @EdgarBbCSA
- 4. What Does Research Tell Us?
- 5. Patterns in Course Design How instructors ACTUALLY use the LMS Source: Blackboard Blog - http://blog.blackboard.com/patterns-in-course-design-how-instructors-actually-use-the-lms/
- 6. Supplemental Use of Blackboard Learn Source: Blackboard Blog - http://blog.blackboard.com/patterns-in-course-design-how-instructors-actually-use-the-lms/
- 7. Complementary Use of Blackboard Learn Source: Blackboard Blog - http://blog.blackboard.com/patterns-in-course-design-how-instructors-actually-use-the-lms/
- 8. Social Use of Blackboard Learn Source: Blackboard Blog - http://blog.blackboard.com/patterns-in-course-design-how-instructors-actually-use-the-lms/
- 9. Evaluative use of Blackboard Learn Source: Blackboard Blog - http://blog.blackboard.com/patterns-in-course-design-how-instructors-actually-use-the-lms/
- 10. Holistic Use of Blackboard Learn Source: Blackboard Blog - http://blog.blackboard.com/patterns-in-course-design-how-instructors-actually-use-the-lms/
- 11. Discussion – How do you use the LMS?
- 12. Discussion – Which is the best archetype?
- 13. Context at NIU
- 14. Research Questions Phase One • What are the most used tools in the LMS? • How has tool usage changed over time? • What factors impact tool usage? Phase Two • What factors impact faculty members’ decisions to adopt, continue to use, or abandon the use of the LMS? (Al-Busaidi & Al-Shihi, 2010)
- 15. Methods Phase One • Sys Admin queries system logs • Refined searches over time based on observation • Dummy-coded tool usage as present or absent (whether it was used vs. how much it was used) • Compared with course modality (e.g., online vs. face-to-face) Phase Two • Distributed a survey to all NIU faculty • Asked about reasons for using Bb, Tools used, and satisfaction/ease of use (TAM constructs) • Coded qualitative data with multiple reviewers • 25% response rate
- 16. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Results – Usage announcements items grades folders files assignments web links plagiarism detection tests discussions
- 17. Results – Usage • Overall, usage has been increasing for all tools • Top tools (at or over 50%): • Announcements • Items • Grades • Folders • Assignments • Files
- 18. Results – Usage • Sharper increases starting in Fall 2015 • Why? • Tightening budgets? • Student printing initiative?
- 19. Results – Usage • Peaks each summer • Why? • More online courses?
- 20. Tool Use for All Courses 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% announcements items grades folders files assignments web links plagiarism detection tests discussions
- 21. Tool Use for In-Person Courses 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% announcements items grades folders files assignments web links plagiarism detection tests discussions
- 22. Tool Use for Online Courses 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% announcements items grades folders files assignments web links plagiarism detection tests discussions
- 23. Courses by Modality 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% PercentofCourses Face-to-Face Courses Online Courses
- 24. Did not Adopt – 5%, n=11 Perceived lack of need I only teach a few seminars Do not have the need Use of Alternative Tool I have my own website on which I post materials Perceived Lack of Training I’ve never had in-depth instruction on how to use it Results – Motivation Adopted – 95%, n=398 Perceived as Required It was mandatory for the course I am teaching Perceived Value For students to keep track of their grades Perceived Ease of Use I was doing everything through a personal server. This was much easier
- 25. Abandoned, n=7 Perceived lack of need I am not teaching right now Perceived Ease of Use Blackboard is way too complicated Hard to navigate and to make it better Results – Motivation Continued to Use, n=377 Perceived Value Effective way to communicate with students Perceived as Required It is mandatory Perceived Ease of Use Easy, efficient, no paper, keep it organized
- 26. Discussion / Interpretation • Overall tool usage increasing • LMS usage seems to be different based on modality • LMS is seen to have value for both faculty and students • LMS is not required by the university, but is perceived as required by many faculty • Despite ubiquity, LMS can be overwhelming – support/training is important still • Further research is needed – partner with us!
- 29. How successful students use LMS tools • My Grades • Course Content • Assessments • Assignments • Discussion
- 30. Discussion – How does this match your perception or experience?
- 31. Contact Information Stephanie Richter Assistant Director srichter@niu.edu @slrichter Edgar Gonzalez Customer Success Advocate Edgar.Gonzalez@blackboard.com @EdgarBbCSA Slides available at facdev.niu.edu/fsi2017research
Editor's Notes
- - LMS has grown from a small innovation into a standard (and critical) tool at most universities. We wanted to learn more about how faculty use it, and why they adopt it vs. continue using it (or stop using it). Once considered innovative and used by few, the learning management system (LMS) has become a critical tool standard at most institutions of higher education. In this session, we will share findings from a study on faculty use and motivation for adopting the LMS, with implications for promoting other technological innovations.
- The data sample for this study included 70,000 courses from 927 institutions, with 3,374,462 unique learners using Blackboard Learn during Spring 2016 in North America. All of this data was anonymized at the individual and institutional levels; only aggregate data was used for analysis. We conducted a cluster analysis and five course patterns emerged. Supplemental – high in content but with very little student interaction Complementary – used primarily for one-way teacher-student communication Social – high peer-to peer interaction through discussion boards Evaluative – heavy use of assessments to facilitate content mastery Holistic – high LMS activity with a balances use of assessments, content, and discussion
- Supplemental – high in content but with very little student interaction
- Complementary – used primarily for one-way teacher-student communication
- Social – high peer-to peer interaction through discussion boards
- Evaluative – heavy use of assessments to facilitate content mastery
- Holistic – high LMS activity with a balances use of assessments, content, and discussion
- At NIU, we have used Bb for over 15 years. It started with a small pilot, but now is used by about 90% of faculty, 95% of students, for over 60% of all courses. We’ve looked at these usage statistics since 2002, and seen growth of 600%! This is interesting, but we wanted to see more nuance and detail – what are they using, and are there patterns of use? Adoption has always been voluntary, so we were curious about motivation The learning management system (LMS) has become a critical tool for nearly all institutions of higher education, and a driving force in online learning. According to a 2015 report by the Educause Center for Analysis and Research (ECAR), 99% of higher education institutions have an LMS in place. The same report found that 85% of faculty and 83% of students use the LMS, and 74% of faculty say the LMS is a very useful tool for enhancing learning. This was not always the case, however. There was a time in the not-so-distant past when using an LMS was considered highly innovative. Understanding the growth and adoption of the LMS is a stepping stone to understanding how faculty may choose to adopt other technological and pedagogical innovations. This study was conducted at a large, research-intensive public university in the Midwest. The university first adopted a learning management system in 2001, and has used the same system for 15 years. Adoption began as a small pilot with a few highly motivated faculty and has grown to nearly universal use. In 2015-2016, the LMS was used by 92% of all instructional staff (including faculty, instructors, and teaching assistants), and 96% of all students. This is clearly substantial growth, but LMS adoption at this institution has been entirely voluntary for faculty. There is no central university expectation or requirement that faculty use the LMS. This makes the university an intriguing site for research into faculty motivation and acceptance of technological innovation.
- Two phases in our study. Phase One looked at actual use of the system, and we wanted to know more about what the top tools were, and how that had changed over time Phase Two looked at the motivations for use. After so much time, why did faculty start using the LMS? Why have they continued to use it (and is that different from why they started? And, if they discontinued using it, why did they do so? Have they used something else? This study was conducted in two phases. For phase one, the research question was "What are the most used tools in the learning management system, and how has this changed over time?". With this question, the goal was to understand what LMS adoption looked like within the institution, and how it had changed. This has broad implications for assessing the depth of adoption, evaluating the ongoing health of the LMS, and coordinating appropriate levels of support for faculty and students. The second phase focused on faculty motivation for use of the LMS. The primary research question was "What motivating factors impact faculty members' decisions to adopt, continue to use, or abandon the use of a learning management system?".
- Talking points - See slide SQL queries against Bb database (similar to what you can do with OpenDB) Took a great deal of time to refine queries because we were using guess and check (based on an old BbWorld presentation to begin with) Looked at overall usage, not quantity of usage (did you use the Grade Center vs. how many grades did you enter?) Beginning to explore differences in course modality by comparing LMS records with data from SIS Phase two – survey Survey questions based on TAM instrument Survey externally validated 25% return rate Multiple coders for validation on qualitative results For phase one, the system administrator gathered usage data from the LMS for each semester from spring 2011 through summer 2016 using SQL queries against the open database, which was possible because the university hosts the LMS locally. The queries generated data files in a comma separated value (csv) format that listed all courses in a given term that used the LMS and how many instances of specific tools were used or created within each course. For example, a given course may have used 5 groups, 48 content items, 14 folders, 26 announcements, 6 assignments, and posted 136 grades. The SQL queries will be published publicly for other institutions to use. We reviewed the data extensively by validating a sample of courses, verifying that the values in the reports matched the courses in the system. We recorded the results of this in a data dictionary that will also be published in conjunction with the queries. For phase two, we surveyed faculty, instructors, and teaching assistants at the institution to gather information on their motivations for using the learning management system. The survey included Likert-type items as well as open-ended questions, and was based on instruments used in prior studies about LMS adoption. The online survey was externally validated prior to use and distributed to all instructors of record at the institution via email.
- The data on LMS usage has shown what the most commonly used tools are. For example, in the Fall 2015 semester, 83% of courses used the announcement tool, 77% had grades posted, 29% had assignments, and 22% had discussion boards with posts. In addition, by using data from several years, it is possible to identify trends in usage. From Fall 2011 to Fall 2015, overall usage of the LMS increased from 78% of faculty, instructors, and teaching assistants to 92%. The majority of individual tool usage increased as well over the same time period. One of the most interesting findings is that the percentage of courses using tools like discussion boards, assignments, and tests increased markedly every summer.
- Because the learning management system is critical to online learning, and in light of recent discussions of innovations such as next generation learning systems, we must understand how and why it is used by faculty. This knowledge will help to guide development of new innovations that are more likely to solve real teaching and learning problems, and ultimately to be adopted and used by faculty. This study is only a beginning at combining objective analysis of usage with insight into faculty motivation for adopting, using, or abandoning the LMS. Further analysis of existing usage data can provide insights into tools that are commonly used in the same course to develop best practices or models of LMS use. This analysis can also examine disciplines and subject fields for commonalities in tools, to provide more targeted support where it is most needed. Both the usage and motivation studies should be extended to other higher education institutions, particularly those with unique characteristics such as private, small, or institutions serving primarily under-served populations. In this presentation, participants will learn about the findings and conclusions we have been able to draw from these first two phases of research, and provide suggestions and inquiries for future research and analysis.
- The results of the survey on faculty motivation for using the LMS are currently being analyzed, but will be available to share at the conference.
- The results of the survey on faculty motivation for using the LMS are currently being analyzed, but will be available to share at the conference.
- We sampled 70,000 courses from 927 institutions, with 3,374,462 unique learners. After filtering the resultant data set included 601,544 learners (16.25%) in 18,810 (26.87%) courses.