Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Open Learning Analytics panel at Open Education Conference 2014

1,759 views

Published on

The past five years have seen a dramatic growth in interest in the emerging field of Learning Analytics (LA), and particularly in the potential the field holds to address major challenges facing education. However, much of the work in the learning analytics landscape today is closed in nature, small in scale, tool- or software-centric, and relatively disconnected from other LA initiatives. This lack of collaboration, openness, and system integration often leads to fragmentation where learning data cannot be aggregated across different sources, institutions only have the option to implement "closed" systems, and cross disciplinary research opportunities are limited. Beyond the immediate concerns this fragmentation creates for educators and learners, a closed approach dramatically limits our ability to build upon successes, learn from failures and move beyond the "pockets of excellence (and failures)? approach that typifies much of the educational technology landscape.

The potential benefits of openness as a core value within the learning analytics community are numerous. Learning initiatives could be informed by large scale research projects. Open-source software, such as dashboards and analytics engines, could be available free of licensing costs and easily enhanced by others, and OERs could become more personalized to match learners' needs. Open data sets and reproducible papers could rapidly spread understanding of analytical approaches, enabling secondary analysis and comparison across research projects. To realize this future, leaders within the learning analytics, open technologies (software, standards, etc.), open research (open data, open predictive models, etc.) and open learning (OER, MOOCs, etc.) fields have established a "network of practice" aimed at connecting subject matter experts, projects, organizations and companies working in these domains. As an initial organizing event, these leaders organized an Open Learning Analytics (OLA) Summit directly following the 2014 Learning Analytics and Knowledge (LAK) conference this past March as means to further the goal of establishing "openness' as a core value of the larger learning analytics movement. Additional details on the Summit and those involved can be found at: http://www.prweb.com/releases/2014/04/prweb11754343.htm.

This panel session will bring together several thought leaders from the Open Learning Analytics community who participated in the Summit to facilitate an interactive dialog with attendees on the intersection of learning analytics and open learning, open technologies, open data, and open research. The presenters represent a broad range of experience with institutional analytics projects, an open source development consortium, the sharing of open learner data, and academic research on open learning environments.

Published in: Data & Analytics
  • Be the first to comment

Open Learning Analytics panel at Open Education Conference 2014

  1. 1. Open Learning Analytics Panel JOSH BARON ST IAN HÅKLEV NORMAN BIER HASH TAG #OPENLA
  2. 2. Panel Session Overview Session Goal: Stimulate discussion around the importance of open learning analytics to the future of the larger open education movement Longer-Term Objective: Form connections between the OpenEd and Open Learning Analytics networks Session Format:  Setting the Context – Open Learning Analytics  Examples from the Real World  Discussion/Q&A
  3. 3. What is Learning Analytics? Academic Analytics Learning Analytics A process for providing higher education institutions with the data necessary to support operational and financial decision making* The use of analytic techniques to help target instructional, curricular, and resources to support the achievement of specific learning goals* Focused on the business of the institution Focused on the student and their learning behaviors Management/executives are the primary audience Learners and instructors are the primary audience * - Source: Analytics in Higher Education: Establishing a Common Language
  4. 4. 2014 Open Learning Analytics Summit Society for Learning Analytics (SoLAR) began exploring openness in learning analytics in 2011 International OLA Summit held in March 2014 Participants identified OLA “knowledge domains” as means to organize future work
  5. 5. OLA Knowledge Domains Open Data and Models  Releasing data sets and models under open licenses Open Research  Publishing research in open-access journals Open-Source Software/Platforms  Open software, standards and APIs Open Strategy and Policy  Open documents on strategy and policy Open Learning Designs  Combine OER & LA to create new models of learning
  6. 6. Openness = Science NORMAN BIER DIRECTOR, OPEN LEARNING INI T IAT IVE CARNEGIE MEL LON UNIVERSI TY
  7. 7. The changing value of content Changing focus in OER community Commoditization of content (Wiley: ‘content is infrastructure’) Instrumenting content is difficult and expensive Well-instrumented content and the tools to analyze student interactions with that content will continue to increase in importance
  8. 8. Problems with Black-Box Systems
  9. 9. Examples Challenges Opportunities CC-OLI Research MOOC Research UC Davis Common measures of outcomes and achievement Simon DataLab Design Analytics Stanford Outcomes Analytics Service Swappable Models Learner Centered
  10. 10. MOOC RESEARCH AND REPRODUCIBLE SCIENCE ST IAN HÅKLEV INST I TUT IONAL RESEARCHER, OPEN UTORONTO UNIVERSI TY OF TORONTO
  11. 11. Supporting three MOOC research projects (MRI)
  12. 12. Tools to collaborate and document
  13. 13. Connecting database with other data
  14. 14. Clicklog: big data, making it queryable, increasing levels of abstraction
  15. 15. 20
  16. 16. Open Data Models & OS Learning Analytics Platform JOSH BARON SENIOR ACADEMIC TECHNOLOGY OF F ICER MARIST COL LEGE
  17. 17. OAAI: Overview and Impact EDUCAUSE Next Generation Learning Challenges (NGLC) Funded by Bill and Melinda Gates Foundations $250,000 over a 15 month period Goal: Leverage Big Data concepts to create an open-source academic early alert system and research “scaling factors”
  18. 18. Student Aptitude Data (SATs, current GPA, etc.) Student Demographic Data (Age, gender, etc.) Sakai Event Log Data Sakai Gradebook Data Step #1: Developed model using historical data Predictive Model Scoring Identifies students “at risk” to not complete course LMS Data SIS Data OAAI Early Alert System Overview Intervention Deployed “Awareness” or Online Academic Support Environment (OASE) “Creating an Open Academic Early Alert System” Model Developed Using Historical Data Academic Alert Report (AAR)
  19. 19. Research Design Deployed OAAI system to 2200 students across four institutions  Two Community Colleges  Two Historically Black Colleges and Universities Design > One instructor teaching 3 sections  One section was control, other 2 were treatment groups Each instructor received an AAR three times during the semester:  Intervals were 25%, 50% and 75% into the semester
  20. 20. Intervention Research Findings Final Course Grades Analysis showed a statistically significant positive impact on final course grades  No difference between treatment groups Saw larger impact in spring then fall Similar trend amount low income students Mean Final Grade for "at Risk" Students 100 90 80 70 60 50 Awareness OASE Control Final Grade (%)
  21. 21. Intervention Research Findings Content Mastery Student in intervention groups were statistically more likely to “master the content” then those in controls.  Content Mastery = Grade of C or better Similar for low income students. Content Mastery for "at Risk" Students 1000 800 600 400 200 0 Yes No Yes No Control Intervention Frequency
  22. 22. More Research Findings… JAYAPRAKASH, S. M. , MOODY, E. W. , LAURÍA, E. J. , REGAN, J. R. , & BARON, J. D. (2014) . EARLY ALERT OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN SOURCE ANALYTICS INITIATIVE. JOURNAL OF LEARNING ANALYTICS, 1(1) , 6 -47.
  23. 23. Strategic Vision: Open Learning Analytics Platform Collection – Standards-based data capture from any potential source using Experience API and/or IMS Caliper/Senor API Storage – Single repository for all learning-related data using Learning Record Store (LRS) standard. Analysis – Flexible Learning Analytics Processor (LAP) that can handle data mining, data processing (ETL), predictive model scoring and reporting. Communication – Dashboard technology for displaying LAP output. Action – LAP output can be fed into other systems to trigger alerts, etc.
  24. 24. Access to Predictive Model and related OS Software… OAAI PREDICT IVE MODEL DOWNLOAD HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/ 8AWCB APEREO LEARNING ANALYT ICS PROCESSOR DOWNLOAD HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/KWCVBQ
  25. 25. Discussion and Q&A
  26. 26. Discussion Questions Do you feel LA will be important to OER and Open Education in the future? How important? Where do you see connections between the OLA network and Open Education? How might we best facilitate making connections across different “networks”? [insert more questions]
  27. 27. Additional Resources European OLA Summit – December 1st (LACE)  http://www.laceproject.eu/ The Asilomar Convention for Learning Research in Higher Education  http://asilomar-highered.info Apereo Learning Analytics Initiative  https://confluence.sakaiproject.org/x/rIB_BQ Society for Learning Analytics and Research  http://solaresearch.org
  28. 28. What is Student Success? X Z Engagement Learning Y Credit: mike.sharkey@phoenix.com

×