From Micro to Macro - Analyzing Activity in the ROLE Sandbox


Published on

Presented at the Third Conference on Learning Analytics and Knowledge (LAK 2013).

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

From Micro to Macro - Analyzing Activity in the ROLE Sandbox

  1. 1. Third Conference on Learning Analytics and Knowledge (LAK 2013) April 8-12, 2013 Leuven, Belgium From Micro to Macro - Analyzing Activity in the ROLE Sandbox Dominik Renzel, Ralf Klamma RWTH Aachen University Advanced Community Information Systems (ACIS) Aachen, Germany {renzel,klamma}@dbis.rwth-aachen.deLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 1 These presentation slides by Dominik Renzel are licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
  2. 2. Advanced Community Information Systems (ACIS) Responsive Web Engineering Community Web Analytics Open Visualization Community and Information Simulation Systems Community Community Support AnalyticsLehrstuhl Informatik 5 Requirements(Information Systems) Prof. Dr. M. Jarke 2 Engineering
  3. 3. Motivation  Increasing adoption of distributed learning services on the Web  Large-scale Web-based learning platforms (MOOC, PLE, hybrids)  Monitoring: unprecedented insights into learner behaviour – Proprietary techniques & data models for specific scenarios (bias/limitation) – New “standards“ for learner behaviour analysis on high semantic level  But: Learner behaviour evolving & inherently hard to model!  Our proposal: step back to existing Web logs  Standardized by-product of regular Web service operation  World-wide adoption  Lowest possible degree of bias  Generic low-level semantics easily liftable to higher-level semantics  Analysis on multiple ecosystem levels from “micro to macro“Lehrstuhl Informatik 5(Information Systems)  Highest level of data interoperability (cross-service analysis) Prof. Dr. M. Jarke 3
  4. 4. Focus & Level of Analysis – Bronfenbrenner‘s EST applied to LA 5 nested & interrelated ecosystems around student:  Microsystem (direct relation to peer/group)  Mesosystem (entirety of microsystems + relations)  Exosystem (networks influencing individual)  Macrosystem (entirety of societal relations incl. norms, rules, etc.)  Chronosystem (temporal dimension of development)  Existing work: often focus on specific system or subsets only…  Any comprehensive LA framework…  should allow analysis & focus on all levels  may not neglect micro data Avoid “washing out“ anomalies!Lehrstuhl Informatik 5  must apply aggregation carefully(Information Systems) Prof. Dr. M. Jarke  should work with historical & real-time data Support targeted & timely intervention! 4
  5. 5. Data Interoperability  Proprietary data “standards“ for monitoring learner activity in LA – High-level semantics for convenient analysis from the beginning  danger of losing anomalies, especially interesting in LA! – Long standardization process with no guarantee for adoption  risks: repeated costly changes to LA installations necessary! – Use of competing/incompatible data formats for different LA frameworks  cross-service analysis complicated/unfeasible!  Why not build on existing Web standards? – Web logs (+ page tagging) provably powerful tools in Web AnalyticsLehrstuhl Informatik 5(Information Systems) – Profit from genericity & widespread use also in LA Prof. Dr. M. Jarke 5 – The Web works incredibly well with these standards! LA might, too…
  6. 6. Opacity & Uncertainty  LA inherently suffers from uncertainty regarding user identity  Web Analytics techniques can improve, but never solve the problem!  Low-level Web log analysis  Not perfect, but always available  Allows sufficiently precise identification of structural/behavioral patternsLehrstuhl Informatik 5(Information Systems)  Allows inferences about opaque parts Prof. Dr. M. Jarke 6
  7. 7. Real-time Web Log Processing PipelineLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 7
  8. 8. Example: Adding widget to Widget-based PLE (ROLE Sandbox) Example Web log entry: <IP> <DATE> POST /space/<space-id>:;tool=<widget-url>Lehrstuhl Informatik 5 Geo Location: Widget Metadata:(Information Systems) Prof. Dr. M. Jarke Longitude/Latitude,City,Region, Country Domains, Activities 8
  9. 9. Analysis Techniques & Benefits for Learning Stakeholders  Analysis techniques possible with simple Web log data – Which learners/learner communities interacted how with which tools, resources in which context where and when? – Single operations  micro-level analysis – Appropriate aggregation  up to macro-level analysis – Relations in log data  actor networks  SNA – Temporal information  time series analysis – Semantic enrichment from external sources  a lot more!!!  Benefits for learning stakeholders – Learning community awareness (e.g. by visualization)Lehrstuhl Informatik 5 – Recommendations (e.g. tools, persons, resources)(Information Systems) Prof. Dr. M. Jarke 9
  10. 10. Conclusions & Future Work LA frameworks for analyzing Web-based learning services to… – Prevent bias/limitation by relying on generic Web standards – Focus & level of analysis to cover micro to macro + chrono – Enable large-scale cross-service analysis by data interoperability – Combine historical & real-time information for intervention planning – Fight uncertainty, esp. regarding learner identity  Argument: Web logs stay superior to “new standards“  Not without limitations, but always available without further instrumentation  Web log processing pipeline incl. data enrichment for LA simple to setup  Multiple analysis techniques applicable and rich information derivable Next steps:  Carry out analysis on ROLE Sandbox dataset recorded from Mar 2012 - now  >8 Mio API requests from > 3000 IPs, >500 Widgets, >1300 PLE Spaces, >3700 shared resourcesLehrstuhl Informatik 5  Apply approach in other scenarios & projects(Information Systems) Prof. Dr. M. Jarke 10
  11. 11. Questions? Contact: renzel@dbis.rwth-aachen.deLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 11