From Micro to Macro - Analyzing Activity in the ROLE Sandbox

Uploaded on

Presented at the Third Conference on Learning Analytics and Knowledge (LAK 2013).

Presented at the Third Conference on Learning Analytics and Knowledge (LAK 2013).

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Third Conference on Learning Analytics and Knowledge (LAK 2013) April 8-12, 2013 Leuven, Belgium From Micro to Macro - Analyzing Activity in the ROLE Sandbox Dominik Renzel, Ralf Klamma RWTH Aachen University Advanced Community Information Systems (ACIS) Aachen, Germany {renzel,klamma}@dbis.rwth-aachen.deLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 1 These presentation slides by Dominik Renzel are licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
  • 2. Advanced Community Information Systems (ACIS) Responsive Web Engineering Community Web Analytics Open Visualization Community and Information Simulation Systems Community Community Support AnalyticsLehrstuhl Informatik 5 Requirements(Information Systems) Prof. Dr. M. Jarke 2 Engineering
  • 3. Motivation  Increasing adoption of distributed learning services on the Web  Large-scale Web-based learning platforms (MOOC, PLE, hybrids)  Monitoring: unprecedented insights into learner behaviour – Proprietary techniques & data models for specific scenarios (bias/limitation) – New “standards“ for learner behaviour analysis on high semantic level  But: Learner behaviour evolving & inherently hard to model!  Our proposal: step back to existing Web logs  Standardized by-product of regular Web service operation  World-wide adoption  Lowest possible degree of bias  Generic low-level semantics easily liftable to higher-level semantics  Analysis on multiple ecosystem levels from “micro to macro“Lehrstuhl Informatik 5(Information Systems)  Highest level of data interoperability (cross-service analysis) Prof. Dr. M. Jarke 3
  • 4. Focus & Level of Analysis – Bronfenbrenner‘s EST applied to LA 5 nested & interrelated ecosystems around student:  Microsystem (direct relation to peer/group)  Mesosystem (entirety of microsystems + relations)  Exosystem (networks influencing individual)  Macrosystem (entirety of societal relations incl. norms, rules, etc.)  Chronosystem (temporal dimension of development)  Existing work: often focus on specific system or subsets only…  Any comprehensive LA framework…  should allow analysis & focus on all levels  may not neglect micro data Avoid “washing out“ anomalies!Lehrstuhl Informatik 5  must apply aggregation carefully(Information Systems) Prof. Dr. M. Jarke  should work with historical & real-time data Support targeted & timely intervention! 4
  • 5. Data Interoperability  Proprietary data “standards“ for monitoring learner activity in LA – High-level semantics for convenient analysis from the beginning  danger of losing anomalies, especially interesting in LA! – Long standardization process with no guarantee for adoption  risks: repeated costly changes to LA installations necessary! – Use of competing/incompatible data formats for different LA frameworks  cross-service analysis complicated/unfeasible!  Why not build on existing Web standards? – Web logs (+ page tagging) provably powerful tools in Web AnalyticsLehrstuhl Informatik 5(Information Systems) – Profit from genericity & widespread use also in LA Prof. Dr. M. Jarke 5 – The Web works incredibly well with these standards! LA might, too…
  • 6. Opacity & Uncertainty  LA inherently suffers from uncertainty regarding user identity  Web Analytics techniques can improve, but never solve the problem!  Low-level Web log analysis  Not perfect, but always available  Allows sufficiently precise identification of structural/behavioral patternsLehrstuhl Informatik 5(Information Systems)  Allows inferences about opaque parts Prof. Dr. M. Jarke 6
  • 7. Real-time Web Log Processing PipelineLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 7
  • 8. Example: Adding widget to Widget-based PLE (ROLE Sandbox) Example Web log entry: <IP> <DATE> POST /space/<space-id>:;tool=<widget-url>Lehrstuhl Informatik 5 Geo Location: Widget Metadata:(Information Systems) Prof. Dr. M. Jarke Longitude/Latitude,City,Region, Country Domains, Activities 8
  • 9. Analysis Techniques & Benefits for Learning Stakeholders  Analysis techniques possible with simple Web log data – Which learners/learner communities interacted how with which tools, resources in which context where and when? – Single operations  micro-level analysis – Appropriate aggregation  up to macro-level analysis – Relations in log data  actor networks  SNA – Temporal information  time series analysis – Semantic enrichment from external sources  a lot more!!!  Benefits for learning stakeholders – Learning community awareness (e.g. by visualization)Lehrstuhl Informatik 5 – Recommendations (e.g. tools, persons, resources)(Information Systems) Prof. Dr. M. Jarke 9
  • 10. Conclusions & Future Work LA frameworks for analyzing Web-based learning services to… – Prevent bias/limitation by relying on generic Web standards – Focus & level of analysis to cover micro to macro + chrono – Enable large-scale cross-service analysis by data interoperability – Combine historical & real-time information for intervention planning – Fight uncertainty, esp. regarding learner identity  Argument: Web logs stay superior to “new standards“  Not without limitations, but always available without further instrumentation  Web log processing pipeline incl. data enrichment for LA simple to setup  Multiple analysis techniques applicable and rich information derivable Next steps:  Carry out analysis on ROLE Sandbox dataset recorded from Mar 2012 - now  >8 Mio API requests from > 3000 IPs, >500 Widgets, >1300 PLE Spaces, >3700 shared resourcesLehrstuhl Informatik 5  Apply approach in other scenarios & projects(Information Systems) Prof. Dr. M. Jarke 10
  • 11. Questions? Contact: renzel@dbis.rwth-aachen.deLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 11