From Micro to Macro - Analyzing Activity in the ROLE Sandbox

  • 214 views
Uploaded on

Presented at the Third Conference on Learning Analytics and Knowledge (LAK 2013).

Presented at the Third Conference on Learning Analytics and Knowledge (LAK 2013).

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
214
On Slideshare
0
From Embeds
0
Number of Embeds
12

Actions

Shares
Downloads
5
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Third Conference on Learning Analytics and Knowledge (LAK 2013) April 8-12, 2013 Leuven, Belgium From Micro to Macro - Analyzing Activity in the ROLE Sandbox Dominik Renzel, Ralf Klamma RWTH Aachen University Advanced Community Information Systems (ACIS) Aachen, Germany {renzel,klamma}@dbis.rwth-aachen.deLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 1 These presentation slides by Dominik Renzel are licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
  • 2. Advanced Community Information Systems (ACIS) Responsive Web Engineering Community Web Analytics Open Visualization Community and Information Simulation Systems Community Community Support AnalyticsLehrstuhl Informatik 5 Requirements(Information Systems) Prof. Dr. M. Jarke 2 Engineering
  • 3. Motivation  Increasing adoption of distributed learning services on the Web  Large-scale Web-based learning platforms (MOOC, PLE, hybrids)  Monitoring: unprecedented insights into learner behaviour – Proprietary techniques & data models for specific scenarios (bias/limitation) – New “standards“ for learner behaviour analysis on high semantic level  But: Learner behaviour evolving & inherently hard to model!  Our proposal: step back to existing Web logs  Standardized by-product of regular Web service operation  World-wide adoption  Lowest possible degree of bias  Generic low-level semantics easily liftable to higher-level semantics  Analysis on multiple ecosystem levels from “micro to macro“Lehrstuhl Informatik 5(Information Systems)  Highest level of data interoperability (cross-service analysis) Prof. Dr. M. Jarke 3
  • 4. Focus & Level of Analysis – Bronfenbrenner‘s EST applied to LA 5 nested & interrelated ecosystems around student:  Microsystem (direct relation to peer/group)  Mesosystem (entirety of microsystems + relations)  Exosystem (networks influencing individual)  Macrosystem (entirety of societal relations incl. norms, rules, etc.)  Chronosystem (temporal dimension of development)  Existing work: often focus on specific system or subsets only…  Any comprehensive LA framework…  should allow analysis & focus on all levels  may not neglect micro data Avoid “washing out“ anomalies!Lehrstuhl Informatik 5  must apply aggregation carefully(Information Systems) Prof. Dr. M. Jarke  should work with historical & real-time data Support targeted & timely intervention! 4
  • 5. Data Interoperability  Proprietary data “standards“ for monitoring learner activity in LA – High-level semantics for convenient analysis from the beginning  danger of losing anomalies, especially interesting in LA! – Long standardization process with no guarantee for adoption  risks: repeated costly changes to LA installations necessary! – Use of competing/incompatible data formats for different LA frameworks  cross-service analysis complicated/unfeasible!  Why not build on existing Web standards? – Web logs (+ page tagging) provably powerful tools in Web AnalyticsLehrstuhl Informatik 5(Information Systems) – Profit from genericity & widespread use also in LA Prof. Dr. M. Jarke 5 – The Web works incredibly well with these standards! LA might, too…
  • 6. Opacity & Uncertainty  LA inherently suffers from uncertainty regarding user identity  Web Analytics techniques can improve, but never solve the problem!  Low-level Web log analysis  Not perfect, but always available  Allows sufficiently precise identification of structural/behavioral patternsLehrstuhl Informatik 5(Information Systems)  Allows inferences about opaque parts Prof. Dr. M. Jarke 6
  • 7. Real-time Web Log Processing PipelineLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 7
  • 8. Example: Adding widget to Widget-based PLE (ROLE Sandbox) Example Web log entry: <IP> <DATE> POST /space/<space-id>:;tool=<widget-url>Lehrstuhl Informatik 5 Geo Location: Widget Metadata:(Information Systems) Prof. Dr. M. Jarke Longitude/Latitude,City,Region, Country Domains, Activities 8
  • 9. Analysis Techniques & Benefits for Learning Stakeholders  Analysis techniques possible with simple Web log data – Which learners/learner communities interacted how with which tools, resources in which context where and when? – Single operations  micro-level analysis – Appropriate aggregation  up to macro-level analysis – Relations in log data  actor networks  SNA – Temporal information  time series analysis – Semantic enrichment from external sources  a lot more!!!  Benefits for learning stakeholders – Learning community awareness (e.g. by visualization)Lehrstuhl Informatik 5 – Recommendations (e.g. tools, persons, resources)(Information Systems) Prof. Dr. M. Jarke 9
  • 10. Conclusions & Future Work LA frameworks for analyzing Web-based learning services to… – Prevent bias/limitation by relying on generic Web standards – Focus & level of analysis to cover micro to macro + chrono – Enable large-scale cross-service analysis by data interoperability – Combine historical & real-time information for intervention planning – Fight uncertainty, esp. regarding learner identity  Argument: Web logs stay superior to “new standards“  Not without limitations, but always available without further instrumentation  Web log processing pipeline incl. data enrichment for LA simple to setup  Multiple analysis techniques applicable and rich information derivable Next steps:  Carry out analysis on ROLE Sandbox dataset recorded from Mar 2012 - now  >8 Mio API requests from > 3000 IPs, >500 Widgets, >1300 PLE Spaces, >3700 shared resourcesLehrstuhl Informatik 5  Apply approach in other scenarios & projects(Information Systems) Prof. Dr. M. Jarke 10
  • 11. Questions? Contact: renzel@dbis.rwth-aachen.deLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke 11