Suthers, D. D., & Rosen, D. (2011). A unified framework for multi-level analysis of distributed learning Proceedings of the First International Conference on Learning Analytics & Knowledge, Banff, Alberta, February 27-March 1, 2011.
The Structure and Logic of the Learning Analytics Field George Siemens, PhD January 9, 2013 Teachers College Columbia University New York
- Publicly funded research university- 38,000 students- One of four research universities in Alberta- Only US accredited Canadian university (MSCHE)- Bachelor, masters, doctoral degrees- Fully online
My Work1. Social network analysis and concept/knowledge development (NSERC/D2L, with Gasevic, Dawson, Haythornthwaite)1. Knowledge mapping and competencies1. Structure/Logic of LA: epistemology/assumptions/trends/deployment/im pact1. Developing post-baccalaureate (Masters, 2014, fingers crossed) on data analytics
Learning analytics is themeasurement, collection, analysis andreporting of data about learners and theircontexts, for purposes of understanding andoptimizing learning and the environments inwhich it occurs.
Scope of focus is important factorEDM: specific variables and factorsLA: systemic and in-context factors(but this distinction is not hard; overlap andblurring is occurring)
Domains of LA activity & impactLearning & knowledge growthNetwork analysis – social and knowledgeContent analysisPersonalization and adaptationPrediction & InterventionSystemic impact 9
Inter-disciplinary emphasisBringing technical, pedagogical, and socialdomains into dialogue with each other.
Analytics around social interactionsAnalytics around learning contentAnalytics in different spaces (digital/F2F)Analytics on interaction with the learning system(university/k-12)Analytics on intervention and adaptationAssessment of analytics
What is happening globally in LAAfrica: end userAustralia: end user, gov’t, networkChina: end userEurope/UK: end user, gov’t, networkHong Kong: end user, gov’tIndia: end user, networkLatin America: end userUSA/Canada: end user, gov’t, network
Logic of analyticsSensemaking and wayfinding Comp683, Stat110 (Blitzstein)
What is research/science?Essentially, discovery (identification) ofconnections
Validity of connection interrogationtechniques (Guba & Lincoln, 1994, 2005)
The research model in learning analytics – Holistic, not reductionist – Focused on systemic level – Impact and in-context evaluation – Social, technical, pedagogical
An EDM paper (best paper 2012 conf)Terms: Student model, CTA, cognitive tutor, instructional techniquesGoal: automated technique for the discovery of better student modelsusing input from previously generated models.Data: DataShop, 300 datasets, 70 million student actionsMethodology: Knowledge components mapped to instructional tasks,LFA algorithm to find better models
A LAK paper (2012 conf)Terms: learning sciences, engagement, design, learning ‘power’, competenciesGoals: “a learning analytics infrastructure for gathering data at scale,managing stakeholder permissions, the range of analytics that itsupports from real time summaries to exploratory research, and aparticular visual analytic which has been shown to havedemonstrable impact on learners.Data: survey data, Learning Warehouse, >40,000.Not auto-tracked, instead: learner self-disclosingMethodology: ELLI visualization and validation using Learning Warehouse(ELLI intends to “provide educators with a practical toolto enable rapid assessment and intervention of a complex quality,to stimulate change in learners”)
A LAK paper (2012 conf)Terms: networks, community structure, SNA, distributed learningGoals: Resolving: “multiple means of participation, each with their ownInteractional and social affordances…Events in these media may be logged in differentformats and databases, disassociating actions that for participants were part of a single unified activity.”Data: Participant interaction in TappedIn, 150k membersMethodology: “developed an abstract transcript representationcalled the Entity-Event-Contingency (EEC) graph that provides aunified analytic artifact ; and an analytic hierarchy derivedfrom the EEC that supports multiple levels of analysis”
ChallengesScope of data captureIdentifying critical elements that matterGenerating multi-dimensional models
“Learning and knowledge creation is oftendistributed across multiple media and sitesin networked environments. Traces of suchactivity may be fragmented across multiplelogs and may not match analytic needs. As aresult, the coherence of distributedinteraction and emergent phenomena areanalytically cloaked” Suthers, Rosen, 2011
Strategy Planning & Metrics & Capacity Systemic resources tools development change allocationData inventory Data/Analytics Analytics goals Faculty/Staff Course team & target areas PD models?Role of data Data sources Educator- Student access Self-directedProblem or controlled learningopportunity toolsStakeholders Budget Enterprise Learning Automated(IR, Academic, tools design discoveryAdmin) Priorities Iterative Process StudentAccess development mapping and models of algorithms evaluationGovernance Stages of Visualization Intelligent deployment curriculumCompliance Policy development