SlideShare a Scribd company logo
1 of 33
Download to read offline
3
Enhancing the Learning Experience by
Leveraging Learner Analytics
A guide to how collecting & assessing learning outcomes
helps you deliver a differentiated elearning experience.
Agenda
1. EdTech Solutions that Achieve Learning Objectives
○ A review of the must-have solution components
○ All-in-one vs. modular solutions
○ How to deliver & track learning events: xAPI/TinCan vs. IMS Caliper
2. Translating Learning Events into Learning Analytics
○ What analytics work best to determine course effectiveness?
○ How to interpret learning data: learner performance vs. course performance
○ How instructors can leverage data for course improvements
○ What’s next: Adaptive learning
1. EdTech Solutions that
Achieve Learning Objectives
WebReader, iOS, Android AppsLMS
Learning Management System
User
Mgmt
Authoring Solution / LCMS
Project ManagementImport
(XML, ePub, PDF)
Authoring
styling, content, activities, media, ...User
Mgmt
Content Distribution
API
LTI
Analytics
Teaching Content
Content Distribution
Content Creation
Analytics
Read, Interact
& Learn
Learning Library
Content Producers / Authors Instructors Learners
Analytics
User
Mgmt
build, manage & distribute
teaching & learning content
manage, assemble & distribute
learning content
learn on learning content
Learning Event Server
1. EdTech Solutions that Achieve Learning Objectives
Global Solution Architecture
User Management
● User invitation,
registration, login
● SSO, access control,
module permissions
● Project teams & project
permissions
➢ Users: Education Publishers, Authors, Business Learning Centers, Communication Dpt, ...
➢ Solution Role: Build, Manage & Distribute teaching, learning or communication content
Project Management
● New project creation
● Project import via various
formats (ePub, PDF, XML, …)
● Project list, sort, search, delete
● Project information & metadata
● Project settings & accessibility
Analytics
Insights for content creators based on end-user interactions to improve content
Project Authoring
● Project theme/layout: colors,
fonts, styles, layout CSS
● Project enrichment: media,
activities, widgets, …
● Print/digital or learner/instructor
specific content
Project distribution
● Project versioning
● Project team-review
● Distribution to learners:
LMS, LTI direct linking,
custom API, direct to
readers
1. EdTech Solutions that Achieve Learning Objectives
The Authoring Solution
User Management
● User invitation,
registration, login, SSO
● Instructor management
● Classrooms / Groups
Content Library
● Content preview, bookmarks
● Content assembling from
learning objects
● External content import
Analytics
Insights for Instructors on content end-user interactions to improve learning
Content Authoring
● Simplified authoring
● Insertion of media, activities
Content Distribution
● Distribution to group of
learners / users
● Assignation, expiration
date, specific settings
➢ Users: Teachers, Instructors, Communication Managers
➢ Solution Role: Manage & Distribute learning or communication content
1. EdTech Solutions that Achieve Learning Objectives
The Learning Management Solution
User Management
● User registration,
login, SSO
● User profile, settings
Learning Library
● List, sort content, detailed info
● Content download for offline reading
● Search content
● Additional content purchase
● Social
Analytics
Summary of interactions with content, progress reports
Content Readers
● Content display with layout
● Media rendering
● Activities & assessments
● Search in content
● Accessibility
● User annotations
● Interaction events
➢ Users: Students, Learners, Employees, Customers, ...
➢ Solution Role: Discover & Interact with the distributed content
Android Apps
Chromebooks
MacOS
iOS apps
Windows
1. EdTech Solutions that Achieve Learning Objectives
The Readers for End-Users
To build & deliver digital content, we see companies follow a similar development path:
To integrate with the legacy ecosystem already in place, specialized solutions need to:
● Be modular with open API available
● Support industry standards: IMS LTI, Caliper in Education/Publishing, Scorm / Tincan for Business Learning
● Integrate with market-leading solutions such as Moodle or Blackboard LMS
Start with an all-in-one
solution, generally a LMS
that can author
& deliver to the web.
Addition & integration of more
specialized solutions.
1
Expansion of digital distribution
=> Richer needs,
workflow industrialization
2
Authoring Solution
Mobile & Web
Distribution Solutions
Analytics Solution?
1. EdTech Solutions that Achieve Learning Objectives
Integrated or Modular Solution?
Analytics are about building a quantitative analysis of the end-users’ behavior with your content.
● Recipients: end-user, instructors, content producers, …
● Types of analytics: aggregate figures, ratio, charts & graphs (dataviz)
Track meaningful end-user
interactions
with content (= events)
Flow events into different user
interfaces
Conclude
most meaningful analytics for
recipient based on events
Build interfaces that provide
easy access to desired
info / analytics
1. EdTech Solutions that Achieve Learning Objectives
Getting Started with Analytics
xAPI (Project TinCan)
Initiative led by the &
Open-source, Apache v2 license
Successor of SCORM since 2013
API Goals:
● Allow applications to share data about human
performance on instructional content & context information
● Store information in a Learning Record Store (LRS)
➔ JSON format over HTTP + OAuth
➔ Service and device agnostic = versatile, transferable
from LMS to mobile devices, wearables, search
engine, gamification, blog, webinar, CRM, ...
24 conformant Learning Record Stores, 39 official adopters
➔ Versatile, but mostly adopted in business learning
Best practices for collecting end-user interaction events led to 2 different analytics standards:
Caliper v1.1
Initiative led by the , started 2015
Closed consortium which controls the Caliper IP
Goals:
● Approach to describing, collecting & exchanging learning
activity data
● Way of measuring learning activities & effectiveness
● Enables new services to measure, report & visualize
➔ JSON format over HTTPS/TLS with API key
➔ Focus around the LMS only => adopted or being
adopted by the main LMS vendors
Only 11 solutions certified by IMS, ex: Blackboard, Canvas,
VitalSource + others having “adopted” ex: Moodle, Elsevier
➔ Focused on learning, mostly in e-education
1. EdTech Solutions that Achieve Learning Objectives
Analytics Standards
Caliper v1.1xAPI
1. EdTech Solutions that Achieve Learning Objectives
Analytics Standards Visualized
xAPI (Project TinCan)
The Activity Provider delivers the learning experiences and
communicates the learning info to the Learning Record Store.
Learning experiences are described through statements:
● Actor: individual/group performing an action within an Activity.
● Verb: action performed during the learning experience.
The xAPI only specifies how to create meaningful Verbs.
● Object: something that the Actor interacted with,
ex: a book, an elearning course, a unit of instruction, an activity
● Result (optional): measured outcome of the statement
ex: score, success, completion, response, activity duration
● Context (optional): additional information ex: the instructor,
inclusion in a broader activity or team activity
● Attachment (optional): evidence of a learning experience
ex: a record of communication, an essay, a video, or a
certificate
● TimeStamp, Authority: device/service delivering the statement
Caliper v1.1
An activity within an Application is tracked through a Sensor
Endpoint that sends events to a Record Store via the Sensor
API.
A Metric Profile describes a type of learning activity with a
controlled & constrained vocabulary.
ex: Annotation, Assessment, Forum, Media, Reading, Outcome, …
● Actor
● Action
● Activity
● Optional: EventTime, target, generated, referrer, edApp, group,
membership, session, federatedSession (LTI), extensions
1. EdTech Solutions that Achieve Learning Objectives
Analytics Standards Cont.
2. Translating Learning Events
into Learning Analytics
Standard Analytics*
1. Content Popularity > Volume & Audience
> Check time spent, # of pages read & # of unique users
> Compare between content subparts & similar content blocks
● Beware that time spent can be rigged; check how pages read are
counted (ie. scroll in the page, assess minimum time spent)
● Check the distribution scenarios; volume can come from content
popularity either among instructors or learners
2. Learner Engagement > Interactions with Content
Digital learning brings the unvalued benefit of engaging learners through
interactivities, making the learning more active and thus efficient.
> Check # of interactions & time spent with videos, audio, quizzes or
widgets within a content block
3. Learner Success > Scores
> Check learner score across different activities to assess their
understanding of a course
> Compare between content blocks to assess their relative difficulty
* Standard learning stats:
- Time spent
- Average result
- Last activity date
- Sessions, ...
2. Translating Learning Events into Learning Analytics
What analytics best determine course effectiveness?
Advanced Analytics
1. Learner Cohort Analysis
> Check % of pages read & volume of interactions by learner over time
If % is plateauing at a low level vs other content, this might be due to not
having enough engaging content or having irrelevant courses.
2. Time-Efficient Learning
> How much time is needed on average for a learner to master a topic
having learned with your content?
Compare average score on the same assessment between readers having
interacted with different content pieces with similar learning objectives.
3. Learner Progression
> Identify correlations between learner result progression over time and
their interaction with specific content or piece of subcontent.
Example of Cohort Analysis
2. Translating Learning Events into Learning Analytics
What analytics best determine course effectiveness?
Analyze Learner Performance Over Time, Across Courses
● Check how learner average performance progressed after a course:
○ Assess before & after the course
○ Compare course performance vs difficulty
● Check correlation between performance with time spent & volume of
engagement:
○ Check performance of most engaged learners vs the others
○ Take note that more engaged learners are often low-level learners
Analyze Performance Distribution Across Learners
● Check how course is performing among low-level, standard &
high-level learners:
○ Assess before & after the course
○ High-level students usually need less teaching content => course
performance is mostly reflected on the performance increase on
low-level & standard learners
How do you clearly distinguish the respective impact of
learning content, instructor methodology & learner work
on learner performance ?
2. Translating Learning Events into Learning Analytics
Interpreting Learning Data: Learner Performance vs.
Course Performance
Learning Content Improvements
● Course Clarity & Efficiency
○ How are learners performing on the content?
Is the score distribution binary or more evenly distributed?
○ Are learners spending too much time on the content in
relation to the performance increase?
➢ Review the course layout and pedagogy
● Course Engagement
○ Is the level of learner engagement matching the level of
other content?
➢ Add interactive elements & activities to engage learner
● Course Level / Difficulty
○ Compare the average score across learners for the
different course sections and across courses vs
expectations
Teaching Improvements
Teaching efficiency is highly correlated with learner engagement / level of
activity and teaching personalization.
● Engage Learners
○ Ensure courses & assignments have a good mix of activities
○ Encourage learners to interact more with the content
● Personalize Teaching
○ Start with a quick test to break down the classroom into ability groups
○ Differentiate assignments for each ability group via digital learning
○ Identify the sticking points & root-causes for low-level learners:
■ Assess pre-requisites
■ Slow down learning pace / try a more progressive learning method,
(ex: step-by-step > application > more complex assessments)
■ Try other pedagogies & learning content types (videos, schema, …)
Which data & actions can improve your teaching ?
2. Translating Learning Events into Learning Analytics
Leveraging Data for Course Improvements
What is Adaptive Learning?
Adaptive learning is about automatically personalizing the learning path of
each learner to optimize the efficiency of their learning experience for
self-defined learning objectives or by assignment from an instructor.
Analyzing learner grades helps you to continually determine:
● 🔽Is a learner struggling? → teach prerequisite knowledge
● = Is a learner progressing as expected? → no action required
● 🔼Is a learner already mastering the knowledge? → save time & push them
learning content for more advanced or other supplementary knowledge
What’s Next: Adaptive Learning
➔ It can also try to identify per learner the most efficient types of learning
content, learning pace, optimal hours of learning, …
➔ A learning profile is built for each learner over time based on all their
interactions with the content & assessment results.
➔ Memory anchoring features, is available, can ensure knowledge is correctly
memorized over time by regularly reviewing it (recurrence of testing is
adapted to each learner).
2. Translating Learning Events into Learning Analytics
Prerequisites & Adaptive Learning Engine
1. Add a Semantic Layer to Your Content
○ Atomic learning objects (LOs) provide more granularity for
personalizing each learner’s learning path
○ LOs to be metattagged with role & target knowledge
ex: “LO 1 teach addition with 2 digits”
○ Build a knowledge tree that links the different knowledge of the
content together to detail which ones are prerequisites to others
2. Track Learner Behavior: Assessment scores, time spent, interactions
with media and interactivities, …
Based on the LO semantic & learner interactions, adaptive learning
engines leverage bayesian or neural network algorithms to dynamically
identify which next LO should be shown to each learner.
Limits & Perspective
● Requires additional work to add semantic layer to learning content
● Fully automated engines (ex: blackbox) work better for self-paced
learning, but are harder to incorporate with an instructor pedagogy
● Works best as an instructor aid with learning recommendations &
insights on learners
2. Translating Learning Events into Learning Analytics
What’s Next: Adaptive Learning
Summary
1. EdTech Solutions that Achieve Learning Objectives
○ The must-have components:
■ A global architecture
■ A combination of authoring, learning management and end-user solutions
○ All-in-one vs. modular: we recommend all-in-one, pending your unique needs
○ xAPI/TinCan vs. IMS Caliper: either, as long as you can build a quantitative analysis of user behavior
2. Translating Learning Events into Learning Analytics
○ Standard analytics: Time spent, # pages read, # unique users, learner scores
○ Advanced analytics: Learner cohort analysis, time efficiency, learner progression
■ Both kinds of data help educators pivot course improvement and student engagement
○ Adaptive learning: the future of personalizing the elearning experience for all learners
25
EVP Global Sales and Marketing, Gutenberg
Technology
Linkedin page: sethpuri/
Twitter ID: @SethPuri
Email: spuri@gutenberg-technology.com
Website: gutenberg-technology.com
Editor, Aggregage
Linkedin page: elearning-learning/
Twitter ID: @eLearningPosts
Email: eve.lyons-berg@aggregage.com
Website: elearninglearning.com
Thank You
Appendix
Caliper v1.1
Caliper / LTI 1.x Workflow
IMS Caliper Adoption / Certification
xAPI Conformant LRS
xAPI Adopters
{
"actor": {
"name": "Sally Glider",
"mbox": "mailto:sally@example.com"
},
"verb": {
"id": "http://adlnet.gov/expapi/verbs/experienced",
"display": {"en-US": "experienced"}
},
"object": {
"id": "http://example.com/activities/solo-hang-gliding",
"definition": {
"type": "http://adlnet.gov/expapi/activities/course",
"name": { "en-US": "Solo Hang Gliding" }
}
},
"context": {
"instructor": {
"name": "Irene Instructor",
"mbox": "mailto:irene@example.com"
},
"contextActivities":{
"parent": { "id": "http://example.com/activities/hang-gliding-class-a" },
"grouping": { "id": "http://example.com/activities/hang-gliding-school" }
}
}
}
xAPI Statement Example

More Related Content

Similar to Bang for Your eLearning Buck: Enhancing the Learning Experience by Leveraging Learner Analytics

IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...
IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...
IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...Margaret Roth
 
M2 Auxiliary resources to optimize activities Telecenter / ICT Center / Lib...
M2 Auxiliary resources to  optimize activities Telecenter /  ICT Center / Lib...M2 Auxiliary resources to  optimize activities Telecenter /  ICT Center / Lib...
M2 Auxiliary resources to optimize activities Telecenter / ICT Center / Lib...TELECENTRE EUROPE
 
Digital Educational Content Quality Assurance Process
Digital Educational Content Quality Assurance ProcessDigital Educational Content Quality Assurance Process
Digital Educational Content Quality Assurance ProcessNikos Palavitsinis, PhD
 
Flipped learning solution for British Council's PTC
Flipped learning solution for British Council's PTCFlipped learning solution for British Council's PTC
Flipped learning solution for British Council's PTCGareth Davies
 
Yet LXi — Learning Experience Interface Overview
Yet LXi — Learning Experience Interface Overview Yet LXi — Learning Experience Interface Overview
Yet LXi — Learning Experience Interface Overview Margaret Roth
 
ePortfolios for Assessment and Measurement
ePortfolios for Assessment and MeasurementePortfolios for Assessment and Measurement
ePortfolios for Assessment and MeasurementPearson
 
Technology Integration in the Classroom - A case study in learning engagement...
Technology Integration in the Classroom - A case study in learning engagement...Technology Integration in the Classroom - A case study in learning engagement...
Technology Integration in the Classroom - A case study in learning engagement...William Welder
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EUmoocs
 
Learning design and learning analytics
Learning design and learning analyticsLearning design and learning analytics
Learning design and learning analyticsRebecca Ferguson
 
14.30 kurilovas bireniene
14.30 kurilovas bireniene14.30 kurilovas bireniene
14.30 kurilovas birenieneprogettoacariss
 
App Circus Project - Group 5.pptx
App Circus Project - Group 5.pptxApp Circus Project - Group 5.pptx
App Circus Project - Group 5.pptxjdean31
 
Macedonia Icamp Short
Macedonia Icamp ShortMacedonia Icamp Short
Macedonia Icamp ShortKai Pata
 
Unveiling the Best Learning Experience Platforms: A Deep Dive
Unveiling the Best Learning Experience Platforms: A Deep DiveUnveiling the Best Learning Experience Platforms: A Deep Dive
Unveiling the Best Learning Experience Platforms: A Deep Divetheapollonai
 
Reflections from various Evaluations of ICT projects - Benita Williams
Reflections from various Evaluations of ICT projects - Benita WilliamsReflections from various Evaluations of ICT projects - Benita Williams
Reflections from various Evaluations of ICT projects - Benita WilliamsBenita Williams
 
Social Networking for Student and Staff Learning
Social Networking for Student and Staff LearningSocial Networking for Student and Staff Learning
Social Networking for Student and Staff LearningAndrew Brasher
 
i-lovelearning London 2016 | Netex 2017 Preview [EN]
i-lovelearning London 2016 | Netex 2017 Preview [EN]i-lovelearning London 2016 | Netex 2017 Preview [EN]
i-lovelearning London 2016 | Netex 2017 Preview [EN]Netex Learning
 
NetGPT 1.0 PRD Presentation2.pdf
NetGPT 1.0 PRD Presentation2.pdfNetGPT 1.0 PRD Presentation2.pdf
NetGPT 1.0 PRD Presentation2.pdfJOHNADEMILUYI3
 
Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...
Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...
Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...Netex Learning
 

Similar to Bang for Your eLearning Buck: Enhancing the Learning Experience by Leveraging Learner Analytics (20)

IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...
IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...
IEEE TAG xAPI Webinar Series: Improving the Learner Experience Through an xAP...
 
M2 Auxiliary resources to optimize activities Telecenter / ICT Center / Lib...
M2 Auxiliary resources to  optimize activities Telecenter /  ICT Center / Lib...M2 Auxiliary resources to  optimize activities Telecenter /  ICT Center / Lib...
M2 Auxiliary resources to optimize activities Telecenter / ICT Center / Lib...
 
Digital Educational Content Quality Assurance Process
Digital Educational Content Quality Assurance ProcessDigital Educational Content Quality Assurance Process
Digital Educational Content Quality Assurance Process
 
E-Learning
E-LearningE-Learning
E-Learning
 
Flipped learning solution for British Council's PTC
Flipped learning solution for British Council's PTCFlipped learning solution for British Council's PTC
Flipped learning solution for British Council's PTC
 
Yet LXi — Learning Experience Interface Overview
Yet LXi — Learning Experience Interface Overview Yet LXi — Learning Experience Interface Overview
Yet LXi — Learning Experience Interface Overview
 
ePortfolios for Assessment and Measurement
ePortfolios for Assessment and MeasurementePortfolios for Assessment and Measurement
ePortfolios for Assessment and Measurement
 
e-content.pdf
e-content.pdfe-content.pdf
e-content.pdf
 
Technology Integration in the Classroom - A case study in learning engagement...
Technology Integration in the Classroom - A case study in learning engagement...Technology Integration in the Classroom - A case study in learning engagement...
Technology Integration in the Classroom - A case study in learning engagement...
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
 
Learning design and learning analytics
Learning design and learning analyticsLearning design and learning analytics
Learning design and learning analytics
 
14.30 kurilovas bireniene
14.30 kurilovas bireniene14.30 kurilovas bireniene
14.30 kurilovas bireniene
 
App Circus Project - Group 5.pptx
App Circus Project - Group 5.pptxApp Circus Project - Group 5.pptx
App Circus Project - Group 5.pptx
 
Macedonia Icamp Short
Macedonia Icamp ShortMacedonia Icamp Short
Macedonia Icamp Short
 
Unveiling the Best Learning Experience Platforms: A Deep Dive
Unveiling the Best Learning Experience Platforms: A Deep DiveUnveiling the Best Learning Experience Platforms: A Deep Dive
Unveiling the Best Learning Experience Platforms: A Deep Dive
 
Reflections from various Evaluations of ICT projects - Benita Williams
Reflections from various Evaluations of ICT projects - Benita WilliamsReflections from various Evaluations of ICT projects - Benita Williams
Reflections from various Evaluations of ICT projects - Benita Williams
 
Social Networking for Student and Staff Learning
Social Networking for Student and Staff LearningSocial Networking for Student and Staff Learning
Social Networking for Student and Staff Learning
 
i-lovelearning London 2016 | Netex 2017 Preview [EN]
i-lovelearning London 2016 | Netex 2017 Preview [EN]i-lovelearning London 2016 | Netex 2017 Preview [EN]
i-lovelearning London 2016 | Netex 2017 Preview [EN]
 
NetGPT 1.0 PRD Presentation2.pdf
NetGPT 1.0 PRD Presentation2.pdfNetGPT 1.0 PRD Presentation2.pdf
NetGPT 1.0 PRD Presentation2.pdf
 
Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...
Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...
Netex Seminar LT2017 | Make your strategy lean and mean for learning in 2017 ...
 

Recently uploaded

Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Osopher
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Projectjordimapav
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQuiz Club NITW
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...Nguyen Thanh Tu Collection
 
4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptxmary850239
 
An Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERPAn Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERPCeline George
 
6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroom6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroomSamsung Business USA
 
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptxDecoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptxDhatriParmar
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management SystemChristalin Nelson
 
Integumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptIntegumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptshraddhaparab530
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQuiz Club NITW
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...DhatriParmar
 
Comparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptxComparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptxAvaniJani1
 
MS4 level being good citizen -imperative- (1) (1).pdf
MS4 level   being good citizen -imperative- (1) (1).pdfMS4 level   being good citizen -imperative- (1) (1).pdf
MS4 level being good citizen -imperative- (1) (1).pdfMr Bounab Samir
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxAnupam32727
 
Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...
Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...
Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...HetalPathak10
 

Recently uploaded (20)

Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
 
prashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Professionprashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Profession
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Project
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
 
4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx
 
An Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERPAn Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERP
 
6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroom6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroom
 
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptxDecoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management System
 
Integumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptIntegumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.ppt
 
Mattingly "AI & Prompt Design" - Introduction to Machine Learning"
Mattingly "AI & Prompt Design" - Introduction to Machine Learning"Mattingly "AI & Prompt Design" - Introduction to Machine Learning"
Mattingly "AI & Prompt Design" - Introduction to Machine Learning"
 
Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...
Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...
Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
 
Comparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptxComparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptx
 
MS4 level being good citizen -imperative- (1) (1).pdf
MS4 level   being good citizen -imperative- (1) (1).pdfMS4 level   being good citizen -imperative- (1) (1).pdf
MS4 level being good citizen -imperative- (1) (1).pdf
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
 
Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...
Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...
Satirical Depths - A Study of Gabriel Okara's Poem - 'You Laughed and Laughed...
 
Introduction to Research ,Need for research, Need for design of Experiments, ...
Introduction to Research ,Need for research, Need for design of Experiments, ...Introduction to Research ,Need for research, Need for design of Experiments, ...
Introduction to Research ,Need for research, Need for design of Experiments, ...
 

Bang for Your eLearning Buck: Enhancing the Learning Experience by Leveraging Learner Analytics

  • 1.
  • 2.
  • 3. 3
  • 4.
  • 5. Enhancing the Learning Experience by Leveraging Learner Analytics A guide to how collecting & assessing learning outcomes helps you deliver a differentiated elearning experience.
  • 6. Agenda 1. EdTech Solutions that Achieve Learning Objectives ○ A review of the must-have solution components ○ All-in-one vs. modular solutions ○ How to deliver & track learning events: xAPI/TinCan vs. IMS Caliper 2. Translating Learning Events into Learning Analytics ○ What analytics work best to determine course effectiveness? ○ How to interpret learning data: learner performance vs. course performance ○ How instructors can leverage data for course improvements ○ What’s next: Adaptive learning
  • 7. 1. EdTech Solutions that Achieve Learning Objectives
  • 8. WebReader, iOS, Android AppsLMS Learning Management System User Mgmt Authoring Solution / LCMS Project ManagementImport (XML, ePub, PDF) Authoring styling, content, activities, media, ...User Mgmt Content Distribution API LTI Analytics Teaching Content Content Distribution Content Creation Analytics Read, Interact & Learn Learning Library Content Producers / Authors Instructors Learners Analytics User Mgmt build, manage & distribute teaching & learning content manage, assemble & distribute learning content learn on learning content Learning Event Server 1. EdTech Solutions that Achieve Learning Objectives Global Solution Architecture
  • 9. User Management ● User invitation, registration, login ● SSO, access control, module permissions ● Project teams & project permissions ➢ Users: Education Publishers, Authors, Business Learning Centers, Communication Dpt, ... ➢ Solution Role: Build, Manage & Distribute teaching, learning or communication content Project Management ● New project creation ● Project import via various formats (ePub, PDF, XML, …) ● Project list, sort, search, delete ● Project information & metadata ● Project settings & accessibility Analytics Insights for content creators based on end-user interactions to improve content Project Authoring ● Project theme/layout: colors, fonts, styles, layout CSS ● Project enrichment: media, activities, widgets, … ● Print/digital or learner/instructor specific content Project distribution ● Project versioning ● Project team-review ● Distribution to learners: LMS, LTI direct linking, custom API, direct to readers 1. EdTech Solutions that Achieve Learning Objectives The Authoring Solution
  • 10. User Management ● User invitation, registration, login, SSO ● Instructor management ● Classrooms / Groups Content Library ● Content preview, bookmarks ● Content assembling from learning objects ● External content import Analytics Insights for Instructors on content end-user interactions to improve learning Content Authoring ● Simplified authoring ● Insertion of media, activities Content Distribution ● Distribution to group of learners / users ● Assignation, expiration date, specific settings ➢ Users: Teachers, Instructors, Communication Managers ➢ Solution Role: Manage & Distribute learning or communication content 1. EdTech Solutions that Achieve Learning Objectives The Learning Management Solution
  • 11. User Management ● User registration, login, SSO ● User profile, settings Learning Library ● List, sort content, detailed info ● Content download for offline reading ● Search content ● Additional content purchase ● Social Analytics Summary of interactions with content, progress reports Content Readers ● Content display with layout ● Media rendering ● Activities & assessments ● Search in content ● Accessibility ● User annotations ● Interaction events ➢ Users: Students, Learners, Employees, Customers, ... ➢ Solution Role: Discover & Interact with the distributed content Android Apps Chromebooks MacOS iOS apps Windows 1. EdTech Solutions that Achieve Learning Objectives The Readers for End-Users
  • 12. To build & deliver digital content, we see companies follow a similar development path: To integrate with the legacy ecosystem already in place, specialized solutions need to: ● Be modular with open API available ● Support industry standards: IMS LTI, Caliper in Education/Publishing, Scorm / Tincan for Business Learning ● Integrate with market-leading solutions such as Moodle or Blackboard LMS Start with an all-in-one solution, generally a LMS that can author & deliver to the web. Addition & integration of more specialized solutions. 1 Expansion of digital distribution => Richer needs, workflow industrialization 2 Authoring Solution Mobile & Web Distribution Solutions Analytics Solution? 1. EdTech Solutions that Achieve Learning Objectives Integrated or Modular Solution?
  • 13. Analytics are about building a quantitative analysis of the end-users’ behavior with your content. ● Recipients: end-user, instructors, content producers, … ● Types of analytics: aggregate figures, ratio, charts & graphs (dataviz) Track meaningful end-user interactions with content (= events) Flow events into different user interfaces Conclude most meaningful analytics for recipient based on events Build interfaces that provide easy access to desired info / analytics 1. EdTech Solutions that Achieve Learning Objectives Getting Started with Analytics
  • 14. xAPI (Project TinCan) Initiative led by the & Open-source, Apache v2 license Successor of SCORM since 2013 API Goals: ● Allow applications to share data about human performance on instructional content & context information ● Store information in a Learning Record Store (LRS) ➔ JSON format over HTTP + OAuth ➔ Service and device agnostic = versatile, transferable from LMS to mobile devices, wearables, search engine, gamification, blog, webinar, CRM, ... 24 conformant Learning Record Stores, 39 official adopters ➔ Versatile, but mostly adopted in business learning Best practices for collecting end-user interaction events led to 2 different analytics standards: Caliper v1.1 Initiative led by the , started 2015 Closed consortium which controls the Caliper IP Goals: ● Approach to describing, collecting & exchanging learning activity data ● Way of measuring learning activities & effectiveness ● Enables new services to measure, report & visualize ➔ JSON format over HTTPS/TLS with API key ➔ Focus around the LMS only => adopted or being adopted by the main LMS vendors Only 11 solutions certified by IMS, ex: Blackboard, Canvas, VitalSource + others having “adopted” ex: Moodle, Elsevier ➔ Focused on learning, mostly in e-education 1. EdTech Solutions that Achieve Learning Objectives Analytics Standards
  • 15. Caliper v1.1xAPI 1. EdTech Solutions that Achieve Learning Objectives Analytics Standards Visualized
  • 16. xAPI (Project TinCan) The Activity Provider delivers the learning experiences and communicates the learning info to the Learning Record Store. Learning experiences are described through statements: ● Actor: individual/group performing an action within an Activity. ● Verb: action performed during the learning experience. The xAPI only specifies how to create meaningful Verbs. ● Object: something that the Actor interacted with, ex: a book, an elearning course, a unit of instruction, an activity ● Result (optional): measured outcome of the statement ex: score, success, completion, response, activity duration ● Context (optional): additional information ex: the instructor, inclusion in a broader activity or team activity ● Attachment (optional): evidence of a learning experience ex: a record of communication, an essay, a video, or a certificate ● TimeStamp, Authority: device/service delivering the statement Caliper v1.1 An activity within an Application is tracked through a Sensor Endpoint that sends events to a Record Store via the Sensor API. A Metric Profile describes a type of learning activity with a controlled & constrained vocabulary. ex: Annotation, Assessment, Forum, Media, Reading, Outcome, … ● Actor ● Action ● Activity ● Optional: EventTime, target, generated, referrer, edApp, group, membership, session, federatedSession (LTI), extensions 1. EdTech Solutions that Achieve Learning Objectives Analytics Standards Cont.
  • 17. 2. Translating Learning Events into Learning Analytics
  • 18. Standard Analytics* 1. Content Popularity > Volume & Audience > Check time spent, # of pages read & # of unique users > Compare between content subparts & similar content blocks ● Beware that time spent can be rigged; check how pages read are counted (ie. scroll in the page, assess minimum time spent) ● Check the distribution scenarios; volume can come from content popularity either among instructors or learners 2. Learner Engagement > Interactions with Content Digital learning brings the unvalued benefit of engaging learners through interactivities, making the learning more active and thus efficient. > Check # of interactions & time spent with videos, audio, quizzes or widgets within a content block 3. Learner Success > Scores > Check learner score across different activities to assess their understanding of a course > Compare between content blocks to assess their relative difficulty * Standard learning stats: - Time spent - Average result - Last activity date - Sessions, ... 2. Translating Learning Events into Learning Analytics What analytics best determine course effectiveness?
  • 19. Advanced Analytics 1. Learner Cohort Analysis > Check % of pages read & volume of interactions by learner over time If % is plateauing at a low level vs other content, this might be due to not having enough engaging content or having irrelevant courses. 2. Time-Efficient Learning > How much time is needed on average for a learner to master a topic having learned with your content? Compare average score on the same assessment between readers having interacted with different content pieces with similar learning objectives. 3. Learner Progression > Identify correlations between learner result progression over time and their interaction with specific content or piece of subcontent. Example of Cohort Analysis 2. Translating Learning Events into Learning Analytics What analytics best determine course effectiveness?
  • 20. Analyze Learner Performance Over Time, Across Courses ● Check how learner average performance progressed after a course: ○ Assess before & after the course ○ Compare course performance vs difficulty ● Check correlation between performance with time spent & volume of engagement: ○ Check performance of most engaged learners vs the others ○ Take note that more engaged learners are often low-level learners Analyze Performance Distribution Across Learners ● Check how course is performing among low-level, standard & high-level learners: ○ Assess before & after the course ○ High-level students usually need less teaching content => course performance is mostly reflected on the performance increase on low-level & standard learners How do you clearly distinguish the respective impact of learning content, instructor methodology & learner work on learner performance ? 2. Translating Learning Events into Learning Analytics Interpreting Learning Data: Learner Performance vs. Course Performance
  • 21. Learning Content Improvements ● Course Clarity & Efficiency ○ How are learners performing on the content? Is the score distribution binary or more evenly distributed? ○ Are learners spending too much time on the content in relation to the performance increase? ➢ Review the course layout and pedagogy ● Course Engagement ○ Is the level of learner engagement matching the level of other content? ➢ Add interactive elements & activities to engage learner ● Course Level / Difficulty ○ Compare the average score across learners for the different course sections and across courses vs expectations Teaching Improvements Teaching efficiency is highly correlated with learner engagement / level of activity and teaching personalization. ● Engage Learners ○ Ensure courses & assignments have a good mix of activities ○ Encourage learners to interact more with the content ● Personalize Teaching ○ Start with a quick test to break down the classroom into ability groups ○ Differentiate assignments for each ability group via digital learning ○ Identify the sticking points & root-causes for low-level learners: ■ Assess pre-requisites ■ Slow down learning pace / try a more progressive learning method, (ex: step-by-step > application > more complex assessments) ■ Try other pedagogies & learning content types (videos, schema, …) Which data & actions can improve your teaching ? 2. Translating Learning Events into Learning Analytics Leveraging Data for Course Improvements
  • 22. What is Adaptive Learning? Adaptive learning is about automatically personalizing the learning path of each learner to optimize the efficiency of their learning experience for self-defined learning objectives or by assignment from an instructor. Analyzing learner grades helps you to continually determine: ● 🔽Is a learner struggling? → teach prerequisite knowledge ● = Is a learner progressing as expected? → no action required ● 🔼Is a learner already mastering the knowledge? → save time & push them learning content for more advanced or other supplementary knowledge What’s Next: Adaptive Learning ➔ It can also try to identify per learner the most efficient types of learning content, learning pace, optimal hours of learning, … ➔ A learning profile is built for each learner over time based on all their interactions with the content & assessment results. ➔ Memory anchoring features, is available, can ensure knowledge is correctly memorized over time by regularly reviewing it (recurrence of testing is adapted to each learner). 2. Translating Learning Events into Learning Analytics
  • 23. Prerequisites & Adaptive Learning Engine 1. Add a Semantic Layer to Your Content ○ Atomic learning objects (LOs) provide more granularity for personalizing each learner’s learning path ○ LOs to be metattagged with role & target knowledge ex: “LO 1 teach addition with 2 digits” ○ Build a knowledge tree that links the different knowledge of the content together to detail which ones are prerequisites to others 2. Track Learner Behavior: Assessment scores, time spent, interactions with media and interactivities, … Based on the LO semantic & learner interactions, adaptive learning engines leverage bayesian or neural network algorithms to dynamically identify which next LO should be shown to each learner. Limits & Perspective ● Requires additional work to add semantic layer to learning content ● Fully automated engines (ex: blackbox) work better for self-paced learning, but are harder to incorporate with an instructor pedagogy ● Works best as an instructor aid with learning recommendations & insights on learners 2. Translating Learning Events into Learning Analytics What’s Next: Adaptive Learning
  • 24. Summary 1. EdTech Solutions that Achieve Learning Objectives ○ The must-have components: ■ A global architecture ■ A combination of authoring, learning management and end-user solutions ○ All-in-one vs. modular: we recommend all-in-one, pending your unique needs ○ xAPI/TinCan vs. IMS Caliper: either, as long as you can build a quantitative analysis of user behavior 2. Translating Learning Events into Learning Analytics ○ Standard analytics: Time spent, # pages read, # unique users, learner scores ○ Advanced analytics: Learner cohort analysis, time efficiency, learner progression ■ Both kinds of data help educators pivot course improvement and student engagement ○ Adaptive learning: the future of personalizing the elearning experience for all learners
  • 25. 25 EVP Global Sales and Marketing, Gutenberg Technology Linkedin page: sethpuri/ Twitter ID: @SethPuri Email: spuri@gutenberg-technology.com Website: gutenberg-technology.com Editor, Aggregage Linkedin page: elearning-learning/ Twitter ID: @eLearningPosts Email: eve.lyons-berg@aggregage.com Website: elearninglearning.com
  • 29. Caliper / LTI 1.x Workflow
  • 30. IMS Caliper Adoption / Certification
  • 33. { "actor": { "name": "Sally Glider", "mbox": "mailto:sally@example.com" }, "verb": { "id": "http://adlnet.gov/expapi/verbs/experienced", "display": {"en-US": "experienced"} }, "object": { "id": "http://example.com/activities/solo-hang-gliding", "definition": { "type": "http://adlnet.gov/expapi/activities/course", "name": { "en-US": "Solo Hang Gliding" } } }, "context": { "instructor": { "name": "Irene Instructor", "mbox": "mailto:irene@example.com" }, "contextActivities":{ "parent": { "id": "http://example.com/activities/hang-gliding-class-a" }, "grouping": { "id": "http://example.com/activities/hang-gliding-school" } } } } xAPI Statement Example