Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos
Juho Kim, Philip J. Guo, Daniel T. Seaton, Piotr Mitros, Krzysztof Z. Gajos, Robert C. Miller
Presented at ACM Learning at Scale 2014, March 4-5 2014, Atlanta, GA, USA.
Learnersourcing: Improving Learning with Collective Learner ActivityJuho Kim
Slides from my thesis defense: "Learnersourcing: Improving Learning with Collective Learner Activity"
Millions of learners today are watching videos on online platforms, such as Khan Academy, YouTube, Coursera, and edX, to take courses and master new skills. But existing video interfaces are not designed to support learning, with limited interactivity and lack of information about learners' engagement and content. Making these improvements requires deep semantic information about video that even state-of-the-art AI techniques cannot fully extract. I take a data-driven approach to address this challenge, using large-scale learning interaction data to dynamically improve video content and interfaces. Specifically, this thesis introduces learnersourcing, a form of crowdsourcing in which learners collectively contribute novel content for future learners while engaging in a meaningful learning experience themselves. I present learnersourcing applications designed for massive open online course videos and how-to tutorial videos, where learners' collective activities 1) highlight points of confusion or importance in a video, 2) extract a solution structure from a tutorial, and 3) improve the navigation experience for future learners. This thesis demonstrates how learnersourcing can enable more interactive, collaborative, and data-driven learning.
CHI2014 Workshop - Leveraging Video Interaction Data and Content Analysis to ...Juho Kim
Video has emerged as a dominant medium for online
education, as witnessed by millions of students learning
from educational videos on Massive Open Online
Courses (MOOCs), Khan Academy, and YouTube. The
large-scale data collected from students’ interactions
with video provide a unique opportunity to analyze and
improve the video learning experience. We combine
click-level interaction data, such as pausing, resuming,
or navigating between points in the video, and video
content analysis, such as visual, text, and speech, to
analyze peaks in viewership and student activity. Such
analysis can reveal points of interest or confusion in the
video, and suggest production and editing
improvements. Furthermore, we envision novel video
interfaces and learning platforms that automatically
adapt to learners’ collective watching behaviors.
CHI2014 - Crowdsourcing Step-by-Step Information Extraction to Enhance Existi...Juho Kim
Millions of learners today use how-to videos to master new skills in a variety of domains. But browsing such videos is often tedious and inefficient because video player interfaces are not optimized for the unique step-by-step structure of such videos. This research aims to improve the learning experience of existing how-to videos with step-by-step annotations.
We first performed a formative study to verify that annotations are actually useful to learners. We created ToolScape, an interactive video player that displays step descriptions and intermediate result thumbnails in the video timeline. Learners in our study performed better and gained more self-efficacy using ToolScape versus a traditional video player.
To add the needed step annotations to existing how-to videos at scale, we introduce a novel crowdsourcing workflow. It extracts step-by-step structure from an existing video, including step times, descriptions, and before and after images. We introduce the Find-Verify-Expand design pattern for temporal and visual annotation, which applies clustering, text processing, and visual analysis algorithms to merge crowd output. The workflow does not rely on domain-specific customization, works on top of existing videos, and recruits untrained crowd workers. We evaluated the workflow with Mechanical Turk, using 75 cooking, makeup, and Photoshop videos on YouTube. Results show that our workflow can extract steps with a quality comparable to that of trained annotators across all three domains with 77% precision and 81% recall.
Presented at AAAI 2016 Spring Symposium: Intelligent systems for supporting distributed human teamwork. Position paper available at: http://juhokim.com/files/AAAISymposium2016-Organic-Crowdsourcing-Systems.pdf
RIMES: Embedding Interactive Multimedia Exercises in Lecture VideosJuho Kim
- RIMES: Embedding Interactive Multimedia Exercises in Lecture Videos
- Juho Kim, Elena L. Glassman, Andrés Monroy-Hernández, Meredith Ringel Morris
- Presented at CHI 2015
- We introduce RIMES, a system that allows teachers to embed interactive multimedia exercises within online lecture videos. Students can record audio, video, and ink-based answers, and teachers can review the responses.
Cobi: A Community-Informed Conference Scheduling Tool. UIST 2013 slidesJuho Kim
Juho Kim, Haoqi Zhang, Paul André, Lydia B. Chilton, Wendy Mackay, Michel Beaudouin-Lafon, Robert C. Miller, and Steven P. Dow. Cobi: A Community-Informed Conference Scheduling Tool. UIST 2013.
http://dl.acm.org/citation.cfm?id=2502034
Learnersourcing: Improving Learning with Collective Learner ActivityJuho Kim
Slides from my thesis defense: "Learnersourcing: Improving Learning with Collective Learner Activity"
Millions of learners today are watching videos on online platforms, such as Khan Academy, YouTube, Coursera, and edX, to take courses and master new skills. But existing video interfaces are not designed to support learning, with limited interactivity and lack of information about learners' engagement and content. Making these improvements requires deep semantic information about video that even state-of-the-art AI techniques cannot fully extract. I take a data-driven approach to address this challenge, using large-scale learning interaction data to dynamically improve video content and interfaces. Specifically, this thesis introduces learnersourcing, a form of crowdsourcing in which learners collectively contribute novel content for future learners while engaging in a meaningful learning experience themselves. I present learnersourcing applications designed for massive open online course videos and how-to tutorial videos, where learners' collective activities 1) highlight points of confusion or importance in a video, 2) extract a solution structure from a tutorial, and 3) improve the navigation experience for future learners. This thesis demonstrates how learnersourcing can enable more interactive, collaborative, and data-driven learning.
CHI2014 Workshop - Leveraging Video Interaction Data and Content Analysis to ...Juho Kim
Video has emerged as a dominant medium for online
education, as witnessed by millions of students learning
from educational videos on Massive Open Online
Courses (MOOCs), Khan Academy, and YouTube. The
large-scale data collected from students’ interactions
with video provide a unique opportunity to analyze and
improve the video learning experience. We combine
click-level interaction data, such as pausing, resuming,
or navigating between points in the video, and video
content analysis, such as visual, text, and speech, to
analyze peaks in viewership and student activity. Such
analysis can reveal points of interest or confusion in the
video, and suggest production and editing
improvements. Furthermore, we envision novel video
interfaces and learning platforms that automatically
adapt to learners’ collective watching behaviors.
CHI2014 - Crowdsourcing Step-by-Step Information Extraction to Enhance Existi...Juho Kim
Millions of learners today use how-to videos to master new skills in a variety of domains. But browsing such videos is often tedious and inefficient because video player interfaces are not optimized for the unique step-by-step structure of such videos. This research aims to improve the learning experience of existing how-to videos with step-by-step annotations.
We first performed a formative study to verify that annotations are actually useful to learners. We created ToolScape, an interactive video player that displays step descriptions and intermediate result thumbnails in the video timeline. Learners in our study performed better and gained more self-efficacy using ToolScape versus a traditional video player.
To add the needed step annotations to existing how-to videos at scale, we introduce a novel crowdsourcing workflow. It extracts step-by-step structure from an existing video, including step times, descriptions, and before and after images. We introduce the Find-Verify-Expand design pattern for temporal and visual annotation, which applies clustering, text processing, and visual analysis algorithms to merge crowd output. The workflow does not rely on domain-specific customization, works on top of existing videos, and recruits untrained crowd workers. We evaluated the workflow with Mechanical Turk, using 75 cooking, makeup, and Photoshop videos on YouTube. Results show that our workflow can extract steps with a quality comparable to that of trained annotators across all three domains with 77% precision and 81% recall.
Presented at AAAI 2016 Spring Symposium: Intelligent systems for supporting distributed human teamwork. Position paper available at: http://juhokim.com/files/AAAISymposium2016-Organic-Crowdsourcing-Systems.pdf
RIMES: Embedding Interactive Multimedia Exercises in Lecture VideosJuho Kim
- RIMES: Embedding Interactive Multimedia Exercises in Lecture Videos
- Juho Kim, Elena L. Glassman, Andrés Monroy-Hernández, Meredith Ringel Morris
- Presented at CHI 2015
- We introduce RIMES, a system that allows teachers to embed interactive multimedia exercises within online lecture videos. Students can record audio, video, and ink-based answers, and teachers can review the responses.
Cobi: A Community-Informed Conference Scheduling Tool. UIST 2013 slidesJuho Kim
Juho Kim, Haoqi Zhang, Paul André, Lydia B. Chilton, Wendy Mackay, Michel Beaudouin-Lafon, Robert C. Miller, and Steven P. Dow. Cobi: A Community-Informed Conference Scheduling Tool. UIST 2013.
http://dl.acm.org/citation.cfm?id=2502034
Mini-presentation within ELI's shortcourse: "Digging into Badges"
AUDIO: play mp3 accompanying these slides at: http://ofcoursesonline.com/badging%20at%20ucf%20audio.mp3
How to ask better questions and how to assess UX using surveys.
This workshop at UXLX 2014 in Lisbon was a deep dive into two important topics in survey design for user research.
We used the four-step model of how people answer questions to work on better questions, then we focused on two special uses of questionnaires in user research: the post-test assessment of satisfaction, and then how to gather information from users for redesign.
Thanks to all the attendees for making this workshop a lot of fun.
Caroline Jarrett @cjforms
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
A presentation for the Google Apps for Learning and Teaching (GALT) conference #galtshef. Two case studies on how Google hangouts have been used to enhance the learning experience.
The chicken or the Elgg? Developing a socially constructed self-paced learnin...Jason Rhode
Rhode, J. F. (2008, May 8). The chicken or the Elgg? Developing a socially constructed self-paced learning environment. Presented at the 2008 Sloan-C Internation Symposium on Emerging Technology Applications for Online Learning, Carefree, AZ.
Empowering YouTube for Higher Education3Play Media
This webinar will teach you how to leverage your YouTube videos to create an engaging learning portal for your students. Justin McCutcheon, the CEO and co-founder of Cattura, will walk you through the tools available for utilizing learning solutions that bridge Google Apps for Education, YouTube, and other Google services to create a rich video experience for your YouTube videos directly inside of your learning management system course.
This webinar will cover:
- An overview of video solutions with Google Apps for Education and YouTube
- Creating a well-organized YouTube course portal
- Delivering lecture capture and flipped classroom content to YouTube
- Making recorded and public YouTube videos accessible with closed captions
- Enriching YouTube videos with time-coded video metadata to drive notes, chapters, tags, and video search
- Creating a complete educational integration between Google Apps for Education and your LMS
- Leveraging Google tools such as YouTube annotations, video clipping, and Google Analytics to improve your educational video content
Pennsylvania State System of Higher Education Virtual Conference, February 16, 2016
Kim Kenward, Justin Melick and Rosemary Cleveland
Our conference presentation will help faculty and instructional designers identify technology resources and assignment design considerations for supporting online student presentations. This session will also provide information on the role of student project partners to facilitate student engagement opportunities for peer review, feedback and building online community.
University of Wisconsin: Captioning and Transcription Policies, Uses and Work...3Play Media
This slideshow comes from a presentation at UB Tech in June, 2014. The presentation was led by Josh Miller of 3Play Media and Patrick Wirth of University of Wisconsin-Extension, Continuing Education, Outreach & E-Learning (CEOEL). 3Play Media provides captioning and transcription services for UW-Extension.
CEOEL serves as the coordinator for University of Wisconsin continuing education programs at all 26 campuses. The presentation covered:
- Overview of CEOEL accessibility policies and procedures
- Overview of media team captioning and transcription workflows
- Discussion of CEOEL strategies for addressing accessibility issues/needs
- Showcase of creative uses for transcripts and captions around marketing initiatives
- Access to media players and templates (audio, video, narrations, etc.)
Using Twitter for Teaching, Learning, and Professional DevelopmentJason Rhode
Have you wondered what Twitter is and what if any practical applications there are for teaching and learning? Perhaps you are among the 30% of faculty who now use Twitter in some capacity and you would like to learn some tips and tricks for better utilizing Twitter in education context. During this online session offered 11/30/2012 we introduced the basics of Twitter and explored best practices for using Twitter in teaching, learning and professional development.
Mini-presentation within ELI's shortcourse: "Digging into Badges"
AUDIO: play mp3 accompanying these slides at: http://ofcoursesonline.com/badging%20at%20ucf%20audio.mp3
How to ask better questions and how to assess UX using surveys.
This workshop at UXLX 2014 in Lisbon was a deep dive into two important topics in survey design for user research.
We used the four-step model of how people answer questions to work on better questions, then we focused on two special uses of questionnaires in user research: the post-test assessment of satisfaction, and then how to gather information from users for redesign.
Thanks to all the attendees for making this workshop a lot of fun.
Caroline Jarrett @cjforms
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
A presentation for the Google Apps for Learning and Teaching (GALT) conference #galtshef. Two case studies on how Google hangouts have been used to enhance the learning experience.
The chicken or the Elgg? Developing a socially constructed self-paced learnin...Jason Rhode
Rhode, J. F. (2008, May 8). The chicken or the Elgg? Developing a socially constructed self-paced learning environment. Presented at the 2008 Sloan-C Internation Symposium on Emerging Technology Applications for Online Learning, Carefree, AZ.
Empowering YouTube for Higher Education3Play Media
This webinar will teach you how to leverage your YouTube videos to create an engaging learning portal for your students. Justin McCutcheon, the CEO and co-founder of Cattura, will walk you through the tools available for utilizing learning solutions that bridge Google Apps for Education, YouTube, and other Google services to create a rich video experience for your YouTube videos directly inside of your learning management system course.
This webinar will cover:
- An overview of video solutions with Google Apps for Education and YouTube
- Creating a well-organized YouTube course portal
- Delivering lecture capture and flipped classroom content to YouTube
- Making recorded and public YouTube videos accessible with closed captions
- Enriching YouTube videos with time-coded video metadata to drive notes, chapters, tags, and video search
- Creating a complete educational integration between Google Apps for Education and your LMS
- Leveraging Google tools such as YouTube annotations, video clipping, and Google Analytics to improve your educational video content
Pennsylvania State System of Higher Education Virtual Conference, February 16, 2016
Kim Kenward, Justin Melick and Rosemary Cleveland
Our conference presentation will help faculty and instructional designers identify technology resources and assignment design considerations for supporting online student presentations. This session will also provide information on the role of student project partners to facilitate student engagement opportunities for peer review, feedback and building online community.
University of Wisconsin: Captioning and Transcription Policies, Uses and Work...3Play Media
This slideshow comes from a presentation at UB Tech in June, 2014. The presentation was led by Josh Miller of 3Play Media and Patrick Wirth of University of Wisconsin-Extension, Continuing Education, Outreach & E-Learning (CEOEL). 3Play Media provides captioning and transcription services for UW-Extension.
CEOEL serves as the coordinator for University of Wisconsin continuing education programs at all 26 campuses. The presentation covered:
- Overview of CEOEL accessibility policies and procedures
- Overview of media team captioning and transcription workflows
- Discussion of CEOEL strategies for addressing accessibility issues/needs
- Showcase of creative uses for transcripts and captions around marketing initiatives
- Access to media players and templates (audio, video, narrations, etc.)
Using Twitter for Teaching, Learning, and Professional DevelopmentJason Rhode
Have you wondered what Twitter is and what if any practical applications there are for teaching and learning? Perhaps you are among the 30% of faculty who now use Twitter in some capacity and you would like to learn some tips and tricks for better utilizing Twitter in education context. During this online session offered 11/30/2012 we introduced the basics of Twitter and explored best practices for using Twitter in teaching, learning and professional development.
10 Tips for Implementing Accessible Online Media3Play Media
In 2016, it is more critical than ever to make your online media accessible: recent lawsuits and updated legal standards are expanding the reach of captioning, transcription, and video description requirements.
In this webinar, Janet Sylvia, Web Accessibility Group Leader and Web Accessibility Trainer, and Lily Bond from 3Play Media will go through 10 tips for implementing accessible online media at your institution. Looking at several different scenarios, they will discuss actionable strategies to help you find a solution that will work for you.
This presentation will cover:
Legal requirements, lawsuits, and standards for online media accessibility
What is required to make a video accessible?
Choosing an accessible video player, platform, or lecture capture system
Prioritizing which content to make accessible
Workflows for captioning, transcription, and video description
Budgeting and building a policy for accessible online media
Ready, Set, Record: Being Present and Engaging Students Online Using YouTubeJason Rhode
During this session at the 7th Annual International Symposium on Emerging Technologies for Online Learning, learn about one instructor's use of YouTube's free and easy-to-use features for recording, editing, captioning, and embedding video into his online course. An overview of the steps for recording, editing, captioning, video in YouTube will be provided as well as examples shared for various approaches for seamlessly incorporating video into any online course. While the session featured the integrated "video anywhere" YouTube features in Blackboard, the principles provided will be applicable to an online course in any learning management system. A summary of feedback survey results from students regarding their experiences with video in the course were shared as well as lessons learned by the instructor for those wishing to follow the same suggested steps for incorporating video in their own course. Accompanying presentation abstract is at http://sloanconsortium.org/conference/2014/et4online/ready-set-record-being-present-and-engaging-students-online-using-youtube and links at http://jasonrhode.com/et4online14
Building Online Community, Interaction, Collaboration, and Engagement through...maritezita
Maritez Apigo's presentation at the Strengthening Student Success Conference on October 8, 2015 at Oakland Marriott City Center
Eliminate the isolation barriers of technology and instead, integrate it to support student success. The presenter will showcase three technology tools and strategies for building community, increasing active participation, supporting social interaction, facilitating collaborative activities, and fostering engagement among students. The presenter will also demonstrate a simplified process for creating your own instructional videos by using free tools such as Screencast-O-Matic and YouTube. VoiceThread allows for asynchronous discussions about media through text, audio and video comments. Popular social media like Facebook extends community and engagement. The techniques modeled and examples shared are applicable to face-to-face, online, or hybrid courses. Tips and tricks for success with these technologies will be shared, and helpful resources for getting started and troubleshooting will be provided. Please bring your iPad or laptop to be able to fully participate in this session.
http://maritez.populr.me/sssc
Strategies and Tips for Engaging Today’s StudentsCengage Learning
Presented by: Pat Galitz, Business Administration faculty member at Southeast Community College
View our one-hour webinar with Pat Galitz as she shares strategies and tips for teaching online discussion–based courses. You’ll see examples of activities and assignments that really work in the online environment as well as innovative techniques for getting students engaged in your online course. We will show you various resources and tools that you can use including examples from 4LTR Press, a student-tested, faculty approved solution from Cengage Learning.
Sustainable Campus-Wide Captioning Practices to Support Course Videos – Is th...D2L Barry
Presentation at 2019 D2L Connection at Normandale CC on April 5,, 2019
Sustainable Campus-Wide Captioning Practices to Support Course Videos – Is this Really Possible? Lesley Blicker, Minnesota State Educational Innovations and Kathleen Coate, Normandale Community College
Interactive E-Lecture Using Video Annotation in Learning GroupsIJERA Editor
Now day‘s users are interested in distance learning as there is rapid growth in digital data due to day today
development in information as well as computer technology. Also its applications or usage have tremendous
response in market. Peoples are attracted towards interactivity in each thing, we found that for e-learning is a
very interactive way to learn and understand things. Currently, YouTube is the global way of video sharing. It is
having certain limitations such as, it having inactivity in online learning. In online study students expecting
some extra guidelines from given resources. In this project we developed video annotation system for foster
active learning. In this project, we achieved active participation of students. There is certain kind of technologies
that extracts some important keywords from textual information. MOOC‘s model is another technology to solve
interaction problem of users in active learning. It also has limitations that it suffered from the problem of
gamification. Our system is interactive as it provides real-time annotations to the video. In our system user can
give their active participation as they have direct interaction to our system. As part of our contribution in this
project we did SVM analysis to provide recommended videos for end users. SVM is Support Vector Machine
algorithm; it classifies the things according to user interest. So, in our system user can search for video and they
get recommended video list for their study.
Similar to Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos (20)
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos
1. Understanding In-Video Dropouts
and Interaction Peaks in
Online Lecture Videos
Juho Kim (MIT CSAIL)
Philip J. Guo (MIT CSAIL, U of Rochester)
Daniel T. Seaton (MIT Office of Digital Learning)
Piotr Mitros (edX)
Krzysztof Z. Gajos (Harvard EECS)
Robert C. Miller (MIT CSAIL)
2014.03.04
Learning at
Scale
3. Classrooms: rich, natural interaction data
Maria Fleischmann / Worldbank on Flickr | CC by-nc-nd
Love Krittaya | public domain
armgov on Flickr | CC by-nc-sa
unknown author | from pc4all.co.kr
5. How do learners use videos?
Data-Driven Approach:
Analyze learners’ interaction
with the video player
6. Why does data matter?
• detailed understanding of video usage
• design implications for
– Instructors
– Video editors
– Platform designers
• new video interfaces and formats
Improved video learning experience
7. How do learners use videos?
• Watch sequentially
• Pause
• Re-watch
• Skip / Skim
15. 36%: dropouts withinrate few seconds
55%: overall dropout first
viewership
36%
19%
55%
video time
Why?
• Auto-play playing unwanted videos
• Misleading video titles / interfaces
18. Tutorial videos lead to more dropouts
than lecture videos.
Lecture
• introduction to concepts
• continuous flow
Tutorial
• supplementary examples
• step-by-step demos
20. Interaction Peaks
Temporal peaks in the number of interaction
events, where a significant number of learners
show similar interaction patterns
interaction
events
video time
29. Five Explanations for
an Interaction Peak
Type 1. Beginning of new material
Type 2. Returning to content
Type 3. Tutorial step
Type 4. Replaying a segment
Type 5. Non-visual explanation
35. 61% of interaction peaks
involved a visual transition.
Peak Category
Frequency
Type 1. Beginning of new material
25%
Type 2. Returning to content
23%
Type 3. Tutorial step
7%
Type 4. Replaying a segment
6%
Type 5. Non-visual explanation
39%
61%
37. Lessons for instructors, video editors,
and platform designers
1. Make shorter videos.
2. Add informative titles and easy navigation.
3. Avoid abrupt visual transitions.
38. Lessons for instructors, video editors,
and platform designers
4. Make interaction peaks more accessible.
5. Enable one-click access for tutorial steps.
39. Next Steps: More Data Streams
• What would transcript / text add to the
analysis? How about acoustic data?
40. Next Steps: Scalability
• Reliably & automatically detect peak types?
• How much data is needed until we see patterns?
viewership
5 learners
video time
viewership
5,000 learners
video time
43. Contributions
• A first MOOC-scale in-video dropout rate
analysis
• A first MOOC-scale in-video interaction peak
analysis
• Categorization of learner activities
responsible for an interaction peak
• Data-driven design implications for video
authoring, editing, and interface design
44. Understanding In-Video Dropouts and
Interaction Peaks in Online Lecture Videos
Juho Kim
MIT CSAIL
juhokim@mit.edu
juhokim.com
47. Vision in learnersourcing
• Feedback loop between
– Learners: natural, pedagogically useful activities
– System: improve interaction using learner data
• Visualize and analyze large-scale video
learning activities
• Use data to inform learning platform design
48. How can we design
online video learning platforms
that are as effective as
in-person classrooms?