Using methodological triangulation to analyse students’ use of recorded lectures

•Download as PPT, PDF•

0 likes•473 views

This document describes a study that used methodological triangulation to analyze students' use of recorded lectures. The study collected data through surveys of over 500 students and log files of over 4,900 students accessing over 5,000 hours of video recordings. It found that while most students reported watching recordings at home and finding them useful, the log files showed some students' self-reports did not always match their actual usage as recorded in the logs. The study concluded that using both survey and log data provides a more accurate picture of how students use recorded lectures than either data source alone.

Education Technology

Using methodological triangulation
to analyse students’ use
of recorded lectures

July 7, 2012
Pierre Gorissen
Fontys University of Applied Sciences
The Netherlands

• PhD Research:
“Facilitating the use of recorded lectures”

• Pierre Gorissen (Fontys University of Applied Sciences)
• Dr. Jan van Bruggen (Open University of the Netherlands)
• Prof. Dr. Wim Jochems (Eindhoven University of Technology)

Recorded Lectures

• Recorded lectures:
• Many
• Long (45 min. average)
• Expensive to create

• On-campus students

Are students using/watching
these recordings?

Multi step approach

#1 Let’s ask them:
- Survey
- Interviews
#2 Measure their use:
- Mediasite reports
- Log files
#3 Compare the results
- Triangulation

#1 Let’s ask them

• 1,122 studenten (203 Fontys / 919 TU/e)
• 7 courses (1 Fontys / 6 TU/e)
• Response rate 46.1% (517 students)

• Follow up interviews (30 minutes) with 14
students

Main results

• Most students watch recordings at home;
• No major technical problems;
• They know where to find the recordings;
• They would prefer for all courses to be
recorded.

Main results

• Reasons to watch:
- Missed one or more lectures;
- Prepare for exams / improving test scores;
- Improve retention of lecture materials;
• Reasons not to watch:
- Already attended the live lecture;
- No time;
- Didn’t feel they missed anything.

Further reading on step #1

Gorissen, P., van Bruggen, J.M. and Jochems, W. (in press) ‘Students
and recorded lectures: survey on current use and demands for
higher education’, Research in Learning Technology.

#2 Measure the use

Logfiles

Combine
Missing Irrelevant Identify Identify Remove
data Combined data data users sessions Outliers
sources dataset

Data cleaning

Mediasite
database
Analyse Filtered
Log results
dataset total
dataset

The log data

• 1,500,000 lines of log data
• 8,000 recordings
• 5,000 hours of video
• 263 different courses
• 4,927 unique student users
• 48,539 learner sessions

Further reading on step #2

Gorissen, P., van Bruggen, J. and Jochems, W. (2012) ‘Usage
reporting on recorded lectures using educational data mining’, Int. J.
Learning Technology, Vol. 7, No. 1, pp.23–40.

#3 Compare the results

Logfiles

Combine
Missing Irrelevant Identify Identify Remove
data Combined data data users sessions Outliers
sources dataset

Data cleaning

Mediasite
database
Survey Compare Analyse Filtered
Log results
results results dataset total
dataset

!?

Reported versus actual use

Reported Actual

n (%) n (%)

Never 13 9.1 6 4.2

< 5 times 22 15.4 35 24.5

5-10 times 51 35.7 43 30.1

> 10 times 57 39.9 59 41.3

Number of times respondents used recorded lectures for the C01 course.

Reported versus actual use

Reported Actual

n (%) n (%)

0% - 10% 2 1.5 27 9.3

10% - 25% 4 3.1 203 69.8

25% - 50% 7 5.4 40 13.7

50% - 75% 26 20.0 13 4.5

75% - 100% 91 70.0 8 2.7

Average percentage of a recording viewed.

Conclusions

• Methodological triangulation is necessary
• Using the logged data, some reports by students:
• can be confirmed
• can be refuted
• can only be confirmed/refuted indirectly
• can not be confirmed/refuted

• …so we need both data sources!

Conclusions

• Don’t simply rely on what students tell you.
It might be just what they think you want to
hear!
• “Views” is a bad measure for use.
• Different recordings or different students
mean different use.

P.Gorissen@ fontys.nl

http://www.linkedin.com/in/PierreGorissen

This document provides an overview of quantitative method design, specifically experimental design. It discusses key concepts in experimental design including random assignment, control over extraneous variables, manipulation of treatment conditions, outcome measures, and threats to validity. It also describes different types of experimental designs including between-group designs like true experiments, quasi-experiments, and factorial designs as well as within-group designs like time series experiments, repeated measures experiments, and single subject experiments. The document provides examples and explanations of how to implement these different experimental designs.

Quantitative research

Tooba Kanwal

Triangulation

Institute of Surveying and Mapping

This document discusses triangulation, which is a surveying technique used to establish horizontal control networks over large areas. It involves measuring angles and lengths within networks of triangles. There are different orders of triangulation based on accuracy and area covered, including primary, secondary, and tertiary triangulation. Key aspects discussed include triangulation station layout and design, angle and distance measurements, controlling errors, and computation of unknown lengths and directions within triangles.

Triangulation

Sheila Shamuganathan

This document discusses different types of triangulation that can be used in research to increase validity and credibility. It defines triangulation as using multiple methods, data sources, investigators, theories or research contexts to check results. Specifically, it outlines data triangulation, investigator triangulation, theory triangulation, methodological triangulation and environmental triangulation. For each type, it provides an example of how it could be applied in practice to strengthen research findings.

Qualitative Research Method - an Introduction (updated jan 2011)

Hora Tjitra

This document provides an introduction to qualitative research methodology. It discusses key aspects of qualitative research such as what qualitative research refers to, common qualitative research paradigms, and differences between qualitative and quantitative research. The document also outlines the qualitative research process from developing research topics and questions to data collection and analysis. It provides examples of common qualitative research designs including field research, case studies, and action research. Data collection techniques in qualitative research like interviews and observation are also examined.

Mixed methods research

Khalid Mahmood

This document provides an overview of mixed methods research. It discusses the three main types of research designs: qualitative, quantitative, and mixed methods. It explains the differences between qualitative and quantitative research in terms of purpose, group studied, variables, data collection/analysis, and results. The document also discusses pragmatism as the philosophy behind mixed methods research and reasons for combining methods. It outlines various ways that qualitative and quantitative methods can be mixed, such as through timing, weighting, and mixing of data. The document concludes by describing six main mixed methods designs and recommending further readings on the topic.

The document describes research into developing software tools to support collaborative work in synthetic biology. User studies were conducted interviewing 32 people across various institutions. Findings from the studies showed needs such as alleviating data explosion, integrating workflow, providing multiple forms of evidence, supporting users from novices to experts, and facilitating collaboration. Prototypes were developed including G-nome Surfer Pro for literature search and primer design, Optimus Primer for primer design, PuppetShow for automating laboratory protocols, and eLabNotebook for recording experiments. Evaluation of the tools found they reduced workload, improved collaboration, and facilitated learning. The software aims to make synthetic biology more accessible and help attract new researchers.

Bb world 2012 Suzan Harkness It’s all about the data increasing student succe...

Suzan Harkness

This document discusses how the University of the District of Columbia used strategic partnerships, professional development, and data analysis to improve student outcomes in online courses. A needs assessment found that most faculty were non-users or minimal users of Blackboard. Workshops and online certification training increased faculty Blackboard skills and engagement over time. Content analysis of courses showed expanded use of Blackboard tools from 2010 to 2012. Partnering with Quality Matters improved online course design and reduced withdrawal and failure rates in online classes compared to traditional courses. The strategic approach focused on quality over quantity and expanded faculty capacity through collaborative professional development.

Workflows, provenance and reporting: a lifecycle perspective at BIH 2013, Rome

Carole Goble

Workflow systems support the design, configuration and execution of repetitive, multi-step pipelines and analytics, well established in many disciplines, notably biology and chemistry, but less so in biodiversity and ecology. From an experimental perspective workflows are a means to handle the work of accessing an ecosystem of software and platforms, manage data and security, and handle errors. From a reporting perspective they are a means to accurately document methodology for reproducibility, comparison, exchange and reuse, and to trace the provenance of results for review, credit, workflow interoperability and impact analysis. Workflows operate in an evolving ecosystem and are assemblages of components in that ecosystem; their provenance trails are snapshots of intermediate and final results. Taking a lifecycle perspective, what are the challenges in workflow design and use with different stakeholders? What needs to be tackled in evolution, resilience, and preservation? And what are the “mitigate or adapt” strategies adopted by workflow systems in the face of changes in the ecosystem/environment, for example when tools are depreciated or datasets become inaccessible in the face of funding shortfalls?

Metadata Quality Issues in Learning Repositories

Nikos Palavitsinis, PhD

Developing National On-Line HIV Prevention Program Evaluation Training

CDC NPIN

The document describes the development of an online HIV prevention program evaluation training for grantees. It details the history of moving from in-person trainings to developing online training modules and webinars. Usage data is provided showing over 1,700 training lessons completed and high satisfaction rates among participants. Lessons learned include webinars promoting online training usage and the need for the training to evolve with changes in HIV prevention.

Students’ perceptions and perceived value of lecture capture in teacher educa...

Ilkka Kukkonen

This document summarizes research on students' perceptions of lecture capture technology in teacher education. A study was conducted surveying over 100 students on their use of recorded lectures. Key findings include: 1) Most students wanted to increase their use of lecture capture and would recommend it to peers, seeing benefits for independent study, reviewing difficult concepts, and exam preparation. 2) Lecture capture seemed most helpful for lower achieving students by supporting understanding and study strategies. 3) Further development is needed to help students learn how to effectively use recordings and integrate lecture capture pedagogically. 4) Additional research was proposed to better understand the impact of lecture capture on learning outcomes and develop engaging formats that supplement live

Multimodal Learning Analytics

Xavier Ochoa

This document discusses multimodal learning analytics (MLA), which examines learning through multiple modalities like video, audio, digital pens, etc. It provides examples of extracting features from these modalities to analyze problem-solving sessions. Video features like total movement, distance from table, and calculator tracking are described. Audio features like speech duration and word counts are mentioned. Digital pen features like strokes, pressure, and shapes are examined. The document concludes that MLA has much potential to explore learning in more realistic settings compared to traditional learning analytics.

Using Metrics and Cluster Analysis in Video Based Learning

Alexandros Kleftodimos

- The document describes research analyzing learner video viewing behaviors in online educational videos using metrics and cluster analysis. - Learners and videos were clustered based on engagement metrics and popularity metrics. For learners, three clusters preferred linear videos while one cluster engaged equally with linear and interactive videos. - Two video clusters differentiated high and low interest videos. Viewing patterns were also clustered, with the largest being sequential viewing and another allowing non-sequential viewing using stop/resume and backward jumps. - Future work includes interviewing learners from each cluster and deploying open source tools to provide more meaningful visualizations and aggregate diverse online content.

Slide 1

Videoguy

This document summarizes a case study on using web-based video conferencing for a professional doctorate interview. It discusses the context of the study, aims to develop guidelines for effective use of this technology in interviews. Key findings address pedagogical, technical and policy factors to consider, such as sound checks, identity validation, location issues, collecting feedback, and a need for training materials and reliable network speeds. The summary proposes establishing induction processes and fixed video conferencing rooms to support this type of work-based learning.

Slide 1

Videoguy

This document summarizes a case study on using web-based video conferencing for a professional doctorate interview. It discusses the context of the study, aims to develop guidelines for effective use of this technology in interviews. Key findings address pedagogical, technical and policy factors to consider, such as sound checks, backup strategies, validating identity, collecting feedback, and establishing dedicated video conferencing rooms. The summary provides guidelines for network testing, induction materials, camera setup and policies to support working remotely.

Understand students’ self-reflections through learning analytics

Vitomir Kovanovic

Using electronic laboratory notebooks in the academic life sciences: a group ...

SC CTSI at USC and CHLA

This document summarizes a webinar on using electronic laboratory notebooks (eLNs). The webinar featured a presentation by Dr. Ulrich Dirnagl on his experience using eLNs to make research teams more efficient. He believes paper notebooks are outdated and that eLNs can help address the reproducibility crisis in research. The webinar covered the benefits of eLNs like collaboration, data sharing, and compliance with regulations. It also reviewed different types of eLNs and pricing models. While implementation challenges exist, eLNs were found to improve oversight, record keeping, and transparency if selected and supported properly.

Evidence-based Semantic WebJust a Dream or the Way to Go?

Dragan Gasevic

The Semantic Web vision emerged with a promise to collect and interlink semantically relevant data from diverse sources in order to to achieve a full potential of the Web. After more than a decade of diligent research, it is the time to start summing up what has been accomplished and how mature Semantic Web research is, so that plans for the future can be charted. One of the key trails of a mature discipline is to have well-designed research methods allowing researchers to establish evidence about the effectiveness of the research ideas. It is equally important to to have knowledge translation methods that allow for transferring the established evidence to decision makers in practice. In this talk, we will first share some experience and challenges in conducting experiments in the area of the Semantic Web. We will next discuss findings of systematic reviews conducted to estimate the level of quality of the existing research results based on the criteria well-known in medical research and recently adopted in empirical software engineering. We will conclude the talk by discussing the importance and potential milestones for the Semantic Web in order to become an evidence-based discipline (similar to medicine or education) capable of producing strong research evidence transferable to practice.

Pain points for preservation services / workflows in repositories

prwheatley

Paul Wheatley discusses key challenges for digital preservation practitioners based on the SPRUCE project. He identifies several themes, including the need for (1) quality assurance tools to check for data flaws, (2) improved characterization for appraisal and ingest preparation, (3) locating preservation-worthy data, (4) checking conformance to institutional policies, and (5) identifying preservation risks. He emphasizes the importance of taking an evidence-based, practitioner-driven approach and reusing existing tools and knowledge rather than reinventing solutions.

Towards Reading Session-based indicators in Educational Reading Analytics

Madjid Sadallah

Your Learners ARE Mobile. Is Your Learning?

Judy Brown

This document discusses mobile learning and how it is becoming increasingly important as more learners use mobile devices. It begins with an introduction to audience response systems that can be used for mobile polling and feedback. It then discusses the growing capabilities and use of mobile phones before examining definitions and examples of mobile learning. The document considers the opportunities mobile learning provides and looks at several case studies. It concludes by discussing future directions such as location-based learning and augmented reality before opening for questions.

Curation-Friendly Tools for the Scientific Researcher

bwestra

Tim Malthus_Towards standards for the exchange of field spectral datasets

TERN Australia

This document discusses the development of standards for the exchange of field spectral datasets. It notes the importance of metadata for determining the quality and representativeness of spectral data obtained in the field. A workshop was held in 2012 to discuss best practices for data collection and exchange and key conclusions included the need for standards to facilitate accurate comparison across studies and the role of thorough metadata. Work is ongoing to enhance the SPECCHIO system for hosting spectral libraries and metadata and establishing it as the international tool for storage and exchange of spectral datasets.

Read Between The Lines: an Annotation Tool for Multimodal Data

Daniele Di Mitri

This is the presentation of Read Between The Lines, the paper which we published at the Learning Analytics & Knowledge Conference 2019 in Tempe, Arizona (#LAK19). Link to the paper available in Open Access ACM library https://dl.acm.org/citation.cfm?id=3303776 Abstract: This paper introduces the Visual Inspection Tool (VIT) which supports researchers in the annotation of multimodal data as well as the processing and exploitation for learning purposes. While most of the existing Multimodal Learning Analytics (MMLA) solutions are tailor-made for specific learning tasks and sensors, the VIT addresses the data annotation for different types of learning tasks that can be captured with a customisable set of sensors in a flexible way. The VIT supports MMLA researchers in 1) triangulating multimodal data with video recordings; 2) segmenting the multimodal data into time-intervals and adding annotations to the time-intervals; 3) downloading the annotated dataset and using it for multimodal data analysis. The VIT is a crucial component that was so far missing in the available tools for MMLA research. By filling this gap we also identified an integrated workflow that characterises current MMLA research. We call this workflow the Multimodal Learning Analytics Pipeline, a toolkit for orchestration, the use and application of various MMLA tools.

Erfaringer med Remote Usability Testing af Jan Stage, AAU

InfinIT - Innovationsnetværket for it

Inspiratiesessie AI in Education Hackathon

Using methodological triangulation to analyse students’ use of recorded lectures

Recommended

Recommended

More Related Content

Similar to Using methodological triangulation to analyse students’ use of recorded lectures

Similar to Using methodological triangulation to analyse students’ use of recorded lectures (20)

More from Pierre Gorissen

More from Pierre Gorissen (20)

Recently uploaded

Recently uploaded (20)

Using methodological triangulation to analyse students’ use of recorded lectures

Editor's Notes