With the introduction of the Teaching Excellence Framework a lot of attention is focussed on measuring learning gains. A vast body of research has found that individual student characteristics influence academic progression over time. This case-study aims to explore how advanced statistical techniques in combination with Big Data can be used to provide potentially new insights into how students are progressing over time, and in particular how students’ socio-demographics (i.e., gender, ethnicity, socioeconomic status, prior educational qualifications) influence students’ learning trajectories
Professor Bart Rienties, Open University UK
https://warwick.ac.uk/services/aro/dar/quality/legacy/anagendaforchange/
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...Bart Rienties
In the last five years there is an increased interest across the globe to define, conceptualise, and measure learning gains. The concept of learning gains, briefly summarised as the improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education, has been hailed by some as an opportunity to measure “excellence” in teaching. However, whether learning gains could be useful for quality assurance can be debated. This interactive workshop aims to provide an open platform to
discuss the opportunities and limitations of learning gains for quality assurance.
Lessons learned from 200K students and 2 GB of learning gains data.Bart Rienties
OfS national conference on learning gains, Birmingham, 12 March 2019
Student Participation: how can learning gain data help students from all backgrounds access, succeed and proceed in higher education @learninggains @officeforstudents
https://abclearninggains.com/
The document compares course evaluation ratings between online and traditional courses. Contrary to expectations, the study found no significant differences in ratings for most items, including those referring to in-class procedures. The only significant difference was that students in online courses perceived a higher workload than those in traditional courses, possibly because online students consider any work for the class as "outside of class." Overall, the results suggest that instructors are viewed similarly in online and traditional courses.
This document outlines a study that aims to develop, implement, and evaluate a blended learning approach within an undergraduate physiotherapy curriculum. The study will assess appropriate teaching strategies and stakeholder attitudes, analyze curriculum alignment and module appropriateness, develop a blended learning module, and implement and evaluate the module. The methodology will include a literature review, surveys, document analysis, and process evaluation. The goal is to promote student-centered, inquiry-based, and self-directed learning through a blended approach.
The document summarizes a study examining the relationships between critical thinking, self-efficacy, self-regulated learning, and academic performance in university students. It discusses prior research showing these factors positively influence learning and performance. The study aimed to examine relationships between critical thinking, self-efficacy, self-regulated learning and course grades/GPA in nutrition and education students. Surveys were used to collect data and analyses found significant associations between self-efficacy and grades for nutrition majors and between critical thinking and grades.
Kara J. Beard studied the effectiveness of a math intervention implemented with 55 students who needed extra support in math, approximately 20% of the students studied. The intervention included modeling, guided practice, and frequent feedback based on articles reviewing acquisition interventions and fluency interventions. The conclusion was that the intervention was effective based on data collected, providing additional practice, modeling, and feedback improved students' math skills.
Constructivist Approach Vs Expository Teaching: Exponential Functionsinventionjournals
This document summarizes a study that compared the effects of expository teaching and constructivist teaching approaches on students' understanding of exponential functions. The study involved 50 10th grade students split into two classes, one taught using expository methods and one taught using constructivist activities. An assessment given after found that both groups struggled with conceptual questions about exponential functions, such as writing the domain and range or determining if an exponential function is one-to-one. The constructivist approach was intended to support active learning but students may have preferred rote memorization due to testing pressures. Overall, neither approach significantly improved students' conceptual grasp of exponential functions.
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...Bart Rienties
In the last five years there is an increased interest across the globe to define, conceptualise, and measure learning gains. The concept of learning gains, briefly summarised as the improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education, has been hailed by some as an opportunity to measure “excellence” in teaching. However, whether learning gains could be useful for quality assurance can be debated. This interactive workshop aims to provide an open platform to
discuss the opportunities and limitations of learning gains for quality assurance.
Lessons learned from 200K students and 2 GB of learning gains data.Bart Rienties
OfS national conference on learning gains, Birmingham, 12 March 2019
Student Participation: how can learning gain data help students from all backgrounds access, succeed and proceed in higher education @learninggains @officeforstudents
https://abclearninggains.com/
The document compares course evaluation ratings between online and traditional courses. Contrary to expectations, the study found no significant differences in ratings for most items, including those referring to in-class procedures. The only significant difference was that students in online courses perceived a higher workload than those in traditional courses, possibly because online students consider any work for the class as "outside of class." Overall, the results suggest that instructors are viewed similarly in online and traditional courses.
This document outlines a study that aims to develop, implement, and evaluate a blended learning approach within an undergraduate physiotherapy curriculum. The study will assess appropriate teaching strategies and stakeholder attitudes, analyze curriculum alignment and module appropriateness, develop a blended learning module, and implement and evaluate the module. The methodology will include a literature review, surveys, document analysis, and process evaluation. The goal is to promote student-centered, inquiry-based, and self-directed learning through a blended approach.
The document summarizes a study examining the relationships between critical thinking, self-efficacy, self-regulated learning, and academic performance in university students. It discusses prior research showing these factors positively influence learning and performance. The study aimed to examine relationships between critical thinking, self-efficacy, self-regulated learning and course grades/GPA in nutrition and education students. Surveys were used to collect data and analyses found significant associations between self-efficacy and grades for nutrition majors and between critical thinking and grades.
Kara J. Beard studied the effectiveness of a math intervention implemented with 55 students who needed extra support in math, approximately 20% of the students studied. The intervention included modeling, guided practice, and frequent feedback based on articles reviewing acquisition interventions and fluency interventions. The conclusion was that the intervention was effective based on data collected, providing additional practice, modeling, and feedback improved students' math skills.
Constructivist Approach Vs Expository Teaching: Exponential Functionsinventionjournals
This document summarizes a study that compared the effects of expository teaching and constructivist teaching approaches on students' understanding of exponential functions. The study involved 50 10th grade students split into two classes, one taught using expository methods and one taught using constructivist activities. An assessment given after found that both groups struggled with conceptual questions about exponential functions, such as writing the domain and range or determining if an exponential function is one-to-one. The constructivist approach was intended to support active learning but students may have preferred rote memorization due to testing pressures. Overall, neither approach significantly improved students' conceptual grasp of exponential functions.
Enhancing (in)formal learning ties in interdisciplinary management courses: a...Bart Rienties
While interdisciplinary courses are regarded as a promising method for students to learn and apply knowledge from other disciplines, there is limited empirical evidence available whether interdisciplinary courses can effectively “create” interdisciplinary students. In this innovative quasi-experimental study amongst 377 Master’s students, in the control condition students were randomised by the teacher into groups, while in the experimental condition students were “balanced” by the teacher into groups based upon their initial social network. Using Social Network Analysis, learning ties after eleven weeks were significantly predicted by the friendship and learning ties established at the beginning of the course, as well as (same) discipline and group allocation. The effects were generally greater than group divisions, irrespective of the two conditions, but substantially smaller than initial social networks. These results indicate that interdisciplinary learning does not occur “automatically” in an interdisciplinary module. This study contributes to effective learning in interdisciplinary learning environments.
Rienties, B., & Héliot, Y. (2016). Enhancing (in)formal learning ties in interdisciplinary management courses: a quasi-experimental social network study. Studies in Higher Education. DOI: 10.1080/03075079.2016.1174986. Impact factor: 1.037.
Full version is available at: http://www.tandfonline.com/doi/full/10.1080/03075079.2016.1174986
Keynote SEC2019 Leeds: The power of learning analytics to impact learning and...Bart Rienties
1. The document discusses the power of learning analytics to impact learning and teaching from a critical perspective.
2. It references research showing that learning design and teachers strongly influence student engagement, satisfaction, and performance based on analyses of over 150 modules.
3. Learning analytics approaches were found to help understand the complexities of learning inside and outside the classroom, and can provide insights to researchers and practitioners to test educational theories.
Three sentences summarizing the key points:
1) While policies aim to improve readiness through rigorous coursework and test scores, students' grades are actually better predictors of their future success than test scores or courses taken. 2) Grades reflect not just content knowledge and skills, but also important noncognitive factors like behaviors, attitudes, strategies, and relationships that support school performance. 3) Noncognitive factors beyond just cognitive abilities and knowledge, such as motivation, self-control, and problem-solving skills, are critical influences on students' academic outcomes.
THE INFLUENCE OF PROBLEM-BASED LEARNING COMMUNITIES ON RESEARCH LITERACY AND ...ijejournal
The current study investigates two Problem-Based Learning (PBL) processes that were carried out in two different Online Learning Communities of 62 pre-service teachers who took a Research Literacy course as a part of their academic obligation. The first one was combined with the moderator based learning
scaffoldings (OLC+M), and the other one with the social based learning scaffoldings (OLC+S). The study seeks to map the differences between these two OLCs in terms of Achievement Goal Motivation and Research Literacy skills as a result of the PBL intervention, and the correlation between these aspects as is expressed in each group. The findings indicated that PBL had a significant positive effect on AGM in both groups, while only the OLC+S showed the significant outperforming in some of the Research Literacy skills, as well as the positive correlation between them and the Mastery Approach component of AGM. The discussion raises possible interpretations of theoretical and practical relationships between Research Literacy skills in the educational field and motivational factors among adult students, as they are expressed in online communication environments.
ASCILITE Webinar: A review of five years of implementation and research in al...Bart Rienties
Date and time: Wednesday 20 September 2017 at 5pm AEST
Abstract: The Open University UK (OU) has been one of few institutions that have explicitly and systematically captured the designs for learning at a large scale. By applying advanced analytical techniques on large and fine-grained datasets, we have been unpacking the complexity of instructional practices, as well as providing empirical evidence of how learning designs influence student behaviour, satisfaction, and performance. This seminar will discuss the implementation of learning design at the OU in the last 5 years, and reviews empirical evidence from several studies that have linked learning design with learning analytics. Recommendations are put forward to support future adoptions of the learning design approach, and potential research trajectories.
https://ascilite.org/get-involved/sigs/learning-analytics-sig/
www.bartrienties.nl
This document summarizes Amy Ballew's Education Specialist project which examined the effects of multiple intelligence-based strategies on fifth grade math students' learning. The study involved two groups of 24 students - a control group that received traditional instruction and an experimental group that received instruction based on multiple intelligences. Pre- and post-tests showed the experimental group had a statistically significant larger increase in scores, supporting the idea that MI strategies can positively impact student learning. The discussion notes this aligns with other findings and suggests MI theory better engages 21st century learners.
Metacognitive Strategies: Instructional Approaches in Teaching and Learning o...IJAEMSJORNAL
The purpose of the study is to determine the effectiveness of the metacognitive strategies as instructional approaches in teaching and learning of Basic Calculus. A number of 48 students consisting of 24 boys and 24 girls were purposively sampled in this study. Pretest-posttest quasi experimental research design was used which applied t-test and descriptive statistics. Both groups were subject to two instruments that were comprised of problem-solving test (pretest and posttest) and observation guide. Experimental group was taught Basic Calculus using metacognitive strategies while the control group was taught Basic Calculus using traditional teaching strategies. Both groups were subject to a pretest. Class observation was done while the two teaching strategies were applied. In the end, the posttest was administered to both groups to identify the effectiveness of the two teaching strategies. The data gathered were treated using paired sample t-test and independent sample t-test. The results of the study showed that the experimental group had significantly higher posttest scores as compared to control group which proved that metacognitive teaching strategies were more effective in improving the performance and problem-solving skills of the students than the traditional teaching strategies. It was also observed that students who taught using metacognitive strategies helped the students to be extremely engaged in Basic Calculus lessons cognitively, behaviorally, and affectively. The study reveals that the significant increase of the students’ learning engagement in Basic Calculus lessons led the students to a corresponding increase in their posttest scores.
The Power of Learning Analytics: Is There Still a Need for Educational Research?Bart Rienties
Across the globe many institutions and organisations have high hopes that learning analytics can play a major role in helping their organisations remain fit-for-purpose, flexible, and innovative. A broad goal of learning analytics is to apply the outcomes of analysing data gathered by monitoring and measuring the learning process. Learning analytics applications in education are expected to provide institutions with opportunities to support learner progression, but more importantly provide personalised, rich learning on a large scale. Substantial progress in learning analytics research has been made in the last few years.
Researchers in learning analytics use a range of advanced computational techniques (e.g., Bayesian modelling, cluster analysis, natural language processing, machine learning) for predicting which learners are likely to fail or succeed, and how to provide appropriate support in a flexible and adaptive manner.
In this keynote, I will argue that unless educational researchers at EARLI embrace some of the key principles, methods, and approaches of learning analytics, educational researchers may be left behind. In particular, a main merit of learning analytics is linking large datasets of actual learning processes and outcomes with learning dispositions and learner characteristics. Using evidence-based approaches rapid insights and advancements are developed how learning designs and learning processes can be optimised to maximise the potential of each learner. For example, our recent research with 151 modules and 133K students at the Open University UK indicates that learning design has a strong impact on student behaviour, satisfaction, and performance. Learning analytics can also drive learning in more “traditional”, face-to-face contexts. For example, by measuring emotions, epistemological expressions, and cross-cultural dialogue, social interactions can be effectively supported by innovative dashboards and adaptive
approaches. I aim to unpack the advantages and limitations of learning analytics and how EARLI researchers can embrace such data-driven research approaches
More info at www.bartrienties.nl
This document reviews empirical studies on the impact of interteaching (IT), a behavioral teaching method, on college student performance. IT incorporates components like prep guides, pair discussions, record sheets, clarifying lectures, and test probes. The review found that IT improved academic performance across various disciplines and class settings. It concluded that IT attributes student success to its emphasis on mental readiness through prep guides and active dialogue during pair discussions. Future research could analyze the impact of specific IT components and their use in different environments and demographics.
This study analyzed social network data from students in a master's program over 10 years to investigate the relationship between social ties and academic performance. Descriptive statistics showed no missing data from the institutional records. Regression analysis found that higher closeness and eccentricity centrality were positively associated with GPA, supporting the hypothesis that greater social capital is linked to better performance. Further, students in the top two quartiles for social capital had significantly higher GPAs than those in the bottom two quartiles, indicating cross-class social ties benefit academic achievement.
Predictors of Success: Linking Student Achievement to School and Educator Successes through Professional Learning
This study show how some schools have seen a dramatic increase in student achievement after developing a strong, online professional learning program.
In this study, an assistant refers to a person who helps the teacher in the classroom but does not have
full teaching responsibilities.
Blended Learning: Refers to a hybrid of face-to-face and online learning where a portion of the content is
delivered online and students have some control over time, place, path and/or pace of their learning.
Communication Skills: Refers to the ability to effectively convey information through the use of verbal and
non-verbal means. It includes listening, speaking, reading and writing.
Online Quizzes: Refers to assessment activities designed to evaluate students’ mastery of lessons delivered
through the internet.
Online Games: Refers to interactive activities
This document summarizes a systematic literature review on monitoring, awareness and reflection in blended learning. It begins with definitions of key terms and motivation for the review. The methodology section describes using a systematic review process. The results section outlines the main findings in 5 areas: learning contexts studied, research problems investigated, solutions provided, evaluation maturity, and open issues. Key findings include blended learning being interpreted differently, existing solutions mainly for teachers in universities, and open areas around stakeholders, data sources, and longer-term evaluations.
The document examines student-centered learning practices in New England high schools. It finds that practices within the personalized learning and student ownership tenets are most prevalent, while anytime/anywhere learning practices lag behind. Schools also report that competency-based learning models are more challenging to adopt than other student-centered practices, facing barriers around annual student advancement and college acceptance of competency-based transcripts.
This literature review examines how online learning aligns with principles of adult learning (andragogy). It summarizes research finding that:
1) Younger adult learners are motivated by competition while older adults are motivated by personal growth and belonging.
2) Incorporating audio into online modules reduces cognitive load and improves learning compared to text-based modules.
3) Adult learners value self-directed learning and their love of learning correlates with online learning success.
Providing personalised student support in blended learning at scaleAbelardo Pardo
Blended learning environments can be used to deploy strategies to increase student engagement in learning experiences. However, for these strategies to be effective, this increase in engagement requires an increase in student support which can pose serious challenges for large cohorts. The increase in technology mediation offers unprecedented opportunities to collect information
about how students interact in a learning environment. Can this data be used to provide student support at scale? Is it feasible to blend data management techniques as part of a learning design to provide personalised suggestions to students? This talk will offer various practical examples of personalised
student support actions in the context of a large flipped classroom.
The power of learning analytics to measure learning gains: an OU, Surrey and ...Bart Rienties
Learning gains has increasingly become apparent within the HE literature, gained traction in government policies in the UK, and are at the heart of Teaching Excellence Framework (TFL). As such, this raises a question to what extent teaching and learning environment can actually predict students’ learning gains using principles of learning analytics. In this presentation, which is joined work with University of Surrey and Oxford Brookes, I will focus on some preliminary findings based upon developing and testing an Affective-Behaviour-Cognition learning gains model using longitudinal approach. The main aim of the research is to examine whether learning gains occur on all three levels of Affective-Behaviour-Cognition model and whether any particular student or course characteristics can predict learning gains or lack of learning and dropout. For more info, see https://abclearninggains.com/
ACTIVE LEARNING STRATEGIES AND HIGHER-ORDER THINKING SKILLS OF GRADE 10 STUDENTSJoe Osborn
This document summarizes a study that examined the effects of two active learning strategies - case-based learning and visual-organization activity - on the higher-order thinking skills of Grade 10 students in the Philippines. The study used an experimental design with two groups of 50 students each that were exposed to one of the two learning strategies. Both groups showed improvement in critical and creative thinking skills after the learning strategies were applied, as measured by pre- and post-tests. The case-based learning group performed significantly better on the critical thinking portion of the post-test compared to the visual-organization group. The study found that both active learning strategies helped improve students' higher-order thinking skills.
Enhancing (in)formal learning ties in interdisciplinary management courses: a...Bart Rienties
While interdisciplinary courses are regarded as a promising method for students to learn and apply knowledge from other disciplines, there is limited empirical evidence available whether interdisciplinary courses can effectively “create” interdisciplinary students. In this innovative quasi-experimental study amongst 377 Master’s students, in the control condition students were randomised by the teacher into groups, while in the experimental condition students were “balanced” by the teacher into groups based upon their initial social network. Using Social Network Analysis, learning ties after eleven weeks were significantly predicted by the friendship and learning ties established at the beginning of the course, as well as (same) discipline and group allocation. The effects were generally greater than group divisions, irrespective of the two conditions, but substantially smaller than initial social networks. These results indicate that interdisciplinary learning does not occur “automatically” in an interdisciplinary module. This study contributes to effective learning in interdisciplinary learning environments.
Rienties, B., & Héliot, Y. (2016). Enhancing (in)formal learning ties in interdisciplinary management courses: a quasi-experimental social network study. Studies in Higher Education. DOI: 10.1080/03075079.2016.1174986. Impact factor: 1.037.
Full version is available at: http://www.tandfonline.com/doi/full/10.1080/03075079.2016.1174986
Keynote SEC2019 Leeds: The power of learning analytics to impact learning and...Bart Rienties
1. The document discusses the power of learning analytics to impact learning and teaching from a critical perspective.
2. It references research showing that learning design and teachers strongly influence student engagement, satisfaction, and performance based on analyses of over 150 modules.
3. Learning analytics approaches were found to help understand the complexities of learning inside and outside the classroom, and can provide insights to researchers and practitioners to test educational theories.
Three sentences summarizing the key points:
1) While policies aim to improve readiness through rigorous coursework and test scores, students' grades are actually better predictors of their future success than test scores or courses taken. 2) Grades reflect not just content knowledge and skills, but also important noncognitive factors like behaviors, attitudes, strategies, and relationships that support school performance. 3) Noncognitive factors beyond just cognitive abilities and knowledge, such as motivation, self-control, and problem-solving skills, are critical influences on students' academic outcomes.
THE INFLUENCE OF PROBLEM-BASED LEARNING COMMUNITIES ON RESEARCH LITERACY AND ...ijejournal
The current study investigates two Problem-Based Learning (PBL) processes that were carried out in two different Online Learning Communities of 62 pre-service teachers who took a Research Literacy course as a part of their academic obligation. The first one was combined with the moderator based learning
scaffoldings (OLC+M), and the other one with the social based learning scaffoldings (OLC+S). The study seeks to map the differences between these two OLCs in terms of Achievement Goal Motivation and Research Literacy skills as a result of the PBL intervention, and the correlation between these aspects as is expressed in each group. The findings indicated that PBL had a significant positive effect on AGM in both groups, while only the OLC+S showed the significant outperforming in some of the Research Literacy skills, as well as the positive correlation between them and the Mastery Approach component of AGM. The discussion raises possible interpretations of theoretical and practical relationships between Research Literacy skills in the educational field and motivational factors among adult students, as they are expressed in online communication environments.
ASCILITE Webinar: A review of five years of implementation and research in al...Bart Rienties
Date and time: Wednesday 20 September 2017 at 5pm AEST
Abstract: The Open University UK (OU) has been one of few institutions that have explicitly and systematically captured the designs for learning at a large scale. By applying advanced analytical techniques on large and fine-grained datasets, we have been unpacking the complexity of instructional practices, as well as providing empirical evidence of how learning designs influence student behaviour, satisfaction, and performance. This seminar will discuss the implementation of learning design at the OU in the last 5 years, and reviews empirical evidence from several studies that have linked learning design with learning analytics. Recommendations are put forward to support future adoptions of the learning design approach, and potential research trajectories.
https://ascilite.org/get-involved/sigs/learning-analytics-sig/
www.bartrienties.nl
This document summarizes Amy Ballew's Education Specialist project which examined the effects of multiple intelligence-based strategies on fifth grade math students' learning. The study involved two groups of 24 students - a control group that received traditional instruction and an experimental group that received instruction based on multiple intelligences. Pre- and post-tests showed the experimental group had a statistically significant larger increase in scores, supporting the idea that MI strategies can positively impact student learning. The discussion notes this aligns with other findings and suggests MI theory better engages 21st century learners.
Metacognitive Strategies: Instructional Approaches in Teaching and Learning o...IJAEMSJORNAL
The purpose of the study is to determine the effectiveness of the metacognitive strategies as instructional approaches in teaching and learning of Basic Calculus. A number of 48 students consisting of 24 boys and 24 girls were purposively sampled in this study. Pretest-posttest quasi experimental research design was used which applied t-test and descriptive statistics. Both groups were subject to two instruments that were comprised of problem-solving test (pretest and posttest) and observation guide. Experimental group was taught Basic Calculus using metacognitive strategies while the control group was taught Basic Calculus using traditional teaching strategies. Both groups were subject to a pretest. Class observation was done while the two teaching strategies were applied. In the end, the posttest was administered to both groups to identify the effectiveness of the two teaching strategies. The data gathered were treated using paired sample t-test and independent sample t-test. The results of the study showed that the experimental group had significantly higher posttest scores as compared to control group which proved that metacognitive teaching strategies were more effective in improving the performance and problem-solving skills of the students than the traditional teaching strategies. It was also observed that students who taught using metacognitive strategies helped the students to be extremely engaged in Basic Calculus lessons cognitively, behaviorally, and affectively. The study reveals that the significant increase of the students’ learning engagement in Basic Calculus lessons led the students to a corresponding increase in their posttest scores.
The Power of Learning Analytics: Is There Still a Need for Educational Research?Bart Rienties
Across the globe many institutions and organisations have high hopes that learning analytics can play a major role in helping their organisations remain fit-for-purpose, flexible, and innovative. A broad goal of learning analytics is to apply the outcomes of analysing data gathered by monitoring and measuring the learning process. Learning analytics applications in education are expected to provide institutions with opportunities to support learner progression, but more importantly provide personalised, rich learning on a large scale. Substantial progress in learning analytics research has been made in the last few years.
Researchers in learning analytics use a range of advanced computational techniques (e.g., Bayesian modelling, cluster analysis, natural language processing, machine learning) for predicting which learners are likely to fail or succeed, and how to provide appropriate support in a flexible and adaptive manner.
In this keynote, I will argue that unless educational researchers at EARLI embrace some of the key principles, methods, and approaches of learning analytics, educational researchers may be left behind. In particular, a main merit of learning analytics is linking large datasets of actual learning processes and outcomes with learning dispositions and learner characteristics. Using evidence-based approaches rapid insights and advancements are developed how learning designs and learning processes can be optimised to maximise the potential of each learner. For example, our recent research with 151 modules and 133K students at the Open University UK indicates that learning design has a strong impact on student behaviour, satisfaction, and performance. Learning analytics can also drive learning in more “traditional”, face-to-face contexts. For example, by measuring emotions, epistemological expressions, and cross-cultural dialogue, social interactions can be effectively supported by innovative dashboards and adaptive
approaches. I aim to unpack the advantages and limitations of learning analytics and how EARLI researchers can embrace such data-driven research approaches
More info at www.bartrienties.nl
This document reviews empirical studies on the impact of interteaching (IT), a behavioral teaching method, on college student performance. IT incorporates components like prep guides, pair discussions, record sheets, clarifying lectures, and test probes. The review found that IT improved academic performance across various disciplines and class settings. It concluded that IT attributes student success to its emphasis on mental readiness through prep guides and active dialogue during pair discussions. Future research could analyze the impact of specific IT components and their use in different environments and demographics.
This study analyzed social network data from students in a master's program over 10 years to investigate the relationship between social ties and academic performance. Descriptive statistics showed no missing data from the institutional records. Regression analysis found that higher closeness and eccentricity centrality were positively associated with GPA, supporting the hypothesis that greater social capital is linked to better performance. Further, students in the top two quartiles for social capital had significantly higher GPAs than those in the bottom two quartiles, indicating cross-class social ties benefit academic achievement.
Predictors of Success: Linking Student Achievement to School and Educator Successes through Professional Learning
This study show how some schools have seen a dramatic increase in student achievement after developing a strong, online professional learning program.
In this study, an assistant refers to a person who helps the teacher in the classroom but does not have
full teaching responsibilities.
Blended Learning: Refers to a hybrid of face-to-face and online learning where a portion of the content is
delivered online and students have some control over time, place, path and/or pace of their learning.
Communication Skills: Refers to the ability to effectively convey information through the use of verbal and
non-verbal means. It includes listening, speaking, reading and writing.
Online Quizzes: Refers to assessment activities designed to evaluate students’ mastery of lessons delivered
through the internet.
Online Games: Refers to interactive activities
This document summarizes a systematic literature review on monitoring, awareness and reflection in blended learning. It begins with definitions of key terms and motivation for the review. The methodology section describes using a systematic review process. The results section outlines the main findings in 5 areas: learning contexts studied, research problems investigated, solutions provided, evaluation maturity, and open issues. Key findings include blended learning being interpreted differently, existing solutions mainly for teachers in universities, and open areas around stakeholders, data sources, and longer-term evaluations.
The document examines student-centered learning practices in New England high schools. It finds that practices within the personalized learning and student ownership tenets are most prevalent, while anytime/anywhere learning practices lag behind. Schools also report that competency-based learning models are more challenging to adopt than other student-centered practices, facing barriers around annual student advancement and college acceptance of competency-based transcripts.
This literature review examines how online learning aligns with principles of adult learning (andragogy). It summarizes research finding that:
1) Younger adult learners are motivated by competition while older adults are motivated by personal growth and belonging.
2) Incorporating audio into online modules reduces cognitive load and improves learning compared to text-based modules.
3) Adult learners value self-directed learning and their love of learning correlates with online learning success.
Providing personalised student support in blended learning at scaleAbelardo Pardo
Blended learning environments can be used to deploy strategies to increase student engagement in learning experiences. However, for these strategies to be effective, this increase in engagement requires an increase in student support which can pose serious challenges for large cohorts. The increase in technology mediation offers unprecedented opportunities to collect information
about how students interact in a learning environment. Can this data be used to provide student support at scale? Is it feasible to blend data management techniques as part of a learning design to provide personalised suggestions to students? This talk will offer various practical examples of personalised
student support actions in the context of a large flipped classroom.
The power of learning analytics to measure learning gains: an OU, Surrey and ...Bart Rienties
Learning gains has increasingly become apparent within the HE literature, gained traction in government policies in the UK, and are at the heart of Teaching Excellence Framework (TFL). As such, this raises a question to what extent teaching and learning environment can actually predict students’ learning gains using principles of learning analytics. In this presentation, which is joined work with University of Surrey and Oxford Brookes, I will focus on some preliminary findings based upon developing and testing an Affective-Behaviour-Cognition learning gains model using longitudinal approach. The main aim of the research is to examine whether learning gains occur on all three levels of Affective-Behaviour-Cognition model and whether any particular student or course characteristics can predict learning gains or lack of learning and dropout. For more info, see https://abclearninggains.com/
ACTIVE LEARNING STRATEGIES AND HIGHER-ORDER THINKING SKILLS OF GRADE 10 STUDENTSJoe Osborn
This document summarizes a study that examined the effects of two active learning strategies - case-based learning and visual-organization activity - on the higher-order thinking skills of Grade 10 students in the Philippines. The study used an experimental design with two groups of 50 students each that were exposed to one of the two learning strategies. Both groups showed improvement in critical and creative thinking skills after the learning strategies were applied, as measured by pre- and post-tests. The case-based learning group performed significantly better on the critical thinking portion of the post-test compared to the visual-organization group. The study found that both active learning strategies helped improve students' higher-order thinking skills.
Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...Tanya Joosten
The document summarizes the program for Day 1 of the ELI 2016 Annual Meeting Leadership Seminar on promoting effective teaching and learning ecosystems through research-proven practice. It lists the speakers for Day 1 as Tanya Joosten, Diane Reddy, and Ray Flemming from the University of Wisconsin-Milwaukee. The seminar will discuss leadership challenges in gathering evidence of key factors that impact student outcomes and brainstorm ways to motivate and incentivize faculty and staff involvement in research. Participants will also explore effective practices for faculty development, instructional design, and institutional research.
LTI series – Learning Analytics with Bart RientiesBart Rienties
Join Bart Rienties, Professor of Learning Analytics at the second LTI Series event
Most institutions, including the OU, are exploring how data can better inform teaching and learning. What can we learn from data, and learning analytics in particular? Should we be afraid about being monitored? Or should we embrace this?
Bart’s research focuses on how the OU can use the power of learning analytics to enhance teaching and learning, and what the potential limitations are for social interaction, cultural discourse, and practice.
This seminar will look at the different models being adopted globally, and use a framework to consider what might be the best approach for the OU.
DATE AND TIME: Thu 25 October 2018, 14:00 – 15:00
LOCATION: The Hub Theatre, Walton Hall, Milton Keynes
Influence of Assessment Process on Students Higher Order Learning in Science ...iosrjce
The study is an attempt to find out the influence of assessment process on students higher order
learning in science subjects in Bangladesh. The main objectives of the study are: (i) to identify the nature of the
question items of science subjects, (ii) to examine what kind of learning is influenced by the question items, and
(iii) to identify the role of science exams on students higher order learning. Findings of the study showed that,
majority of the science question items are mostly knowledge based. The nature of the items mainly demanded
the memorizing ability of the learners and it can be said that simple learning or straightforward learning skills
like memorizing is influenced heavily by the question items. It is also found unlikely but truly that; the question
items do not play any significant role on students higher order learning. The foregoing discussions suggest that
the nature of the assessment process used at the examinations make a bad or negative impact on students
learning. Therefore, the nature of the question items of the science examination should be changed for
influencing the students higher order learning and it should cover all the sub-domain of the cognitive domain of
learning. The items should be designed in such a way that it encourages the students for self-thinking
SRHE2016: Multilevel Modelling of Learning Gains: The Impact of Module Partic...Bart Rienties
Jekaterina Rogaten1
, Bart Rienties1
, Denise Whitelock1
, Simon Cross1
, Allison Littlejohn1
, Rhona
Sharpe2
, Simon Lygo-Baker3
, Ian Scott2
, Steven Warburton3
, Ian Kinchin3
1The Open University UK, UK,
2Oxford Brooks University, UK,
3University of Surrey, UK
Research Domain: Learning, teaching and assessment (LTA)
In the UK, the introduction of the Teaching Excellence Framework (TEF) has increased interest in
appropriate and valid measurement approaches of learning gains in Higher Education. Usually
learning gains are measured using pre-post testing, but this study examines whether academic
performance can be effectively used as proxy to estimate students’ learning progress. Academic
performance of 21,192 online learners from two major faculties was retrieved from university
database. A three-level growth-curve model was estimated and results showed that 16% to 46% of
variance in students’ initial academic performance, and 51% to 77% of variance in their subsequent
learning gains was due to them studying at a particular module. In addition, the results illustrate that
students who studied in modules with initial high student achievements exhibited lower learning gains
than students learning in modules with low initial student achievements. The importance of
assessment and learning design for learning gains are outlined.
www.abclearninggains.com @learninggains
The disconnect between data collection and analysis across sectors of academic institutions makes it challenging to incorporate data into curricular design. Understanding the factors related to student persistence and success is unlikely to occur by focusing only on one sector at a time. Facilitating evidence-based course design might begin with the creation of a tool that allows real-time exploration of data across sectors for integration into the traditional course/curricular design. Our paper describes how data from institutional, learning, and what we call “developmental” analytics can be incorporated into course and curricular design by using a purposefully built analysis tool that permits the exploration of student and course objects. This Browser of Student and Course Objects (BoSCO) is being built in a faculty driven-process and can be used as a bridge between the analytics space and the course/curriculum design environments.
Xavier Prat-Resina has a PhD in Physical Chemistry. He is a faculty member at the “Center for Learning Innovation” in the UofM Rochester campus. His interests are the design of web materials to enhance student learning and to analyze student and course data to optimize the academic curriculum.
-It takes institutional, learning and developmental data to assess a curriculum.
-BoSCO is an agile tool that may encourage teachers to use analytics for course and curricular design.
-Evidence-based course design requires the involvement of many sectors of the academic institution.
This summary analyzes an article about educational negligence. The article discusses how negligence in education can negatively impact students' school lives and future ambitions. It addresses the issue of educational negligence in three main points:
1) Educational negligence can take many forms from poor teaching standards to ignoring student needs and attendance issues. It seriously impacts students.
2) A study in South Korea found that neglecting culturally diverse students was linked to relationship issues, dropout rates increasing over time. Neglect harmed peer and teacher relationships for these students.
3) To address educational negligence, teachers must ensure all students complete assignments by attending all lessons to fully cover required material. Parents and teachers must work together to meet students' needs for
This document summarizes research evaluating the development of effective teacher qualities in students participating in the Scottish Teachers for a New Era (STNE) programme. Key findings include:
1) STNE students demonstrated more sophisticated epistemic beliefs, constructivist teaching preferences, emotional intelligence, and commitment to inclusion compared to earlier cohorts.
2) Students showed significant growth over four years in these qualities, indicating the programme's positive impact.
3) STNE students performed well academically and during school experience, associated with strengths in pedagogic content knowledge, reflection, and inclusive mindsets.
4) While most skills improved, subject knowledge and technology use need more focus. The programme enhanced skills but students
Inspiring change in assessment and feedbackTansy Jessop
1) The document summarizes a mixed methods study exploring assessment and feedback practices across university programmes. It identified variations in assessment patterns, an over-reliance on high-stakes summative assessment compared to formative assessment, disconnected feedback practices, and a lack of clarity around learning goals and standards.
2) To address these issues, the study employed strategies through its TESTA programme such as rebalancing assessment, collaborative peer processes, linking formative and summative assessment, and helping students and staff internalize goals and standards.
3) Early results suggest the TESTA programme improved student satisfaction, but further research is still needed to determine its long term impact on student learning outcomes.
This document summarizes a JTC event from May 2013 focused on inclusive education and the role of technology. It discusses creating universally designed learning environments and flexible pathways for students through innovative uses of technology. School jurisdictions agreed to implement a research project exploring assistive technology and inclusive practices. The purpose is to better understand how to support learner participation and achievement for diverse students through technology and pedagogy. A developmental evaluation approach will be used to understand contexts and iteratively inform the initiative.
This study examined the effect of co-curricular activities on the academic achievement of secondary school students in Abbottabad, Pakistan. 200 students were divided into experimental and control groups. The experimental groups participated in physical and non-physical co-curricular activities for 40 minutes daily for 12 weeks, while the control groups did not. Pre- and post-tests were used to measure the students' academic achievement before and after the activities. The results showed that the experimental groups performed significantly better than the control groups on the post-test in government boys', girls', and private girls' schools, but not in the private boys' school. The study concluded that co-curricular activities can positively impact academic achievement.
This document provides background information on factors affecting mathematics performance of high school students. It discusses several key factors from previous studies, including student interest, study habits, teacher personality traits and teaching skills, and instructional materials. The conceptual framework outlines how these input factors may influence student mathematics performance outcomes. The study aims to determine the extent to which student-related factors and teacher-related factors impact mathematics performance of high school students at a particular university.
The document provides guidance on how to write an action research proposal, including tips for developing a title, context and rationale, research questions, proposed intervention, research methodology, and work plan. It outlines the necessary components of each section and considerations for writing them effectively. Sample text is also included to demonstrate how each section could be structured.
Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...johnroseadams1
This document summarizes an interview with Dr. Stephanie McKendry about her research on replacing a successful campus-based pre-entry program for nursing students with a virtual version.
[1] Dr. McKendry conducted research through action research cycles involving interviews and evaluations. Her research found that replacing campus activities with online versions is limited and may threaten inclusivity by disenfranchising some learners.
[2] Interviews with students who attended the campus-based pre-entry program found that the "face time" and socialization aspects were most valuable in building confidence and community. Students were not confident that an online version could replicate these benefits.
[3] While blended learning can supplement
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...Tanya Joosten
EDUCATION SESSION
Systematic Review And Environmental Scan On Digital Learning At Minority Serving Institutions
Date: Tuesday, November 10th
Time: 6:00 PM to 6:45 PM
Conference Session: Concurrent Session 8
Session Modality: Virtual
Lead Presenter: Tanya Joosten (National Research Center for Distance Education and Technological Advancements (DETA) and the University of Wisconsin-Milwaukee)
Co-presenter: Kate Lee-McCarthy (The Online Learning Consortium (OLC))
Track: Research, Evaluation, and Learning Analytics
Location: Zoom Room 1
Session Duration: 45min
Brief Abstract:
Through the Every Learner Everywhere Partnership, the Online Learning Consortium (OLC) and the National Research Center for Distance Education and Technological Advancements (DETA) have completed a review of research done in online and digital learning at minority serving institutions and/or community colleges, with a focus on Black, Latinx and Tribal population outcomes. Come join us and learn about our findings, hear about the next steps in our process, participate in future research, and continue the conversation in equity and inclusion.
This document summarizes research on blended and digital learning. It finds that blended learning can be as effective as traditional learning, though some students may struggle more online. Research shows blended learning develops independence and skills beyond the classroom. However, teaching methods often rely more on lectures than interaction. Strong design, social connection, and teaching quality are keys to student success online. Sources offer further guidance on implementing blended models.
This document provides a summary of the key factors affecting mathematics performance identified in the related literature. It discusses several factors including student interest, study habits, teacher personality traits, teaching skills, and instructional materials. Effective study habits require practice and perfect practice. Instructional materials and teaching strategies are important determinants of math teaching methods. Students' beliefs about their ability and whether it can be improved also impact performance. A teacher's competence relies on possessing key personality traits and using varied teaching approaches helps student engagement. The literature shows relationships between these factors and mathematics achievement.
Presentations morning session 22 January 2018 HEFCE open event “Using data to...Bart Rienties
With the Teaching Excellence Framework being implemented across England, a lot of higher education institutions have started to ask questions about what it means to be “excellent” in teaching. In particular, with the rich and complex data that all educational institutions gather that could potentially capture learning gains, what do we actually know about our students’ learning journeys? What kinds of data could be used to infer whether our students are actually making affective (e.g., motivation), behavioural (e.g., engagement), and/or cognitive learning gains? Please join us on 22 January 2018 in lovely Milton Keynes at a free OU- and HEFCE-supported event on Using data to increase learning gains and teaching excellence.
10.30-11.00 Welcome and Coffee
11.00-11.30 Lightning presentations by participants, outlining insights about learning gains
1130-1300 Insights from the ABC-Learning Gains project
Dr Jekaterina Rogaten (OU): Reviewing affective, behavioural and cognitive learning gains in higher education of 54 learning gains studies
Prof Bart Rienties & Dr Jekaterina Rogaten (OU): Are assessment scores good proxies of estimating learning gains: a large-scale study amongst humanities and science students
Prof Rhona Sharpe (University of Surrey) & Dr Simon Cross (OU): Insights from 45 qualitative interviews with different learning gain paths of high and low achievers
Dr Ian Scott (Oxford Brookes) & Dr Simon Lygo-Baker (OU): Making sense of learning trajectories: a qualitative perspective
The document discusses the need to broaden the pipeline of students in K-12 mathematics to address several issues, including declining student interest in STEM fields and an aging technical workforce. It introduces BEST, a public-private partnership aimed at building a stronger and more diverse workforce in STEM. BEST analyzes research on pre-K-12 programs to identify effective practices and design principles, such as defined outcomes, persistence, personalization, challenging content, and engaged adults. BEST then disseminates its findings and empowers districts to implement changes to broaden the STEM pipeline.
Similar to Lessons learned from 200K students and 2 GB of learning gains data. (20)
Applying and translating learning design and analytics approaches in your ins...Bart Rienties
This interactive workshop delivered by the University of Zagreb, Faculty of Organisation and Informatics (UZ) and the Open University UK (OU) will build on two large-scale implementations of learning design and learning analytics, and how you could potentially implement similar approaches in your institution. The OU has been implementing learning design for nearly 20 years as a structured design, specification, and review process for blended and online courses. The learning design is focused on "what students do" as part of their learning, rather than on "what teachers do" or on what will be taught.
Building on this work, UZ has recently developed the Balanced Design Planning (BDP) tool specifically for educators working in hybrid and blended contexts. The tool is more focused on intended learning outcomes and automated learning analytics and is currently being developed, tested, evaluated and implemented with 1000+ practitioners from dozens of institutions in 20+ countries as part of four European projects (eDesk, Teach4EDU, RAPIDE, iLED), and is publicly available for other institutions to use for free. It has been shown by studies conducted by OU and UZ that when these learning design (LD) approaches are used, they help educators to make real-time informed decisions based on learning analytics (LA) and improve the predictive modelling of student behaviour.
Attendees should bring their laptop for this workshop session.
Bart Rienties, Professor in Learning Analytics, Institute of Education Technology, The Open University
Keynote Presentation: Implementing learning analytics and learning design at ...Bart Rienties
The University of the Roller Coaster
How can Higher Education function in a world struggling to save itself from climate change, pandemics and war? How can it drive innovation and shape the future as the pace of technological change constantly increases? How can it re-invent itself to respond imaginatively to the new challenges facing humanity?
We are living in an uncertain, unpredictable world with no “back to normal” any more. So, how can we re-imagine higher education when nothing can be taken for granted? What kind of technologies can help universities to adapt? What lessons can we learn from recent successes and failures? What 'best practice' examples point the way into the future? How can we shape the development of institutions, so that they are neither “ivory towers” nor “competence factories"? How can we encourage future-oriented universities in which both pedagogy and research are fit for the challenges ahead?
In the Academic Plenary, our experts will examine the threats and opportunities facing higher education today and ask how we can design new approaches that prepare staff and students to thrive in the University of the Roller Coaster.
Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...Bart Rienties
This keynote will help you:
-Understand where to start with learning analytics
-Understand how to effectively support your staff to use data
-Critically review whether learning analytics is something for your organisation
https://www.terrapinn.com/exhibition/edutech-europe/speaker-bart-RIENTIES.stm
How can you use learning analytics in your own research and practice: an intr...Bart Rienties
While many “brick-and-mortar” universities had to rapidly shift online provision during the pandemic, a range of online and distance learning universities have been teaching in blended and online formats for years. Obviously with every single click potentially interesting data might become available about how and perhaps why learners are engaging with learning materials and activities. A blossoming field of learning analytics has emerged since 2011 trying to make sense of these increased data flows. The Open University UK (OU) has been trailblazing innovative learning across the globe for 50 years. Since 2014 the OU has gradually moved from small-scale experimentation to large-scale adoption of learning analytics throughout all 400+ modules and qualifications available within the OU for its 170.000+ online learners.
This keynote will explore how you as researcher, practitioner, and/or policy maker could start to use learning analytics to better understand your educational practice. Using examples from small-scale experiments and large-scale adoptions of predictive learning analytics I will explore together with EDEN RW participants which approaches and methods in learning analytics might be useful to consider. No prior knowledge or experience of learning analytics is expected, and join me on a journey of how you could potentially use data from your learners and teachers to further improve and finetune your blended and online provision.
Dr. Bart Rienties is Professor of Learning Analytics and programme lead of the learning analytics and learning design research programme at the Institute of Educational Technology at the Open University UK. He leads a group of academics who provide university-wide learning analytics and learning design solutions and conduct evidence-based research of how students and professionals learn. His primary research interests are focussed on Learning Analytics, Professional Development, and the role of motivation in learning. Furthermore, Bart is interested in broader internationalisation aspects of higher education. He has successfully led a range of institutional/national/European projects, and has received a range of awards for his educational innovation projects. He has published over 285 academic outputs, and is the 2nd most published author on Networks in Education in period 1969-2020 (Saqr et al. 2022), the 3rd most cited author on higher education internationalisation in Asia in the period 2013-2018 (Can & Hou, 2021), the 4th most cited author and contributor in Learning Analytics in the period 2011-2018 (Adeniji, 2019), the 5th most published author on internationalisation in the period 1900-2018 (Jing et al. 2020) and the 7th most published author on social network analysis in social sciences in the period 1999-2018 (Su et al. 2020), and the 14th most published author on educational technology in the period 2015-2018 (West & Bodily, 2020).
SAAIR: Implementing learning analytics at scale in an online world: lessons l...Bart Rienties
Workshop objectives:
Explore how institutions like Open University UK have implemented learning analytics at scale. Workshop activities:
Presentation from the facilitator and interactive with questions via pollev, chat, and Zoom. Facilitator biography:
Dr. Bart Rienties is Professor of Learning Analytics and programme lead of the learning analytics and learning design research programme at the Institute of Educational Technology at the Open University UK. He leads a group of academics who provide university-wide learning analytics and learning design solutions and conduct evidence-based research of how students and professionals learn. As educational psychologist, he conducts multi-disciplinary research on work-based and collaborative learning environments and focuses on the role of social interaction in learning, which is published in leading academic journals and books. His primary research interests are focussed on Learning Analytics, Professional Development, and the role of motivation in learning. Furthermore, Bart is interested in broader internationalisation aspects of higher education. He has successfully led a range of institutional/national/European projects, and has received a range of awards for his educational innovation projects. He has published over 250 academic outputs, and is the 4th most cited author and contributor in Learning Analytics in the period 2011-2018 (Adeniji, 2019), the 5th most published author on internationalisation in the period 1900-2018 (Jing et al. 2020) and the 3rd most cited author on higher education internationalisation in Asia in the period 2013-2018 (Can & Hou, 2021), the 7th most published author on social network analysis in social sciences in the period 1999-2018 (Su et al. 2020), and the 14th most published author on educational technology in the period 2015-2018 (West & Bodily, 2020). More info at https://iet.open.ac.uk/people/bart.rienties
OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...Bart Rienties
This online event will be a showcase of leading research in the field of open learning, conducted by Doctoral Scholars of The Open University and Leverhulme Trust’s Open World Learning programme, whose work is being recognised with the launch of a new open-access Open World Learning Book.
The event will feature an opening panel discussion on the achievements of our Doctoral Scholars, a collection of themed break-out sessions where scholars will share their research studies and their social impacts, and close with a roundtable where our scholars will consider the future of open learning.
Learning in the 21st century is undergoing both subtle and radical transformation due to the impact of digital, innovative, network technologies. Open learning provides unprecedented access to educational information, providing support to learners worldwide. However, it is not the technologies themselves that represent the biggest change, but the opportunities for access to formal and informal learning.
The Open World Learning programme has been funded by the Leverhulme Trust and The Open University to provide 18 Scholars the opportunity to identify changes in open learning which may exclude, rather than include those who would most benefit. Despite technological advancements, the main challenges to open learning are access-related. Our Open World Learning Scholars have been researching the barriers to access for those whose experiences open learning can benefit most and addressing issues where possible.
Hosted by Professor Bart Rienties, Programme Lead of the Open World Learning programme at the OU's Institute of Educational Technology, this two-hour event will provide a knowledge exchange platform to learn from our Open World Learning Doctoral Scholars and celebrate their exceptional achievements with the Open World Learning Book Launch.
We hope you join us and register to attend our free event. Follow us on the IETatOU Twitter and visit the IET website where a series of digital and social content will be shared highlighting the work of our Open World Learning scholars.
Visit us here: https://iet.open.ac.uk | https://twitter.com/ietatou
The document discusses lessons learned from implementing learning analytics and learning design at scale over 10 years at the Open University UK. Some key points:
1. Change is slow but can be enhanced with clear senior support, bottom-up support from teachers, and evidence-based research to change perspectives.
2. Both predictive learning analytics since 2013 and learning design since 2005 have provided insights but their impact is often forgotten or underestimated.
3. Factors like faculty engagement, teachers as champions, evidence generation, and digital literacy were critical to successfully implementing predictive learning analytics at scale.
4. Research has found learning design provides important context for learning analytics and can improve courses by closing the loop between design and enhanced learning
Education 4.0 and Computer Science: A European perspectiveBart Rienties
This systematic literature review aimed at identifying the pedagogical approaches, aligned with Education 4.0, used to support teaching computer science courses with undergraduate and graduate students in Europe. A three-step coding process was conducted to identify and analyse 20 papers. Quantitative and qualitative analysis of the selected papers revealed a three-cluster solution with common characteristics that could be used to describe those pedagogical approaches. The review also showed that the term Education 4.0 is still relatively new and has not been conceptualised in terms of computer science courses, although the characteristics of Education 4.0 are visible throughout the pedagogical approaches.
Bart Rienties, Rebecca Ferguson, Christothea Herodotou, Francisco Iniesto, Julia Sargent, Igor Balaban, Henry Muccini, Sirje Virkus
AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...Bart Rienties
The Open University UK (OU) has been implementing learning analytics and learning design on a large scale since 2012. With its 170+ students and 4000+ teaching staff, the OU has been at the forefront of testing, implementing, and evaluating the impact of learning analytics and learning design on students outcome and retention. A range of reviews and scholarly repositories (e.g., Web of Science) indicate that the OU is the largest contributor to academic output in learning analytics and learning design in the world. However, despite the large uptake of learning analytics at the OU there are a range of complex issues in terms of buy-in from staff, data infrastructures, ethics and privacy, student engagement, and perhaps most importantly how to make sense of big and small data in a complex organisation like the OU. During his talk Bart will be presenting on the implementation and learnings.
Keynote Data Matters JISC What is the impact? Six years of learning analytics...Bart Rienties
The Open University (OU) was an early adopter of learning analytics, and after six years has had the opportunity to reflect on the impact of large scale adoption across the institution.
Has there been an impact on student retention/progress/completion?
How are the positives (or negatives) reflected in student satisfaction surveys?
What worked, what didn't, and with this benefit of hindsight what is, or should be, next?
What have we learned from 6 years of implementing learning analytics amongst ...Bart Rienties
By Professor Bart Rienties, Head of Academic Professional Development, Institute of Educational Technology, The Open University, UK
Abstract
The Open University UK (OU) has been implementing learning analytics since 2014, starting with one or two modules to its current practice of large-scale implementation across all its 400+ modules and 170.000+ students and 4000+ teaching staff. While a range of reviews (e.g., Adenij, 2019) and scholarly repositories (e.g., Web of Science) indicate that the OU is the largest contributor to academic output in learning analytics in the world, behind the flashy publications and practitioner outputs there are a range of complex issues in terms of ethics and privacy, data infrastructures, buy-in from staff, student engagement, and how to make sense of big data in a complex organisation like the OU.
Based upon large-scale big data research we found some interesting tensions in both design and educational theory, such as:
– 69% of engagement by students on a week by week basis is determined by how teachers are designing courses (i.e., learning design and instructional design indeed directly influence behaviour and cognition), but many teachers seem reluctant to change their learning design based upon data of what works and what does not work (e.g., making sense of data, agency);
– How teachers engage with predictive learning analytics (PLA) significantly improves student outcomes, but only a minority of teachers actually use PLA;
– Some disadvantaged groups engage more actively in OU courses, but nonetheless perform lower than non-disadvantaged students.
During this CELDA keynote I would like to share some of my own reflections of how the OU has implemented learning analytics, and how these insights are helping towards a stronger evidence-base for data-informed change. Furthermore, by sharing some of the lessons learned from implementing learning analytics on a large scale I hope to provide some dos and don’ts in terms of how you might consider to use data in your own practice and context.
Using Learning analytics to support learners and teachers at the Open UniversityBart Rienties
In this seminar Prof Bart Rienties will reflect on how the Open University UK has become a leading institution in implementing learning analytics at scale amongst its 170K students and 5K staff. Furthermore, he will discuss how learning analytics is being adopted at other UK institutions, and what the implications for higher education might be in these Covid19 times.
https://www.kent.ac.uk/cshe/news-events.html
Using student data to transform teaching and learningBart Rienties
This document summarizes a webinar given by Dr. Bart Rienties on using student data and learning analytics to transform teaching and learning. Some key points:
- Learning analytics aims to measure, collect, analyze and report data about learners to understand and optimize learning. Social learning analytics focuses on how learners build knowledge together.
- The Open University has been a world leader in collecting and analyzing large-scale student data to provide actionable insights for students, teachers, and institutional benefit. Studies have shown the importance of linking learning analytics outcomes to student satisfaction, retention, and learning design.
- Practitioners want learning analytics solutions that are integrated across an entire learning journey from initial inquiry through modules to
Lecture series: Using trace data or subjective data, that is the question dur...Bart Rienties
In this lecture series Bart Rienties (Professor of Learning Analytics, head of Academic Professional Development) will discuss how from the safety of your home you could use existing trace data to explore interactions between people (e.g., Twitter data, engagement data in a virtual learning environment, public data sets), and what the affordances and limitations of these trace data might be. Furthermore, he will discuss how other ways of collecting subjective data (e.g., surveys, interviews) might strengthen our understandings of complex interactions between people.
There are no prior requirements to join, and everyone is welcome. For those with a technical background you may enjoy this recent paper in PLOS ONE https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0233977. For those with a non-technical background, you may enjoy this paper https://journals.sfu.ca/flr/index.php/journal/article/view/348
How to analyse questionnaire data: an advanced sessionBart Rienties
This document outlines an advanced workshop on analyzing questionnaire data. The objectives are to familiarize participants with psychometric and linguistic techniques for analyzing questionnaire data, including computing constructs, factor analysis, reliability, validity, and advanced statistical techniques. It discusses what a questionnaire is, the questionnaire design process, strengths and limitations of questionnaires, and provides case studies on using questionnaires to measure constructs like academic motivation and student adjustment. The document provides information on collecting questionnaire data, checking reliability and validity, and using statistical analyses to test hypotheses and predict outcomes.
Questionnaire design for beginners (Bart Rienties)Bart Rienties
This document provides an introduction to questionnaire design. It discusses the objectives of using questionnaires which are to understand why they are used, the process of constructing them, and key features of good question design. It also covers strengths and limitations of questionnaires, the survey process, maximizing response rates, and types of questions. The document aims to provide guidance on best practices for designing and implementing effective questionnaires.
Learning analytics adoption in Higher Education: Reviewing six years of exper...Bart Rienties
The document summarizes a webinar by Dr. Bart Rienties on his work implementing learning analytics at scale at the Open University over the past 6 years. Some key points:
1. The Open University is a world leader in collecting and analyzing large-scale student data to provide actionable insights for students and teachers.
2. Analytics4Action supports the university-wide approach to learning analytics and provided insights into interventions for students and modules.
3. Iterative use of learning analytics establishes the need for student and module interventions, with faster feedback loops leading to better outcomes.
4. Legal, ethical and privacy challenges around learning analytics interventions must be addressed, including student consent and transparency.
«Learning Analytics at the Open University and the UK»Bart Rienties
In this seminar, Prof Bart Rienties will reflect on how the Open University UK has become a leading institution in implementing learning analytics at scale amongst its 170K students and 5K staff. Furthermore, he will discuss how learning analytics is being adopted at other UK institutions, and what the implications for higher education might be.
eMadrid seminar on «Review and challenges in Learning Analytics»
Presentation LMU Munich: The power of learning analytics to unpack learning a...Bart Rienties
The power of learning analytics to unpack learning and teaching: a critical perspective
Ludwig-Maximilians-Universität München
Fakultät für Psychologie und Pädagogik
Educational Technology - opportunities and pitfalls How to make the most use...Bart Rienties
The keynote presentation covered opportunities and limitations of educational technology based on learning analytics research. It included three research exemplars: 1) a study that found students' self-reported internet searching skills did not match their actual online behavior, 2) a randomized study showing how internationalized course content can encourage participation in diverse groups, and 3) a project linking multiple datasets across 150+ modules to predict student outcomes. The talk concluded by emphasizing the need to consider ethics and standardization as more educational data becomes available and harvested for learning analytics.
Strategies for Effective Upskilling is a presentation by Chinwendu Peace in a Your Skill Boost Masterclass organisation by the Excellence Foundation for South Sudan on 08th and 09th June 2024 from 1 PM to 3 PM on each day.
How to Fix the Import Error in the Odoo 17Celine George
An import error occurs when a program fails to import a module or library, disrupting its execution. In languages like Python, this issue arises when the specified module cannot be found or accessed, hindering the program's functionality. Resolving import errors is crucial for maintaining smooth software operation and uninterrupted development processes.
This presentation was provided by Steph Pollock of The American Psychological Association’s Journals Program, and Damita Snow, of The American Society of Civil Engineers (ASCE), for the initial session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session One: 'Setting Expectations: a DEIA Primer,' was held June 6, 2024.
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...PECB
Denis is a dynamic and results-driven Chief Information Officer (CIO) with a distinguished career spanning information systems analysis and technical project management. With a proven track record of spearheading the design and delivery of cutting-edge Information Management solutions, he has consistently elevated business operations, streamlined reporting functions, and maximized process efficiency.
Certified as an ISO/IEC 27001: Information Security Management Systems (ISMS) Lead Implementer, Data Protection Officer, and Cyber Risks Analyst, Denis brings a heightened focus on data security, privacy, and cyber resilience to every endeavor.
His expertise extends across a diverse spectrum of reporting, database, and web development applications, underpinned by an exceptional grasp of data storage and virtualization technologies. His proficiency in application testing, database administration, and data cleansing ensures seamless execution of complex projects.
What sets Denis apart is his comprehensive understanding of Business and Systems Analysis technologies, honed through involvement in all phases of the Software Development Lifecycle (SDLC). From meticulous requirements gathering to precise analysis, innovative design, rigorous development, thorough testing, and successful implementation, he has consistently delivered exceptional results.
Throughout his career, he has taken on multifaceted roles, from leading technical project management teams to owning solutions that drive operational excellence. His conscientious and proactive approach is unwavering, whether he is working independently or collaboratively within a team. His ability to connect with colleagues on a personal level underscores his commitment to fostering a harmonious and productive workplace environment.
Date: May 29, 2024
Tags: Information Security, ISO/IEC 27001, ISO/IEC 42001, Artificial Intelligence, GDPR
-------------------------------------------------------------------------------
Find out more about ISO training and certification services
Training: ISO/IEC 27001 Information Security Management System - EN | PECB
ISO/IEC 42001 Artificial Intelligence Management System - EN | PECB
General Data Protection Regulation (GDPR) - Training Courses - EN | PECB
Webinars: https://pecb.com/webinars
Article: https://pecb.com/article
-------------------------------------------------------------------------------
For more information about PECB:
Website: https://pecb.com/
LinkedIn: https://www.linkedin.com/company/pecb/
Facebook: https://www.facebook.com/PECBInternational/
Slideshare: http://www.slideshare.net/PECBCERTIFICATION
How to Setup Warehouse & Location in Odoo 17 InventoryCeline George
In this slide, we'll explore how to set up warehouses and locations in Odoo 17 Inventory. This will help us manage our stock effectively, track inventory levels, and streamline warehouse operations.
it describes the bony anatomy including the femoral head , acetabulum, labrum . also discusses the capsule , ligaments . muscle that act on the hip joint and the range of motion are outlined. factors affecting hip joint stability and weight transmission through the joint are summarized.
Main Java[All of the Base Concepts}.docxadhitya5119
This is part 1 of my Java Learning Journey. This Contains Custom methods, classes, constructors, packages, multithreading , try- catch block, finally block and more.
Walmart Business+ and Spark Good for Nonprofits.pdfTechSoup
"Learn about all the ways Walmart supports nonprofit organizations.
You will hear from Liz Willett, the Head of Nonprofits, and hear about what Walmart is doing to help nonprofits, including Walmart Business and Spark Good. Walmart Business+ is a new offer for nonprofits that offers discounts and also streamlines nonprofits order and expense tracking, saving time and money.
The webinar may also give some examples on how nonprofits can best leverage Walmart Business+.
The event will cover the following::
Walmart Business + (https://business.walmart.com/plus) is a new shopping experience for nonprofits, schools, and local business customers that connects an exclusive online shopping experience to stores. Benefits include free delivery and shipping, a 'Spend Analytics” feature, special discounts, deals and tax-exempt shopping.
Special TechSoup offer for a free 180 days membership, and up to $150 in discounts on eligible orders.
Spark Good (walmart.com/sparkgood) is a charitable platform that enables nonprofits to receive donations directly from customers and associates.
Answers about how you can do more with Walmart!"
Lessons learned from 200K students and 2 GB of learning gains data.
1. Lessons learned from 200K
students and 2 GB of learning
gains data
https://twitter.com/LearningGains
https://abclearninggains.com/
The results of the ABC project were made possible due to Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Jekaterina Rogaten, Bart Rienties, George Roberts, Ian Scott, Rhona Sharpe, Steve Warburton, and
Denise Whitelock. Please contract bart.rienties@open.ac.uk if you want to know more about ABC learning gains
2. Research Team
Prof Bart Rienties
Open University
Dr Jekaterina Rogaten
Open University
Dr Simon Cross
Open University Dr Ian Scott
Oxford Brookes
Prof Ian Kinchin
University of Surrey
Prof Denise Whitelock
Open University
Prof Allison Littlejohn
Open University
Prof Rhona Sharpe
University of Surrey
Dr Simon Lygo-Baker
University of Surrey
Dr George Roberts
Oxford Brookes
3. Agenda for today
1. What have we learned from systematic literature review of
52 studies focussed on ABC learning gains (n = 42000+
students)?
2. What have we learned from 200K students and 2 GB of
learning gains data in terms of ABC learning gains?
5. How are learning gains measured: a
systematic analysis
52 studies selected: 42000+
students
The concept of learning gain is
primarily used to examine the effect
of any particular educational
‘intervention’
There is a gradual increase in
studies examining learning gains all
across the world
All learning gains can be classified
into ABC
53%
16% 21%
10% Behaviour-Cognitive
Learning Gains
Affective-Behaviour-Cognitive
Learning Gains
Cognitive Learning Gains
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (2018). Reviewing affective, behavioural, and cognitive learning gains in higher education.
Affective-Cognitive
Learning Gains
Year
Numberofstudies
6. What type of learning gains are there
Affective learning gains:
• Attitude
• Confidence
• Enjoyment
• Enthusiasm for a topic
• Feeling comfortable with
complex ideas
• Interest in a topic
• Motivation
• Satisfaction
• Self-efficacy
Cognitive learning gains:
• Students’ ability to evaluate and
create knowledge
• Analytical ability
• Autonomous cognition
• Critical thinking
• Ethical thinking
• Creative and higher order thinking
Discipline specific skills
• Knowledge and understanding of the
topic,
• Oral and written communication
• Problem solving
• Scientific reasoning
• Statistical and research
kills/knowledge
Behavioural learning gains:
•Ability to work independently
•Applied conceptual understanding
•Effort and engagement
•Leadership skills
•Team/group working skills
•Practical competence
•Resource management
•Responsibility
•Preparation skills
•Time management skills
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (2018). Reviewing affective, behavioural, and cognitive learning gains in higher education. Assessment & Evaluation in
7. Affective learning gains
Affective learning gains were
measured in 19 studies (e.g.,
Moorer, 2009; Strayhorn,
2010) comprising 28 student
samples totalling 3,333
higher education students.
Self-reported affective
learning gains there were
mainly studies that reported
relatively high learning gains
of > 40%, ranging from 39-
98%
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (2018). Reviewing affective, behavioural, and cognitive learning gains in higher education. Assessment & Evaluation in
8. Behavioural learning gains
Behavioural learning gains
were measured in 13 studies
(e.g., Casem, 2006; Varsavsky
et al., 2014) comprising 23
student samples totalling
4,268 higher education
students.
With the exception of one
study (Stolk and Martello
2015), the remaining 12
studies used a cross-
sectional design for
measuring behavioural
learning gains
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (2018). Reviewing affective, behavioural, and cognitive learning gains in higher education. Assessment & Evaluation in
9. Cognitive learning gains
Cognitive learning gains were
measured in 22 studies, comprising
39 student samples, totalling 18,024
higher education students.
Pre-post testing was used in four
studies, and two studies used a
form of pre-post testing through
reflection all totalling to seven
student samples.
Only in one sample (Stolk and
Martello 2015) did students report
lower cognitive ability at the post-
test than at the pre-test, but the
difference was not significant.
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (2018). Reviewing affective, behavioural, and cognitive learning gains in higher education. Assessment & Evaluation in
10. Cognitive learning gains
Follow-up analyses of treatment
studies which compared a change
in curricular or module design
(treatment) enhanced students
learning in comparison to traditional
lectures (control) found that
students performed better in the
treatment condition <g> = 0.39
than in control condition <g> =
0.26.
Again a wide range of learning
gains were found, whereby average
normalised learning gains g ranged
from -.20 to .81,
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (2018). Reviewing affective, behavioural, and cognitive learning gains in higher education. Assessment & Evaluation in
11.
12.
13. 2. What have we learned from 200K students and 2 GB of
learning gains data in terms of ABC learning gains?
• Affective: 1 cycle of data from OU, 1 cycle from OB
• Behavioural: 1 cycle of data from OU, 1 cycle from US
• Cognitive: 2 cycles of data from OU, 1 cycle from US and OB
• Communication with OFS with types of data collected
OU OB US
data Grades and
demographics data
Grades and
demographics data
Grades and
demographics data
File size 1.45GB 5,59 MB 213 MB +26.78 MB
Number of students 166,722 2,653 (21 – 241 per
department)
25,825 (171 – 4276)
Number of
qualifications/departments
246 18 21
14. Affective learning gains
Using student satisfaction data for proxies for
affective learning gains was not an appropriate
approach.
First, there was a lack of consistent data over
time for sufficiently large numbers of students.
Second, substantial variation in student
satisfaction rates across modules, so changes in
measured affective learning gains are more
likely to arise from differences in sequences of
modules
using student satisfaction data for proxies for affective learning gains was not an appropriate approach. Although there are several studies highlighting the importance of affective learning gains12-14
Third, those who completed the
student satisfaction surveys
were not representative for the
wider student population.
Fourth, when comparing the
approaches across the
institutions, the lack of
standardisation of student
satisfaction approaches,
constructs, and items made it
impossible to compare potential
differences in learning gains
across institutions over time.
15. Behavioural learning gains
Engagement data from VLE not good
proxy for behaviour learning gains
First of all, engagement of students in a
respective module is strongly dependent
by the learning design.
Second, even if proxies for engagement
could be identified, our research showed
that the types of engagement will heavily
be influenced by the type of learning
design
using student satisfaction data for proxies for affective learning gains was not an appropriate approach. Although there are several studies highlighting the importance of affective learning gains12-14
Third, related research looking at fine-grained
analyses of what students are actually studying,
and when, showed substantial variation in
engagement and successful learning
approaches
In other words, our longitudinal analyses
showed that our LMS proxies of engagement
were not effective for understanding how
students made behavioural learning gains over
time.
16. Using Grades as proxies for cognitive
learning gains
David Boud44:
”The most problematic feature of current marking practice is that it is not possible to associate
any reported mark with what a student can or cannot do. The meaning of the mark is not
described in terms of the standards to be reached as articulated in the stated learning outcomes.
Outside its immediate context, it is not clear what meaning should be attached to a mark. Marks
act as obscuring devices”.
18. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rogaten, J., Rienties, B., & Whitelock, D. (2017). Assessing Learning Gains. In D. Joosten-ten Brinke & M. Laanpere (Eds.), Technology Enhanced Assessment. TEA 2016. Communications in Computer and
19. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rogaten, J., & Rienties, B. (2018). Which first-year students are making most learning gains in STEM subjects? Higher Education Pedagogies, 3(1), 161-172. doi: 10.1080/23752696.2018.1484671.
20. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rogaten, J., Clow, D., Edwards, C., Gaved, M., Rienties, B. (Accepted with minor revision: 12-07-2018). Do we need to re-imagine university assessment in a digital world? A big data exploration. Margaret
21. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rienties, B., Rogaten, J., Nguyen, Q., Edwards, C., Gaved, M., Holt, D., . . . Ullmann, T. (2017). Scholarly insight Spring 2017: a Data wrangler perspective. Milton Keynes: Open University UK.
22. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rienties, B., Rogaten, J., Nguyen, Q., Edwards, C., Gaved, M., Holt, D., . . . Ullmann, T. (2017). Scholarly insight Spring 2017: a Data wrangler perspective. Milton Keynes: Open University UK.
23. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rienties, B., Rogaten, J., Nguyen, Q., Edwards, C., Gaved, M., Holt, D., . . . Ullmann, T. (2017). Scholarly insight Spring 2017: a Data wrangler perspective. Milton Keynes: Open University UK.
24. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rienties, B., Rogaten, J., Nguyen, Q., Edwards, C., Gaved, M., Holt, D., . . . Ullmann, T. (2017). Scholarly insight Spring 2017: a Data wrangler perspective. Milton Keynes: Open University UK.
25. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rienties, B., Rogaten, J., Nguyen, Q., Edwards, C., Gaved, M., Holt, D., . . . Ullmann, T. (2017). Scholarly insight Spring 2017: a Data wrangler perspective. Milton Keynes: Open University UK.
26. Cognitive learning gains
Cognitive learning gains were
measured in five ways:
1. Cognitive learning gains within
modules
2. Cognitive learning gains from first
to second module
3. Cognitive learning gains within a
qualification
4. Cognitive learning gains across
different qualifications
5. Cognitive learning gains between
institutions
Rienties, B., Rogaten, J., Nguyen, Q., Edwards, C., Gaved, M., Holt, D., . . . Ullmann, T. (2017). Scholarly insight Spring 2017: a Data wrangler perspective. Milton Keynes: Open University UK.
The proportion of variance due to
differences
OU OB US
Level 3: Between qualifications 12% 8% 22%
Level 2: Between students 45% 67% 22%*
Level 1 Between modules (i.e., within-
student level between modules any
one student completed)
43% 25% 56%
Number of students (n) 18329 1990 1547
Table 1 Proportion of variance explained by qualification, student
characteristics, and across modules (OU, OB, US)
27. Our qualitative data shows a different
picture
using student satisfaction data for proxies for affective learning gains was not an appropriate approach. Although there are several studies highlighting the importance of affective learning gains12-14
28. What students think they gain?
I think I am more openly critical
(in the positive sense)
Day to day when I have my book I have very different
approach from recording my notes for example
[in my new job], there will reports and
planning to be drawn and I think that this will
be an aspect of my job where I can say yes
the OU study and discipline I’ve received
from the OU has actually contributed to that.
I observe things better, work into deeper and
work on the whole picture rather than narrow.
I think more logically and more ‘why did that
happen, why did that happen’, there is more
questioning, instead of just to accept things.
I am much better at time management, I am much more
organised now and planning things in advance.
now I say, ‘you know what, I can do that in future’.
I feel more confident and I am happier
because I am doing something I have always
wanted to be doing and something that
interests me
I think I can go confidently to
speak what I learned. But
even to a job that isn’t directly
related to this subject area. I
could talk about my
experiences, my time
management, team working,
computer skills as I feel much
more confident, I can say,
‘actually I have done this’.
Which was one of the
reasons I wanted to a degree.
29. Do grades matter?
How well do your grades represent your progress?
probably in the same way that many other people when
they look at their own assignment results and exam results
…. I feel that I am doing fairly well but I’d always like to
improve myself to my results.
I get quite upset when I get around 70s
… because I am putting so much effort I want my grades to reflect it.
They usually go up. But it is Marginal. 5 marks across all the TMAs
that’s the variance, it just varies very slightly
Even if it is 1-2 marks I say what did I do differently and
I go back to tutor to see what did I do differently. What
happened, what caused it?
Well there are questions with the text books, exercises. So
if I get correct answer, I know I am doing fine. When I say
correct answer that’s not the end product that’s the whole
answer check through it
“I suppose you could say… the skills you learn, like group work, presenting and being able to talk to people…
I would say the main way that you think about [achievement], it’s just the grade because… that’s what is going
on your CV… and affect what job you get. … I’d say the skills you learn as well as becoming an all-rounded person
are quite important as well”.
32. Implications for practice
• Substantial freedom for students to select “unique” pathways:
some programmes and qualifications have relatively fixed and
structured pathways. Other programmes and qualifications
offer students wide and far reaching freedom to choose (one
qualification had 84 potential pathways to complete a degree).
However, institutions provide limited to no structural support
which pathways would fit students’ needs and abilities.
Recommendation 1: Institutions needs to improve how we communicate to our
students which modules fit with their needs and abilities, and be more explicit about
successful pathways for students to obtain a qualification.
Rogaten, J., Clow, D., Edwards, C., Gaved, M., Rienties, B. (Accepted with minor revision: 12-07-2018). Do we need to re-imagine university assessment in a digital world? A big data exploration. Margaret Bearman, Phillip Dawson, Rola
Ajjawi, Joanna Tai, David Boud (Eds). Re-imagining University Assessment in a Digital World. Springer.
33. Implications for practice
• Alignment of modules within a qualification: students
experience substantially different learning designs,
assessment practices, and workload fluctuations when
transitioning from one module to another.
Recommendation 2: Institutions need to improve how we communicate and
manage the students’ expectations of the learning designs and assessment
practices from one module to another.
Recommendation 3: In the longer term, it would be beneficial to align the
module designs across a qualification based upon evidence-based practice
and what works, thereby allowing smooth transitions from one module to
another in a qualification.
Rogaten, J., Clow, D., Edwards, C., Gaved, M., Rienties, B. (Accepted with minor revision: 12-07-2018). Do we need to re-imagine university assessment in a digital world? A big data exploration. Margaret Bearman, Phillip Dawson, Rola
34. Implications for practice
• Alignment of marking within and across modules within and across
qualifications: “embedded expectations”, norms and practice influence
marking practices. Across some qualifications there appears to be a
widespread deliberate approach of making early assessment relatively easy,
both within modules (particularly the first assignment) and within
qualifications (particularly the first module). This approach is intended to
reduce drop-out, but may have unintended consequences.
• Furthermore, given that in most modules teachers are marking relatively
small numbers of students, potential misalignments might be present which
may not be immediately apparent when just looking at average grades and
the normal distribution curves.
• Another potential explanation is the increasing difficulty of the material
being assessed may not be completely accounted for in the marks awarded.
Final-year-equivalent modules rightly contain much more difficult material
than entry modules.
Rogaten, J., Clow, D., Edwards, C., Gaved, M., Rienties, B. (Accepted with minor revision: 12-07-2018). Do we need to re-imagine university assessment in a digital world? A big data exploration. Margaret Bearman, Phillip Dawson, Rola
35. Implications for practice
Recommendation 4: It is essential that grades are aligned not only within a module
but also across a qualification. For exam boards we recommend to include cross-
checks of previous performance of students (e.g., correlation analyses) and
longitudinal analyses of historical data to determine whether previously successful
students were again successful, and whether they maintained a successful learning
journey after a respective module.
Recommendation 5: We recommend that clearer guidelines and grade descriptors
across a qualification are developed, which are clearly communicated to staff and
students.
Recommendation 6: Given that many students follow modules from different
qualifications, it is important to develop coherent university-wide grade descriptors
and align marking across qualifications.
Rogaten, J., Clow, D., Edwards, C., Gaved, M., Rienties, B. (Accepted with minor revision: 12-07-2018). Do we need to re-imagine university assessment in a digital world? A big data exploration. Margaret Bearman, Phillip Dawson, Rola
36. Lessons learned from 200K
students and 2 GB of learning
gains data
https://twitter.com/LearningGains
https://abclearninggains.com/
The results of the ABC project were made possible due to Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Jekaterina Rogaten, Bart Rienties, George Roberts, Ian Scott, Rhona Sharpe, Steve Warburton, and
Denise Whitelock. Please contract bart.rienties@open.ac.uk if you want to know more about ABC learning gains
Editor's Notes
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Level 1 – Grade: repeated measures on students and tell us about students learning trajectory
Level 2 – student: between students variations
Level 3 – Course: between course variation
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)
Web of Science core collection
Science Citation Index Expanded (SCI-EXPANDED)
Social Sciences Citation Index (SSCI)
Arts & Humanities Citation Index (A&HCI)
Conference Proceedings Citation Index- Science (CPCI-S)
Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
Book Citation Index– Science (BKCI-S)
Book Citation Index– Social Sciences & Humanities (BKCI-SSH) Emerging Sources Citation Index (ESCI)