SlideShare a Scribd company logo
Discussion: A cross-institutional
perspective on merits and
challenges of learning gains for
Teaching Excellence Framework
Bart Rienties
https://twitter.com/LearningGains
https://abclearninggains.com/
The results of the ABC project were made possible due to Jekaterina Rogaten, Bart Rienties, Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Ian Scott, Rhona Sharpe, Steve Warburton, and Denise
Whitelock. Please contract Jekaterina.rogaten@open.ac.uk if you want to know more about ABC learning gains
Learning gains is a complex topic
Learning gains is a complex topic
Learning gains: different conceptualisations
• Conceptualisation of learning gains
substantially different across the five
papers
• From private vs public
• From affective to cognitive
• From during a degree until employment
• From student or staff perspective
Learning gains: different methods and approaches
• Qualitative
• Interviews (Paper 1, 4, 5)
• Sorting exercise (Paper 4)
• Personal reflections (Paper 1)
• Quantitative (Paper 3, 2)
• (Affective) survey instruments (Paper 3)
• Grades/degree outcomes (Paper 3, 2)
• Different level of analysis
• Thematic analysis
• Semi-structured interviews
• ANOVA/correlations (Paper 3)
• Regressions and Multi-level modelling (Paper 3, 2)
• Longitudinal (Paper 2, 3, 5)
• Randomised Control Trials (Paper 5) vs convenience sampling
Do we need fine-grained techniques?
• “I learned nothing in College. Ok, ok, maybe that’s an exaggeration. I learned
some things in college. I learned that I can’t stomach cheap beer; I’d rather
play drinking games with a box of wine. I learned what kind of guys I should
never date again. I learned how to live with kids my own age and quite frankly
I do best living alone. Most importantly, I met two English professors who had
a huge impact on my life, who taught me not just how to write but also to
believe in myself. Meeting them made my college years worth it. For the most
part, as I reflect on my undergraduate experience, I feel that it was a colossal
waste of time. http://www.huffingtonpost.com/sarah-rose-attman/i-learned-
nothing-in-coll_b_5606640.html. (Hughes, Paper 1)
Do we need multi-level analysis?
• “[A] limitation of [common learning gains approaches] is that general linear
models are based on the assumption that observed data of one student are
independent of observed data of another. This assumption is not true in
situations where students from the same discipline, identical module, and
same university have similar experiences (e.g., easy assessment in week 1,
difficult assessment in week 6, and moderately hard final assessment in week
15) which means the variance in their scores are not independent and are
linked. This means that simple linear models to measure learning gains may
provide inaccurate and misleading outcomes.” (Rogaten et al., paper 2)
Need for coherence: questions to authors
• Which learning gains?
• Can we develop a common, shared perspective of what learning gains are?
• How to measure learning gains?
• What approaches and measurements are appropriate for capturing and
measuring learning gains?
• From apples to oranges to busses and e-cigarettes
• How are we going to take into consideration individual differences (e.g., prior
qualifications, socio-economic) and learning design differences (e.g., different
assessments, support structures) when comparing learning gains?
• How are we going to compare learning gains across different
disciplines, contexts, and universities?
Discussion: A cross-institutional
perspective on merits and
challenges of learning gains for
Teaching Excellence Framework
Bart Rienties
https://twitter.com/LearningGains
https://abclearninggains.com/
The results of the ABC project were made possible due to Jekaterina Rogaten, Bart Rienties, Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Ian Scott, Rhona Sharpe, Steve Warburton, and Denise
Whitelock. Please contract Jekaterina.rogaten@open.ac.uk if you want to know more about ABC learning gains

More Related Content

What's hot

22 January 2018 HEFCE open event “Using data to increase learning gains and t...
22 January 2018 HEFCE open event “Using data to increase learning gains and t...22 January 2018 HEFCE open event “Using data to increase learning gains and t...
22 January 2018 HEFCE open event “Using data to increase learning gains and t...
Bart Rienties
 
Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...
Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...
Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...
Bart Rienties
 
Enhancing (in)formal learning ties in interdisciplinary management courses: a...
Enhancing (in)formal learning ties in interdisciplinary management courses: a...Enhancing (in)formal learning ties in interdisciplinary management courses: a...
Enhancing (in)formal learning ties in interdisciplinary management courses: a...
Bart Rienties
 
ASCILITE Webinar: A review of five years of implementation and research in al...
ASCILITE Webinar: A review of five years of implementation and research in al...ASCILITE Webinar: A review of five years of implementation and research in al...
ASCILITE Webinar: A review of five years of implementation and research in al...
Bart Rienties
 
Attendance and student performance arp (1)
Attendance and student performance arp (1)Attendance and student performance arp (1)
Attendance and student performance arp (1)
Cindy Paynter
 
Learning Design and Analytics in Learning and Teaching: The Role of Big Data
Learning Design and Analytics in Learning and Teaching: The Role of Big DataLearning Design and Analytics in Learning and Teaching: The Role of Big Data
Learning Design and Analytics in Learning and Teaching: The Role of Big Data
Bart Rienties
 
Presentation 2
Presentation 2Presentation 2
Presentation 2
CYUT
 
Lamar research institute_collaboration_3-22-2013_final
Lamar research institute_collaboration_3-22-2013_finalLamar research institute_collaboration_3-22-2013_final
Lamar research institute_collaboration_3-22-2013_final
Lamar University
 
An analytical review of the literature on e-feedback
An analytical review of the literature on e-feedbackAn analytical review of the literature on e-feedback
An analytical review of the literature on e-feedback
anyakirax
 
Assessment & Feedback Literature Review
Assessment & Feedback Literature ReviewAssessment & Feedback Literature Review
Assessment & Feedback Literature Review
Morse Project
 
Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...
Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...
Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...
Bart Rienties
 
Meaningful Conversations about EdTech: Transforming Student Learning
Meaningful Conversations about EdTech: Transforming Student LearningMeaningful Conversations about EdTech: Transforming Student Learning
Meaningful Conversations about EdTech: Transforming Student Learning
Bradford Wheeler
 
AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...
AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...
AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...
Bart Rienties
 
Using web quests to increase student learning2
Using web quests to increase student learning2Using web quests to increase student learning2
Using web quests to increase student learning2
hlmohs
 
Keynote presentation OOFHEC2016: Bart Rienties
Keynote presentation OOFHEC2016: Bart RientiesKeynote presentation OOFHEC2016: Bart Rienties
Keynote presentation OOFHEC2016: Bart Rienties
EADTU
 
Technology-enabled learning-oriented assessment
Technology-enabled learning-oriented assessmentTechnology-enabled learning-oriented assessment
Technology-enabled learning-oriented assessment
David Carless
 
How can we engage mathematics ITE students with research?
How can we engage mathematics ITE students with research?How can we engage mathematics ITE students with research?
How can we engage mathematics ITE students with research?
Christian Bokhove
 
Data Driven Instrruction
Data Driven InstrructionData Driven Instrruction
Data Driven Instrruction
Tman
 
Ces Ed Tech Presentation 5 26
Ces Ed Tech Presentation  5 26Ces Ed Tech Presentation  5 26
Ces Ed Tech Presentation 5 26
jcrandall1027
 
Oral Defense presentation
Oral Defense presentationOral Defense presentation
Oral Defense presentation
Dwayne Squires
 

What's hot (20)

22 January 2018 HEFCE open event “Using data to increase learning gains and t...
22 January 2018 HEFCE open event “Using data to increase learning gains and t...22 January 2018 HEFCE open event “Using data to increase learning gains and t...
22 January 2018 HEFCE open event “Using data to increase learning gains and t...
 
Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...
Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...
Keynote EARLI SIG17 The power of learning analytics: a need to move towards n...
 
Enhancing (in)formal learning ties in interdisciplinary management courses: a...
Enhancing (in)formal learning ties in interdisciplinary management courses: a...Enhancing (in)formal learning ties in interdisciplinary management courses: a...
Enhancing (in)formal learning ties in interdisciplinary management courses: a...
 
ASCILITE Webinar: A review of five years of implementation and research in al...
ASCILITE Webinar: A review of five years of implementation and research in al...ASCILITE Webinar: A review of five years of implementation and research in al...
ASCILITE Webinar: A review of five years of implementation and research in al...
 
Attendance and student performance arp (1)
Attendance and student performance arp (1)Attendance and student performance arp (1)
Attendance and student performance arp (1)
 
Learning Design and Analytics in Learning and Teaching: The Role of Big Data
Learning Design and Analytics in Learning and Teaching: The Role of Big DataLearning Design and Analytics in Learning and Teaching: The Role of Big Data
Learning Design and Analytics in Learning and Teaching: The Role of Big Data
 
Presentation 2
Presentation 2Presentation 2
Presentation 2
 
Lamar research institute_collaboration_3-22-2013_final
Lamar research institute_collaboration_3-22-2013_finalLamar research institute_collaboration_3-22-2013_final
Lamar research institute_collaboration_3-22-2013_final
 
An analytical review of the literature on e-feedback
An analytical review of the literature on e-feedbackAn analytical review of the literature on e-feedback
An analytical review of the literature on e-feedback
 
Assessment & Feedback Literature Review
Assessment & Feedback Literature ReviewAssessment & Feedback Literature Review
Assessment & Feedback Literature Review
 
Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...
Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...
Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...
 
Meaningful Conversations about EdTech: Transforming Student Learning
Meaningful Conversations about EdTech: Transforming Student LearningMeaningful Conversations about EdTech: Transforming Student Learning
Meaningful Conversations about EdTech: Transforming Student Learning
 
AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...
AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...
AI in Education Amsterdam Data Science (ADS) What have we learned after a dec...
 
Using web quests to increase student learning2
Using web quests to increase student learning2Using web quests to increase student learning2
Using web quests to increase student learning2
 
Keynote presentation OOFHEC2016: Bart Rienties
Keynote presentation OOFHEC2016: Bart RientiesKeynote presentation OOFHEC2016: Bart Rienties
Keynote presentation OOFHEC2016: Bart Rienties
 
Technology-enabled learning-oriented assessment
Technology-enabled learning-oriented assessmentTechnology-enabled learning-oriented assessment
Technology-enabled learning-oriented assessment
 
How can we engage mathematics ITE students with research?
How can we engage mathematics ITE students with research?How can we engage mathematics ITE students with research?
How can we engage mathematics ITE students with research?
 
Data Driven Instrruction
Data Driven InstrructionData Driven Instrruction
Data Driven Instrruction
 
Ces Ed Tech Presentation 5 26
Ces Ed Tech Presentation  5 26Ces Ed Tech Presentation  5 26
Ces Ed Tech Presentation 5 26
 
Oral Defense presentation
Oral Defense presentationOral Defense presentation
Oral Defense presentation
 

Similar to Discussant SRHE Symposium "A cross-institutional perspective on merits and challenges of learning gains for Teaching Excellence Framework"

Personal Digital Inquiry: Connecting Learners in Ways That Matter
Personal Digital Inquiry: Connecting Learners in Ways That MatterPersonal Digital Inquiry: Connecting Learners in Ways That Matter
Personal Digital Inquiry: Connecting Learners in Ways That Matter
Julie Coiro
 
Implications of the student learning journey for teaching
Implications of the student learning journey for teachingImplications of the student learning journey for teaching
Implications of the student learning journey for teaching
Tansy Jessop
 
Academic Honesty in the Digital Age
Academic Honesty in the Digital AgeAcademic Honesty in the Digital Age
Academic Honesty in the Digital Age
The Association of International Schools of India
 
Academic honesty in the digital age sept 2014
Academic honesty in the digital age sept 2014Academic honesty in the digital age sept 2014
Academic honesty in the digital age sept 2014
Mick Purcell
 
Who cares? 2016 GCP Collaboratory Plenary
Who cares? 2016 GCP Collaboratory PlenaryWho cares? 2016 GCP Collaboratory Plenary
Who cares? 2016 GCP Collaboratory Plenary
bumbaugh
 
Project-Based Learning for Theory of Knowledge (TOK)
Project-Based Learning for Theory of Knowledge (TOK)Project-Based Learning for Theory of Knowledge (TOK)
Project-Based Learning for Theory of Knowledge (TOK)
jenniferjoelle
 
What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...
What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...
What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...
Jerilyn Veldof
 
Evolving the Loop: How ePortfolios drive a 21st century learning college
Evolving the Loop: How ePortfolios drive a 21st century learning collegeEvolving the Loop: How ePortfolios drive a 21st century learning college
Evolving the Loop: How ePortfolios drive a 21st century learning college
Niesha Ziehmke
 
Academic honesty in the digital age march 2014
Academic honesty in the digital age march 2014 Academic honesty in the digital age march 2014
Academic honesty in the digital age march 2014
mick purcell
 
Nais2 10presentation 100301185336 Phpapp02[1]
Nais2 10presentation 100301185336 Phpapp02[1]Nais2 10presentation 100301185336 Phpapp02[1]
Nais2 10presentation 100301185336 Phpapp02[1]
ConorCusack
 
Tony Wagner NAIS presentation
Tony Wagner NAIS presentationTony Wagner NAIS presentation
Tony Wagner NAIS presentation
guest0410aba
 
Tony Wagner NAIS presentation
Tony Wagner NAIS presentationTony Wagner NAIS presentation
Tony Wagner NAIS presentation
Jonathan Martin
 
Change Leadership In Education
Change Leadership In EducationChange Leadership In Education
Change Leadership In Education
Mark Luffman
 
Surprisingly simple strategies to engage students in complex learning
Surprisingly simple strategies to engage students in complex learningSurprisingly simple strategies to engage students in complex learning
Surprisingly simple strategies to engage students in complex learning
Tansy Jessop
 
Tony Wagner: The Global Achievement Gap
Tony Wagner: The Global Achievement GapTony Wagner: The Global Achievement Gap
Tony Wagner: The Global Achievement Gap
Lee Carlton
 
Nais presentation
Nais presentationNais presentation
Nais presentation
Lee Carlton
 
Introduction to Personal Digital Inquiry in Grades K-8
Introduction to Personal Digital Inquiry in Grades K-8Introduction to Personal Digital Inquiry in Grades K-8
Introduction to Personal Digital Inquiry in Grades K-8
Julie Coiro
 
JTC May 2013 - Flexible Pathways Session
JTC May 2013 - Flexible Pathways SessionJTC May 2013 - Flexible Pathways Session
JTC May 2013 - Flexible Pathways Session
Alberta Education
 
HE Course and Module Evaluation Conference - Kate Little
HE Course and Module Evaluation Conference - Kate LittleHE Course and Module Evaluation Conference - Kate Little
HE Course and Module Evaluation Conference - Kate Little
Association of University Administrators
 
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...
Bart Rienties
 

Similar to Discussant SRHE Symposium "A cross-institutional perspective on merits and challenges of learning gains for Teaching Excellence Framework" (20)

Personal Digital Inquiry: Connecting Learners in Ways That Matter
Personal Digital Inquiry: Connecting Learners in Ways That MatterPersonal Digital Inquiry: Connecting Learners in Ways That Matter
Personal Digital Inquiry: Connecting Learners in Ways That Matter
 
Implications of the student learning journey for teaching
Implications of the student learning journey for teachingImplications of the student learning journey for teaching
Implications of the student learning journey for teaching
 
Academic Honesty in the Digital Age
Academic Honesty in the Digital AgeAcademic Honesty in the Digital Age
Academic Honesty in the Digital Age
 
Academic honesty in the digital age sept 2014
Academic honesty in the digital age sept 2014Academic honesty in the digital age sept 2014
Academic honesty in the digital age sept 2014
 
Who cares? 2016 GCP Collaboratory Plenary
Who cares? 2016 GCP Collaboratory PlenaryWho cares? 2016 GCP Collaboratory Plenary
Who cares? 2016 GCP Collaboratory Plenary
 
Project-Based Learning for Theory of Knowledge (TOK)
Project-Based Learning for Theory of Knowledge (TOK)Project-Based Learning for Theory of Knowledge (TOK)
Project-Based Learning for Theory of Knowledge (TOK)
 
What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...
What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...
What's a Library to Do? Transforming the One-Shot Library Workshop for the Ne...
 
Evolving the Loop: How ePortfolios drive a 21st century learning college
Evolving the Loop: How ePortfolios drive a 21st century learning collegeEvolving the Loop: How ePortfolios drive a 21st century learning college
Evolving the Loop: How ePortfolios drive a 21st century learning college
 
Academic honesty in the digital age march 2014
Academic honesty in the digital age march 2014 Academic honesty in the digital age march 2014
Academic honesty in the digital age march 2014
 
Nais2 10presentation 100301185336 Phpapp02[1]
Nais2 10presentation 100301185336 Phpapp02[1]Nais2 10presentation 100301185336 Phpapp02[1]
Nais2 10presentation 100301185336 Phpapp02[1]
 
Tony Wagner NAIS presentation
Tony Wagner NAIS presentationTony Wagner NAIS presentation
Tony Wagner NAIS presentation
 
Tony Wagner NAIS presentation
Tony Wagner NAIS presentationTony Wagner NAIS presentation
Tony Wagner NAIS presentation
 
Change Leadership In Education
Change Leadership In EducationChange Leadership In Education
Change Leadership In Education
 
Surprisingly simple strategies to engage students in complex learning
Surprisingly simple strategies to engage students in complex learningSurprisingly simple strategies to engage students in complex learning
Surprisingly simple strategies to engage students in complex learning
 
Tony Wagner: The Global Achievement Gap
Tony Wagner: The Global Achievement GapTony Wagner: The Global Achievement Gap
Tony Wagner: The Global Achievement Gap
 
Nais presentation
Nais presentationNais presentation
Nais presentation
 
Introduction to Personal Digital Inquiry in Grades K-8
Introduction to Personal Digital Inquiry in Grades K-8Introduction to Personal Digital Inquiry in Grades K-8
Introduction to Personal Digital Inquiry in Grades K-8
 
JTC May 2013 - Flexible Pathways Session
JTC May 2013 - Flexible Pathways SessionJTC May 2013 - Flexible Pathways Session
JTC May 2013 - Flexible Pathways Session
 
HE Course and Module Evaluation Conference - Kate Little
HE Course and Module Evaluation Conference - Kate LittleHE Course and Module Evaluation Conference - Kate Little
HE Course and Module Evaluation Conference - Kate Little
 
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...
How learning gains and Quality Assurance are (mis)Aligned: An Interactive Wor...
 

More from Bart Rienties

Applying and translating learning design and analytics approaches in your ins...
Applying and translating learning design and analytics approaches in your ins...Applying and translating learning design and analytics approaches in your ins...
Applying and translating learning design and analytics approaches in your ins...
Bart Rienties
 
Keynote Presentation: Implementing learning analytics and learning design at ...
Keynote Presentation: Implementing learning analytics and learning design at ...Keynote Presentation: Implementing learning analytics and learning design at ...
Keynote Presentation: Implementing learning analytics and learning design at ...
Bart Rienties
 
Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...
Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...
Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...
Bart Rienties
 
How can you use learning analytics in your own research and practice: an intr...
How can you use learning analytics in your own research and practice: an intr...How can you use learning analytics in your own research and practice: an intr...
How can you use learning analytics in your own research and practice: an intr...
Bart Rienties
 
SAAIR: Implementing learning analytics at scale in an online world: lessons l...
SAAIR: Implementing learning analytics at scale in an online world: lessons l...SAAIR: Implementing learning analytics at scale in an online world: lessons l...
SAAIR: Implementing learning analytics at scale in an online world: lessons l...
Bart Rienties
 
OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...
OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...
OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...
Bart Rienties
 
Keynote Learning & Innovation Maastricht University
Keynote Learning & Innovation Maastricht UniversityKeynote Learning & Innovation Maastricht University
Keynote Learning & Innovation Maastricht University
Bart Rienties
 
Education 4.0 and Computer Science: A European perspective
Education 4.0 and Computer Science: A European perspectiveEducation 4.0 and Computer Science: A European perspective
Education 4.0 and Computer Science: A European perspective
Bart Rienties
 
Keynote Data Matters JISC What is the impact? Six years of learning analytics...
Keynote Data Matters JISC What is the impact? Six years of learning analytics...Keynote Data Matters JISC What is the impact? Six years of learning analytics...
Keynote Data Matters JISC What is the impact? Six years of learning analytics...
Bart Rienties
 
What have we learned from 6 years of implementing learning analytics amongst ...
What have we learned from 6 years of implementing learning analytics amongst ...What have we learned from 6 years of implementing learning analytics amongst ...
What have we learned from 6 years of implementing learning analytics amongst ...
Bart Rienties
 
Using Learning analytics to support learners and teachers at the Open University
Using Learning analytics to support learners and teachers at the Open UniversityUsing Learning analytics to support learners and teachers at the Open University
Using Learning analytics to support learners and teachers at the Open University
Bart Rienties
 
Using student data to transform teaching and learning
Using student data to transform teaching and learningUsing student data to transform teaching and learning
Using student data to transform teaching and learning
Bart Rienties
 
Lecture series: Using trace data or subjective data, that is the question dur...
Lecture series: Using trace data or subjective data, that is the question dur...Lecture series: Using trace data or subjective data, that is the question dur...
Lecture series: Using trace data or subjective data, that is the question dur...
Bart Rienties
 
How to analyse questionnaire data: an advanced session
How to analyse questionnaire data: an advanced sessionHow to analyse questionnaire data: an advanced session
How to analyse questionnaire data: an advanced session
Bart Rienties
 
Questionnaire design for beginners (Bart Rienties)
Questionnaire design for beginners (Bart Rienties)Questionnaire design for beginners (Bart Rienties)
Questionnaire design for beginners (Bart Rienties)
Bart Rienties
 
Learning analytics adoption in Higher Education: Reviewing six years of exper...
Learning analytics adoption in Higher Education: Reviewing six years of exper...Learning analytics adoption in Higher Education: Reviewing six years of exper...
Learning analytics adoption in Higher Education: Reviewing six years of exper...
Bart Rienties
 
«Learning Analytics at the Open University and the UK»
 «Learning Analytics at the Open University and the UK» «Learning Analytics at the Open University and the UK»
«Learning Analytics at the Open University and the UK»
Bart Rienties
 
Presentation LMU Munich: The power of learning analytics to unpack learning a...
Presentation LMU Munich: The power of learning analytics to unpack learning a...Presentation LMU Munich: The power of learning analytics to unpack learning a...
Presentation LMU Munich: The power of learning analytics to unpack learning a...
Bart Rienties
 
Educational Technology - opportunities and pitfalls How to make the most use...
Educational Technology - opportunities and pitfalls  How to make the most use...Educational Technology - opportunities and pitfalls  How to make the most use...
Educational Technology - opportunities and pitfalls How to make the most use...
Bart Rienties
 
Unpacking academic and social adjustment of internationalisation at a distanc...
Unpacking academic and social adjustment of internationalisation at a distanc...Unpacking academic and social adjustment of internationalisation at a distanc...
Unpacking academic and social adjustment of internationalisation at a distanc...
Bart Rienties
 

More from Bart Rienties (20)

Applying and translating learning design and analytics approaches in your ins...
Applying and translating learning design and analytics approaches in your ins...Applying and translating learning design and analytics approaches in your ins...
Applying and translating learning design and analytics approaches in your ins...
 
Keynote Presentation: Implementing learning analytics and learning design at ...
Keynote Presentation: Implementing learning analytics and learning design at ...Keynote Presentation: Implementing learning analytics and learning design at ...
Keynote Presentation: Implementing learning analytics and learning design at ...
 
Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...
Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...
Edutech_Europe Keynote Presentation: Implementing learning analytics and lear...
 
How can you use learning analytics in your own research and practice: an intr...
How can you use learning analytics in your own research and practice: an intr...How can you use learning analytics in your own research and practice: an intr...
How can you use learning analytics in your own research and practice: an intr...
 
SAAIR: Implementing learning analytics at scale in an online world: lessons l...
SAAIR: Implementing learning analytics at scale in an online world: lessons l...SAAIR: Implementing learning analytics at scale in an online world: lessons l...
SAAIR: Implementing learning analytics at scale in an online world: lessons l...
 
OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...
OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...
OU/Leverhulme Open World Learning: Knowledge Exchange and Book Launch Event p...
 
Keynote Learning & Innovation Maastricht University
Keynote Learning & Innovation Maastricht UniversityKeynote Learning & Innovation Maastricht University
Keynote Learning & Innovation Maastricht University
 
Education 4.0 and Computer Science: A European perspective
Education 4.0 and Computer Science: A European perspectiveEducation 4.0 and Computer Science: A European perspective
Education 4.0 and Computer Science: A European perspective
 
Keynote Data Matters JISC What is the impact? Six years of learning analytics...
Keynote Data Matters JISC What is the impact? Six years of learning analytics...Keynote Data Matters JISC What is the impact? Six years of learning analytics...
Keynote Data Matters JISC What is the impact? Six years of learning analytics...
 
What have we learned from 6 years of implementing learning analytics amongst ...
What have we learned from 6 years of implementing learning analytics amongst ...What have we learned from 6 years of implementing learning analytics amongst ...
What have we learned from 6 years of implementing learning analytics amongst ...
 
Using Learning analytics to support learners and teachers at the Open University
Using Learning analytics to support learners and teachers at the Open UniversityUsing Learning analytics to support learners and teachers at the Open University
Using Learning analytics to support learners and teachers at the Open University
 
Using student data to transform teaching and learning
Using student data to transform teaching and learningUsing student data to transform teaching and learning
Using student data to transform teaching and learning
 
Lecture series: Using trace data or subjective data, that is the question dur...
Lecture series: Using trace data or subjective data, that is the question dur...Lecture series: Using trace data or subjective data, that is the question dur...
Lecture series: Using trace data or subjective data, that is the question dur...
 
How to analyse questionnaire data: an advanced session
How to analyse questionnaire data: an advanced sessionHow to analyse questionnaire data: an advanced session
How to analyse questionnaire data: an advanced session
 
Questionnaire design for beginners (Bart Rienties)
Questionnaire design for beginners (Bart Rienties)Questionnaire design for beginners (Bart Rienties)
Questionnaire design for beginners (Bart Rienties)
 
Learning analytics adoption in Higher Education: Reviewing six years of exper...
Learning analytics adoption in Higher Education: Reviewing six years of exper...Learning analytics adoption in Higher Education: Reviewing six years of exper...
Learning analytics adoption in Higher Education: Reviewing six years of exper...
 
«Learning Analytics at the Open University and the UK»
 «Learning Analytics at the Open University and the UK» «Learning Analytics at the Open University and the UK»
«Learning Analytics at the Open University and the UK»
 
Presentation LMU Munich: The power of learning analytics to unpack learning a...
Presentation LMU Munich: The power of learning analytics to unpack learning a...Presentation LMU Munich: The power of learning analytics to unpack learning a...
Presentation LMU Munich: The power of learning analytics to unpack learning a...
 
Educational Technology - opportunities and pitfalls How to make the most use...
Educational Technology - opportunities and pitfalls  How to make the most use...Educational Technology - opportunities and pitfalls  How to make the most use...
Educational Technology - opportunities and pitfalls How to make the most use...
 
Unpacking academic and social adjustment of internationalisation at a distanc...
Unpacking academic and social adjustment of internationalisation at a distanc...Unpacking academic and social adjustment of internationalisation at a distanc...
Unpacking academic and social adjustment of internationalisation at a distanc...
 

Recently uploaded

C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
mulvey2
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptxBeyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
EduSkills OECD
 
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdfمصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
سمير بسيوني
 
The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
History of Stoke Newington
 
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdfREASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
giancarloi8888
 
SWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptxSWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptx
zuzanka
 
B. Ed Syllabus for babasaheb ambedkar education university.pdf
B. Ed Syllabus for babasaheb ambedkar education university.pdfB. Ed Syllabus for babasaheb ambedkar education university.pdf
B. Ed Syllabus for babasaheb ambedkar education university.pdf
BoudhayanBhattachari
 
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDFLifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Vivekanand Anglo Vedic Academy
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
Himanshu Rai
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
RAHUL
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
deepaannamalai16
 
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) CurriculumPhilippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
MJDuyan
 
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
National Information Standards Organization (NISO)
 
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptxChapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Denish Jangid
 
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.pptLevel 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Henry Hollis
 
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
zuzanka
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
iammrhaywood
 
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptxBIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
RidwanHassanYusuf
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
PECB
 

Recently uploaded (20)

C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptxBeyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
 
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdfمصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
 
The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
 
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdfREASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
 
SWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptxSWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptx
 
B. Ed Syllabus for babasaheb ambedkar education university.pdf
B. Ed Syllabus for babasaheb ambedkar education university.pdfB. Ed Syllabus for babasaheb ambedkar education university.pdf
B. Ed Syllabus for babasaheb ambedkar education university.pdf
 
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDFLifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
 
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) CurriculumPhilippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
 
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
 
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptxChapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptx
 
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.pptLevel 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
 
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
 
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptxBIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
 

Discussant SRHE Symposium "A cross-institutional perspective on merits and challenges of learning gains for Teaching Excellence Framework"

  • 1. Discussion: A cross-institutional perspective on merits and challenges of learning gains for Teaching Excellence Framework Bart Rienties https://twitter.com/LearningGains https://abclearninggains.com/ The results of the ABC project were made possible due to Jekaterina Rogaten, Bart Rienties, Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Ian Scott, Rhona Sharpe, Steve Warburton, and Denise Whitelock. Please contract Jekaterina.rogaten@open.ac.uk if you want to know more about ABC learning gains
  • 2. Learning gains is a complex topic
  • 3. Learning gains is a complex topic
  • 4. Learning gains: different conceptualisations • Conceptualisation of learning gains substantially different across the five papers • From private vs public • From affective to cognitive • From during a degree until employment • From student or staff perspective
  • 5. Learning gains: different methods and approaches • Qualitative • Interviews (Paper 1, 4, 5) • Sorting exercise (Paper 4) • Personal reflections (Paper 1) • Quantitative (Paper 3, 2) • (Affective) survey instruments (Paper 3) • Grades/degree outcomes (Paper 3, 2) • Different level of analysis • Thematic analysis • Semi-structured interviews • ANOVA/correlations (Paper 3) • Regressions and Multi-level modelling (Paper 3, 2) • Longitudinal (Paper 2, 3, 5) • Randomised Control Trials (Paper 5) vs convenience sampling
  • 6. Do we need fine-grained techniques? • “I learned nothing in College. Ok, ok, maybe that’s an exaggeration. I learned some things in college. I learned that I can’t stomach cheap beer; I’d rather play drinking games with a box of wine. I learned what kind of guys I should never date again. I learned how to live with kids my own age and quite frankly I do best living alone. Most importantly, I met two English professors who had a huge impact on my life, who taught me not just how to write but also to believe in myself. Meeting them made my college years worth it. For the most part, as I reflect on my undergraduate experience, I feel that it was a colossal waste of time. http://www.huffingtonpost.com/sarah-rose-attman/i-learned- nothing-in-coll_b_5606640.html. (Hughes, Paper 1)
  • 7. Do we need multi-level analysis? • “[A] limitation of [common learning gains approaches] is that general linear models are based on the assumption that observed data of one student are independent of observed data of another. This assumption is not true in situations where students from the same discipline, identical module, and same university have similar experiences (e.g., easy assessment in week 1, difficult assessment in week 6, and moderately hard final assessment in week 15) which means the variance in their scores are not independent and are linked. This means that simple linear models to measure learning gains may provide inaccurate and misleading outcomes.” (Rogaten et al., paper 2)
  • 8. Need for coherence: questions to authors • Which learning gains? • Can we develop a common, shared perspective of what learning gains are? • How to measure learning gains? • What approaches and measurements are appropriate for capturing and measuring learning gains? • From apples to oranges to busses and e-cigarettes • How are we going to take into consideration individual differences (e.g., prior qualifications, socio-economic) and learning design differences (e.g., different assessments, support structures) when comparing learning gains? • How are we going to compare learning gains across different disciplines, contexts, and universities?
  • 9. Discussion: A cross-institutional perspective on merits and challenges of learning gains for Teaching Excellence Framework Bart Rienties https://twitter.com/LearningGains https://abclearninggains.com/ The results of the ABC project were made possible due to Jekaterina Rogaten, Bart Rienties, Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Ian Scott, Rhona Sharpe, Steve Warburton, and Denise Whitelock. Please contract Jekaterina.rogaten@open.ac.uk if you want to know more about ABC learning gains

Editor's Notes

  1. Add reference
  2. Add reference
  3. Add reference
  4. Add reference
  5. Add reference
  6. Add reference