SlideShare a Scribd company logo
Incorporating digitally derived
information in large scale
educational assessment
programmes
Harvey Goldstein
University of Bristol
Large scale assessment
• Issues:
• Assessment for individual learning vs assessment for understanding
systems (‘accountability’) and educational processes: the latter will be
addressed.
• Information about attainment (tests, records) in relation to contextual
information such as SES, extra-institutional activities.
• Digital information may occur through tests themselves or through
contextual information e.g. derived from linked records either available
administratively or acquired through interrogating social media.
• A distinguishing feature of digital data is that it tends to raise new issues of
data quality, ethical issues and the volume of data to be processed.
Digitised (computerised) testing I
• Much assessment is now carried out using ‘adaptive’ algorithms that
present respondents with tasks based upon prior responses and/or
individual characteristics
• These typically depend strongly on a model for the relationship between
the response and one (possibly >1) underlying (latent) trait so that
information about the trait is continually updated to determine the next
task.
• The claimed advantage fro such a procedure is that it is ’efficient’ in terms
of time taken to reach a required accuracy on the trait, and that it is
implicitly ‘tailored’ so that the ‘difficulty’ of tasks is adjusted to match the
respondents ‘ability’.
• A major problem is the assumption of ‘unidimensionality’ and there is
anyway no guarantee that ‘convergence’ to the true trait value is assured.
Digitised (computerised) testing II
• More creative use of digitised testing is possible:
• Interactive test items can be utilised to elicit creativity
• Real life ‘project tasks’ can be simulated where projects are not feasible
• Feedback during testing can be utilised to ascertain responsiveness to such
feedback
• Choice of tasks can be implemented easily
• Large numbers of respondents can be used.
• Longitudinal information can be stored and built on.
• A formative assessment system can utilise the flexibility of a student
database.
Digitised contextual information. Acquisition
• Contextual information may be acquired either by direct
interrogation, e.g. via questionnaires or
• By linking respondents to existing databases
• Administrative data such as school records, health data, demographic family
data etc.
• Self generated data such as those on social media
• Issues:
• Ethical concerns about consent
• Data quality, especially from social media
• Vast quantities of potential data – how to select.
Digitally acquired information - quality
• If data sourced from admin records the quality of these may not be
apparent, nor the quality of the ‘linkage’ procedure.
• If sourced from social media, the context in which the data are
produced may be important, for example depending on whom the
perceived audience may be.
• This is not the case with traditionally collected data where the
environment is known and often strictly controlled such as in
experiments or interviews – there is research about the effect of data
collection environments on data quality and meaning.
International studies and digital data
• Studies such as PISA collect data largely in traditional ways within
fairly well controlled environments. Digital skills, such as reading
digital texts, navigating online etc have been incorporated.
• Is there a role for linking traditional forms of assessment to digital
media data (and other databases)?
• Would require cooperation with service providers such as facebook and
twitter
• Consent needed and safeguards against disclosure.
• Could provide interesting insights into students’ use of time and relationships
with traditional learning
Acquiring information digitally in general
• PISA already has plans for further assessing ‘digital literacy’ through
understanding internet ‘navigation’ and uses of online materials.
• Providing students with digital devices that monitor activities is a
further step and is already possible:
• Physical activities
• Learning activities
• Recording of life events and feelings
• Would seem to be worth developing:
• Can be controlled and quality assessed
• Can be motivating
• Can provide data unobtainable in other ways
Overview
• Large scale and especially international assessment systems cannot
ignore student use of digital information.
• A useful area for research is the use of ‘wearable’ devices to capture
dynamically student’s interactions with the real and virtual worlds.
• Devices to monitor physical activity are now common.
• A challenge exists to develop devices to monitor interactions with
others and to record different types of mental activity.
• A challenge also exists for data analysts to model such data.
Thank you for listening

More Related Content

What's hot

The Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and ToolsThe Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and Tools
Christina Sax
 
Exploring Digital Assessment Strategies in a Digital Age
Exploring Digital Assessment Strategies in a Digital AgeExploring Digital Assessment Strategies in a Digital Age
Exploring Digital Assessment Strategies in a Digital Age
Greig Krull
 
Tracking learners digital experience: the benefits and impacts
Tracking learners digital experience: the benefits and impactsTracking learners digital experience: the benefits and impacts
Tracking learners digital experience: the benefits and impacts
Sarah Knight
 
OLC Innovate: Why Isn’t There More Cross-Institutional Research?
OLC Innovate: Why Isn’t There More Cross-Institutional Research?OLC Innovate: Why Isn’t There More Cross-Institutional Research?
OLC Innovate: Why Isn’t There More Cross-Institutional Research?
Tanya Joosten
 
Learning Analytics
Learning AnalyticsLearning Analytics
Learning Analytics
Stian Håklev
 
SHEILA-CRLI seminar
SHEILA-CRLI seminarSHEILA-CRLI seminar
SHEILA-CRLI seminar
Yi-Shan Tsai
 
Using A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic ResponseUsing A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic Response
Tanya Joosten
 
Points of Strength & Distinction at Assiut University Faculty of Education (A...
Points of Strength & Distinction at Assiut University Faculty of Education (A...Points of Strength & Distinction at Assiut University Faculty of Education (A...
Points of Strength & Distinction at Assiut University Faculty of Education (A...
memogreat
 
Innovating pedagogy
Innovating pedagogyInnovating pedagogy
Innovating pedagogy
Rebecca Ferguson
 
Building Data Literacy Among Middle School Administrators and Teachers
Building Data Literacy Among Middle School Administrators and TeachersBuilding Data Literacy Among Middle School Administrators and Teachers
Building Data Literacy Among Middle School Administrators and Teachers
North Carolina Association for Middle Level Education
 
Moocs: what the research tells us
Moocs: what the research tells usMoocs: what the research tells us
Moocs: what the research tells us
Rebecca Ferguson
 
Peter Chatterton
Peter ChattertonPeter Chatterton
Peter Chatterton
Lucia Garcia
 
Online text assignment
Online text assignmentOnline text assignment
Online text assignment
ebrebru
 
SOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the artSOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the art
Rebecca Ferguson
 
The ethics of MOOC research: why we should involve learners
The ethics of MOOC research: why we should involve learnersThe ethics of MOOC research: why we should involve learners
The ethics of MOOC research: why we should involve learners
Rebecca Ferguson
 
Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...
Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...
Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...
Tanya Joosten
 
Closing the Gap With STEM Education: Why, What, and How
Closing the Gap With STEM Education: Why, What, and HowClosing the Gap With STEM Education: Why, What, and How
Closing the Gap With STEM Education: Why, What, and How
North Carolina Association for Middle Level Education
 
Excellence rome - v03 2016
Excellence   rome - v03 2016Excellence   rome - v03 2016
Excellence rome - v03 2016
Ebba Ossiannilsson
 
Learning Analytics: What is it? Why do it? And how?
Learning Analytics: What is it? Why do it?  And how?Learning Analytics: What is it? Why do it?  And how?
Learning Analytics: What is it? Why do it? And how?
Timothy Harfield
 
Learning analytics: the state of the art and the future
Learning analytics: the state of the art and the futureLearning analytics: the state of the art and the future
Learning analytics: the state of the art and the future
Rebecca Ferguson
 

What's hot (20)

The Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and ToolsThe Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and Tools
 
Exploring Digital Assessment Strategies in a Digital Age
Exploring Digital Assessment Strategies in a Digital AgeExploring Digital Assessment Strategies in a Digital Age
Exploring Digital Assessment Strategies in a Digital Age
 
Tracking learners digital experience: the benefits and impacts
Tracking learners digital experience: the benefits and impactsTracking learners digital experience: the benefits and impacts
Tracking learners digital experience: the benefits and impacts
 
OLC Innovate: Why Isn’t There More Cross-Institutional Research?
OLC Innovate: Why Isn’t There More Cross-Institutional Research?OLC Innovate: Why Isn’t There More Cross-Institutional Research?
OLC Innovate: Why Isn’t There More Cross-Institutional Research?
 
Learning Analytics
Learning AnalyticsLearning Analytics
Learning Analytics
 
SHEILA-CRLI seminar
SHEILA-CRLI seminarSHEILA-CRLI seminar
SHEILA-CRLI seminar
 
Using A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic ResponseUsing A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic Response
 
Points of Strength & Distinction at Assiut University Faculty of Education (A...
Points of Strength & Distinction at Assiut University Faculty of Education (A...Points of Strength & Distinction at Assiut University Faculty of Education (A...
Points of Strength & Distinction at Assiut University Faculty of Education (A...
 
Innovating pedagogy
Innovating pedagogyInnovating pedagogy
Innovating pedagogy
 
Building Data Literacy Among Middle School Administrators and Teachers
Building Data Literacy Among Middle School Administrators and TeachersBuilding Data Literacy Among Middle School Administrators and Teachers
Building Data Literacy Among Middle School Administrators and Teachers
 
Moocs: what the research tells us
Moocs: what the research tells usMoocs: what the research tells us
Moocs: what the research tells us
 
Peter Chatterton
Peter ChattertonPeter Chatterton
Peter Chatterton
 
Online text assignment
Online text assignmentOnline text assignment
Online text assignment
 
SOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the artSOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the art
 
The ethics of MOOC research: why we should involve learners
The ethics of MOOC research: why we should involve learnersThe ethics of MOOC research: why we should involve learners
The ethics of MOOC research: why we should involve learners
 
Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...
Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...
Promoting Effective Teaching and Learning Ecosystems via Research Proven Prac...
 
Closing the Gap With STEM Education: Why, What, and How
Closing the Gap With STEM Education: Why, What, and HowClosing the Gap With STEM Education: Why, What, and How
Closing the Gap With STEM Education: Why, What, and How
 
Excellence rome - v03 2016
Excellence   rome - v03 2016Excellence   rome - v03 2016
Excellence rome - v03 2016
 
Learning Analytics: What is it? Why do it? And how?
Learning Analytics: What is it? Why do it?  And how?Learning Analytics: What is it? Why do it?  And how?
Learning Analytics: What is it? Why do it? And how?
 
Learning analytics: the state of the art and the future
Learning analytics: the state of the art and the futureLearning analytics: the state of the art and the future
Learning analytics: the state of the art and the future
 

Similar to Traditional Large Scale Educational Assessment and the Incorporation of Digitally Derived Information

Sdal air education workforce analytics workshop jan. 7 , 2014.pptx
Sdal air education workforce analytics workshop jan. 7 , 2014.pptxSdal air education workforce analytics workshop jan. 7 , 2014.pptx
Sdal air education workforce analytics workshop jan. 7 , 2014.pptx
kimlyman
 
Big Data for Student Learning
Big Data for Student LearningBig Data for Student Learning
Big Data for Student Learning
Marie Bienkowski
 
Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014
Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014
Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014
ICPSR
 
[DSC Europe 22] Machine learning algorithms as tools for student success pred...
[DSC Europe 22] Machine learning algorithms as tools for student success pred...[DSC Europe 22] Machine learning algorithms as tools for student success pred...
[DSC Europe 22] Machine learning algorithms as tools for student success pred...
DataScienceConferenc1
 
Supporting staff to teach effectively online
Supporting staff to teach effectively onlineSupporting staff to teach effectively online
Supporting staff to teach effectively online
Jisc
 
From Reporting to Insight to Action
From Reporting to Insight to ActionFrom Reporting to Insight to Action
From Reporting to Insight to Action
Ellen Wagner
 
Ellen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to WorkEllen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to Work
Alexandra M. Pickett
 
Needs Assessment
Needs AssessmentNeeds Assessment
Needs Assessment
Leila Zaim
 
Measuring What Matters: Noncognitive Skills - GRIT
Measuring What Matters: Noncognitive Skills - GRITMeasuring What Matters: Noncognitive Skills - GRIT
Measuring What Matters: Noncognitive Skills - GRIT
SmarterServices Owen
 
How data informs decision making 2
How data informs decision making 2How data informs decision making 2
How data informs decision making 2
jaccalder
 
eAssessment Policy and Practice
eAssessment Policy and PracticeeAssessment Policy and Practice
eAssessment Policy and Practice
drdjwalker
 
RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...
ASIS&T
 
Module 3 - Improving Current Business with External Data- Online
Module 3 - Improving Current Business with External Data- Online Module 3 - Improving Current Business with External Data- Online
Module 3 - Improving Current Business with External Data- Online
caniceconsulting
 
Education data mining presentation
Education data mining presentationEducation data mining presentation
Education data mining presentation
Nishabhanot1
 
Computing: planning, assessment and resources
Computing: planning, assessment and resourcesComputing: planning, assessment and resources
Computing: planning, assessment and resources
Miles Berry
 
BUSINESS RESEARCH METHODS-DATA COLLECTION METHODS
BUSINESS RESEARCH METHODS-DATA COLLECTION METHODSBUSINESS RESEARCH METHODS-DATA COLLECTION METHODS
BUSINESS RESEARCH METHODS-DATA COLLECTION METHODS
mariaboaler1
 
Big Data and Student Retention
Big Data and Student RetentionBig Data and Student Retention
Big Data and Student Retention
Vince Kellen, Ph.D.
 
Ensuring data quality
Ensuring data qualityEnsuring data quality
Ensuring data quality
IUPUI
 
Online Assessment
Online AssessmentOnline Assessment
Online Assessment
EADTU
 
Data collection instruments (2)
Data collection instruments (2)Data collection instruments (2)
Data collection instruments (2)
pesantez666
 

Similar to Traditional Large Scale Educational Assessment and the Incorporation of Digitally Derived Information (20)

Sdal air education workforce analytics workshop jan. 7 , 2014.pptx
Sdal air education workforce analytics workshop jan. 7 , 2014.pptxSdal air education workforce analytics workshop jan. 7 , 2014.pptx
Sdal air education workforce analytics workshop jan. 7 , 2014.pptx
 
Big Data for Student Learning
Big Data for Student LearningBig Data for Student Learning
Big Data for Student Learning
 
Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014
Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014
Instructional Data Sets from Q-step Launch Event (Univ of Exeter) 3-20-2014
 
[DSC Europe 22] Machine learning algorithms as tools for student success pred...
[DSC Europe 22] Machine learning algorithms as tools for student success pred...[DSC Europe 22] Machine learning algorithms as tools for student success pred...
[DSC Europe 22] Machine learning algorithms as tools for student success pred...
 
Supporting staff to teach effectively online
Supporting staff to teach effectively onlineSupporting staff to teach effectively online
Supporting staff to teach effectively online
 
From Reporting to Insight to Action
From Reporting to Insight to ActionFrom Reporting to Insight to Action
From Reporting to Insight to Action
 
Ellen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to WorkEllen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to Work
 
Needs Assessment
Needs AssessmentNeeds Assessment
Needs Assessment
 
Measuring What Matters: Noncognitive Skills - GRIT
Measuring What Matters: Noncognitive Skills - GRITMeasuring What Matters: Noncognitive Skills - GRIT
Measuring What Matters: Noncognitive Skills - GRIT
 
How data informs decision making 2
How data informs decision making 2How data informs decision making 2
How data informs decision making 2
 
eAssessment Policy and Practice
eAssessment Policy and PracticeeAssessment Policy and Practice
eAssessment Policy and Practice
 
RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...
 
Module 3 - Improving Current Business with External Data- Online
Module 3 - Improving Current Business with External Data- Online Module 3 - Improving Current Business with External Data- Online
Module 3 - Improving Current Business with External Data- Online
 
Education data mining presentation
Education data mining presentationEducation data mining presentation
Education data mining presentation
 
Computing: planning, assessment and resources
Computing: planning, assessment and resourcesComputing: planning, assessment and resources
Computing: planning, assessment and resources
 
BUSINESS RESEARCH METHODS-DATA COLLECTION METHODS
BUSINESS RESEARCH METHODS-DATA COLLECTION METHODSBUSINESS RESEARCH METHODS-DATA COLLECTION METHODS
BUSINESS RESEARCH METHODS-DATA COLLECTION METHODS
 
Big Data and Student Retention
Big Data and Student RetentionBig Data and Student Retention
Big Data and Student Retention
 
Ensuring data quality
Ensuring data qualityEnsuring data quality
Ensuring data quality
 
Online Assessment
Online AssessmentOnline Assessment
Online Assessment
 
Data collection instruments (2)
Data collection instruments (2)Data collection instruments (2)
Data collection instruments (2)
 

More from CITE

Keynote 1: Teaching and Learning Computational Thinking at Scale
Keynote 1: Teaching and Learning Computational Thinking at ScaleKeynote 1: Teaching and Learning Computational Thinking at Scale
Keynote 1: Teaching and Learning Computational Thinking at Scale
CITE
 
Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...
Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...
Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...
CITE
 
Scaling up Assessment for Learning
Scaling up Assessment for LearningScaling up Assessment for Learning
Scaling up Assessment for Learning
CITE
 
Seminar on policy study on e-Learning in Informal Learning contexts
Seminar on policy study on e-Learning in Informal Learning contextsSeminar on policy study on e-Learning in Informal Learning contexts
Seminar on policy study on e-Learning in Informal Learning contexts
CITE
 
Seminar on policy study on e-Learning in Formal & Open Learning contexts
Seminar on policy study on e-Learning in Formal & Open Learning contextsSeminar on policy study on e-Learning in Formal & Open Learning contexts
Seminar on policy study on e-Learning in Formal & Open Learning contexts
CITE
 
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
CITE
 
G:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-Fred
G:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-FredG:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-Fred
G:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-Fred
CITE
 
Dr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based LearningDr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based Learning
CITE
 
Analogy, Causality, and Discovery in Science: The engines of human thought
Analogy, Causality, and Discovery in Science: The engines of human thoughtAnalogy, Causality, and Discovery in Science: The engines of human thought
Analogy, Causality, and Discovery in Science: The engines of human thought
CITE
 
Educating the Scientific Brain and Mind: Insights from The Science of Learnin...
Educating the Scientific Brain and Mind: Insights from The Science of Learnin...Educating the Scientific Brain and Mind: Insights from The Science of Learnin...
Educating the Scientific Brain and Mind: Insights from The Science of Learnin...
CITE
 
Science of Learning — Why it matters to schools and families?
Science of Learning — Why it matters to schools and families?Science of Learning — Why it matters to schools and families?
Science of Learning — Why it matters to schools and families?
CITE
 
Understanding the self through self bias
Understanding the self through self biasUnderstanding the self through self bias
Understanding the self through self bias
CITE
 
The implementation of "Reading Battle" in Lam Tin Methodist Primary School
The implementation of "Reading Battle" in Lam Tin Methodist Primary SchoolThe implementation of "Reading Battle" in Lam Tin Methodist Primary School
The implementation of "Reading Battle" in Lam Tin Methodist Primary School
CITE
 
Strengthening students' reading comprehension ability (both Chinese and Engli...
Strengthening students' reading comprehension ability (both Chinese and Engli...Strengthening students' reading comprehension ability (both Chinese and Engli...
Strengthening students' reading comprehension ability (both Chinese and Engli...
CITE
 
Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"
Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"
Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"
CITE
 
Xiao Hu "Learning Analytics Initiatives"
Xiao Hu "Learning Analytics Initiatives"Xiao Hu "Learning Analytics Initiatives"
Xiao Hu "Learning Analytics Initiatives"
CITE
 
Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...
Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...
Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...
CITE
 
Phil Winne "Learning Analytics for Learning Science When N = me"
Phil Winne "Learning Analytics for Learning Science When N = me"Phil Winne "Learning Analytics for Learning Science When N = me"
Phil Winne "Learning Analytics for Learning Science When N = me"
CITE
 
Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...
Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...
Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...
CITE
 
Chi-Un Lei "Text Mining and Educational Discourse"
Chi-Un Lei "Text Mining and Educational Discourse"Chi-Un Lei "Text Mining and Educational Discourse"
Chi-Un Lei "Text Mining and Educational Discourse"
CITE
 

More from CITE (20)

Keynote 1: Teaching and Learning Computational Thinking at Scale
Keynote 1: Teaching and Learning Computational Thinking at ScaleKeynote 1: Teaching and Learning Computational Thinking at Scale
Keynote 1: Teaching and Learning Computational Thinking at Scale
 
Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...
Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...
Keynote 2: Social Epistemic Cognition in Engineering Learning: Theory, Pedago...
 
Scaling up Assessment for Learning
Scaling up Assessment for LearningScaling up Assessment for Learning
Scaling up Assessment for Learning
 
Seminar on policy study on e-Learning in Informal Learning contexts
Seminar on policy study on e-Learning in Informal Learning contextsSeminar on policy study on e-Learning in Informal Learning contexts
Seminar on policy study on e-Learning in Informal Learning contexts
 
Seminar on policy study on e-Learning in Formal & Open Learning contexts
Seminar on policy study on e-Learning in Formal & Open Learning contextsSeminar on policy study on e-Learning in Formal & Open Learning contexts
Seminar on policy study on e-Learning in Formal & Open Learning contexts
 
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
 
G:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-Fred
G:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-FredG:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-Fred
G:\CITERS2015\29May2015\2 Invited-Talk-2-Sidorko-Fred
 
Dr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based LearningDr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based Learning
 
Analogy, Causality, and Discovery in Science: The engines of human thought
Analogy, Causality, and Discovery in Science: The engines of human thoughtAnalogy, Causality, and Discovery in Science: The engines of human thought
Analogy, Causality, and Discovery in Science: The engines of human thought
 
Educating the Scientific Brain and Mind: Insights from The Science of Learnin...
Educating the Scientific Brain and Mind: Insights from The Science of Learnin...Educating the Scientific Brain and Mind: Insights from The Science of Learnin...
Educating the Scientific Brain and Mind: Insights from The Science of Learnin...
 
Science of Learning — Why it matters to schools and families?
Science of Learning — Why it matters to schools and families?Science of Learning — Why it matters to schools and families?
Science of Learning — Why it matters to schools and families?
 
Understanding the self through self bias
Understanding the self through self biasUnderstanding the self through self bias
Understanding the self through self bias
 
The implementation of "Reading Battle" in Lam Tin Methodist Primary School
The implementation of "Reading Battle" in Lam Tin Methodist Primary SchoolThe implementation of "Reading Battle" in Lam Tin Methodist Primary School
The implementation of "Reading Battle" in Lam Tin Methodist Primary School
 
Strengthening students' reading comprehension ability (both Chinese and Engli...
Strengthening students' reading comprehension ability (both Chinese and Engli...Strengthening students' reading comprehension ability (both Chinese and Engli...
Strengthening students' reading comprehension ability (both Chinese and Engli...
 
Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"
Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"
Gobert, Dede, Martin, Rose "Panel: Learning Analytics and Learning Sciences"
 
Xiao Hu "Learning Analytics Initiatives"
Xiao Hu "Learning Analytics Initiatives"Xiao Hu "Learning Analytics Initiatives"
Xiao Hu "Learning Analytics Initiatives"
 
Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...
Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...
Tiffany Barnes "Making a meaningful difference: Leveraging data to improve le...
 
Phil Winne "Learning Analytics for Learning Science When N = me"
Phil Winne "Learning Analytics for Learning Science When N = me"Phil Winne "Learning Analytics for Learning Science When N = me"
Phil Winne "Learning Analytics for Learning Science When N = me"
 
Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...
Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...
Xiao Hu "Overview of the Space of Learning Analytics and Educational Data Min...
 
Chi-Un Lei "Text Mining and Educational Discourse"
Chi-Un Lei "Text Mining and Educational Discourse"Chi-Un Lei "Text Mining and Educational Discourse"
Chi-Un Lei "Text Mining and Educational Discourse"
 

Recently uploaded

How to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 WebsiteHow to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 Website
Celine George
 
1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx
UmeshTimilsina1
 
C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
Scholarhat
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
Celine George
 
Introduction to Google Productivity Tools for Office and Personal Use
Introduction to Google Productivity Tools for Office and Personal UseIntroduction to Google Productivity Tools for Office and Personal Use
Introduction to Google Productivity Tools for Office and Personal Use
Excellence Foundation for South Sudan
 
NAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource BookNAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource Book
lakitawilson
 
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptxKesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
artenzmartenkai
 
A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...
Association for Project Management
 
View Inheritance in Odoo 17 - Odoo 17 Slides
View Inheritance in Odoo 17 - Odoo 17  SlidesView Inheritance in Odoo 17 - Odoo 17  Slides
View Inheritance in Odoo 17 - Odoo 17 Slides
Celine George
 
Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)
Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)
Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)
Cátedra Banco Santander
 
10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf
10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf
10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf
SSRCreations
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
DrRavindrakshirsagar1
 
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour International
 
Imagination in Computer Science Research
Imagination in Computer Science ResearchImagination in Computer Science Research
Imagination in Computer Science Research
Abhik Roychoudhury
 
How to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POSHow to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POS
Celine George
 
E-learning Odoo 17 New features - Odoo 17 Slides
E-learning Odoo 17  New features - Odoo 17 SlidesE-learning Odoo 17  New features - Odoo 17 Slides
E-learning Odoo 17 New features - Odoo 17 Slides
Celine George
 
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 SlidesHow to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
Celine George
 
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
Nguyen Thanh Tu Collection
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
DrRavindrakshirsagar1
 
SD_Integrating 21st Century Skills in Classroom-based Assessment.pptx
SD_Integrating 21st Century Skills in Classroom-based Assessment.pptxSD_Integrating 21st Century Skills in Classroom-based Assessment.pptx
SD_Integrating 21st Century Skills in Classroom-based Assessment.pptx
elwoodprias1
 

Recently uploaded (20)

How to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 WebsiteHow to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 Website
 
1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx
 
C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
 
Introduction to Google Productivity Tools for Office and Personal Use
Introduction to Google Productivity Tools for Office and Personal UseIntroduction to Google Productivity Tools for Office and Personal Use
Introduction to Google Productivity Tools for Office and Personal Use
 
NAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource BookNAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource Book
 
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptxKesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
 
A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...
 
View Inheritance in Odoo 17 - Odoo 17 Slides
View Inheritance in Odoo 17 - Odoo 17  SlidesView Inheritance in Odoo 17 - Odoo 17  Slides
View Inheritance in Odoo 17 - Odoo 17 Slides
 
Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)
Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)
Cómo crear video-tutoriales con ScreenPal (2 de julio de 2024)
 
10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf
10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf
10th Social Studies Enrichment Material (Abhyasa Deepika) EM.pdf
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
 
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
 
Imagination in Computer Science Research
Imagination in Computer Science ResearchImagination in Computer Science Research
Imagination in Computer Science Research
 
How to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POSHow to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POS
 
E-learning Odoo 17 New features - Odoo 17 Slides
E-learning Odoo 17  New features - Odoo 17 SlidesE-learning Odoo 17  New features - Odoo 17 Slides
E-learning Odoo 17 New features - Odoo 17 Slides
 
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 SlidesHow to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
 
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
 
SD_Integrating 21st Century Skills in Classroom-based Assessment.pptx
SD_Integrating 21st Century Skills in Classroom-based Assessment.pptxSD_Integrating 21st Century Skills in Classroom-based Assessment.pptx
SD_Integrating 21st Century Skills in Classroom-based Assessment.pptx
 

Traditional Large Scale Educational Assessment and the Incorporation of Digitally Derived Information

  • 1. Incorporating digitally derived information in large scale educational assessment programmes Harvey Goldstein University of Bristol
  • 2. Large scale assessment • Issues: • Assessment for individual learning vs assessment for understanding systems (‘accountability’) and educational processes: the latter will be addressed. • Information about attainment (tests, records) in relation to contextual information such as SES, extra-institutional activities. • Digital information may occur through tests themselves or through contextual information e.g. derived from linked records either available administratively or acquired through interrogating social media. • A distinguishing feature of digital data is that it tends to raise new issues of data quality, ethical issues and the volume of data to be processed.
  • 3. Digitised (computerised) testing I • Much assessment is now carried out using ‘adaptive’ algorithms that present respondents with tasks based upon prior responses and/or individual characteristics • These typically depend strongly on a model for the relationship between the response and one (possibly >1) underlying (latent) trait so that information about the trait is continually updated to determine the next task. • The claimed advantage fro such a procedure is that it is ’efficient’ in terms of time taken to reach a required accuracy on the trait, and that it is implicitly ‘tailored’ so that the ‘difficulty’ of tasks is adjusted to match the respondents ‘ability’. • A major problem is the assumption of ‘unidimensionality’ and there is anyway no guarantee that ‘convergence’ to the true trait value is assured.
  • 4. Digitised (computerised) testing II • More creative use of digitised testing is possible: • Interactive test items can be utilised to elicit creativity • Real life ‘project tasks’ can be simulated where projects are not feasible • Feedback during testing can be utilised to ascertain responsiveness to such feedback • Choice of tasks can be implemented easily • Large numbers of respondents can be used. • Longitudinal information can be stored and built on. • A formative assessment system can utilise the flexibility of a student database.
  • 5. Digitised contextual information. Acquisition • Contextual information may be acquired either by direct interrogation, e.g. via questionnaires or • By linking respondents to existing databases • Administrative data such as school records, health data, demographic family data etc. • Self generated data such as those on social media • Issues: • Ethical concerns about consent • Data quality, especially from social media • Vast quantities of potential data – how to select.
  • 6. Digitally acquired information - quality • If data sourced from admin records the quality of these may not be apparent, nor the quality of the ‘linkage’ procedure. • If sourced from social media, the context in which the data are produced may be important, for example depending on whom the perceived audience may be. • This is not the case with traditionally collected data where the environment is known and often strictly controlled such as in experiments or interviews – there is research about the effect of data collection environments on data quality and meaning.
  • 7. International studies and digital data • Studies such as PISA collect data largely in traditional ways within fairly well controlled environments. Digital skills, such as reading digital texts, navigating online etc have been incorporated. • Is there a role for linking traditional forms of assessment to digital media data (and other databases)? • Would require cooperation with service providers such as facebook and twitter • Consent needed and safeguards against disclosure. • Could provide interesting insights into students’ use of time and relationships with traditional learning
  • 8. Acquiring information digitally in general • PISA already has plans for further assessing ‘digital literacy’ through understanding internet ‘navigation’ and uses of online materials. • Providing students with digital devices that monitor activities is a further step and is already possible: • Physical activities • Learning activities • Recording of life events and feelings • Would seem to be worth developing: • Can be controlled and quality assessed • Can be motivating • Can provide data unobtainable in other ways
  • 9. Overview • Large scale and especially international assessment systems cannot ignore student use of digital information. • A useful area for research is the use of ‘wearable’ devices to capture dynamically student’s interactions with the real and virtual worlds. • Devices to monitor physical activity are now common. • A challenge exists to develop devices to monitor interactions with others and to record different types of mental activity. • A challenge also exists for data analysts to model such data.
  • 10. Thank you for listening