This document outlines a 3-year research plan on learning analytics at Seoul National University. In the first year, the researchers will develop a model for learning analytics by reviewing existing literature and metrics, collecting input from experts, and classifying teaching and learning activities into measurable units. They will then propose a national learning metrics framework. In the second year, the researchers will apply the model by collecting and analyzing learning data to inform application and revision. The third year involves expanding the learning analytics service based on the previous years' work.
Research program educationaldataanalytics4personalisedt&l-2017Demetrios G. Sampson
Educational Data Analytics for Personalised Teaching and Learning
Keynote Speaker
2017 Symposium on Taiwan-Estonia Research Cooperation, Taipei, Taiwan
6-9 March 2017
Keynote address Analytics4Action Evaluation Framework: a review of evidence-...Bart Rienties
Bart Rienties is a Reader in Learning Analytics at the Institute of Educational Technology at the Open University UK. He is programme director Learning Analytics within IET and Chair of Analytics4Action project, which focuses on evidence-based research on interventions on OU modules to enhance student experience. As educational psychologist, he conducts multi-disciplinary research on work-based and collaborative learning environments and focuses on the role of social interaction in learning, which is published in leading academic journals and books. His primary research interests are focussed on Learning Analytics, Computer-Supported Collaborative Learning, and the role of motivation in learning. Furthermore, Bart is interested in broader internationalisation aspects of higher education. He successfully led a range of institutional/national/European projects and received several awards for his educational innovation projects.
Keynote H818 The Power of (In)formal learning: a learning analytics approachBart Rienties
A special thanks to Avinash Boroowa, Simon Cross, Lee Farrington-Flint, Christothea Herodotou, Lynda Prescott, Kevin Mayles, Tom Olney, Lisette Toetenel, John Woodthorpe and others…A special thanks to Prof Belinda Tynan for her continuous support on analytics at the OU UK
Promoting Data Literacy at the Grassroots (ACRL 2015, Portland, OR)Adam Beauchamp
Presentation given at ACRL 2015, with Christine Murray, on teaching undergraduate students to discover and evaluate datasets for secondary data analysis.
Teaching and Learning Analytics to Support the Classroom Teacher Inquiry
Invited Tutorial
IEEE Global Engineering Education Conference (EDUCON2017), University of Piraeus, Greece
26-28 April 2017
Invited Tutorial
Συνέδριο ΕΤΠΕ2017, ΑΙΣΠΕΤΕ, Ελλάδα
21-23 April 2017
Invited Tutorial
8th IEEE International Conference on Technology for Education (T4E 2016), IIT Bombay, Mumbai, India
1 December 2016
Research program educationaldataanalytics4personalisedt&l-2017Demetrios G. Sampson
Educational Data Analytics for Personalised Teaching and Learning
Keynote Speaker
2017 Symposium on Taiwan-Estonia Research Cooperation, Taipei, Taiwan
6-9 March 2017
Keynote address Analytics4Action Evaluation Framework: a review of evidence-...Bart Rienties
Bart Rienties is a Reader in Learning Analytics at the Institute of Educational Technology at the Open University UK. He is programme director Learning Analytics within IET and Chair of Analytics4Action project, which focuses on evidence-based research on interventions on OU modules to enhance student experience. As educational psychologist, he conducts multi-disciplinary research on work-based and collaborative learning environments and focuses on the role of social interaction in learning, which is published in leading academic journals and books. His primary research interests are focussed on Learning Analytics, Computer-Supported Collaborative Learning, and the role of motivation in learning. Furthermore, Bart is interested in broader internationalisation aspects of higher education. He successfully led a range of institutional/national/European projects and received several awards for his educational innovation projects.
Keynote H818 The Power of (In)formal learning: a learning analytics approachBart Rienties
A special thanks to Avinash Boroowa, Simon Cross, Lee Farrington-Flint, Christothea Herodotou, Lynda Prescott, Kevin Mayles, Tom Olney, Lisette Toetenel, John Woodthorpe and others…A special thanks to Prof Belinda Tynan for her continuous support on analytics at the OU UK
Promoting Data Literacy at the Grassroots (ACRL 2015, Portland, OR)Adam Beauchamp
Presentation given at ACRL 2015, with Christine Murray, on teaching undergraduate students to discover and evaluate datasets for secondary data analysis.
Teaching and Learning Analytics to Support the Classroom Teacher Inquiry
Invited Tutorial
IEEE Global Engineering Education Conference (EDUCON2017), University of Piraeus, Greece
26-28 April 2017
Invited Tutorial
Συνέδριο ΕΤΠΕ2017, ΑΙΣΠΕΤΕ, Ελλάδα
21-23 April 2017
Invited Tutorial
8th IEEE International Conference on Technology for Education (T4E 2016), IIT Bombay, Mumbai, India
1 December 2016
The power of learning analytics to measure learning gains: an OU, Surrey and ...Bart Rienties
Learning gains has increasingly become apparent within the HE literature, gained traction in government policies in the UK, and are at the heart of Teaching Excellence Framework (TFL). As such, this raises a question to what extent teaching and learning environment can actually predict students’ learning gains using principles of learning analytics. In this presentation, which is joined work with University of Surrey and Oxford Brookes, I will focus on some preliminary findings based upon developing and testing an Affective-Behaviour-Cognition learning gains model using longitudinal approach. The main aim of the research is to examine whether learning gains occur on all three levels of Affective-Behaviour-Cognition model and whether any particular student or course characteristics can predict learning gains or lack of learning and dropout. For more info, see https://abclearninggains.com/
Προσκεκλημένη Ομιλία
"Αναλυτική Εκπαιδευτικών Δεδομένων στην Σχολική Πράξη"
5ο Πανελλήνιο Επιστημονικό Συνέδριο
«Ένταξη και Χρήση των ΤΠΕ στην Εκπαιδευτική Διαδικασία»
21-23 Απριλίου 2017
Discussant SRHE Symposium "A cross-institutional perspective on merits and ch...Bart Rienties
In the UK, the introduction of the Teaching Excellence Framework (TEF) has increased interest in
appropriate and valid measurement approaches of learning gains in Higher Education. Learning gains
are defined as growth or change in knowledge, skills, and abilities of learners over time. While the UK
government and other organisations like HEFCE expect tremendous opportunities for learning gains
to “objectively” measure the value added of higher education across institutions, empirical evidence of
the robustness, reliability, and validity of learning gains literature outside the UK is mixed. At SRHE,
we will discuss the affordances, lived experiences, and limitations of using different measurements,
conceptualisations, and methodologies of learning gains. We aim to set an evidence-based agenda of
how HEIs can effectively start to measure and implement notions of learning gains, while at the same
time discussing potential limitations and caveats.
www.abclearninggains.com @learninggains
Learning design meets learning analytics: Dr Bart Rienties, Open UniversityBart Rienties
8th UK Learning Analytics Network Meeting, The Open University, 2nd November 2016
1) The power of 151 Learning Designs on 113K+ students at the OU?
2) How can we use learning design to empower teachers?
3) How can Early Alert Systems improve Student Engagement and Academic Success? (Amara Atif, Macquarie University)
4) What evidence is there that learning design makes a difference over time and how students engage?
Creating a New Generation of Science LeadersStephen Best
This presentation documents the activities and processes used to develop teacher leaders in high needs schools by the Michigan Mathematics and Science Teacher Leadership Collaborative.
Global experiences with e-learning and dataBart Rienties
Pedagogically informed designs of learning are increasingly of interest to researchers in blended and online learning, as learning design is shown to have an impact on student behaviour and outcomes. Although learning design is widely studied, often these studies are individual courses or programmes and few empirical studies have connected learning designs of a substantial number of courses with learning behaviour. In this study we linked 151 modules and 111.256 students with students' behaviour (<400 million minutes of online behaviour), satisfaction and performance at the Open University UK using multiple regression models. Our findings strongly indicate the importance of learning design in predicting and understanding Virtual Learning Environment behaviour and performance of students in blended and online environments. In line with proponents of social learning theories, our primary predictor for academic retention was the time learners spent on communication activities, controlling for various institutional and disciplinary factors. Where possible, appropriate and well designed communication tasks that align with the learning objectives of the course may be a way forward to enhance academic retention.
SRHE2016: Multilevel Modelling of Learning Gains: The Impact of Module Partic...Bart Rienties
Jekaterina Rogaten1
, Bart Rienties1
, Denise Whitelock1
, Simon Cross1
, Allison Littlejohn1
, Rhona
Sharpe2
, Simon Lygo-Baker3
, Ian Scott2
, Steven Warburton3
, Ian Kinchin3
1The Open University UK, UK,
2Oxford Brooks University, UK,
3University of Surrey, UK
Research Domain: Learning, teaching and assessment (LTA)
In the UK, the introduction of the Teaching Excellence Framework (TEF) has increased interest in
appropriate and valid measurement approaches of learning gains in Higher Education. Usually
learning gains are measured using pre-post testing, but this study examines whether academic
performance can be effectively used as proxy to estimate students’ learning progress. Academic
performance of 21,192 online learners from two major faculties was retrieved from university
database. A three-level growth-curve model was estimated and results showed that 16% to 46% of
variance in students’ initial academic performance, and 51% to 77% of variance in their subsequent
learning gains was due to them studying at a particular module. In addition, the results illustrate that
students who studied in modules with initial high student achievements exhibited lower learning gains
than students learning in modules with low initial student achievements. The importance of
assessment and learning design for learning gains are outlined.
www.abclearninggains.com @learninggains
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...Marko Teräs
Presentation at the 28th ICDE World Conference on Online Learning of a national-level learning analytics research and development project funded by the Finnish Ministry of Education and Culture. Student and teacher needs analysis results for LA pilot development and for policy recommendations.
Slides from Keynote presentation at the University of Southern California's 2015 Teaching with Technology annual conference.
"9:15 am – ANN Auditorium
Key Note: What Do We Mean by Learning Analytics?
Leah Macfadyen, Director for Evaluation and Learning Analytics, University of British Columbia
Executive Board, SoLAR (Society for Learning Analytics Research)
Leah Macfadyen will define and explore the emerging and interdisciplinary field of learning analytics in the context of quantified and personalized learning. Leah will use actual examples and case studies to illustrate the range of stakeholders learning analytics may serve, the diverse array of questions they may be used to address, and the potential impact of learning analytics in higher education."
A reflection on where we are with learning analytics as a new multi-discipline research area. Reflections from the Learning Analytics Conference 2013 with respect to Assessment.
OCLC ALISE Library & Information Science Research Grant ProgramLynn Connaway
Connaway, L. S. (2018). OCLC ALISE Library & Information Science Research Grant Program. Presented at ALISE 2018 Conference, February 8, 2018, Denver, Colorado.
Presentation at the HEA-funded workshop 'Making undergraduate social science count: engaging sociology and criminology students in quantitative research methods'.
This workshop aimed to encourage pedagogical reflection and debate on the teaching of quantitative methods to sociology/criminology undergraduates and provide delegates with opportunities for the sharing of best practice in this area. The event included dissemination of the outputs of two recent HEA-funded projects on teaching research methods in the social sciences. Delegates were also introduced to some new and existing quantitative datasets and resources and explore the potential for integrating these across the undergraduate curriculum.
This presentation is part of a related blog post that provides an overview of the event: http://bit.ly/1iBrVMR
For further details of the HEA's work on teaching research methods in the Social Sciences, please see: http://bit.ly/15go0mh
The power of learning analytics to measure learning gains: an OU, Surrey and ...Bart Rienties
Learning gains has increasingly become apparent within the HE literature, gained traction in government policies in the UK, and are at the heart of Teaching Excellence Framework (TFL). As such, this raises a question to what extent teaching and learning environment can actually predict students’ learning gains using principles of learning analytics. In this presentation, which is joined work with University of Surrey and Oxford Brookes, I will focus on some preliminary findings based upon developing and testing an Affective-Behaviour-Cognition learning gains model using longitudinal approach. The main aim of the research is to examine whether learning gains occur on all three levels of Affective-Behaviour-Cognition model and whether any particular student or course characteristics can predict learning gains or lack of learning and dropout. For more info, see https://abclearninggains.com/
Προσκεκλημένη Ομιλία
"Αναλυτική Εκπαιδευτικών Δεδομένων στην Σχολική Πράξη"
5ο Πανελλήνιο Επιστημονικό Συνέδριο
«Ένταξη και Χρήση των ΤΠΕ στην Εκπαιδευτική Διαδικασία»
21-23 Απριλίου 2017
Discussant SRHE Symposium "A cross-institutional perspective on merits and ch...Bart Rienties
In the UK, the introduction of the Teaching Excellence Framework (TEF) has increased interest in
appropriate and valid measurement approaches of learning gains in Higher Education. Learning gains
are defined as growth or change in knowledge, skills, and abilities of learners over time. While the UK
government and other organisations like HEFCE expect tremendous opportunities for learning gains
to “objectively” measure the value added of higher education across institutions, empirical evidence of
the robustness, reliability, and validity of learning gains literature outside the UK is mixed. At SRHE,
we will discuss the affordances, lived experiences, and limitations of using different measurements,
conceptualisations, and methodologies of learning gains. We aim to set an evidence-based agenda of
how HEIs can effectively start to measure and implement notions of learning gains, while at the same
time discussing potential limitations and caveats.
www.abclearninggains.com @learninggains
Learning design meets learning analytics: Dr Bart Rienties, Open UniversityBart Rienties
8th UK Learning Analytics Network Meeting, The Open University, 2nd November 2016
1) The power of 151 Learning Designs on 113K+ students at the OU?
2) How can we use learning design to empower teachers?
3) How can Early Alert Systems improve Student Engagement and Academic Success? (Amara Atif, Macquarie University)
4) What evidence is there that learning design makes a difference over time and how students engage?
Creating a New Generation of Science LeadersStephen Best
This presentation documents the activities and processes used to develop teacher leaders in high needs schools by the Michigan Mathematics and Science Teacher Leadership Collaborative.
Global experiences with e-learning and dataBart Rienties
Pedagogically informed designs of learning are increasingly of interest to researchers in blended and online learning, as learning design is shown to have an impact on student behaviour and outcomes. Although learning design is widely studied, often these studies are individual courses or programmes and few empirical studies have connected learning designs of a substantial number of courses with learning behaviour. In this study we linked 151 modules and 111.256 students with students' behaviour (<400 million minutes of online behaviour), satisfaction and performance at the Open University UK using multiple regression models. Our findings strongly indicate the importance of learning design in predicting and understanding Virtual Learning Environment behaviour and performance of students in blended and online environments. In line with proponents of social learning theories, our primary predictor for academic retention was the time learners spent on communication activities, controlling for various institutional and disciplinary factors. Where possible, appropriate and well designed communication tasks that align with the learning objectives of the course may be a way forward to enhance academic retention.
SRHE2016: Multilevel Modelling of Learning Gains: The Impact of Module Partic...Bart Rienties
Jekaterina Rogaten1
, Bart Rienties1
, Denise Whitelock1
, Simon Cross1
, Allison Littlejohn1
, Rhona
Sharpe2
, Simon Lygo-Baker3
, Ian Scott2
, Steven Warburton3
, Ian Kinchin3
1The Open University UK, UK,
2Oxford Brooks University, UK,
3University of Surrey, UK
Research Domain: Learning, teaching and assessment (LTA)
In the UK, the introduction of the Teaching Excellence Framework (TEF) has increased interest in
appropriate and valid measurement approaches of learning gains in Higher Education. Usually
learning gains are measured using pre-post testing, but this study examines whether academic
performance can be effectively used as proxy to estimate students’ learning progress. Academic
performance of 21,192 online learners from two major faculties was retrieved from university
database. A three-level growth-curve model was estimated and results showed that 16% to 46% of
variance in students’ initial academic performance, and 51% to 77% of variance in their subsequent
learning gains was due to them studying at a particular module. In addition, the results illustrate that
students who studied in modules with initial high student achievements exhibited lower learning gains
than students learning in modules with low initial student achievements. The importance of
assessment and learning design for learning gains are outlined.
www.abclearninggains.com @learninggains
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...Marko Teräs
Presentation at the 28th ICDE World Conference on Online Learning of a national-level learning analytics research and development project funded by the Finnish Ministry of Education and Culture. Student and teacher needs analysis results for LA pilot development and for policy recommendations.
Slides from Keynote presentation at the University of Southern California's 2015 Teaching with Technology annual conference.
"9:15 am – ANN Auditorium
Key Note: What Do We Mean by Learning Analytics?
Leah Macfadyen, Director for Evaluation and Learning Analytics, University of British Columbia
Executive Board, SoLAR (Society for Learning Analytics Research)
Leah Macfadyen will define and explore the emerging and interdisciplinary field of learning analytics in the context of quantified and personalized learning. Leah will use actual examples and case studies to illustrate the range of stakeholders learning analytics may serve, the diverse array of questions they may be used to address, and the potential impact of learning analytics in higher education."
A reflection on where we are with learning analytics as a new multi-discipline research area. Reflections from the Learning Analytics Conference 2013 with respect to Assessment.
OCLC ALISE Library & Information Science Research Grant ProgramLynn Connaway
Connaway, L. S. (2018). OCLC ALISE Library & Information Science Research Grant Program. Presented at ALISE 2018 Conference, February 8, 2018, Denver, Colorado.
Presentation at the HEA-funded workshop 'Making undergraduate social science count: engaging sociology and criminology students in quantitative research methods'.
This workshop aimed to encourage pedagogical reflection and debate on the teaching of quantitative methods to sociology/criminology undergraduates and provide delegates with opportunities for the sharing of best practice in this area. The event included dissemination of the outputs of two recent HEA-funded projects on teaching research methods in the social sciences. Delegates were also introduced to some new and existing quantitative datasets and resources and explore the potential for integrating these across the undergraduate curriculum.
This presentation is part of a related blog post that provides an overview of the event: http://bit.ly/1iBrVMR
For further details of the HEA's work on teaching research methods in the Social Sciences, please see: http://bit.ly/15go0mh
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EUmoocs
This hands-on workshop will work with learning design tools and with massive open online courses (MOOCs) on the FutureLearn platform to explore how learning design can be used to influence the choice and design of learning analytics. This workshop will be of interest to people who are involved in the design or presentation of online courses, and to those who want to find out more about learning design, learning analytics or MOOCs. Participants will find it helpful to have registered for FutureLearn and explored the platform for a short time in advance of the workshop.
This presentation was given during the EMMA Summer School, that took place in Ischia (Italy) on 4-11 July 2015.
More info on the website: http://project.europeanmoocs.eu/project/get-involved/summer-school/
Follow our MOOCs: http://platform.europeanmoocs.eu/MOOCs
Design and deliver your MOOC with EMMA: http://project.europeanmoocs.eu/project/get-involved/become-an-emma-mooc-provider/
The 7 Cs of Learning Design - presented at the Fourth International Conference of E-Learning and Distance Learning - Riyadh, Saudi Arabia - February - March 2015
Learning analytics futures: a teaching perspectiveRebecca Ferguson
Talk given by Rebecca Ferguson on 22 November 2018 int Universita Ca'Foscario Venezia at the event Nuovi orizzonti della ricerca pedagogica: evidence-based learning e learning analytics
European Perspectives on Learning Analytics: LAK15 LACE panelLACE Project
Panel presentation at Learning Analytics and Knowledge 2015 (LAK15) in Poughkeepsie, NY, USA by a team of speakers from the LACE project.
Since the emergence of learning analytics in North America, researchers and practitioners have worked to develop an international community. The organization of events such as SoLAR Flares and LASI Locals, as well as the move of LAK in 2013 from North America to Europe, has supported this aim. There are now thriving learning analytics groups in North American, Europe and Australia, with smaller pockets of activity emerging on other continents. Nevertheless, much of the work carried out outside these forums, or published in languages other than English, is still inaccessible to most people in the community. This panel, organized by Europe’s Learning Analytics Community Exchange (LACE) project, brings together researchers from five European countries to examine the field from European perspectives. In doing so, it will identify the benefits and challenges associated with sharing and developing practice across national boundaries.
Massive open online courses or MOOCs were predicted to achieve world domination and completely transformation of higher education. Today, these predictions are seen to have been overblown. But with several years of experience now behind them, MOOC providers and users are adjusting both their perceptions about online learning and the courses themselves. Mainly based on empirical research articles and reports and interviews with K-MOOC providers, this paper examines impacts of MOOCs on higher education and analyze K-MOOC as an illustrative case. For this, it asks such questions as: 1) have MOOCs expanded higher education and provided access for all, especially for the socially marginalized groups? 2) have MOOCs improved the quality of campus-based higher education? 3) have MOOCs reduced the costs to the providers and users? It will conclude with discussion of the emerging issues and future directions.
Instructional Contents Delivery through SPAT format in Mobile Environment: ...Ilju Rha
Instructional Contents Delivery through SPAT format in Mobile Environment: Introduction to L.i.B study system.
SPAT represents Still Picture+Audio+Text format digital knowledge unit. The slide was presented for Global Knowledge Alliances.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
How to Create Map Views in the Odoo 17 ERPCeline George
The map views are useful for providing a geographical representation of data. They allow users to visualize and analyze the data in a more intuitive manner.
This is a presentation by Dada Robert in a Your Skill Boost masterclass organised by the Excellence Foundation for South Sudan (EFSS) on Saturday, the 25th and Sunday, the 26th of May 2024.
He discussed the concept of quality improvement, emphasizing its applicability to various aspects of life, including personal, project, and program improvements. He defined quality as doing the right thing at the right time in the right way to achieve the best possible results and discussed the concept of the "gap" between what we know and what we do, and how this gap represents the areas we need to improve. He explained the scientific approach to quality improvement, which involves systematic performance analysis, testing and learning, and implementing change ideas. He also highlighted the importance of client focus and a team approach to quality improvement.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
The Art Pastor's Guide to Sabbath | Steve ThomasonSteve Thomason
What is the purpose of the Sabbath Law in the Torah. It is interesting to compare how the context of the law shifts from Exodus to Deuteronomy. Who gets to rest, and why?
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
2024.06.01 Introducing a competency framework for languag learning materials ...
National level data metrics framework development in Kouth Korea -Iljr Rha
1. September 17, 2015
Prof. Ilju Rha, Ph. D.
& Prof. Cheolil Lim, Prof. Young Hoan Cho & Mina Yoo
Seoul National University, Seoul, Korea
SeoulNationalUniversity
13. Seoul National University 13
*Introduction of Research
Seoul National University
Start-up Mega Planning
14. Seoul National University
*Introduction of Research: 3-year plan
14
1st-year
Modeling of Learning
Analytics
plan of collecting learning
data
Modeling of learning data
analysis
Deduction of the
application plan based on
learning data analysis
2nd-year
Application of Learning
Analytics
Collecting 1st-year
modeling based learning
data, analyzing,
application, and revision
Development of models
for LOD based learning
analytics service in the
field of education
3rd-year
Expansion of the service
based on learning data
Application of models for
LOD based learning
analytics service in 2nd -
year
Application and revision
of adaptive and
personalized learning
prescription service
15. Seoul National University
*Introduction of Research-1st year
Research Title : A basic research on learning analytics
model and plan of expansion
Research Period : Sep 11, 2014 ~ Jan 31, 2015
(5 months)
Institutions : Seoul National University
Director : Prof. Ilju Rha, Ph.D. Dept. of Education,
College of Education, Seoul National University
Co-researchers : Prof. Cheolil Lim, Ph.D.,
Prof. Younghoan Cho, Ph.D.
Assistant researchers : 6 graduate students
15
16. 16Verbert et al. (2012). p. 137. (Figure 3. Learner action model)
18. Seoul National University 18
*Research Methods
Literature Review
Case studies of learning analytics
Experts advisory councils and seminars
19. Seoul National University 19
*Research Methods
Forums with specialists
Visiting schools and focus group interviews with teachers
Interviews with teachers for review of the application plans
and learning analytics
Research group meetings
20. Seoul National University
Derivation from learners’ activities
Conformity to the International Standard
Easy communication among researchers
Meaningful units of activities in the field of education
20
*Research Question - The plan of collecting learning data
1
2
3
4
What are plans for collecting learning data?
21. Seoul National University 21
* The plan of collecting learning data (1)
• The development of learning activities metrics
Review of IMS learning activities metrics
• Utilization of applicable
metadata
- Composition of learners’
activities and the data object
• Focus on the elements of
learning design
- tasks, resources, support
22. Seoul National University 22
* The plan of collecting learning data (2)
• The development of learning activities metrics
– Review of the classification of teaching-learning activities
• Ascher(1976)
• Classification of teaching-
learning activities in F2F
environment
• Horton(2006)
• Linkage between activities
and media in E-learning
environment
• Watkinson(2005)
• Classification of E-learning
activities into 75 elements
• Lim, Lim, and Kim(2008)
• Classification of digital text
books into 49 teaching-
learning activities
Supplement and modification
• tools, basic activities, classification of
multiple activities
• Tools : Management of learning goals
• Basic activities : Writing, mind
mapping
• Multiple activities : Hands-on
learning
23. Seoul National University 23
* The plan of collecting learning data (3)
• The development of learning activities metrics
– Review of models of instruction and lesson plans for utilizing
digital textbooks
• Kim et al. (2011). Classroom
centered teaching-learning
activities in 21st century.
• Rho et al. (2013). A research
on models of instruction for
utilizing digital textbooks.
Supplement and modification
• Basic activities : writing, mind
mapping, speaking
24. Seoul National University 24
* The plan of collecting learning data (4)
• The development of learning activities metrics
– Expert reviews and modification
Classification List of experts
Educational Technology
Prof. Insung Jung, Ph.D., International Christian
University, Japan
Prof. Il-Hyun Jo, Ph.D., Ehwa Womans University,
Korea
Prof. Yeonwook Im, Ph.D., Hanyang Cyber university,
Korea
Educational Psychology
Prof. Jongho Shin, Ph.D., Seoul National University,
Korea
Computer Engineering
Prof. Jang-Mook Kang, Ph.D., Korea University,
Korea
Education Field Principal Manjong Yang, Jeongmok Elementary School
Supplement and modification
• Tools : Access management
• Basic Activities : Download of writing resources, citation of researches
25. Seoul National University
* Process - The development of learning activities metrics
• The development of learning activities metrics
– Classifying every learning activities for measurable units
– Taxonomy for utilizing units of activities composing learning
behaviors as the analyzed data
The development of
learning activities
metrics
IMS learning
activities metrics
Models of instructions and
lesson plans for utilizing
digital text books
Classifying teaching-
learning types in
domestic and
international level
25
26. Seoul National University 26
* Process - The development of learning activities metrics
Retrieved from http://www.imsglobal.org/IMSLearningAnalyticsWP.pdf
• IMS Metrics profiles
27. Seoul National University 27
* Process - The development of learning activities metrics
The Difference
The Ideal Vision
Needs
Negotiation
Mega
“National Level”
Macro
“K-12 Education”
Micro
“Institutional and In
dividual Metrics”
Policy Makers
(MOE, Province,
School & Support
Institution)
Practitioners
(Teachers, Researchers,
Technicians)
Process
Inputs
Integration
“NDM framework”
28. Seoul National University 28
* Results - The development of learning activities metrics
Proposed
National Learning Metrics
Framework
29. T
O
O
L
S
Goal
Management Scheduling Media Social Annotation Access
Management
Reading Lectures Writing Discussions Messaging Quiz
Speaking Projects Research Assessment Mind Map Gaming
C
O
M
B
I
N
E
D
Tutoring Collaboration Field StudyHomework
• Annotations
• Page/block use
• Media use
• Lookups
• Frameset use
• Scrub marks
• View time
• Weblink refs
• Assoc context
• Goal setting
• Subordinate
goal setting
• Event pattern
• Frequency
• Assoc context
• Event patterns
• Event profile
• Time utilization
A
C
T
I
V
I
T
Y
B
A
S
I
C
A
C
T
I
V
I
T
Y
• Media type
• Frameset use
• Scrub marks
• View time
• Usage context
• Connections
• Assoc context
• Message profile
• frequency
• Highlights
• Notes
• Marks
• Tags
• Assoc context
• Input
• Contents
• Attachments
• Post mark
• Frequency
• Participation
• Collaboration
• Assoc context
• Outbound pool
• Inbound pool
• Attachments
• Scores
• Attempts
• Remediations
• Assoc refs
• Questioning
• Answering
• Presentation
• Communication
• Scores
• Attempts
• Remediations
• Assoc refs
• Topics
• Assoc context
• Frequency
• Feedback
• Connections
• Assoc context
• Message profile
• Frequency
• Deliverables
• Structure
• Milestone perf
• Group profile
• Patterns
• Searches
• Patterns
• Citations
• Topics
• Scores
• Patterns(item)
• Time utilization
• Attempts
• Completion
• Drawing
• Frequency
• Participation
• Collaboration
• Progress
• Cognition
• Attempts
• Hints
• Collaboration
· Clicking (navigation, operation) · Swiping (navigation, close up, resizing)
· Typing (note taking, memo, searching, communication) · Pen writing (highlights, memo, using color pens)
PERFORMANCECONTEXT
• Activity usage
time on task
• Session time
• Last access
• Activity affinity
• Content affinity
• Task patterns
• Correlations
• Institution
• Course/Section
• Learner profile
• Course context
• Path/Sequence
• Usage context
• Grades
• Progress
• Rubrics
‐ Course goals
‐ Topic objects
‐ Qualitative
evaluation
‐ Quantitative
scores
• Patterns
• Correlations
Learning Activity Metrics
INPUT PROCESS OUTPUT
• Time Stamp
(Log in & Logout)
• Duration
• Frequency
LEVEL OF
METRICS
UTILIZATION
· Government · Policy maker · School district · School · Teacher
ENGAGEMENT
• Assoc context
• Frequency
• Participation
• Collaboration
29
30. T
O
O
L
S
Goal
Management Scheduling Media Social Annotation Access
Management
Reading Lectures Writing Discussions Messaging Quiz
Speaking Projects Research Assessment Mind Map Gaming
C
O
M
B
I
N
E
D
Tutoring Collaboration Field StudyHomework
• Annotations
• Page/block use
• Media use
• Lookups
• Frameset use
• Scrub marks
• View time
• Weblink refs
• Assoc context
• Goal setting
• Subordinate
goal setting
• Event pattern
• Frequency
• Assoc context
• Event patterns
• Event profile
• Time utilization
A
C
T
I
V
I
T
Y
B
A
S
I
C
A
C
T
I
V
I
T
Y
• Media type
• Frameset use
• Scrub marks
• View time
• Usage context
• Connections
• Assoc context
• Message profile
• frequency
• Highlights
• Notes
• Marks
• Tags
• Assoc context
• Input
• Contents
• Attachments
• Post mark
• Frequency
• Participation
• Collaboration
• Assoc context
• Outbound pool
• Inbound pool
• Attachments
• Scores
• Attempts
• Remediations
• Assoc refs
• Questioning
• Answering
• Presentation
• Communication
• Scores
• Attempts
• Remediations
• Assoc refs
• Topics
• Assoc context
• Frequency
• Feedback
• Connections
• Assoc context
• Message profile
• Frequency
• Deliverables
• Structure
• Milestone perf
• Group profile
• Patterns
• Searches
• Patterns
• Citations
• Topics
• Scores
• Patterns(item)
• Time utilization
• Attempts
• Completion
• Drawing
• Frequency
• Participation
• Collaboration
• Progress
• Cognition
• Attempts
• Hints
• Collaboration
· Clicking (navigation, operation) · Swiping (navigation, close up, resizing)
· Typing (note taking, memo, searching, communication) · Pen writing (highlights, memo, using color pens)
PERFORMANCECONTEXT
• Activity usage
time on task
• Session time
• Last access
• Activity affinity
• Content affinity
• Task patterns
• Correlations
• Institution
• Course/Section
• Learner profile
• Course context
• Path/Sequence
• Usage context
• Grades
• Progress
• Rubrics
‐ Course goals
‐ Topic objects
‐ Qualitative
evaluation
‐ Quantitative
scores
• Patterns
• Correlations
Learning Activity Metrics
INPUT PROCESS OUTPUT
• Time Stamp
(Log in & Logout)
• Duration
• Frequency
LEVEL OF
METRICS
UTILIZATION
· Government · Policy maker · School district · School · Teacher
ENGAGEMENT
• Assoc context
• Frequency
• Participation
• Collaboration
30
31. T
O
O
L
S
Goal
Management Scheduling Media Social Annotation Access
Management
Reading Lectures Writing Discussions Messaging Quiz
Speaking Projects Research Assessment Mind Map Gaming
C
O
M
B
I
N
E
D
Tutoring Collaboration Field StudyHomework
• Annotations
• Page/block use
• Media use
• Lookups
• Frameset use
• Scrub marks
• View time
• Weblink refs
• Assoc context
• Goal setting
• Subordinate
goal setting
• Event pattern
• Frequency
• Assoc context
• Event patterns
• Event profile
• Time utilization
A
C
T
I
V
I
T
Y
B
A
S
I
C
A
C
T
I
V
I
T
Y
• Media type
• Frameset use
• Scrub marks
• View time
• Usage context
• Connections
• Assoc context
• Message profile
• frequency
• Highlights
• Notes
• Marks
• Tags
• Assoc context
• Input
• Contents
• Attachments
• Post mark
• Frequency
• Participation
• Collaboration
• Assoc context
• Outbound pool
• Inbound pool
• Attachments
• Scores
• Attempts
• Remediations
• Assoc refs
• Questioning
• Answering
• Presentation
• Communication
• Scores
• Attempts
• Remediations
• Assoc refs
• Topics
• Assoc context
• Frequency
• Feedback
• Connections
• Assoc context
• Message profile
• Frequency
• Deliverables
• Structure
• Milestone perf
• Group profile
• Patterns
• Searches
• Patterns
• Citations
• Topics
• Scores
• Patterns(item)
• Time utilization
• Attempts
• Completion
• Drawing
• Frequency
• Participation
• Collaboration
• Progress
• Cognition
• Attempts
• Hints
• Collaboration
· Clicking (navigation, operation) · Swiping (navigation, close up, resizing)
· Typing (note taking, memo, searching, communication) · Pen writing (highlights, memo, using color pens)
PERFORMANCECONTEXT
• Activity usage
time on task
• Session time
• Last access
• Activity affinity
• Content affinity
• Task patterns
• Correlations
• Institution
• Course/Section
• Learner profile
• Course context
• Path/Sequence
• Usage context
• Grades
• Progress
• Rubrics
‐ Course goals
‐ Topic objects
‐ Qualitative
evaluation
‐ Quantitative
scores
• Patterns
• Correlations
Learning Activity Metrics
INPUT PROCESS OUTPUT
• Time Stamp
(Log in & Logout)
• Duration
• Frequency
LEVEL OF
METRICS
UTILIZATION
· Government · Policy maker · School district · School · Teacher
ENGAGEMENT
• Assoc context
• Frequency
• Participation
• Collaboration
31
32. T
O
O
L
S
Goal
Management Scheduling Media Social Annotation Access
Management
Reading Lectures Writing Discussions Messaging Quiz
Speaking Projects Research Assessment Mind Map Gaming
C
O
M
B
I
N
E
D
Tutoring Collaboration Field StudyHomework
• Annotations
• Page/block use
• Media use
• Lookups
• Frameset use
• Scrub marks
• View time
• Weblink refs
• Assoc context
• Goal setting
• Subordinate
goal setting
• Event pattern
• Frequency
• Assoc context
• Event patterns
• Event profile
• Time utilization
A
C
T
I
V
I
T
Y
B
A
S
I
C
A
C
T
I
V
I
T
Y
• Media type
• Frameset use
• Scrub marks
• View time
• Usage context
• Connections
• Assoc context
• Message profile
• frequency
• Highlights
• Notes
• Marks
• Tags
• Assoc context
• Input
• Contents
• Attachments
• Post mark
• Frequency
• Participation
• Collaboration
• Assoc context
• Outbound pool
• Inbound pool
• Attachments
• Scores
• Attempts
• Remediations
• Assoc refs
• Questioning
• Answering
• Presentation
• Communication
• Scores
• Attempts
• Remediations
• Assoc refs
• Topics
• Assoc context
• Frequency
• Feedback
• Connections
• Assoc context
• Message profile
• Frequency
• Deliverables
• Structure
• Milestone perf
• Group profile
• Patterns
• Searches
• Patterns
• Citations
• Topics
• Scores
• Patterns(item)
• Time utilization
• Attempts
• Completion
• Drawing
• Frequency
• Participation
• Collaboration
• Progress
• Cognition
• Attempts
• Hints
• Collaboration
· Clicking (navigation, operation) · Swiping (navigation, close up, resizing)
· Typing (note taking, memo, searching, communication) · Pen writing (highlights, memo, using color pens)
PERFORMANCECONTEXT
• Activity usage
time on task
• Session time
• Last access
• Activity affinity
• Content affinity
• Task patterns
• Correlations
• Institution
• Course/Section
• Learner profile
• Course context
• Path/Sequence
• Usage context
• Grades
• Progress
• Rubrics
‐ Course goals
‐ Topic objects
‐ Qualitative
evaluation
‐ Quantitative
scores
• Patterns
• Correlations
Learning Activity Metrics
INPUT PROCESS OUTPUT
• Time Stamp
(Log in & Logout)
• Duration
• Frequency
LEVEL OF
METRICS
UTILIZATION
· Government · Policy maker · School district · School · Teacher
ENGAGEMENT
• Assoc context
• Frequency
• Participation
• Collaboration
32
33. T
O
O
L
S
Goal
Management Scheduling Media Social Annotation Access
Management
Reading Lectures Writing Discussions Messaging Quiz
Speaking Projects Research Assessment Mind Map Gaming
C
O
M
B
I
N
E
D
Tutoring Collaboration Field StudyHomework
• Annotations
• Page/block use
• Media use
• Lookups
• Frameset use
• Scrub marks
• View time
• Weblink refs
• Assoc context
• Goal setting
• Subordinate
goal setting
• Event pattern
• Frequency
• Assoc context
• Event patterns
• Event profile
• Time utilization
A
C
T
I
V
I
T
Y
B
A
S
I
C
A
C
T
I
V
I
T
Y
• Media type
• Frameset use
• Scrub marks
• View time
• Usage context
• Connections
• Assoc context
• Message profile
• frequency
• Highlights
• Notes
• Marks
• Tags
• Assoc context
• Input
• Contents
• Attachments
• Post mark
• Frequency
• Participation
• Collaboration
• Assoc context
• Outbound pool
• Inbound pool
• Attachments
• Scores
• Attempts
• Remediations
• Assoc refs
• Questioning
• Answering
• Presentation
• Communication
• Scores
• Attempts
• Remediations
• Assoc refs
• Topics
• Assoc context
• Frequency
• Feedback
• Connections
• Assoc context
• Message profile
• Frequency
• Deliverables
• Structure
• Milestone perf
• Group profile
• Patterns
• Searches
• Patterns
• Citations
• Topics
• Scores
• Patterns(item)
• Time utilization
• Attempts
• Completion
• Drawing
• Frequency
• Participation
• Collaboration
• Progress
• Cognition
• Attempts
• Hints
• Collaboration
· Clicking (navigation, operation) · Swiping (navigation, close up, resizing)
· Typing (note taking, memo, searching, communication) · Pen writing (highlights, memo, using color pens)
PERFORMANCECONTEXT
• Activity usage
time on task
• Session time
• Last access
• Activity affinity
• Content affinity
• Task patterns
• Correlations
• Institution
• Course/Section
• Learner profile
• Course context
• Path/Sequence
• Usage context
• Grades
• Progress
• Rubrics
‐ Course goals
‐ Topic objects
‐ Qualitative
evaluation
‐ Quantitative
scores
• Patterns
• Correlations
Learning Activity Metrics
INPUT PROCESS OUTPUT
• Time Stamp
(Log in & Logout)
• Duration
• Frequency
LEVEL OF
METRICS
UTILIZATION
· Government · Policy maker · School district · School · Teacher
ENGAGEMENT
• Assoc context
• Frequency
• Participation
• Collaboration
Engagement Elements Sub-elements
Basic events
Clicking (C) C1. Navigation C2. Operation
Typing (W)
W1. Note taking W2. Memo
W3. Searching W4. Communication
Swiping (S) S1. Navigation S2. Close up S3. Resizing
Pen writing (P) P1. Highlights P2. Memo P3. Using Color pens
Time (T) T1. Timestamp T2. Duration T3. Interval
Data input &
output
Login-Logout, Installation, Download, Save, Print, Record, Capture, Bookmark
Elements of Engagement
33
34. T
O
O
L
S
Goal
Management Scheduling Media Social Annotation Access
Management
Reading Lectures Writing Discussions Messaging Quiz
Speaking Projects Research Assessment Mind Map Gaming
C
O
M
B
I
N
E
D
Tutoring Collaboration Field StudyHomework
• Annotations
• Page/block use
• Media use
• Lookups
• Frameset use
• Scrub marks
• View time
• Weblink refs
• Assoc context
• Goal setting
• Subordinate
goal setting
• Event pattern
• Frequency
• Assoc context
• Event patterns
• Event profile
• Time utilization
A
C
T
I
V
I
T
Y
B
A
S
I
C
A
C
T
I
V
I
T
Y
• Media type
• Frameset use
• Scrub marks
• View time
• Usage context
• Connections
• Assoc context
• Message profile
• frequency
• Highlights
• Notes
• Marks
• Tags
• Assoc context
• Input
• Contents
• Attachments
• Post mark
• Frequency
• Participation
• Collaboration
• Assoc context
• Outbound pool
• Inbound pool
• Attachments
• Scores
• Attempts
• Remediations
• Assoc refs
• Questioning
• Answering
• Presentation
• Communication
• Scores
• Attempts
• Remediations
• Assoc refs
• Topics
• Assoc context
• Frequency
• Feedback
• Connections
• Assoc context
• Message profile
• Frequency
• Deliverables
• Structure
• Milestone perf
• Group profile
• Patterns
• Searches
• Patterns
• Citations
• Topics
• Scores
• Patterns(item)
• Time utilization
• Attempts
• Completion
• Drawing
• Frequency
• Participation
• Collaboration
• Progress
• Cognition
• Attempts
• Hints
• Collaboration
· Clicking (navigation, operation) · Swiping (navigation, close up, resizing)
· Typing (note taking, memo, searching, communication) · Pen writing (highlights, memo, using color pens)
PERFORMANCECONTEXT
• Activity usage
time on task
• Session time
• Last access
• Activity affinity
• Content affinity
• Task patterns
• Correlations
• Institution
• Course/Section
• Learner profile
• Course context
• Path/Sequence
• Usage context
• Grades
• Progress
• Rubrics
‐ Course goals
‐ Topic objects
‐ Qualitative
evaluation
‐ Quantitative
scores
• Patterns
• Correlations
Learning Activity Metrics
INPUT PROCESS OUTPUT
• Time Stamp
(Log in & Logout)
• Duration
• Frequency
LEVEL OF
METRICS
UTILIZATION
· Government · Policy maker · School district · School · Teacher
ENGAGEMENT
• Assoc context
• Frequency
• Participation
• Collaboration
34
35. Seoul National University 35
* Conclusion
The proposed national data metrics
• Tentative attribute in theoretical level
• Systematic data collection for learning analytics in an aspect of
educational utilization of big data
• Learning analytics in national level
• Learning activities are the basis of learning analytics
• Used literature reviews and needs analysis methodology
• Practical implementation is needed to investigate its feasibility
36. Seoul National University 36
* Future Plan
1st-year
Modeling of Learning
Analytics
plan of collecting learning
data
Modeling of learning data
analysis
Deduction of the
application plan based on
learning data analysis
2nd-year
Application of Learning
Analytics
Collecting 1st-year
modeling based learning
data, analyzing,
application, and revision
Development of models
for LOD based learning
analytics service in the
field of education
3rd-year
Expansion of the service
based on learning data
Application of models for
LOD based learning
analytics service in 2nd -
year
Application and revision
of adaptive and
personalized learning
prescription service
37. Seoul National University 37
Q. What should we do?
Learning analytics!
Q.. What kinds of data?
Q… How to collect data?
Q…. How to analyze data?
Q….. Is it the most optimal?
* Future Plan
39. Seoul National University
Thank You!
Prof. Ilju Rha, Ph.D.
iljurha@snu.ac.kr
Prof. Cheolil Lim, Prof. Young Hoan Cho &
Mina Yoo
Dept. of Education, College of Education
Seoul National University