Providing Cognitive Scaffolding
within Computer-Supported
Adaptive Learning Environment
for Material Science Education
(ICL-2018, Kos)
Fedor Dudyrev, Head of the Center for Vocational Education Research, National
Research University Higher School of Economics
Alexey Neznanov, senior researcher IL ISSA, Faculty of Computer Science,
National Research University Higher School of Economics
Olga Maksimenkova, junior research fellow IL ISSA, Faculty of Computer Science,
National Research University Higher School of Economics
© 2018, Dudyrev F., Maksimenkova O., Neznanov A. 1
Context
• Vocational students studying technical subjects
• Low level of knowledge and skills in basic school courses , esp.. math and
physics
• Learning outcomes freshmen who attend TVET system vary from 2 to 6 points
in 10 grades scale
• Percentage of early school leavers in TVET system is very high
• Negative social environment
2© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
From John Elwood, License: CC BY-NC-ND
From 4mygodsglory, License: CC BY-ND
From Sandra Mathison, License: CC BY-SA-NC
Personalized Learning as an Approach
• Learning objectives, instructional
approaches, and instructional content
may all vary based on learner needs
• Learning activities are made available
that are meaningful and relevant to
learners, driven by their interests and
(often) self-initiated
• The instructor takes into account the
student's troubles and mistakes and
provides quick, dosed and targeted
assistance
• The pace of learning and the
instructional approach are optimized
for the needs of each learner
© 2018, Dudyrev F., Maksimenkova O., Neznanov A. 3
Scaffolding as a Theoretical Framework for
Personalized Learning (R.Wood, J.Bruner,
L.Vygotsky)
© 2018, Dudyrev F., Maksimenkova O., Neznanov A. 4
Learner: More Knowledgeable Other (MKO)
in particular instructor:
Stimulates learners cognitive interest and
involves him in the problem-solving
process
Tries to solve a complex cognitive
problem on his own
Ensures the feasibility of the decision
process, breaking the impossible task into
easier subtasks
Requests assistance to more
knowledgeable other (MKO) and receives
feedback
Protects the student from frustration and
loss of interest, encouraging and
motivating him
Internalizes the way to solve the problem
and moves on to more complex tasks
Ensures the internalization of the basic
steps of reasoning process and gradually
delegates to the learner additional
degrees of independence
Adaptive Learning System
which Provides Scaffolding
Requirements:
• Explicit description of learning
outcomes
• Ontology controlled knowledge
assessment
• Scaffolding types
• Automatic mistakes/gap detection
• Instant self-assessment
• Formative feedback
• Checking relevance of educational
materials
• Engaging active learning
techniques
• Reinforcing the training
• Scalability
5© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
From flikr, license: CC BY-NC
Adaptive Learning System Architecture
6© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Pedagogical
Intervention
Activation
Subsystem
Automatic Item
Generation
Subsystem
Learning Process Management Module
Relevant
Materials
Searching
Subsystem
Educational Materials Module
Assessment Materials Bank
Learning
Management
Subsystem
EDM Tools
Review and
validation
Subsystem
Analytics Module
Ontology Management Module
Ontology
Querying
Subsystem
EdMachine
metaontology
Learning
processes
ontology
Domain
ontology
Learning
materials DB
References DB
Analytics DB
Assessment
materials DB
Learning
activity DB
“Scaffolding square”
from Technical Point of View
Pedagogical Intervention Activation Subsystem
• Logic of scaffolding
Relevant Materials Searching Subsystem
• Linkage between Scaffolding actions and Educational materials
Ontology Querying Subsystem
• Linkage between Scaffolding actions and
Domain ontology + Learning process ontology
Automatic Item Generation Subsystem
• Obligate tool for automatic adaptive learning: Adaptive assessment,
Prerequisite checking, Additional gap detection as part of formative feedback
7© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Ontologies and Ontology Modelling
• Technical ontology –
formalized computer-readed
domain knowledge
• From 1993 – Gruber T.:
Explicit specification of a
conceptualization
• The most cited definition
8© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
From wikipedia, license: CC BY-SA
Specific Integrated Domain Ontology
for MSE
Mathematicalfoundations
• Formal Concept Analysis (FCA)
•Mens K. (In)Formal Concept Analysis
(https://www.slideshare.net/kim.mens/formalcon
ceptanalysis)
•Priss U. Formal Concept Analysis
Homepage
(http://www.upriss.org.uk/fca/fca.html)
• Descriptive/description logic
•Krötzsch M., Simancik F., Horrocks I.
A Description Logic Primer
(http://arxiv.org/abs/1201.4089)
•MLWiki – Descriptive Logic
(http://mlwiki.org/index.php/Descriptive_Logic)
Structure
• Thesaurus – classical set of
taxonomies + basic subclass of
descriptive logic
• Collections of:
•Distinctor – special object for
explanation of distinction between
entities
•Scale – definition of scaling multivalued
attribute (datatype) into binary
attribute
•Formal context (binary, as a main
source of data about objects) and
associated multivalued context (as a
source)
9© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Thesaurus Nodes
• Entity description
• Id: GUID // Identifier for computer
• Creator: UserId
• DateOfCreation: ISO_Time
• StatusOfVisibility: Enum = (IsActive, IsDeleted, IsReplaced)
• StatusOfApproval: Enum = (IsDraft, IsProposed, IsApproved)
• ReplacedBy: GUID
• Name: UTF8 // Main title of node
• Synonyms: Collection[UTF8]
• Sense: UTF8
• Comments: UTF8
• Collection of relations
• RelationType: SR_Id
• Operand: GUID
10© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Main Semantic Relations
• Requirement: SKOS compatibility
• Simple knowledge organization system (SKOS) – W3C Standard
(http://www.w3.org/TR/2009/REC-skos-reference-20090818/)
• Basic “class-class” relations
• “Is-A” (generalization)
• Example: <stainless steel> IS-A <steel>
• “Part-Of” (aggregation)
• Example: <housing> PART-OF <machine>
• Basic “instance-class” relation
• “Instans-Of” (instantiation)
• Example: <4Х5В2ФС> INSTANS-OF <stainless steel>
• Additional relations
• “Defined-In” (source of definition)
• Example: <4Х5В2ФС> DEFINED-IN <ГОСТ_5950-2000>
• …
11© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Table Representation of Thesaurus
Fragment
12© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Scales
• Formal scale
• Id: GUID
• Caption: UTF8
• Hint: UTF8
• Source: UTF8 (text with refs to thesaurus)
• ScaleType: Enum = (Nominal, Ordinal, Interval, Ratio)
• IsOrdered: Boolean // explicit order of elements for any type
• Cardinality: Integer
• Names: array [0..Cardinality-1] of UTF8
• Values: array [0..Cardinality-1] of Variant (may be NULL for nominal scale)
13© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Table Representation of a Scale
14© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Formal Context
15© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
• Multivalued context ↔ Binary context
• Main derived artifact – formal concept lattice (not for this presentation )
• Example of small binary context for geometric figures:
Automatic Item Generation (AIG)
• One of the most important part of scaffolding engine
• Goal – additional identification of knowledge gap
• Method
• Define current objects and attributes from the error
• Provide basic description of the error to student
• Generate new questions with:
• The same object and different attributes
• The same attribute and different object
• The same object and different object
• The same attribute and different attribute
• Clarify the source of error – most problematic object and attribute
16© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Scaffolding Place and Time
• In-topic self-assessment
• At any time, student can select objects and attributes + level of adaptivity
• Scaffolding is fully automatic
• In-topic assessment
• At the end of lesson, student can select level of scaffolding
• End-of-topic grading
• At the end of topic
• Summative assessment
• Scaffolding is used as part of formative feedback
17© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Scaffolding Levels (MKO Abilities)
• Reason 1 – detected error in process of in-topic (self-) assessment
• Levels
1. Indication of the error
2. Indication of a specific place (entity) of the error
3. Indication of invalid values of attributes (maybe after additional questions)
•  Importance of AIG!
4. References to sources of definitions
5. Indication of “distinctions” between entities (after additional questions)
•  Importance of AIG!
6. Reminder and link to prerequisites
• We have done all we can…
• Reason 2 – low rank at the end of a topic
• The other report
18© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Conclusion
• Ontology controlled knowledge assessment in the field of MSE with
automatic generation of several types of assignments
• Automatic knowledge gaps detection with permanent availability of
instant self-assessment according to constructivist pedagogic
approach
• Automatic pedagogical interventions after assessment actions with
formative feedback
• [future work] Educational content relevance estimation using Bayes
inference on learning logs
19© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
Contacts
• Contacts:
• Fedor Dudyrev
• E-mail: fdudyrev@hse.ru
• Web-site: https://www.hse.ru/org/persons/87525217
• Alexey Neznanov
• E-mail: aneznanov@hse.ru
• Web-site: http://hse.ru/staff/aneznanov
• Olga Maksimenkova
• E-mail: omaksimenkova@hse.ru
• Web-site: http://hse.ru/staff/maksimenkova
20© 2018, Dudyrev F., Maksimenkova O., Neznanov A.

Providing Cognitive Scaffolding within Computer-Supported Adaptive Learning Environment for Material Science Education

  • 1.
    Providing Cognitive Scaffolding withinComputer-Supported Adaptive Learning Environment for Material Science Education (ICL-2018, Kos) Fedor Dudyrev, Head of the Center for Vocational Education Research, National Research University Higher School of Economics Alexey Neznanov, senior researcher IL ISSA, Faculty of Computer Science, National Research University Higher School of Economics Olga Maksimenkova, junior research fellow IL ISSA, Faculty of Computer Science, National Research University Higher School of Economics © 2018, Dudyrev F., Maksimenkova O., Neznanov A. 1
  • 2.
    Context • Vocational studentsstudying technical subjects • Low level of knowledge and skills in basic school courses , esp.. math and physics • Learning outcomes freshmen who attend TVET system vary from 2 to 6 points in 10 grades scale • Percentage of early school leavers in TVET system is very high • Negative social environment 2© 2018, Dudyrev F., Maksimenkova O., Neznanov A. From John Elwood, License: CC BY-NC-ND From 4mygodsglory, License: CC BY-ND From Sandra Mathison, License: CC BY-SA-NC
  • 3.
    Personalized Learning asan Approach • Learning objectives, instructional approaches, and instructional content may all vary based on learner needs • Learning activities are made available that are meaningful and relevant to learners, driven by their interests and (often) self-initiated • The instructor takes into account the student's troubles and mistakes and provides quick, dosed and targeted assistance • The pace of learning and the instructional approach are optimized for the needs of each learner © 2018, Dudyrev F., Maksimenkova O., Neznanov A. 3
  • 4.
    Scaffolding as aTheoretical Framework for Personalized Learning (R.Wood, J.Bruner, L.Vygotsky) © 2018, Dudyrev F., Maksimenkova O., Neznanov A. 4 Learner: More Knowledgeable Other (MKO) in particular instructor: Stimulates learners cognitive interest and involves him in the problem-solving process Tries to solve a complex cognitive problem on his own Ensures the feasibility of the decision process, breaking the impossible task into easier subtasks Requests assistance to more knowledgeable other (MKO) and receives feedback Protects the student from frustration and loss of interest, encouraging and motivating him Internalizes the way to solve the problem and moves on to more complex tasks Ensures the internalization of the basic steps of reasoning process and gradually delegates to the learner additional degrees of independence
  • 5.
    Adaptive Learning System whichProvides Scaffolding Requirements: • Explicit description of learning outcomes • Ontology controlled knowledge assessment • Scaffolding types • Automatic mistakes/gap detection • Instant self-assessment • Formative feedback • Checking relevance of educational materials • Engaging active learning techniques • Reinforcing the training • Scalability 5© 2018, Dudyrev F., Maksimenkova O., Neznanov A. From flikr, license: CC BY-NC
  • 6.
    Adaptive Learning SystemArchitecture 6© 2018, Dudyrev F., Maksimenkova O., Neznanov A. Pedagogical Intervention Activation Subsystem Automatic Item Generation Subsystem Learning Process Management Module Relevant Materials Searching Subsystem Educational Materials Module Assessment Materials Bank Learning Management Subsystem EDM Tools Review and validation Subsystem Analytics Module Ontology Management Module Ontology Querying Subsystem EdMachine metaontology Learning processes ontology Domain ontology Learning materials DB References DB Analytics DB Assessment materials DB Learning activity DB
  • 7.
    “Scaffolding square” from TechnicalPoint of View Pedagogical Intervention Activation Subsystem • Logic of scaffolding Relevant Materials Searching Subsystem • Linkage between Scaffolding actions and Educational materials Ontology Querying Subsystem • Linkage between Scaffolding actions and Domain ontology + Learning process ontology Automatic Item Generation Subsystem • Obligate tool for automatic adaptive learning: Adaptive assessment, Prerequisite checking, Additional gap detection as part of formative feedback 7© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 8.
    Ontologies and OntologyModelling • Technical ontology – formalized computer-readed domain knowledge • From 1993 – Gruber T.: Explicit specification of a conceptualization • The most cited definition 8© 2018, Dudyrev F., Maksimenkova O., Neznanov A. From wikipedia, license: CC BY-SA
  • 9.
    Specific Integrated DomainOntology for MSE Mathematicalfoundations • Formal Concept Analysis (FCA) •Mens K. (In)Formal Concept Analysis (https://www.slideshare.net/kim.mens/formalcon ceptanalysis) •Priss U. Formal Concept Analysis Homepage (http://www.upriss.org.uk/fca/fca.html) • Descriptive/description logic •Krötzsch M., Simancik F., Horrocks I. A Description Logic Primer (http://arxiv.org/abs/1201.4089) •MLWiki – Descriptive Logic (http://mlwiki.org/index.php/Descriptive_Logic) Structure • Thesaurus – classical set of taxonomies + basic subclass of descriptive logic • Collections of: •Distinctor – special object for explanation of distinction between entities •Scale – definition of scaling multivalued attribute (datatype) into binary attribute •Formal context (binary, as a main source of data about objects) and associated multivalued context (as a source) 9© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 10.
    Thesaurus Nodes • Entitydescription • Id: GUID // Identifier for computer • Creator: UserId • DateOfCreation: ISO_Time • StatusOfVisibility: Enum = (IsActive, IsDeleted, IsReplaced) • StatusOfApproval: Enum = (IsDraft, IsProposed, IsApproved) • ReplacedBy: GUID • Name: UTF8 // Main title of node • Synonyms: Collection[UTF8] • Sense: UTF8 • Comments: UTF8 • Collection of relations • RelationType: SR_Id • Operand: GUID 10© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 11.
    Main Semantic Relations •Requirement: SKOS compatibility • Simple knowledge organization system (SKOS) – W3C Standard (http://www.w3.org/TR/2009/REC-skos-reference-20090818/) • Basic “class-class” relations • “Is-A” (generalization) • Example: <stainless steel> IS-A <steel> • “Part-Of” (aggregation) • Example: <housing> PART-OF <machine> • Basic “instance-class” relation • “Instans-Of” (instantiation) • Example: <4Х5В2ФС> INSTANS-OF <stainless steel> • Additional relations • “Defined-In” (source of definition) • Example: <4Х5В2ФС> DEFINED-IN <ГОСТ_5950-2000> • … 11© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 12.
    Table Representation ofThesaurus Fragment 12© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 13.
    Scales • Formal scale •Id: GUID • Caption: UTF8 • Hint: UTF8 • Source: UTF8 (text with refs to thesaurus) • ScaleType: Enum = (Nominal, Ordinal, Interval, Ratio) • IsOrdered: Boolean // explicit order of elements for any type • Cardinality: Integer • Names: array [0..Cardinality-1] of UTF8 • Values: array [0..Cardinality-1] of Variant (may be NULL for nominal scale) 13© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 14.
    Table Representation ofa Scale 14© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 15.
    Formal Context 15© 2018,Dudyrev F., Maksimenkova O., Neznanov A. • Multivalued context ↔ Binary context • Main derived artifact – formal concept lattice (not for this presentation ) • Example of small binary context for geometric figures:
  • 16.
    Automatic Item Generation(AIG) • One of the most important part of scaffolding engine • Goal – additional identification of knowledge gap • Method • Define current objects and attributes from the error • Provide basic description of the error to student • Generate new questions with: • The same object and different attributes • The same attribute and different object • The same object and different object • The same attribute and different attribute • Clarify the source of error – most problematic object and attribute 16© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 17.
    Scaffolding Place andTime • In-topic self-assessment • At any time, student can select objects and attributes + level of adaptivity • Scaffolding is fully automatic • In-topic assessment • At the end of lesson, student can select level of scaffolding • End-of-topic grading • At the end of topic • Summative assessment • Scaffolding is used as part of formative feedback 17© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 18.
    Scaffolding Levels (MKOAbilities) • Reason 1 – detected error in process of in-topic (self-) assessment • Levels 1. Indication of the error 2. Indication of a specific place (entity) of the error 3. Indication of invalid values of attributes (maybe after additional questions) •  Importance of AIG! 4. References to sources of definitions 5. Indication of “distinctions” between entities (after additional questions) •  Importance of AIG! 6. Reminder and link to prerequisites • We have done all we can… • Reason 2 – low rank at the end of a topic • The other report 18© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 19.
    Conclusion • Ontology controlledknowledge assessment in the field of MSE with automatic generation of several types of assignments • Automatic knowledge gaps detection with permanent availability of instant self-assessment according to constructivist pedagogic approach • Automatic pedagogical interventions after assessment actions with formative feedback • [future work] Educational content relevance estimation using Bayes inference on learning logs 19© 2018, Dudyrev F., Maksimenkova O., Neznanov A.
  • 20.
    Contacts • Contacts: • FedorDudyrev • E-mail: fdudyrev@hse.ru • Web-site: https://www.hse.ru/org/persons/87525217 • Alexey Neznanov • E-mail: aneznanov@hse.ru • Web-site: http://hse.ru/staff/aneznanov • Olga Maksimenkova • E-mail: omaksimenkova@hse.ru • Web-site: http://hse.ru/staff/maksimenkova 20© 2018, Dudyrev F., Maksimenkova O., Neznanov A.