Bodong Chen (@bodong_c)
Assistant Professor, University of Minnesota
September 9, 2015, KBSI2015, Trieste, Italy
Towards M...
Idea Improvement in KB
▷ Why important?
▷ What is idea improvement?
Existing indicators
▷ Scientific sophistication (Zhang 2007)
▷ Levels of questioning (Law 2013)
▷ Lexical measures (Zhang ...
Multimodality
Multimodal interaction
▷ Human-Computer Interaction
▷ Multimodal Learning Analytics
Multiple modalities medi...
Two directions
Mining multimodal data in
existing KB environments
▷ Lexical
▷ Syntactic
▷ Graphic
Designing next-generatio...
1. Mining multimodal data
in existing environments
1.1. Lexical
▷ Text 1 – by a Grade 6 student: “My theory is solar eclipses are when the moon,
earth and sun are in a perfe...
Text 1 Text 2
Sentence count, number of sentences 4 4
Word count, number of words 100 87
Sentence length, number of words,...
1.2. Syntactic
Text 1 Text 2
Words before main verb, mean 3.5 7
Sentence syntax similarity, adjacent sentences, mean 0.09 ...
Xerox Incremental Parser
Source: CIC, UTS
1.3. Graphic
▷ Layout of note
network / view
structure
low centralization; long path length
vs.
high centralization; short...
x
y
1.3. Graphic
▷ View “landscape”
▷ Organizing
behaviors by
students
Other possibilities
▷ Affective states reflected in logs
(Baker 2010; McDaniel 2007; Pardos 2013)
▷ Certainty and identity...
2. Designing multimodal
knowledge building
environments
Motivations
▷ Pervasive knowledge building
▷ “One discourse, multiple entry points”
▷ Environments mediate knowledge
repre...
2.1. Sociometrics
(Onnela et al., 2014)
Physical proximity
(radio signal)
Speech time
(microphone)
Body movement
(accelero...
2.2. Mobile & mobile sensing
▷ Multimedia
○ Citizen science (Riesch, 2014)
▷ Multimodal sensors
▷ Context awareness
○ Loca...
Towards a holistic view
▷ “The Shadow Rabbit”
▷ Bridge different modalities
(Shum, 2015)
Scatterplot of reading and writing activities in six classes
Towards a holistic view
▷ “The Shadow Rabbit”
▷ Bridge different modalities
▷ Support multimodal
representations of ideas
...
“Thank you!
Slides: https://goo.gl/pRxuV9
Email: chenbd@umn.edu
Twitter: @bodong_c
Upcoming SlideShare
Loading in …5
×

KBSI 2015: Towards multimodal indicators of idea improvement in knowledge building

439 views

Published on

An invited talk about potential multimodal indicators for assessing idea improvement in group knowledge building. Trieste, Italy. Early thoughts; comments welcomed.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
439
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

KBSI 2015: Towards multimodal indicators of idea improvement in knowledge building

  1. 1. Bodong Chen (@bodong_c) Assistant Professor, University of Minnesota September 9, 2015, KBSI2015, Trieste, Italy Towards Multimodal Indicators of Idea Improvement: A Review of Possibilities
  2. 2. Idea Improvement in KB ▷ Why important? ▷ What is idea improvement?
  3. 3. Existing indicators ▷ Scientific sophistication (Zhang 2007) ▷ Levels of questioning (Law 2013) ▷ Lexical measures (Zhang & Sun 2011) ▷ Connectedness of concepts (Oshima 2013) ▷ ...
  4. 4. Multimodality Multimodal interaction ▷ Human-Computer Interaction ▷ Multimodal Learning Analytics Multiple modalities mediating meaning-making ▷ Communication ▷ Linguistics Goal of this presentation: Explore emerging possibilities from other communities, with attention to both conceptions.
  5. 5. Two directions Mining multimodal data in existing KB environments ▷ Lexical ▷ Syntactic ▷ Graphic Designing next-generation multimodal KB environments ▷ Sociometrics ▷ Mobile sensing
  6. 6. 1. Mining multimodal data in existing environments
  7. 7. 1.1. Lexical ▷ Text 1 – by a Grade 6 student: “My theory is solar eclipses are when the moon, earth and sun are in a perfect line. Why solar eclipses don't happen every month is because the moon's axis around the earth is slightly tilted. The moon needs to be so close to the earth so that it covers the space of the sun. You have to be in a certain spot in the earth to see a full solar eclipse but there is an arc the sun and moon make across the earth but from other spots on the arc you would only see a partial eclipse.” ▷ Text 2 – from Wikipedia: “As seen from the Earth, a solar eclipse is a type of eclipse that occurs when the Moon passes between the Sun and Earth, and the Moon fully or partially blocks the Sun. This can happen only at new moon, when the Sun and the Moon are in conjunction as seen from Earth in an alignment referred to as syzygy. In a total eclipse, the disk of the Sun is fully obscured by the Moon. In partial and annular eclipses, only part of the Sun is obscured.”
  8. 8. Text 1 Text 2 Sentence count, number of sentences 4 4 Word count, number of words 100 87 Sentence length, number of words, mean 25 21.75 Word length, number of letters, mean 3.75 4 Text Easability PC Narrativity, z score 0.54 0.08 WordNet verb overlap 0.61 0.30 LSA overlap, adjacent sentences, mean 0.56 0.63 Causal verb incidence 10 34.48
  9. 9. 1.2. Syntactic Text 1 Text 2 Words before main verb, mean 3.5 7 Sentence syntax similarity, adjacent sentences, mean 0.09 0.24 Noun phrase density, incidence 320 333.33 Verb phrase density, incidence 220 172.41 Adverbial phrase density, incidence 10 45.98 Preposition phrase density, incidence 100 172.41
  10. 10. Xerox Incremental Parser Source: CIC, UTS
  11. 11. 1.3. Graphic ▷ Layout of note network / view structure low centralization; long path length vs. high centralization; short path length
  12. 12. x y 1.3. Graphic ▷ View “landscape” ▷ Organizing behaviors by students
  13. 13. Other possibilities ▷ Affective states reflected in logs (Baker 2010; McDaniel 2007; Pardos 2013) ▷ Certainty and identity in text (Blikstein 2013) ▷ Sentiment in text (Chamlertwat 2012)
  14. 14. 2. Designing multimodal knowledge building environments
  15. 15. Motivations ▷ Pervasive knowledge building ▷ “One discourse, multiple entry points” ▷ Environments mediate knowledge representation and meaning-making
  16. 16. 2.1. Sociometrics (Onnela et al., 2014) Physical proximity (radio signal) Speech time (microphone) Body movement (accelerometer) Social intimacy Talkativeness Turn-taking Personality Energy level
  17. 17. 2.2. Mobile & mobile sensing ▷ Multimedia ○ Citizen science (Riesch, 2014) ▷ Multimodal sensors ▷ Context awareness ○ Location ○ Surroundings ○ Proximity ▷ Augmented reality
  18. 18. Towards a holistic view ▷ “The Shadow Rabbit” ▷ Bridge different modalities (Shum, 2015)
  19. 19. Scatterplot of reading and writing activities in six classes
  20. 20. Towards a holistic view ▷ “The Shadow Rabbit” ▷ Bridge different modalities ▷ Support multimodal representations of ideas (Shum, 2015)
  21. 21. “Thank you! Slides: https://goo.gl/pRxuV9 Email: chenbd@umn.edu Twitter: @bodong_c

×