Examining Learner-computer Interactions: Advanced Lab-based Research Methods Slides before 1st Section Divider Motivation ...
Outline <ul><li>Examining learner-computer interactions </li></ul><ul><ul><li>Focus on detecting learning, even when nothi...
Outline <ul><li>Based on a specific study </li></ul><ul><ul><li>PhD research </li></ul></ul><ul><ul><li>Will broaden this ...
pptPlex Section Divider [Motivation of the research] The slides after this divider will be grouped into a section and give...
Acknowledgements <ul><li>Dr. James Aczel, Dr. Barbara Hodgson and Prof. Eileen Scanlon </li></ul><ul><li>Prof. Josie Taylo...
Acknowledgements <ul><li>Prof. Diana Laurillard and Prof. Margaret Cox </li></ul><ul><li>Pascal Mangold of Mangold Softwar...
The 2003 MSc Study
Methodological challenges <ul><li>“ You need times ‘cause you need it to that (points on the screen) times twenty” </li></...
Traditional approaches to analysing video data <ul><li>Methodological </li></ul><ul><ul><li>Reflexivity (e.g. Camera effec...
Digital Video and digital data <ul><li>Advantages </li></ul><ul><ul><li>Consistent record than observation notes, capture ...
pptPlex Section Divider [Advanced lab-based methods] The slides after this divider will be grouped into a section and give...
The data capture setup 19 February 2007 LKL Seminar (J.San-Diego@ioe.ac.uk)
Data capture and analysis tools
INTERACT ™
Protocols <ul><li>Think-aloud  </li></ul><ul><ul><li>Ericsson and Simon (1984) </li></ul></ul><ul><li>Eye-tracking </li></...
pptPlex Section Divider [Illustrative study] The slides after this divider will be grouped into a section and given the la...
The study design <ul><li>Data collection </li></ul><ul><ul><li>18 students with A-level Maths or higher </li></ul></ul><ul...
Standard external representations and instantiations
Instantiations <ul><li>Static : Non-moving, non-changing, non-interactive </li></ul><ul><li>Dynamic : Capable of animation...
The data
pptPlex Section Divider [Strategy as a unit of analysis] The slides after this divider will be grouped into a section and ...
Main research question <ul><li>How do representations instantiated in different ways influence learners’ cognitive process...
Strategies <ul><li>strategic theories </li></ul><ul><ul><li>strategies are attempts to solve problems </li></ul></ul><ul><...
Hypotheses <ul><li>Strategies with each standard external representation can be characterised at different levels of granu...
Strategies identified <ul><li>Representation-specific </li></ul><ul><ul><li>Algebraic, graphic and numeric </li></ul></ul>...
pptPlex Section Divider [Illustrative Analyses and Some Findings] The slides after this divider will be grouped into a sec...
Representations-specific strategies by task X instantiation
Frequency of participants for each strategy
The  chart  shows the participants’ imagining strategies graphed by instantiation across the three tasks
Areas Of Interest
“ Aha! moments” Participant’s talk: Aha! They are the same distance away.
“ Invisible path” Participant’s talk: This is going from minus two…
“ Invisible region” Participant’s talk: I’m trying to imagine what happens as the line tends to infinity…
Re-representation Participant's talk:  I don’t know what to call it… Err… I’ll just draw it
‘ Freeze frames’
Attention paid to representations
Focus of attention
Findings relating to difficulties 00:14:13:22 P4: It will never ever comes cross… Something... it never comes across
Bringing the evidence together 19 February 2007 LKL Seminar (J.San-Diego@ioe.ac.uk)
Other examples of evidence
Other examples of evidence 19 February 2007 LKL Seminar (J.San-Diego@ioe.ac.uk) Participant's talk:  I don’t know what to ...
pptPlex Section Divider [Technology Enhanced Research] The slides after this divider will be grouped into a section and gi...
<ul><li>Current project </li></ul><ul><ul><li>hapTEL (Haptic Technology Enhanced Learning) </li></ul></ul><ul><ul><li>PhD ...
The Future… <ul><li>Current focus on attention </li></ul><ul><ul><li>Mobiles </li></ul></ul><ul><ul><li>Windows </li></ul>...
From Marvin Minsky  (The Society Of Mind) <ul><li>It often does more harm than good to force definitions on things we don'...
Upcoming SlideShare
Loading in …5
×

TER Workshop J P San Diego

1,152 views

Published on

Examining learner-computer interactions: advanced lab-based research methods

Jonathan P. San Diego of King's College London presented an approach to examining learner-computer interactions using strategy as a unit of analysis developed within his PhD. He showed some of the data collection and analysis techniques that include capturing attention via eye-tracking, capturing sketches via tablet computers, integrating the analysis of multiple video feeds, and using strategy as a unit of analysis. Jonathan also gave some of his reflections on potential future uses of these research techniques.

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,152
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007 For this part of my presentation, I would like to thank some of the people who contributed to the completion of this set of techniques
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • Five hypotheses are generated using the research literature about strategies, and computer-based representations while in this section, a hypothesis is generated about the value of looking at gazes, utterances and sketches:
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • LKL Seminar 19 Feb 2007
  • TER Workshop J P San Diego

    1. 1. Examining Learner-computer Interactions: Advanced Lab-based Research Methods Slides before 1st Section Divider Motivation of the research Strategy as a unit of analysis Illustrative Analyses and Some Findings Unused Section Space 2 Technology Enhanced Research Unused Section Space 1 Advanced lab-based methods Illustrative study J.P. San Diego and J.C. Aczel
    2. 2. Outline <ul><li>Examining learner-computer interactions </li></ul><ul><ul><li>Focus on detecting learning, even when nothing is being explicitly “taught” </li></ul></ul><ul><ul><li>Trying to understand why and how learning is occurring </li></ul></ul><ul><ul><li>Within the learning context </li></ul></ul><ul><ul><li>Largely visual </li></ul></ul><ul><li>Advanced lab-based research methods </li></ul><ul><ul><li>Not just pointing a camera at a screen </li></ul></ul><ul><ul><li>Or asking “what are you learning?” </li></ul></ul><ul><ul><li>Advanced data collection: eye-tracking, sketches, gestures, physiological measures </li></ul></ul><ul><ul><li>Advanced data analysis: handling multiple video streams, software analytics and strategy as unit of analysis </li></ul></ul>
    3. 3. Outline <ul><li>Based on a specific study </li></ul><ul><ul><li>PhD research </li></ul></ul><ul><ul><li>Will broaden this out to reflections of other methods and applications </li></ul></ul>
    4. 4. pptPlex Section Divider [Motivation of the research] The slides after this divider will be grouped into a section and given the label you type above. Feel free to move this slide to any position in the deck.
    5. 5. Acknowledgements <ul><li>Dr. James Aczel, Dr. Barbara Hodgson and Prof. Eileen Scanlon </li></ul><ul><li>Prof. Josie Taylor and Dr. Richard Cox </li></ul><ul><li>Prof. Marian Petre, Prof. John Mason, Dr. Ann Jones, Dr. Patrick McAndrew and Dr. Denise Whitelock </li></ul><ul><li>Dr. Ekaterini Tzanidou, Dr. Geke vanDijk, Dr. Miquel Prats and Ms. Anesa Hosein </li></ul><ul><li>The participants of the study </li></ul>
    6. 6. Acknowledgements <ul><li>Prof. Diana Laurillard and Prof. Margaret Cox </li></ul><ul><li>Pascal Mangold of Mangold Software & Consulting GbHM </li></ul><ul><li>Microsoft Research in Cambridge through Dr. Fabien Petitcolas </li></ul><ul><li>IET-IT (Will Woods, P. Downs, D. Perry & S. Hammond) and CALRG Colleagues </li></ul><ul><li>OU-LTS (Mr. Collin Thomas) </li></ul><ul><li>Prof. Jeff Johnson, Prof. Chris Earl, Dr. Peter Lloyd and Dr. Georgy Holden, </li></ul>
    7. 7. The 2003 MSc Study
    8. 8. Methodological challenges <ul><li>“ You need times ‘cause you need it to that (points on the screen) times twenty” </li></ul>“ Oh OK I can see what it is doing (the graph) It is going towards there”
    9. 9. Traditional approaches to analysing video data <ul><li>Methodological </li></ul><ul><ul><li>Reflexivity (e.g. Camera effect), selectivity (transcript as data versus video as data) </li></ul></ul><ul><li>Technical </li></ul><ul><ul><li>Selecting, setting up, and operating video equipment </li></ul></ul><ul><li>Practical </li></ul><ul><ul><li>Data storage, transcription and coding </li></ul></ul><ul><li>Ethical </li></ul><ul><ul><li>Anonymity and privacy </li></ul></ul>
    10. 10. Digital Video and digital data <ul><li>Advantages </li></ul><ul><ul><li>Consistent record than observation notes, capture difficult-to-record events, multi-perspective, multi-observers, offers flexibility, stimulus for discussion </li></ul></ul><ul><li>Recent developments </li></ul><ul><ul><li>Variety of media, logs in video forms, video search technologies, processing power of computers, sensors, eye-tracking, haptics, sketch recognition, etc. </li></ul></ul>
    11. 11. pptPlex Section Divider [Advanced lab-based methods] The slides after this divider will be grouped into a section and given the label you type above. Feel free to move this slide to any position in the deck.
    12. 12. The data capture setup 19 February 2007 LKL Seminar (J.San-Diego@ioe.ac.uk)
    13. 13. Data capture and analysis tools
    14. 14. INTERACT ™
    15. 15. Protocols <ul><li>Think-aloud </li></ul><ul><ul><li>Ericsson and Simon (1984) </li></ul></ul><ul><li>Eye-tracking </li></ul><ul><ul><li>Yoon and Narayanan (2003), Hansen et al . (2001) </li></ul></ul><ul><li>Sketching </li></ul><ul><ul><li>Pirrie (1996, 1997), Cox (1996) </li></ul></ul>
    16. 16. pptPlex Section Divider [Illustrative study] The slides after this divider will be grouped into a section and given the label you type above. Feel free to move this slide to any position in the deck.
    17. 17. The study design <ul><li>Data collection </li></ul><ul><ul><li>18 students with A-level Maths or higher </li></ul></ul><ul><li>3 comparable tasks </li></ul><ul><ul><li>External mathematical representations </li></ul></ul><ul><li>Each task presented in either static, dynamic and interactive forms </li></ul>
    18. 18. Standard external representations and instantiations
    19. 19. Instantiations <ul><li>Static : Non-moving, non-changing, non-interactive </li></ul><ul><li>Dynamic : Capable of animation through alpha-numeric inputs </li></ul><ul><li>Interactive : Directly manipulable graphs </li></ul>
    20. 20. The data
    21. 21. pptPlex Section Divider [Strategy as a unit of analysis] The slides after this divider will be grouped into a section and given the label you type above. Feel free to move this slide to any position in the deck.
    22. 22. Main research question <ul><li>How do representations instantiated in different ways influence learners’ cognitive processes? </li></ul>
    23. 23. Strategies <ul><li>strategic theories </li></ul><ul><ul><li>strategies are attempts to solve problems </li></ul></ul><ul><ul><li>theories are conjectured expectations, dispositions, or assumptions (articulated or not), of some sort of reality in a particular context </li></ul></ul><ul><ul><li>a strategy can be considered as theoretical, in a sense, in that it incorporates expectations about some state of affairs </li></ul></ul><ul><ul><li>theory can be considered strategic, in a sense, in that some are instrumentally better adapted to reality than others </li></ul></ul><ul><li>Donald T. Campbell: Blind-Variation-and-Selective-Retention </li></ul><ul><ul><li>a mechanism for introducing variation [thought trials]; </li></ul></ul><ul><ul><li>a consistent selection pressure [concerns] </li></ul></ul><ul><ul><li>a mechanism for preserving and reproducing the selected variations </li></ul></ul><ul><li>Learning </li></ul><ul><ul><li>processes of discontinuous trial-and-improvement of strategic theories under the selection pressures provided by concerns </li></ul></ul>
    24. 24. Hypotheses <ul><li>Strategies with each standard external representation can be characterised at different levels of granularity. </li></ul><ul><li>Learners’ choice of strategies depends not just on the standard external representations given but also on the instantiation. </li></ul><ul><li>Mental constructions of images with graphical representations vary between instantiations. </li></ul><ul><li>Attention paid with each standard external representation varies between instantiations. </li></ul><ul><li>Expression of inferences varies depending on the instantiation </li></ul><ul><li>Analyses of strategies based on gazes, actions, utterances and sketches can identify factors contributing to strategy choice in a way that is not possible with traditional observation techniques. </li></ul>
    25. 25. Strategies identified <ul><li>Representation-specific </li></ul><ul><ul><li>Algebraic, graphic and numeric </li></ul></ul><ul><li>Imagining </li></ul><ul><ul><li>Pen, mouse, mental, gaze, gesture </li></ul></ul><ul><li>Re-representing </li></ul><ul><ul><li>Visual, textual, symbolic </li></ul></ul>
    26. 26. pptPlex Section Divider [Illustrative Analyses and Some Findings] The slides after this divider will be grouped into a section and given the label you type above. Feel free to move this slide to any position in the deck.
    27. 27. Representations-specific strategies by task X instantiation
    28. 28. Frequency of participants for each strategy
    29. 29. The chart shows the participants’ imagining strategies graphed by instantiation across the three tasks
    30. 30. Areas Of Interest
    31. 31. “ Aha! moments” Participant’s talk: Aha! They are the same distance away.
    32. 32. “ Invisible path” Participant’s talk: This is going from minus two…
    33. 33. “ Invisible region” Participant’s talk: I’m trying to imagine what happens as the line tends to infinity…
    34. 34. Re-representation Participant's talk: I don’t know what to call it… Err… I’ll just draw it
    35. 35. ‘ Freeze frames’
    36. 36. Attention paid to representations
    37. 37. Focus of attention
    38. 38. Findings relating to difficulties 00:14:13:22 P4: It will never ever comes cross… Something... it never comes across
    39. 39. Bringing the evidence together 19 February 2007 LKL Seminar (J.San-Diego@ioe.ac.uk)
    40. 40. Other examples of evidence
    41. 41. Other examples of evidence 19 February 2007 LKL Seminar (J.San-Diego@ioe.ac.uk) Participant's talk: I don’t know what to call it… Err… I’ll just draw it
    42. 42. pptPlex Section Divider [Technology Enhanced Research] The slides after this divider will be grouped into a section and given the label you type above. Feel free to move this slide to any position in the deck.
    43. 43. <ul><li>Current project </li></ul><ul><ul><li>hapTEL (Haptic Technology Enhanced Learning) </li></ul></ul><ul><ul><li>PhD Student (Arash Shahriari-rad) </li></ul></ul><ul><ul><ul><li>TER and Formative feedback </li></ul></ul></ul>The Future 19 February 2007 LKL Seminar (J.San-Diego@ioe.ac.uk)
    44. 44. The Future… <ul><li>Current focus on attention </li></ul><ul><ul><li>Mobiles </li></ul></ul><ul><ul><li>Windows </li></ul></ul><ul><ul><li>Books </li></ul></ul><ul><li>Jo Iacovides & games </li></ul><ul><ul><li>jaw tension (EMG) </li></ul></ul><ul><ul><li>skin conductance (GSR) </li></ul></ul><ul><ul><li>heart-rate (EKG) </li></ul></ul><ul><ul><li>brainwaves (EEG) </li></ul></ul>
    45. 45. From Marvin Minsky (The Society Of Mind) <ul><li>It often does more harm than good to force definitions on things we don't understand. Besides, only in logic and mathematics do definitions ever capture concepts perfectly. </li></ul><ul><li>The things we deal with in practical life are usually too complicated to be represented by neat, compact expressions. </li></ul><ul><li>Especially when it comes to understanding minds, we still know so little that we can't be sure our ideas about psychology are even aimed in the right directions. </li></ul><ul><li>In any case, one must not mistake defining things for knowing what they are. </li></ul>

    ×