SubvocalizationToward Hearing the Inner Thoughts of Developers<br />Chris Parnin<br />Georgia Tech<br />
Overview<br />+<br />=<br />Subvocal speech<br />EMG (electromyography)<br />Cognition?<br />
Problem<br />
Questions<br />Does a tool or language reduce cognitive effort?<br />Can we identify moments of learning uncertainty?<br /...
Some Attempts of <br />Measuring Cognition<br />
fMRI<br />12-30 secondstasks<br />While crammed in the middle of a jet engine<br />
EEG<br />
History of Reading?<br /> ”His heart sought out the meaning, but his voice was silent and his tongue was still”<br />300s ...
Kymograph<br />
Inner Speech (Brain)<br />
Speech Plans and Acts<br />
fMRI processing inner speech..<br />
BRAIN<br />
Measure with EMG<br />
Sokolov’s EMG Recordings (1970s)<br />
Reasons why People Subvocalize<br />Attention<br />Focus, maintain<br />Boost<br />Multiple modalities<br />Activation<br ...
Research Questions<br />When do developers subvocalize?<br />Can subvocalization be used to measure difficulty of programmi...
Tasks<br />Make block fall all the way down with space key.<br />Make monster reappear if missed shot.<br />
An instrumented session…<br />
Moments of interest and difficulty<br />Formulating<br />Problem<br />Found <br />useful code<br />Editing<br />Formulatin...
Difference in tasks at frequency bands<br />microvolts<br />Frequency bands<br />
One more thing…<br />
Silent Speech Interfaces<br />
State of the Art<br />?<br />“Things Programmers Think When Coding?”<br />Jorgensen<br />6 words<br />(8% error)<br />Schu...
Questions?<br />Detect Hmm?<br />
Subvocalization -- Towards Hearing the Inner Thoughts of Developers
Upcoming SlideShare
Loading in …5
×

Subvocalization -- Towards Hearing the Inner Thoughts of Developers

2,726
-1

Published on

Background and initial experiments performing subvocalization studies of programmers with EMG recordings.

Published in: Technology
2 Comments
0 Likes
Statistics
Notes
  • These slides don't really stand up on their own without the speaker :p

    This was a summer project. More detail here:
    http://www.cc.gatech.edu/~vector/papers/emg.pdf

    20-23 explains an experiment with measuring subvocalization during two programming tasks. I measured programmer activity: navigation events, clicks, edits, searches, etc, and looked for correlations with the subvocal events. Next, I looked for differences in the amount of subvocalization between the two programming tasks. The graph is the difference in the number of events in a frequency band. (harder task having more events)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Interesting topic. Don't quite get slides 20-23. Is this a project you are working on?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
2,726
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
2
Likes
0
Embeds 0
No embeds

No notes for slide
  • Can we look back in history for some inspiration?
  • Early Russian Scientistshttp://en.wikipedia.org/wiki/Lev_Vygotsky
  • http://www.acsu.buffalo.edu/~duchan/new_history/hist19c/science.htmlhttp://www.macalester.edu/linguistics/lab.html
  • Some anatomy
  • http://www.google.ca/url?sa=t&amp;source=web&amp;cd=5&amp;ved=0CEAQFjAE&amp;url=http%3A%2F%2Fwww-bmu.psychiatry.cam.ac.uk%2FPUBLICATION_STORE%2Fshergill02mod.pdf&amp;ei=ZxMBTpm3Dae40AGWroiKDg&amp;usg=AFQjCNEjDZExvuXwdVDIgsmZFAmtzUmhzg
  • Boost =&gt;Multiple modalitiesAttention =&gt; Focus and maintain attentionActivation =&gt; Turn on concepts helping comprehension and memory
  • Hmm… events
  • http://www.metrovision.fr/mv-pu-notice-us.html
  • Subvocalization -- Towards Hearing the Inner Thoughts of Developers

    1. 1. SubvocalizationToward Hearing the Inner Thoughts of Developers<br />Chris Parnin<br />Georgia Tech<br />
    2. 2. Overview<br />+<br />=<br />Subvocal speech<br />EMG (electromyography)<br />Cognition?<br />
    3. 3. Problem<br />
    4. 4. Questions<br />Does a tool or language reduce cognitive effort?<br />Can we identify moments of learning uncertainty?<br />Can we distinguish cognitive processes?<br />Can we find news ways to interact?<br />
    5. 5. Some Attempts of <br />Measuring Cognition<br />
    6. 6. fMRI<br />12-30 secondstasks<br />While crammed in the middle of a jet engine<br />
    7. 7. EEG<br />
    8. 8. History of Reading?<br /> ”His heart sought out the meaning, but his voice was silent and his tongue was still”<br />300s AD <br />
    9. 9.
    10. 10. Kymograph<br />
    11. 11. Inner Speech (Brain)<br />
    12. 12.
    13. 13. Speech Plans and Acts<br />
    14. 14. fMRI processing inner speech..<br />
    15. 15. BRAIN<br />
    16. 16. Measure with EMG<br />
    17. 17. Sokolov’s EMG Recordings (1970s)<br />
    18. 18. Reasons why People Subvocalize<br />Attention<br />Focus, maintain<br />Boost<br />Multiple modalities<br />Activation<br />comprehension, memory<br />
    19. 19. Research Questions<br />When do developers subvocalize?<br />Can subvocalization be used to measure difficulty of programming task?<br />Distinguish visual or verbal cognitive processing?<br />
    20. 20. Tasks<br />Make block fall all the way down with space key.<br />Make monster reappear if missed shot.<br />
    21. 21. An instrumented session…<br />
    22. 22. Moments of interest and difficulty<br />Formulating<br />Problem<br />Found <br />useful code<br />Editing<br />Formulating<br />solution<br />Found <br />useful code<br />Evaluating<br />
    23. 23. Difference in tasks at frequency bands<br />microvolts<br />Frequency bands<br />
    24. 24. One more thing…<br />
    25. 25. Silent Speech Interfaces<br />
    26. 26. State of the Art<br />?<br />“Things Programmers Think When Coding?”<br />Jorgensen<br />6 words<br />(8% error)<br />Schultz<br />101 words<br />(10% error)<br />2003<br />2010<br />ICPC 2012<br />
    27. 27.
    28. 28. Questions?<br />Detect Hmm?<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×