Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
SubvocalizationToward Hearing the Inner Thoughts of Developers<br />Chris Parnin<br />Georgia Tech<br />
Overview<br />+<br />=<br />Subvocal speech<br />EMG (electromyography)<br />Cognition?<br />
Problem<br />
Questions<br />Does a tool or language reduce cognitive effort?<br />Can we identify moments of learning uncertainty?<br /...
Some Attempts of <br />Measuring Cognition<br />
fMRI<br />12-30 secondstasks<br />While crammed in the middle of a jet engine<br />
EEG<br />
History of Reading?<br /> ”His heart sought out the meaning, but his voice was silent and his tongue was still”<br />300s ...
Kymograph<br />
Inner Speech (Brain)<br />
Speech Plans and Acts<br />
fMRI processing inner speech..<br />
BRAIN<br />
Measure with EMG<br />
Sokolov’s EMG Recordings (1970s)<br />
Reasons why People Subvocalize<br />Attention<br />Focus, maintain<br />Boost<br />Multiple modalities<br />Activation<br ...
Research Questions<br />When do developers subvocalize?<br />Can subvocalization be used to measure difficulty of programmi...
Tasks<br />Make block fall all the way down with space key.<br />Make monster reappear if missed shot.<br />
An instrumented session…<br />
Moments of interest and difficulty<br />Formulating<br />Problem<br />Found <br />useful code<br />Editing<br />Formulatin...
Difference in tasks at frequency bands<br />microvolts<br />Frequency bands<br />
One more thing…<br />
Silent Speech Interfaces<br />
State of the Art<br />?<br />“Things Programmers Think When Coding?”<br />Jorgensen<br />6 words<br />(8% error)<br />Schu...
Questions?<br />Detect Hmm?<br />
Subvocalization -- Towards Hearing the Inner Thoughts of Developers
Upcoming SlideShare
Loading in …5
×

Subvocalization -- Towards Hearing the Inner Thoughts of Developers

3,648 views

Published on

Background and initial experiments performing subvocalization studies of programmers with EMG recordings.

Published in: Technology
  • These slides don't really stand up on their own without the speaker :p

    This was a summer project. More detail here:
    http://www.cc.gatech.edu/~vector/papers/emg.pdf

    20-23 explains an experiment with measuring subvocalization during two programming tasks. I measured programmer activity: navigation events, clicks, edits, searches, etc, and looked for correlations with the subvocal events. Next, I looked for differences in the amount of subvocalization between the two programming tasks. The graph is the difference in the number of events in a frequency band. (harder task having more events)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Interesting topic. Don't quite get slides 20-23. Is this a project you are working on?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Subvocalization -- Towards Hearing the Inner Thoughts of Developers

  1. 1. SubvocalizationToward Hearing the Inner Thoughts of Developers<br />Chris Parnin<br />Georgia Tech<br />
  2. 2. Overview<br />+<br />=<br />Subvocal speech<br />EMG (electromyography)<br />Cognition?<br />
  3. 3. Problem<br />
  4. 4. Questions<br />Does a tool or language reduce cognitive effort?<br />Can we identify moments of learning uncertainty?<br />Can we distinguish cognitive processes?<br />Can we find news ways to interact?<br />
  5. 5. Some Attempts of <br />Measuring Cognition<br />
  6. 6. fMRI<br />12-30 secondstasks<br />While crammed in the middle of a jet engine<br />
  7. 7. EEG<br />
  8. 8. History of Reading?<br /> ”His heart sought out the meaning, but his voice was silent and his tongue was still”<br />300s AD <br />
  9. 9.
  10. 10. Kymograph<br />
  11. 11. Inner Speech (Brain)<br />
  12. 12.
  13. 13. Speech Plans and Acts<br />
  14. 14. fMRI processing inner speech..<br />
  15. 15. BRAIN<br />
  16. 16. Measure with EMG<br />
  17. 17. Sokolov’s EMG Recordings (1970s)<br />
  18. 18. Reasons why People Subvocalize<br />Attention<br />Focus, maintain<br />Boost<br />Multiple modalities<br />Activation<br />comprehension, memory<br />
  19. 19. Research Questions<br />When do developers subvocalize?<br />Can subvocalization be used to measure difficulty of programming task?<br />Distinguish visual or verbal cognitive processing?<br />
  20. 20. Tasks<br />Make block fall all the way down with space key.<br />Make monster reappear if missed shot.<br />
  21. 21. An instrumented session…<br />
  22. 22. Moments of interest and difficulty<br />Formulating<br />Problem<br />Found <br />useful code<br />Editing<br />Formulating<br />solution<br />Found <br />useful code<br />Evaluating<br />
  23. 23. Difference in tasks at frequency bands<br />microvolts<br />Frequency bands<br />
  24. 24. One more thing…<br />
  25. 25. Silent Speech Interfaces<br />
  26. 26. State of the Art<br />?<br />“Things Programmers Think When Coding?”<br />Jorgensen<br />6 words<br />(8% error)<br />Schultz<br />101 words<br />(10% error)<br />2003<br />2010<br />ICPC 2012<br />
  27. 27.
  28. 28. Questions?<br />Detect Hmm?<br />

×