The document presents a study on the relationship between developers' emotions and their progress, using biometric sensors to track emotional states and performance during coding tasks. Key findings indicate that positive emotions correlate with perceived progress, while feelings of frustration hinder it, suggesting the importance of emotional awareness in developer productivity. The study discusses methods for leveraging biometric data to support developers effectively, especially when they are either stuck or in a productive flow state.

1. Sebastian Müller and Thomas Fritz
Stuck and Frustrated or in Flow and Happy:
Sensing Developers’ Emotions and Progress
ICSE 2015, Florence, Italy

4. Benefits of Knowing
Emotions and Progress
4
Support developers when stuck and
making no progress
Avoid interruptions when in flow and
making a lot of progress

5. Biometric Sensing of
Emotions & Progress
5
positive/negative emotions (e.g. Lin et al)
happiness, sadness (e.g. Li et al)
positive/negative emotions (e.g. Haag et al)
frustration (e.g. Scheirer et al)
positive/negative emotions (e.g. Muldner et al)
EEG
(Electroencephalography)
frustration (e.g. Kapoor et al)
anxiety (e.g. Doehring)
Pupil sizes
Eye blinks
positive/negative emotions (e.g. Haag et al)
boredom, anxiety (e.g. Chanel et al)
positive/negative emotions (e.g. Drachen et al)
happiness (e.g. Steptoe et al)
EDA
(Electro-dermal activity)
Skin temperature
HR(V)
(Heart rate (variability))

6. Research Questions
6
Correlation of emotions and progress?
Aspects and practices that affect developers’ emotions
and progress?
Biometric sensors to determine developers’ emotions
and progress?

7. Study Participants & Tasks
17 participants: 6 professionals and 11 PhD students
• on average 7.1 years of prof. development experience
• only minor differences between professionals and students
Two change tasks
• Task 1: small program to interact with StackExchange API
• Task 2: new feature in JHotDraw
7

8. Lab Study Setup
8

9. Lab Study Setup
8
Neurosky
headband
Empatica
wristband
Eyetribe
eye tracker

10. Study Method
9
Fish tank video as a baseline for biometric measurements

11. Study Method
9
Fish tank video as a baseline for biometric measurements
Work on two change tasks, each for 30 minutes

12. Study Method
9
Fish tank video as a baseline for biometric measurements
Work on two change tasks, each for 30 minutes
• progress
• valence: ‘’very unpleasant’’ to ‘’very pleasant’’
arousal: ‘’very calm’’ to ‘’very excited’’
• reasons for the current state of emotions / progress & aspects
that would help to improve
Assessments every 5 minutes or at signs of strong negative/
positive emotions

13. Study Method
9
Fish tank video as a baseline for biometric measurements
Work on two change tasks, each for 30 minutes
After working on both change tasks: set of emotion inducing pictures
• progress
• valence: ‘’very unpleasant’’ to ‘’very pleasant’’
arousal: ‘’very calm’’ to ‘’very excited’’
• reasons for the current state of emotions / progress & aspects
that would help to improve
Assessments every 5 minutes or at signs of strong negative/
positive emotions

14. Data Collection
10
biometric data from 3 sensors
213 data points
• emotion and progress ratings
• answers to the questions we asked
2 emotion ratings for the set of emotion
inducing pictures

15. Research Questions
11
Correlation of emotions and progress?
Aspects and practices that affect developers’ emotions
and progress
Biometric sensors to determine developers’ emotions
and progress

16. Correlation Emotions & Progress
12
Linear Mixed Model
Dependent variable: Self-reported progress
Fixed effects: Valence, arousal
Random effects: Participant, task, measurement time

17. Correlation Emotions & Progress
13
Linear Mixed Model
Dependent variable: Self-reported progress
Fixed effects: Valence, arousal
Random effects: Participant, task, measurement time
Random effects: Measurement time no effect, task medium effect and
participant highest effect
Effect Estimate Deviance explained
Valence 0.66 28.03
Arousal 0.10 1.09

18. Correlation Emotions & Progress
14

19. Correlation Emotions & Progress
14
Subject 1
Significant correlation between
progress and valence

20. Correlation Emotions & Progress
14
Subject 6Subject 1
Significant correlation between
progress and valence
No significant correlation
between progress and valence

21. Correlation Emotions & Progress
14
Subject 6Subject 1
Significant correlation between
progress and valence
No significant correlation
between progress and valence
significant correlations for 12 subjects,
but not for the other 5 subjects

22. Research Questions
15
Correlation of emotions and progress?
Aspects and practices that affect developers’ emotions
and progress?
Biometric sensors to determine developers’ emotions
and progress

23. Top 3 Reasons for a Change in
Emotions / Progress
16
Increase in emotions/progress # Cases # Subjects
localize relevant code 21 (11.3%) 14 (82.4%)
(better) understand parts of the code 18 (9.7%) 13 (76.5%)
next steps are clear 12 (6.5%) 9 (52.9%)

24. Top 3 Reasons for a Change in
Emotions / Progress
16
Increase in emotions/progress # Cases # Subjects
localize relevant code 21 (11.3%) 14 (82.4%)
(better) understand parts of the code 18 (9.7%) 13 (76.5%)
next steps are clear 12 (6.5%) 9 (52.9%)
”I’ve found a starting point almost
immediately. I have a good feeling that I’ll
make significant progress very soon.” (S1)

25. Top 3 Reasons for a Change in
Emotions / Progress
17
Decrease in emotions/progress # Cases # Subjects
difficulty in understanding how parts of the code / API works 33 (17.7%) 12 (70.6%)
difficulty in localizing relevant code 15 (8.1%) 8 (47.1%)
not being sure about next steps 9 (4.8%) 9 (52.9%)

26. Top 3 Reasons for a Change in
Emotions / Progress
17
Decrease in emotions/progress # Cases # Subjects
difficulty in understanding how parts of the code / API works 33 (17.7%) 12 (70.6%)
difficulty in localizing relevant code 15 (8.1%) 8 (47.1%)
not being sure about next steps 9 (4.8%) 9 (52.9%)
”It’s going too slow. I think it’s very cumbersome
when so much time is needed to understand the
project and find a starting point.” (S9)

27. Emotions and Progress
are Often Intertwined
18
”I finally figured out how to do it. I’m really
happy and I’m not feeling completely stuck
anymore.” (S6)
”I can’t make any progress.
That’s annoying.” (S13)

28. Research Questions
19
Correlation of emotions and progress?
Aspects and practices that affect developers’ emotions
and progress
Biometric sensors to determine developers’ emotions
and progress?

29. Electro-dermal Activity (EDA)
and Valence
20
1 2 3 4

30. Machine Learning Approach
21
Data Cleaning and Feature Extraction
• Noise cleaning
• Feature extraction following established methods
{Min, Max}Attention; {Min, Max}Meditation, …
∆(α); ∆(β); ∆(γ); ∆(δ); ∆(θ), …
{Min, Max}PeakAmpl; ∆NumPhasicPeaks/Min, …
MinPupilSize; MaxPupilSize, …
EEG
(Electroencephalography)
Pupil sizes
Eye blinks
EDA
(Electro-dermal activity)
Skin temperature
HR(V)
(Heart rate (variability))
MeanTemp; ∆MeanTemp, …
∆MeanHR; ∆VarianceHR, …
∆Eyeblinks/Min

31. Machine Learning Approach
22
Data Cleaning and Feature Extraction
• Noise cleaning
• Feature extraction following established methods
Data Labeling
• binary split of emotions (positive/negative) and progress (low/high)

32. Machine Learning Approach
23
Data Cleaning and Feature Extraction
• Noise cleaning
• Feature extraction following established methods
Data Labeling
• binary split of emotions (positive/negative) and progress (low/high)
Machine Learning
• Leave-one-out approach
• Classifier: decision tree S01
T1
S17
T1
S01
T2
S17
T2
T1
...S01 S17
Test data Training data
......
T2

33. Machine Learning Results
24
Prediction Correct Impr. Precision Recall Features
Emotion 71.36% 18.76% 64.32% 82.03%
α, β/θ, MinPupilSize,
∆MeanHR
Progress 67.70% 32.93% 67.85% 68.29%
α, β/θ,
∆MeanTempPeakAmpl,
MaxPupilSize,
∆MeanPupilSize,
∆MeanSCL

34. Contributions and Outlook
25
Study on developers’ emotions and progress with 17 participants
• Emotions and perceived progress are strongly correlated
• Biometric sensors can be used to predict emotions and progress
Future challenges
• Individual differences
• Privacy concerns
New opportunities for developer support
• Avoid interruptions when in flow and making a lot of progress
• Support when stuck and making no progress

35. Individual Machine Learning
Results
26
Accuracy(%)
0
25
50
75
100
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 S16 S17
Emotions Progress

36. Individual Machine Learning
Results
27
Accuracy(%)
0
25
50
75
100
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 S16 S17
Emotions Progress

37. Individual Machine Learning
Results
28
Accuracy(%)
0
25
50
75
100
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 S16 S17
Emotions Progress

38. Individual Machine Learning
Results
29
Accuracy(%)
0
25
50
75
100
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 S16 S17
Emotions Progress

39. Biometric Features
3030
EEG: {Min, Max}Attention; {Min, Max}Meditation; ∆{Mean, Stdev}
Attention; ∆{Mean, Stdev}Meditation;
∆(α); ∆(β); ∆(γ); ∆(δ); ∆(θ); ∆(α/β); ∆(α/γ); ∆(α/δ); ∆(α/θ); ∆(β/α);
∆(β/γ); ∆(β/δ); ∆(β/θ); ∆(γ/α); ∆(γ/β); ∆(γ/δ); ∆(γ/θ); ∆(δ/α); ∆(δ/β);
∆(δ/γ); ∆(δ/θ); ∆(θ/α); ∆(θ/β); ∆(θ/γ); ∆(θ/δ); ∆(θ/(α+β)); ∆(β/(α+θ))
EDA: {Min, Max}PeakAmpl; ∆NumPhasicPeaks/Min;
∆MeanPhasicPeakAmpl; ∆SumPhasicPeakAmpl/Min
∆MeanSCL; ∆AUCPhasic
Skin temperature: MeanTemp; ∆MeanTemp; {Max, Mean}
TempPeakAmpl; SumTempPeakAmpl/Min
HR(V): ∆MeanHR; ∆VarianceHR; SDNN; RMSSD; NN20; NN50;
pNN20; pNN50
Pupil size: MinPupilSize; MaxPupilSize; ∆{Mean, Stdev}PupilSize;
∆NumPupilSizeOutliers
Eye blinks: ∆Eyeblinks/Min