-
1.
Towards a Theory of
Developer Satisfaction and Productivity
Margaret-Anne Storey, Thomas Zimmermann, Chris Bird,
Jacek Czerwonka, Brendan Murphy, Eirini Kalliamvakou
July 8th, ICSE 2020 Journal First
IEEE Transactions on Software Engineering
@margaretstorey
mstorey@uvic.ca
Special Thanks
Cassandra Petrachenko
Engineers at Microsoft
-
2.
A Sabbatical at Microsoft Engineering System Team Spring
2017
-
3.
Theory about Job Satisfaction and Performance
(Judge et al. 2001)
3
Job
Satisfaction
Job
Performance
Factors
-
4.
4
Work context variables
Overall Job
Satisfaction
Productivity
Satisfaction
Social and
Technical
Factors
Social and
Technical
Factors
Challenges
Goal: develop a developer satisfaction and productivity theory
?
-
5.
A literature review and
onsite interviews:
30 factors,
15 challenges
Jan-March 2017
Second survey
465 developers
satisfaction, importance
of 44 factors, and
impact of 24 challenges
Oct 2017
First survey
591 engineers
revealed additional
14 factors, 9 challenges
April 2017
Methodology5
-
6.
The final survey with 44 factors, Fall 20176
-
7.
7
Challenges Factors
A link indicates satisfaction
with a factor was significantly
lower in the presence of a
challenge:
-
8.
8
Engineering system
Hardware
Engineering
processes
Collaboration tools
Engineering
tools
Composite factors
Work environmentPhysical
environment
Interruptions
# people in workspace
Private
office
Impactful Work
Learning skills for future
Work
vision
Work type
Can achieve
goals
Impact of work
Work culture
Manager
Team
culture
Collaborative team
Organization
culture
Important contributor
Skills are well used
Important team member
-
9.
Linear regression models with standardized coefficients
shows how different factors influence overall job satisfaction
9
-
10.
10
Linear regression models with standardized coefficients
shows how different factors influence perceived productivity
-
11.
11 Instantiating the theory for a specific cohort of developers
-
12.
External validity
Single company, but
diverse dev contexts
Internal validity
Ambiguity in questions
Response bias
Construct validity
Self reported productivity
Perceived importance
of factors
Limitations12
Transferability and credibility: Our findings align with other rigorous research
[Murphy-Hill, Graziotin] and resonated with our study subjects throughout the study.
-
13.
Developer
Satisfaction
Perceived
Productivity
Impactful work
Important contributor
13
Appreciation
Work culture
Work-life balance
Autonomy
Can complete tasks
Engineering system
Technical skills
Work environment
Developer satisfaction and productivity theory @margaretstorey
mstorey@uvic.ca
-
14.
14 Hello ICSE colleagues and friends….
(Transcript)
Today, I have the pleasure of presenting a journal-first paper about a theory of developer satisfaction and productivity, joint work with Tom Zimmermann, Chris Bird, Jacek Czerwonka, Brendan Murphy and Eirini Kalliamvakou.
This research started when I visited Microsoft for a sabbatical in 2017. I was hosted by the 1ES team that is responsible for building the engineering systems for most of Microsoft.
One of their mandates is to understand developer satisfaction with engineering tools, but they also wanted to know what other factors may affect engineering satisfaction and productivity.
Image from : https://www.archdaily.com/909841/microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing/5c45f358284dd1c2410000dd-microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing-photo
Understanding knowledge worker job satisfaction and the relationship with job performance has been studied for several decades.
Early in this research, many assumed there was no relationship or at best a unidirectional relationship between satisfaction and performance.
However, a meta review by Judge et al. showed that there is a bidirectional relationship between the two, and that other factors mediate that relationship. They also showed that the correlation between job satisfaction and performance is likely higher for more complex work. Consequently, we expected this theory would hold for software developers, but we didn’t know what specific factors may associate more closely with satisfaction and performance.
Image attribution: https://lmi.workforcegps.org/resources/2015/06/18/11/23/State_Level_Job-to-Job_Flows
So our goal for this study was to identify which factors as well as which challenges may impact satisfaction and productivity.
We also wanted to identify if work context variables may play a role in these relationships (such as the kind of work a developer does).
Our research methodology followed three phases.
First, weeks of onsite informal interviews and a literature review led to a preliminary set of 30 factors and 15 challenges.
We then conducted a survey, answered by 591 engineers, to confirm these factors and challenges, and identified an additional 14 factors and 9 challenges from open-ended responses.
We conducted a second survey, answered by 465 developers, to probe how important they thought these factors were to developers, and how satisfied they were with the factors. We also identified the impact of the 24 challenges on developer satisfaction and productivity.
(see the paper’s supplementary materials for the origin of all factors/challenges)
Notes:
Response rate of 14% for first survey, 13% for the second.
However, of the 640 for the second survey, only 465 were developers.
More precisely 10 factors emerged from the first survey, but we also added a further 3 factors to the survey from a paper on work characteristics by Shaw and Gupta on work complexity.
This is how the second survey looked for the importance (and similarly for the satisfaction) question with a 5-point likert scale.
No new factors emerged from this survey.
Looking at the challenges, we found that the presence of some challenges was associated with lower satisfaction of several factors.
For example, the challenge of a poor manager was correlated with lower satisfaction across 16 different factors.
We anticipated that some of the 44 factors may be related. We clustered the factors into composite factors, using a correlation cut-off of 0.50 and identified 20 composite factors, for example, we clustered factors that relate to the engineering system, doing impactful work, culture of work, their work environment, and feeling they are an important team contributor.
Note: We refined the factors into composite factors by averaging their scores in the remaining analysis.
We built a linear regression model with standardized coefficients showing how these composite factors influence overall job satisfaction for developers.
The main factors in this model are doing impactful work, feeling they are an important team contributor and having a good work culture, and we see that higher perceived productivity is associated with higher job satisfaction.
Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.581. (RQ3.1)
A second linear regression model shows how the composite factors influence perceived productivity for developers.
The main influencing factors were work autonomy, the engineering system and doing impactful work, and again we see job satisfaction as a factor in the model for perceived productivity.
Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.5585. (RQ3.1).
As we built our models, we also considered different work context variables.
Here we show which challenges and factors appear in a model that focused on developers who code less than 37% of their time.
Like any study, there are limitations that we discuss in our paper. We note that our findings align with other studies that consider similar constructs or different settings. Our findings resonated with developers and management at Microsoft, and helped them understand that improving developer satisfaction and productivity requires attention to many different factors.
To conclude, we found that for the developers we studied at Microsoft, doing impactful work and being an important contributor are associated with both satisfaction and productivity. While appreciation, a good work culture, and work life balance are important for satisfaction, and autonomy over one’s work, ability to complete tasks, a good engineering system, strong technical skills, and a positive work environment are associated with higher productivity. Most importantly, we also found that a developer that feels more productive may feel more satisfied and vice-versa, and so steps taken to improve productivity that may lower satisfaction may ultimately lower productivity -- so care has to be paid to both aspects.
Thank you for listening
Missing friends and colleagues from last year!
