Presentation of IEEE TSE Journal First paper at ICSE 2020
Developer satisfaction and work productivity are important
considerations for software companies. Enhanced developer satisfaction may improve the attraction, retention and health of employees, while higher productivity should reduce costs and increase customer satisfaction through faster software improvements. Many researchers and companies assume that perceived productivity and job satisfaction are related and may be used as proxies for one another, but these claims are a current topic of debate. There are also many social and technical
factors that may impact satisfaction and productivity, but which factors have the most impact is not clear, especially for specific development contexts. Through our research, we developed a theory articulating a bidirectional relationship between software developer job satisfaction and perceived productivity, and identified what additional social and technical
factors, challenges and work context variables influence this relationship. The constructs and relationships in our theory were derived in part from related literature in software engineering and knowledge work, and we validated and extended these concepts through a rigorously designed survey instrument. We instantiate our theory with a large software company, which suggests a number of propositions about the relative impact of various factors and challenges on developer satisfaction and perceived productivity. Our survey instrument and analysis approach
can be applied to other development settings, while our findings lead to concrete recommendations for practitioners and researchers.
Margaret-Anne Storey, Tom Zimmermann, Chris Bird, Jacek Czerwonka, Brendan Murphy and Eirini Kalliamvakou
(Transcript) Today, I have the pleasure of presenting a journal-first paper about a theory of developer satisfaction and productivity, joint work with Tom Zimmermann, Chris Bird, Jacek Czerwonka, Brendan Murphy and Eirini Kalliamvakou.
This research started when I visited Microsoft for a sabbatical in 2017. I was hosted by the 1ES team that is responsible for building the engineering systems for most of Microsoft.
One of their mandates is to understand developer satisfaction with engineering tools, but they also wanted to know what other factors may affect engineering satisfaction and productivity.
Image from : https://www.archdaily.com/909841/microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing/5c45f358284dd1c2410000dd-microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing-photo
Understanding knowledge worker job satisfaction and the relationship with job performance has been studied for several decades. Early in this research, many assumed there was no relationship or at best a unidirectional relationship between satisfaction and performance.
However, a meta review by Judge et al. showed that there is a bidirectional relationship between the two, and that other factors mediate that relationship. They also showed that the correlation between job satisfaction and performance is likely higher for more complex work. Consequently, we expected this theory would hold for software developers, but we didn’t know what specific factors may associate more closely with satisfaction and performance.
So our goal for this study was to identify which factors as well as which challenges may impact satisfaction and productivity. We also wanted to identify if work context variables may play a role in these relationships (such as the kind of work a developer does).
Our research methodology followed three phases. First, weeks of onsite informal interviews and a literature review led to a preliminary set of 30 factors and 15 challenges. We then conducted a survey, answered by 591 engineers, to confirm these factors and challenges, and identified an additional 14 factors and 9 challenges from open-ended responses. We conducted a second survey, answered by 465 developers, to probe how important they thought these factors were to developers, and how satisfied they were with the factors. We also identified the impact of the 24 challenges on developer satisfaction and productivity. (see the paper’s supplementary materials for the origin of all factors/challenges)
Notes: Response rate of 14% for first survey, 13% for the second. However, of the 640 for the second survey, only 465 were developers. More precisely 10 factors emerged from the first survey, but we also added a further 3 factors to the survey from a paper on work characteristics by Shaw and Gupta on work complexity.
This is how the second survey looked for the importance (and similarly for the satisfaction) question with a 5-point likert scale. No new factors emerged from this survey.
Looking at the challenges, we found that the presence of some challenges was associated with lower satisfaction of several factors. For example, the challenge of a poor manager was correlated with lower satisfaction across 16 different factors.
We anticipated that some of the 44 factors may be related. We clustered the factors into composite factors, using a correlation cut-off of 0.50 and identified 20 composite factors, for example, we clustered factors that relate to the engineering system, doing impactful work, culture of work, their work environment, and feeling they are an important team contributor.
Note: We refined the factors into composite factors by averaging their scores in the remaining analysis.
We built a linear regression model with standardized coefficients showing how these composite factors influence overall job satisfaction for developers. The main factors in this model are doing impactful work, feeling they are an important team contributor and having a good work culture, and we see that higher perceived productivity is associated with higher job satisfaction.
Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.581. (RQ3.1)
A second linear regression model shows how the composite factors influence perceived productivity for developers. The main influencing factors were work autonomy, the engineering system and doing impactful work, and again we see job satisfaction as a factor in the model for perceived productivity.
Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.5585. (RQ3.1).
As we built our models, we also considered different work context variables. Here we show which challenges and factors appear in a model that focused on developers who code less than 37% of their time.
Like any study, there are limitations that we discuss in our paper. We note that our findings align with other studies that consider similar constructs or different settings. Our findings resonated with developers and management at Microsoft, and helped them understand that improving developer satisfaction and productivity requires attention to many different factors.
To conclude, we found that for the developers we studied at Microsoft, doing impactful work and being an important contributor are associated with both satisfaction and productivity. While appreciation, a good work culture, and work life balance are important for satisfaction, and autonomy over one’s work, ability to complete tasks, a good engineering system, strong technical skills, and a positive work environment are associated with higher productivity. Most importantly, we also found that a developer that feels more productive may feel more satisfied and vice-versa, and so steps taken to improve productivity that may lower satisfaction may ultimately lower productivity -- so care has to be paid to both aspects.
Thank you for listening
Missing friends and colleagues from last year!
Towards a Theory of Developer Satisfaction and Productivity
Towards a Theory of
Developer Satisfaction and Productivity
Margaret-Anne Storey, Thomas Zimmermann, Chris Bird,
Jacek Czerwonka, Brendan Murphy, Eirini Kalliamvakou
July 8th, ICSE 2020 Journal First
IEEE Transactions on Software Engineering
Engineers at Microsoft
A Sabbatical at Microsoft Engineering System Team Spring
Theory about Job Satisfaction and Performance
(Judge et al. 2001)
Work context variables
Goal: develop a developer satisfaction and productivity theory
A literature review and
of 44 factors, and
impact of 24 challenges
14 factors, 9 challenges
A link indicates satisfaction
with a factor was significantly
lower in the presence of a
# people in workspace
Learning skills for future
Impact of work
Skills are well used
Important team member
Linear regression models with standardized coefficients
shows how different factors influence overall job satisfaction
Linear regression models with standardized coefficients
shows how different factors influence perceived productivity
11 Instantiating the theory for a specific cohort of developers
Single company, but
diverse dev contexts
Ambiguity in questions
Self reported productivity
Transferability and credibility: Our findings align with other rigorous research
[Murphy-Hill, Graziotin] and resonated with our study subjects throughout the study.
Can complete tasks
Developer satisfaction and productivity theory @margaretstorey