Towards a Theory of Developer Satisfaction and Productivity

No notes for slide

(Transcript)
Today, I have the pleasure of presenting a journal-first paper about a theory of developer satisfaction and productivity, joint work with Tom Zimmermann, Chris Bird, Jacek Czerwonka, Brendan Murphy and Eirini Kalliamvakou.

This research started when I visited Microsoft for a sabbatical in 2017. I was hosted by the 1ES team that is responsible for building the engineering systems for most of Microsoft.

One of their mandates is to understand developer satisfaction with engineering tools, but they also wanted to know what other factors may affect engineering satisfaction and productivity.

Image from : https://www.archdaily.com/909841/microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing/5c45f358284dd1c2410000dd-microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing-photo

Understanding knowledge worker job satisfaction and the relationship with job performance has been studied for several decades.
Early in this research, many assumed there was no relationship or at best a unidirectional relationship between satisfaction and performance.

However, a meta review by Judge et al. showed that there is a bidirectional relationship between the two, and that other factors mediate that relationship. They also showed that the correlation between job satisfaction and performance is likely higher for more complex work. Consequently, we expected this theory would hold for software developers, but we didn’t know what specific factors may associate more closely with satisfaction and performance.

Image attribution: https://lmi.workforcegps.org/resources/2015/06/18/11/23/State_Level_Job-to-Job_Flows

So our goal for this study was to identify which factors as well as which challenges may impact satisfaction and productivity.
We also wanted to identify if work context variables may play a role in these relationships (such as the kind of work a developer does).

Our research methodology followed three phases.
First, weeks of onsite informal interviews and a literature review led to a preliminary set of 30 factors and 15 challenges.
We then conducted a survey, answered by 591 engineers, to confirm these factors and challenges, and identified an additional 14 factors and 9 challenges from open-ended responses.
We conducted a second survey, answered by 465 developers, to probe how important they thought these factors were to developers, and how satisfied they were with the factors. We also identified the impact of the 24 challenges on developer satisfaction and productivity.
(see the paper’s supplementary materials for the origin of all factors/challenges)

Notes:
Response rate of 14% for first survey, 13% for the second.
However, of the 640 for the second survey, only 465 were developers.
More precisely 10 factors emerged from the first survey, but we also added a further 3 factors to the survey from a paper on work characteristics by Shaw and Gupta on work complexity.

This is how the second survey looked for the importance (and similarly for the satisfaction) question with a 5-point likert scale.
No new factors emerged from this survey.

Looking at the challenges, we found that the presence of some challenges was associated with lower satisfaction of several factors.
For example, the challenge of a poor manager was correlated with lower satisfaction across 16 different factors.

We anticipated that some of the 44 factors may be related. We clustered the factors into composite factors, using a correlation cut-off of 0.50 and identified 20 composite factors, for example, we clustered factors that relate to the engineering system, doing impactful work, culture of work, their work environment, and feeling they are an important team contributor.

Note: We refined the factors into composite factors by averaging their scores in the remaining analysis.

We built a linear regression model with standardized coefficients showing how these composite factors influence overall job satisfaction for developers.
The main factors in this model are doing impactful work, feeling they are an important team contributor and having a good work culture, and we see that higher perceived productivity is associated with higher job satisfaction.

Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.581. (RQ3.1)

A second linear regression model shows how the composite factors influence perceived productivity for developers.
The main influencing factors were work autonomy, the engineering system and doing impactful work, and again we see job satisfaction as a factor in the model for perceived productivity.

Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.5585. (RQ3.1).

As we built our models, we also considered different work context variables.
Here we show which challenges and factors appear in a model that focused on developers who code less than 37% of their time.

Like any study, there are limitations that we discuss in our paper. We note that our findings align with other studies that consider similar constructs or different settings. Our findings resonated with developers and management at Microsoft, and helped them understand that improving developer satisfaction and productivity requires attention to many different factors.

To conclude, we found that for the developers we studied at Microsoft, doing impactful work and being an important contributor are associated with both satisfaction and productivity. While appreciation, a good work culture, and work life balance are important for satisfaction, and autonomy over one’s work, ability to complete tasks, a good engineering system, strong technical skills, and a positive work environment are associated with higher productivity. Most importantly, we also found that a developer that feels more productive may feel more satisfied and vice-versa, and so steps taken to improve productivity that may lower satisfaction may ultimately lower productivity -- so care has to be paid to both aspects.

Thank you for listening

Missing friends and colleagues from last year!

Towards a Theory of Developer Satisfaction and Productivity

Margaret-Anne Storey

Towards a Theory of Developer Satisfaction and Productivity