Recommended
More Related Content
What's hot
What's hot (20)
Similar to Towards a Theory of Developer Satisfaction and Productivity
Similar to Towards a Theory of Developer Satisfaction and Productivity (20)
More from Margaret-Anne Storey
More from Margaret-Anne Storey (20)
Recently uploaded
Recently uploaded (20)
Towards a Theory of Developer Satisfaction and Productivity
- 1. Towards a Theory of Developer Satisfaction and Productivity Margaret-Anne Storey, Thomas Zimmermann, Chris Bird, Jacek Czerwonka, Brendan Murphy, Eirini Kalliamvakou July 8th, ICSE 2020 Journal First IEEE Transactions on Software Engineering @margaretstorey mstorey@uvic.ca Special Thanks Cassandra Petrachenko Engineers at Microsoft
- 2. A Sabbatical at Microsoft Engineering System Team Spring 2017
- 3. Theory about Job Satisfaction and Performance (Judge et al. 2001) 3 Job Satisfaction Job Performance Factors
- 4. 4 Work context variables Overall Job Satisfaction Productivity Satisfaction Social and Technical Factors Social and Technical Factors Challenges Goal: develop a developer satisfaction and productivity theory ?
- 5. A literature review and onsite interviews: 30 factors, 15 challenges Jan-March 2017 Second survey 465 developers satisfaction, importance of 44 factors, and impact of 24 challenges Oct 2017 First survey 591 engineers revealed additional 14 factors, 9 challenges April 2017 Methodology5
- 6. The final survey with 44 factors, Fall 20176
- 7. 7 Challenges Factors A link indicates satisfaction with a factor was significantly lower in the presence of a challenge:
- 8. 8 Engineering system Hardware Engineering processes Collaboration tools Engineering tools Composite factors Work environmentPhysical environment Interruptions # people in workspace Private office Impactful Work Learning skills for future Work vision Work type Can achieve goals Impact of work Work culture Manager Team culture Collaborative team Organization culture Important contributor Skills are well used Important team member
- 9. Linear regression models with standardized coefficients shows how different factors influence overall job satisfaction 9
- 10. 10 Linear regression models with standardized coefficients shows how different factors influence perceived productivity
- 11. 11 Instantiating the theory for a specific cohort of developers
- 12. External validity Single company, but diverse dev contexts Internal validity Ambiguity in questions Response bias Construct validity Self reported productivity Perceived importance of factors Limitations12 Transferability and credibility: Our findings align with other rigorous research [Murphy-Hill, Graziotin] and resonated with our study subjects throughout the study.
- 13. Developer Satisfaction Perceived Productivity Impactful work Important contributor 13 Appreciation Work culture Work-life balance Autonomy Can complete tasks Engineering system Technical skills Work environment Developer satisfaction and productivity theory @margaretstorey mstorey@uvic.ca
- 14. 14 Hello ICSE colleagues and friends….
Editor's Notes
- (Transcript) Today, I have the pleasure of presenting a journal-first paper about a theory of developer satisfaction and productivity, joint work with Tom Zimmermann, Chris Bird, Jacek Czerwonka, Brendan Murphy and Eirini Kalliamvakou.
- This research started when I visited Microsoft for a sabbatical in 2017. I was hosted by the 1ES team that is responsible for building the engineering systems for most of Microsoft. One of their mandates is to understand developer satisfaction with engineering tools, but they also wanted to know what other factors may affect engineering satisfaction and productivity. Image from : https://www.archdaily.com/909841/microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing/5c45f358284dd1c2410000dd-microsoft-is-investing-500-dollars-million-in-seattle-affordable-housing-photo
- Understanding knowledge worker job satisfaction and the relationship with job performance has been studied for several decades. Early in this research, many assumed there was no relationship or at best a unidirectional relationship between satisfaction and performance. However, a meta review by Judge et al. showed that there is a bidirectional relationship between the two, and that other factors mediate that relationship. They also showed that the correlation between job satisfaction and performance is likely higher for more complex work. Consequently, we expected this theory would hold for software developers, but we didn’t know what specific factors may associate more closely with satisfaction and performance. Image attribution: https://lmi.workforcegps.org/resources/2015/06/18/11/23/State_Level_Job-to-Job_Flows
- So our goal for this study was to identify which factors as well as which challenges may impact satisfaction and productivity. We also wanted to identify if work context variables may play a role in these relationships (such as the kind of work a developer does).
- Our research methodology followed three phases. First, weeks of onsite informal interviews and a literature review led to a preliminary set of 30 factors and 15 challenges. We then conducted a survey, answered by 591 engineers, to confirm these factors and challenges, and identified an additional 14 factors and 9 challenges from open-ended responses. We conducted a second survey, answered by 465 developers, to probe how important they thought these factors were to developers, and how satisfied they were with the factors. We also identified the impact of the 24 challenges on developer satisfaction and productivity. (see the paper’s supplementary materials for the origin of all factors/challenges) Notes: Response rate of 14% for first survey, 13% for the second. However, of the 640 for the second survey, only 465 were developers. More precisely 10 factors emerged from the first survey, but we also added a further 3 factors to the survey from a paper on work characteristics by Shaw and Gupta on work complexity.
- This is how the second survey looked for the importance (and similarly for the satisfaction) question with a 5-point likert scale. No new factors emerged from this survey.
- Looking at the challenges, we found that the presence of some challenges was associated with lower satisfaction of several factors. For example, the challenge of a poor manager was correlated with lower satisfaction across 16 different factors.
- We anticipated that some of the 44 factors may be related. We clustered the factors into composite factors, using a correlation cut-off of 0.50 and identified 20 composite factors, for example, we clustered factors that relate to the engineering system, doing impactful work, culture of work, their work environment, and feeling they are an important team contributor. Note: We refined the factors into composite factors by averaging their scores in the remaining analysis.
- We built a linear regression model with standardized coefficients showing how these composite factors influence overall job satisfaction for developers. The main factors in this model are doing impactful work, feeling they are an important team contributor and having a good work culture, and we see that higher perceived productivity is associated with higher job satisfaction. Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.581. (RQ3.1)
- A second linear regression model shows how the composite factors influence perceived productivity for developers. The main influencing factors were work autonomy, the engineering system and doing impactful work, and again we see job satisfaction as a factor in the model for perceived productivity. Note: The level of statistical significance is indicated with asterisks: (*) for p < :05, (**) for p < :01, and (***) for p < :001. Adjusted R-squared: 0.5585. (RQ3.1).
- As we built our models, we also considered different work context variables. Here we show which challenges and factors appear in a model that focused on developers who code less than 37% of their time.
- Like any study, there are limitations that we discuss in our paper. We note that our findings align with other studies that consider similar constructs or different settings. Our findings resonated with developers and management at Microsoft, and helped them understand that improving developer satisfaction and productivity requires attention to many different factors.
- To conclude, we found that for the developers we studied at Microsoft, doing impactful work and being an important contributor are associated with both satisfaction and productivity. While appreciation, a good work culture, and work life balance are important for satisfaction, and autonomy over one’s work, ability to complete tasks, a good engineering system, strong technical skills, and a positive work environment are associated with higher productivity. Most importantly, we also found that a developer that feels more productive may feel more satisfied and vice-versa, and so steps taken to improve productivity that may lower satisfaction may ultimately lower productivity -- so care has to be paid to both aspects. Thank you for listening
- Missing friends and colleagues from last year!