Slides of the talk given at the Doctoral Symposium of ICSE 2019. https://2019.icse-conferences.org/track/icse-2019-Doctoral-Symposium#program

1. Hasso Plattner Institute
University of Potsdam, Germany
christoph.matthies@hpi.de
@chrisma0
Feedback in Scrum:
Data-Informed Retrospectives
Christoph Matthies
Doctoral Symp., Canada, May ’19

2. Motivation
2
Software Engineering in General
Software engineering must shed the folkloric advice [...],
replace them with [...] empirical methods
– Bertrand Meyer [Meyer, 2013]
“
”[Meyer, 2013] B. Meyer, H. Gall, M. Harman, and G. Succi, “Empirical Answers to Fundamental Software
Engineering Problems (Panel),” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software
Engineering, ser. ESEC/FSE 2013. New York, USA: ACM, 2013, pp. 14–18.
Picture: https://commons.wikimedia.org/wiki/File:Bertrand_Meyer_recent.jpg

3. Motivation
3
The Role of Data in Scrum
Scrum is founded on empirical process control theory [...].
Three pillars [...]: transparency, inspection, and adaptation.
– The Scrum Guide [Schwaber, 2017]
“
”[Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” 2017,
[online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
Picture: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland

4. Main Research Topic
4
Likely PhD Thesis Topic
Supporting agile teams
in their process adaptation efforts
using transparency
and inspection of
their own project data

5. Related Work
5
[Svensson, 2019]
[Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”,
20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948

6. Unfulfilled Potential of DDDM
6
[Svensson, 2019]
■ Survey of software practitioners
■ How is data used in the company for making decisions?
[Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”,
20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948

7. Software Project Data
7
Mining Repositories of Teams [Kalliamvakou et al., 2016]
■ Project data is continuously produced by development teams
■ Holds insights into team processes
code code analyses
Project Data
documentation
Primary purpose: Communication Opportunity: Process Insights
...
[Kalliamvakou et al., 2016] Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D. M., Damian, D. “An in-depth study of the promises and
perils of mining GitHub”. Empirical Software Engineering, 21(5), pp. 2035–2071. 2016. https://doi.org/10.1007/s10664-015-9393-5

8. Agile Process Improvement
8
The Retrospective Meeting
■ Scrum’s dedicated process improvement meeting
■ Feedback on the product as well as the process

9. The Retrospective
9
Tracking Retrospective Action Items
Did we improve
what we planned?
commits,
reviews
test runs
tickets
static
analysis
Retrospective
Meeting
Project Data
Evidence of last
iteration’s work

10. Current Research Hypothesis
10
Towards Data-Informed Process Improvement
■ Development data is already created by Agile teams during
regular development activities.
■ It holds extensive information on how team members
work and collaborate.
■ Teams can use analyses of this data to inform and track
their process improvement steps.

11. Related Work
11
Mining Software Repositories
■ Draw from MSR techniques [Dyer et al., 2013]
■ However, mostly focus on large amounts of code
□ “What do README files look like?” [Prana et al., 2018]
□ “most widely used open source license?” [Dyer et al., 2013]
■ Little research: Few repositories,
intricate knowledge of creators / users
[Prana et al., 2018] Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. “Categorizing the Content of
GitHub README Files”. Empirical Software Engineering. 2018. https://doi.org/10.1007/s10664-018-9660-3
[Dyer et al., 2013] Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. “Boa: A language and infrastructure for
analyzing ultra-large-scale software repositories”. In Proceedings - International Conference on Software
Engineering. pp. 422–431. 2013. IEEE.

12. Contributions So Far
12
■ Development data of student teams provided actionable insights
□ into team processes [1,2]
□ for exercise improvement [3]
□ for improving teaching efforts [4,5]
■ Measurements from course experience and from literature
[1] Matthies, C., Kowark, T., Richly, K., Uflacker, M., & Plattner, H. “How Surveys, Tutors, and Software Help to Assess Scrum Adoption”. In
Proceedings of the 38th International Conference on Software Engineering Companion - ICSE ’16. pp. 313–322 2016
[2] Matthies, C., Kowark, T., Uflacker, M., & Plattner, H. “Agile Metrics for a University Software Engineering Course”. In 2016 IEEE Frontiers in
Education Conference (FIE). pp. 1–5. 2016.
[3] Matthies, C., Treffer, A., & Uflacker, M. “Prof. CI: Employing Continuous Integration Services and GitHub Workflows to Teach Test-Driven
Development”. In 2017 IEEE Frontiers in Education Conference (FIE). pp. 1–8. 2017
[4] Matthies, C. “Scrum2kanban: Integrating Kanban and Scrum in a University Software Engineering Capstone Course”. In Proceedings of the 2nd
International Workshop on Software Engineering Education for Millennials - SEEM ’18. pp. 48–55. 2018
[5] Matthies, C., Teusner, R., & Hesse, G. “Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts”. In 2018 IEEE
Frontiers in Education Conference (FIE). pp. 1–9. 2018

13. Next Steps
13
Application in Industry
■ Learnings not directly transferable to industry
□ Experienced professionals working full-time
□ Custom development processes
■ Study challenges of improving processes in industry
□ How are Retrospectives implemented in industry?
□ What are the outcomes of Retrospectives?
□ Can / are action items tracked?

14. Current Industry Study
14
Interviews with Agile Facilitators
■ Initial interviews in companies (Wikimedia, Signavio, Nokia HERE, SAP Teams)
□ Project data usage: None to Jira with custom plugins
□ Little usage of data for process improvement (except Kanban cycle time)
□ No mentions of using data for tracking retro issues:
“regression tests for processes”
■ Interest in application of project data analysis
for everything (also for management)
■ Retrospectives not as mature as assumed

15. Next Steps in Industry
15
Interviews with Agile Facilitators
■ Is project data being used or considered useful?
■ Collect and organize the Retrospective outcomes in industry
□ Action items which are directly related to data vs.
those that are not, e.g. interpersonal issues.
■ Form further hypotheses on how teams can
be supported with tools for process improvement

16. Summary
16

17. Image Credits
17
In order of appearance
■ retrospective meeting by Shocho from the Noun Project (CC BY 3.0 US)
■ Mortar Board by Mike Chum from the Noun Project (CC BY 3.0 US)
■ Target by Arthur Shlain from the Noun Project (CC BY 3.0 US)
■ Paper By LUTFI GANI AL ACHMAD, ID the Noun Project (CC BY 3.0 US)