Presentation slides for the LASD'21 paper "Experience vs Data: A Case for More Data-Informed Retrospective Activities" Matthies, C., Dobrigkeit, F. (2021). Experience vs Data: A Case for More Data-Informed Retrospective Activities. In: Przybyłek, A., Miler, J., Poth, A., Riel, A. (eds) Lean and Agile Software Development. LASD 2021. Lecture Notes in Business Information Processing, vol 408. Springer, Cham. https://doi.org/10.1007/978-3-030-67084-9_8

1. Hasso Plattner Institute
University of Potsdam, Germany
christoph.matthies@hpi.de
@chrisma0
Experience vs Data: A Case for More
Data-informed Retrospective Activities
Christoph Matthies, Franziska Dobrigkeit
virtual conference, January ’21

2. Scrum Development Method
2
Iterative Process Improvement: The Retrospective Meeting

3. Retrospective Meetings
3
Definition
opportunity for the team to inspect itself
– The Scrum Guide [Schwaber, 2017]
“ ”
[Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” (2017)
[online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
Image: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland
■ Time and space to discuss and improve development process
■ Looking back after a development iteration

4. 4
Team Retrospectives
Two separate (?) worlds: Perceptions & Project Data

5. 5
Motivation
A Case for More Data-informed Retrospective Activities
■ Decisions based on easily collectible team perceptions
■ Modern development practices (& automated tools) allow
insights into teams’ processes via project data [Zaitsev, 2020]
□ version control (what was changed, why, when?)
□ communication tools (what are others working on?)
□ software builds (what is the testing status?)
[Zaitsev, 2020] Anna Zaitsev, Uri Gal, and Barney Tan. “Coordination artifacts in Agile Software Development”. In:
Information and Organization 30.2 (June 2020), p. 100288. issn: 14717727. doi: 10.1016/j.infoandorg.2020.100288.

6. Retrospective Activities
6
Agendas that Structure Retrospectives
■ Proposed meeting agendas to structure Retros [Jovanovic, 2015]
■ Generalized phases: set the stage, gather data, generate insight,
decide what to do, close [Derby, 2006]
■ Multiple collection efforts [Loeffler, 2017], most comprehensive :
The Retromat online tool, retromat.org [Baldauf, 2018]
[Jovanovic, 2015] Jovanovic, M., Mesquida, A.L., Mas, A.: Process improvement with retrospective gaming in agile
software development. In: Systems, Software and Services Process Improvement. pp. 287–294. Springer (2015)
[Derby, 2006] Derby, E., Larsen, D.: Agile Retrospectives: Making Good Teams Great. Pragmatic Bookshelf Series,
Pragmatic Bookshelf (2006)
[Loeffler, 2017] Marc Loeffler. Improving Agile Retrospectives: Helping Teams Become More Efficient. Addison-Wesley
Professional, 2017, p. 270. isbn: 978-0134678344.
[Baldauf, 2018] Corinna Baldauf. Retromat - Run great agile retrospectives! Leanpub.com, 2018, p. 239.

7. Research Goals
■ Focus: integration of project data sources into Agile Retrospectives
■ Analysis of 140 Retromat activities
■ Review types of data being employed
to identify possible improvements.
■ Highlight activities already relying on software project data
and those that can be augmented
7
Research Questions

8. Research Steps
8
Review of Retrospective Activities and Project Data Usage
1. Extract activities from Retromat that provide or generate
inputs for subsequent use
2. Review descriptions, identify data points being collected
3. Categorize data sources
4. Analyze in detail activities which already
(or are close to) taking project data into account

9. Activity Extraction
9
Review of Retromat Activities for the Five Retrospective Phases
■ Extracted 35 activities for gather data phase
■ Reviewed all other phases for mentions of “data gathering”
□ Data collection and analysis steps often intertwined
□ Activity's main focus broader than data collection
□ Result: 4 additional activities [1]
e.g. Snow Mountain (using Scrum burndown chart)
[1] https://retromat.org/en/?id=70-84-106-118

10. Retrospective Activity Inputs
10
■ Analyzed textual descriptions of activities
■ Manually tagged with the specific data points collected as inputs
■ Generalized physical representations, index cards ➔ notes
■ Examples:
□ Numerical ratings of performed meetings
□ Notes on what team members wish the team would learn
The Kinds of Data That Teams Work With During an Activity

11. Data Source Classification
11
■ Distinguish whether gathered data is
□ drawn solely from perceptions
□ extracted from project data
□ is ambiguous and depends on project context
Analyzing the Types of Data That Is Being Used

12. Data Source Classification
12
■ 86% (30 of 35) of gather data activities
do not mention project data
■ Often discussion prompts, e.g. mad, sad, glad
□ By default use perceptions and experiences
Analyzing the Types of Data That Is Being Used

13. Activities Reliant on Data
13
■ Overall nine activities with (possible) project data connections
■ Five depend on interpretation and project context
■ Four make direct mentions of specific development artifacts
□ Tiny fraction of 140 Retromat activities
□ Use data produced in regular tasks of modern software devs
□ Collectable with minimal overhead [Ortu, 2015]
Retromat Activities That Take Project Data Into Account
[Ortu, 2015] Marco Ortu et al. “The JIRA Repository Dataset”. In: Proceedings of the 11th International Conference on Predictive Models
and Data Analytics in Software Engineering. ACM Press, 2015, pp. 1–4. isbn: 9781450337151. doi: 10.1145/2810146.2810147.

14. Activities Reliant on Data
14
The Four Activities That Already Explicitly Rely on Project Data

15. Possibly Data-informed
15
Example: Agile Self-Assessment [SeAs]
■ Assessments on team state, based on a predefined checklist
■ Depends on checklist, e.g. “minimal time from pushing code to test”
or “we deliver what the business needs most”
■ Modification: Use checklist featuring measurements based on
Agile practice usage and project data [Matthies, 2016]
Activities That May Use Project Data, Depending on Context
[SeAs] https://retromat.org/en/?id=35
[Matthies, 2016] Christoph Matthies et al. “Agile metrics for a university software engineering course”. In: 2016 IEEE Frontiers in Education
Conference. IEEE, 2016, pp. 1–5. isbn: 978-1-5090-1790-4. doi: 10.1109/FIE.2016.7757684.

16. Conclusions
16
■ Perceptions vary strongly between team members [Derby, 2006]
■ Project data analyses in Retrospectives enable additional view &
“evidence-based decision making” [Fitzgerald, 2014]
■ Initial Retrospective activities based on data are already exist
■ Team members may focus on interpreting existing data
instead of reconstructing details of the last iteration
Take-Away Messages
[Derby, 2006] Esther Derby and Diana Larsen. Agile retrospectives: Making Good Teams Great. Pragmatic Bookshelf, 2006, p. 200. isbn: 0-9776166-4-9
[Fitzgerald, 2014] Brian Fitzgerald, Mariusz Musia l, and Klaas-Jan Stol. “Evidence-based decision making in lean software project management”. In:
Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014. ACM Press, 2014, pp. 93–102.

17. Future Work
17
The Research Questions and Ideas That Remain Open
■ Extend the toolbox of activities with more data-informed ones
■ Create & evaluate new data-informed Retrospective activities
based exclusively on (analysis/visualizations) of project data
□ Mining Software Repositories
□ Agile Process Measurements
Image: https://erikbern.com/2016/12/05/the-half-life-of-code.html

18. Summary
18
christoph.matthies@hpi.de @chrisma0
HPI, University of Potsdam, Germany

19. ■ sprint retrospective by Eucalyp from the Noun Project (CC-BY 3.0)
■ Meeting by Shocho from the Noun Project (CC-BY 3.0)
■ tools by Nhor from the Noun Project (CC-BY 3.0)
■ Games by Icons Producer from the Noun Project (CC-BY 3.0)
■ Meeting by Chanut is Industries from the Noun Project (CC-BY 3.0)
■ Target by Arthur Shlain from the Noun Project (CC-BY 3.0)
■ Research by Eucalyp from the Noun Project (CC-BY 3.0)
■ Books by sandra from the Noun Project (CC-BY 3.0)
■ Inputs by jngll from the Noun Project (CC-BY 3.0)
■ data classification by Chanut is Industries from the Noun Project (CC-BY 3.0)
■ Data by priyanka from the Noun Project (CC-BY 3.0)
■ Checklist by unlimicon from the Noun Project (CC-BY 3.0)
■ conclusion by Justin Blake from the Noun Project (CC-BY 3.0)
Image Credits
19
In order of appearance