Note: You can find a higher resolution slide deck here https://drive.google.com/file/d/1Umv5nfMMU_wJeGPekdBfMVbqp-BsM7ko/view?usp=sharing Abstract: "Never had a PR over 300 LoC that didn't look good to me". We've all been there. The PR is so big you don't even bother commenting. It's already too late to build the quality in. You make a sad face, comment "LGTM", and click approve. That's still the case in lots of teams but feels like after a long time the industry, on average, learned the value of the Small Batches idea from Lean applied to PRs. And that's a good thing; it's a step in the right direction. We do a bit of coding, raise a PR and then ask for feedback. People are more likely to engage on smaller PRs. It takes them less time to review it, and they have a feeling that they can still course-correct if something goes astray. PRs go sooner out of the door, and we feel productive as a team. But, here's the surprise. What if I told you that teams doing small PRs (with async code reviews) actually have way lower throughput than teams doing big PRs. "Wait, what?!" Yes. I got this surprising systemic insight from analyzing PRs across a bunch of very active repositories and I'll present you with the results of the study. On the bigger PRs side of the spectrum, we tend to lose quality, while on the smaller PRs end of the spectrum we lose throughput. We're forced to make a trade-off between speed and quality. But! There's a parallel universe of ways of working where you get to have your cake and eat it too. Where you can have both throughput and quality. That universe is called co-creation patterns (Pair and Mob programming). Join me on a journey where I'll show you data invalidating the assumption that two or more people sitting at the same computer will hurt our throughput and why the opposite is actually the case. Bio: Dragan is currently a principal engineer at HelloFresh, one of the unicorns of Berlin's thriving start-up scene. Before HelloFresh he helped Careem/Uber build the largest loyalty program in the MENA region, drive architecture, technical strategy, and shape engineering culture. Typically on the search for better ways of working, exploring ends of the spectrums, and helping teams and organizations try out counter-intuitive ideas that initially don't make a lot of sense, but end up as completely opposite of that. It's been a long time since he fell in love with eXtreme Programming, Domain-Driven Design, and software as a craft (founder of Software Crafting Serbia community). Last couple of years he enjoys endless discussions connecting the Theory of Constraints, Systems Thinking, Lean and socio-technical topics.

1. Async Code Reviews Are Killing
Your Company’s Throughput

2. ● Principal@HelloFresh
● Ex: @Careem/Uber,
@GetYourGuide
● Rants on
○ draganstepanovic.com
○ @d_stepanovic
● I put mayo on 🍕 😱

4. PR-based async code review

5. Meet people where they are

6. What was I curious to see?
● Engagement
● Wait Time
● Size

7. Engagement

8. “Never had a 300+ LoC PR that
didn’t look good to me”
~ Anonymous Developer

9. Why was I curious about the engagement?
● Feedback delayed and ‘choked’ by the system
● High-latency, low-throughput feedback
vs
Low-latency, high-throughput feedback
● Engagement by size?
○ Only counting non-trivial comments (no LGTM)

11. Lack of engagement/feedback =>
No ability to build the quality in

12. Wait time

15. A couple of important assumptions and
approximations
● Processing Time and Flow Efficiency on the bigger size PRs end of the
spectrum inaccurate because of git rewrite practice
● Wait Time way more accurate
● Wait time can have processing time
● Processing time can have wait time as well
● PR size is measured through simple LoC changed

19. Cost of code
review per
LoC
size

21. www.draganstepanovic.com/2020/11/16/pr-size-cannot-be-reduced.html

23. starts plummeting here
cumulative_lead_time(15 PRs, 20 LoC) >> cumulative_lead_time(1 PRs, 300 LoC)

24. Throughput ↓

26. "A 100% utilized
highway is a
parking lot."
~ Mary Poppendieck

27. Throughput ↓
Quality ↓

29. https://lastcallmedia.com/blog/why-devops-most-important-tech-strategy-today

30. There’s always a trade-off
There’s always a trade-off
Some trade-offs exist only
because of a flawed underlying
assumption

31. https://sircharlescaryinc.com/having-your-cake-and-eating-it-too/

32. cost of code
review/LoC
size
throughput
size

33. Cost of code
review per
LoC
size
Actors’ reaction
time
size
Availability of
actors
size
Actors = Author + Reviewers
Throughput
size

35. Availability of
actors
size

36. In order to not exponentially lose the
throughput while reducing the
average size of a PR
People need to get exponentially
closer and closer in time

38. You cannot be
interrupted if you’re not
doing anything else

39. https://i.ytimg.com/vi/2TUza5C2uJ8/maxresdefault.jpg

40. Enter Co-creation patterns
https://en.wikipedia.org/wiki/Mob_programming https://www.codefellows.org/blog/6-reasons-for-pair-programming/

41. https://lastcallmedia.com/blog/why-devops-most-important-tech-strategy-today

42. How would this scatter look like
had we have continuous code
review (after each line of code)?

43. How would this scatter look like
had we have continuous code
review (after each line of code)?

44. Both Throughput AND Quality

45. PR Score

47. Size ↓
Wait time per size ↓
Engagement per size↑ (or not ↓)
What are we trying to optimize for?

48. PR Score
@staticmethod
def calculate_for(size, wait_time, engagement):
return math.log(1 + Score.absolute(engagement, size, wait_time))
@staticmethod
def absolute(engagement, size, wait_time):
return size * wait_time.in_seconds() / (1 + engagement)
1 ↑
0
= 0
= 0

49. The optimal size of Pull Request is one
LoC that is reviewed immediately.
It's called Pair/Mob programming.

50. Continuous code review (pairing/mobbing)

51. How would the world look like had we
paired (for PRs up to 100 LoC)?

52. We’ve been told all along that
we’ll achieve more if we limit
and delay our interactions
Hope you now have (also)
a data-informed reason to
not believe that
We’ve been told all along that
we’ll achieve more if we limit
and delay our interactions

53. ● If you’re interested in trying out the tool for free drop me a line at:
dragan.stepanovic@gmail.com
○ The need for the tool is what is charged
○ Helps you change the process so you don’t have the need for the tool anymore
○ The same way that TDD renders obsolete the need for code coverage tools
● Only Github supported currently
● I’m writing a book (intersection of XP, Lean, ToC, Systems Thinking)
● If you’d like me to speak to your community about this topic, Small Batches and/or Intro to
ToC, Systemic Advantages of co-creation, feel free to reach out

54. @d_stepanovic