Async Code Reviews Are Killing
Your Company’s Throughput

● Principal@HelloFresh
● Ex: @Careem/Uber,
@GetYourGuide
● Rants on
draganstepanovic.com
@d_stepanovic
● I put mayo on 🍕 😱

Meet people where they are

PR-based async code review

What was I curious to see?
● Engagement
● Wait Time
● Size

Engagement

“Never had a 300+ LoC PR that
didn’t look good to me”
~ Anonymous Developer

Why was I curious about the engagement?
● Feedback delayed and ‘choked’ by the system
● High-latency, low-throughput feedback
vs
Low-latency, high-throughput feedback
● Engagement by size?
○ Only counting non-trivial comments (no LGTM)

Lack of engagement/feedback =>
No ability to build the quality in

Wait time

A couple of important assumptions and facts
● Processing Time and Flow Efficiency on the bigger size PRs end of the
spectrum inaccurate because of git rewrite practice
● Wait Time way more accurate
● Wait time can have processing time
● Processing time can have wait time as well
● PR size is measured through simple LoC changed

Cost of code review per LoC
size

www.draganstepanovic.com/2020/11/16/pr-size-cannot-be-reduced.html

Throughput ↓

Throughput ↓
Quality ↓

Cost of code review per LoC
size
How do we keep cost of code review per LoC constant so
the throughput doesn’t plummet? 🤔

Cost of code
review per
LoC
size
Actors’ reaction
time
size
Availability of
actors
size
Actors = Author + Reviewers
Throughput
size

Availability of
actors
size

With the decrease in size of PR,
people need to get exponentially
closer and closer in time in order not
to lose throughput

You cannot be
interrupted if you’re not
doing anything else

Enter Co-creation patterns

How would this scatter look like
had we had continuous code
review after each line of code?

How would this scatter look like
had we had continuous code
review after each line of code?

PR Score

Size ↓
Wait time per size ↓
Engagement per size↑ (or not ↓)
What are we trying to optimize for?

PR Score
@staticmethod
def calculate_for(size, wait_time, engagement):
return math.log(1 + Score.absolute(engagement, size, wait_time))
@staticmethod
def absolute(engagement, size, wait_time):
return size * wait_time.in_seconds() / (1 + engagement)
1 ↑
0
= 0
= 0

Continuous code review (pairing/mobbing)

How would the world look like had we
paired?

We’ve been told all along that
we’ll achieve more if we limit
and delay our interactions
Hope you now have (also)
a data-informed reason to
not believe that
We’ve been told all along that
we’ll achieve more if we limit
and delay our interactions

● If you’re interested in trying out the tool for free?
dragan.stepanovic@gmail.com
● The idea of the tool is to provide you value to change your process so you don’t need the
tool anymore
● Only Github supported currently
● I’m writing a book (intersection of XP, Lean, ToC, Systems Thinking)
● If you’d like me to speak to your community about this topic, Small Batches and/or Intro to
ToC, Systemic Advantages of co-creation, let me know

@d_stepanovic