Dependencies,
Distributed code
and engineering velocity
Mike McGarr
@SonOfGarr
© J. Michael McGarr, 2017
I have a problem...
© J. Michael McGarr, 2017
...that impacts every engineer at Netflix.
© J. Michael McGarr, 2017
Affects you too.
© J. Michael McGarr, 2017
I don't know the solution.
© J. Michael McGarr, 2017
direction
© J. Michael McGarr, 2017
So what is this problem?
© J. Michael McGarr, 2017
SHARED
CODE© J. Michael McGarr, 2017
cloud microservices
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
client libraries
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
transitive dependencies
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
consuming libraries
should be
easy© J. Michael McGarr, 2017
software isn't static
© J. Michael McGarr, 2017
semantic versions
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
(*) 3.1.13
3.1.14
3.1.15
3.1.16
3.2.0
3.2.1
3.2.2
3.2.3
4.0.0-RC1
© J. Michael McGarr, 2017
Which version is good?
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
(*) 3.1.13 ❌
3.1.14 ❌
3.1.15 ❌
3.1.16 ✔
3.2.0 ❌
3.2.1 ❌
3.2.2 ✔
3.2.3 ❌
4.0.0-RC1 ❓
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
Semantic versioning is insufficient
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
Which version of guava
will you get?
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
Pinning versions is
technical debt
© J. Michael McGarr, 2017
Just give me a
version that
works!© J. Michael McGarr, 2017
consuming libraries
is actually
hard© J. Michael McGarr, 2017
Well, at least
publishing
is the easy, right?
© J. Michael McGarr, 2017
Need to change foobar-client:3.2.2
→ Bug fix? 3.2.3
→ Behavior change? 3.2.0
→ API change? 4.0.0
© J. Michael McGarr, 2017
Who will I break?
© J. Michael McGarr, 2017
Who is consuming
my library?
© J. Michael McGarr, 2017
Who is using this
API
that I want to change?
© J. Michael McGarr, 2017
Publishers lack
organization-wide
visibility
© J. Michael McGarr, 2017
There are
solutions out
there...
© J. Michael McGarr, 2017
Option #1:
Just deal with pain
© J. Michael McGarr, 2017
Option #2:
Share nothing
© J. Michael McGarr, 2017
Option #2:
Share nothing little
© J. Michael McGarr, 2017
Option #3:
Monorepo
© J. Michael McGarr, 2017
→ all code in a single repository
→ source integration
→ head is production
→ no versions
→ atomic API changes
→ lock step deployments
© J. Michael McGarr, 2017
constraints
© J. Michael McGarr, 2017
monorepo depends on gates
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
Our approach...
© J. Michael McGarr, 2017
Can we get the benefits
of a monorepo,
without the constraints?
© J. Michael McGarr, 2017
Publisher feedback
Managed source
Distributed refactoring
© J. Michael McGarr, 2017
Publisher feedback
© J. Michael McGarr, 2017
Astrid
Niagara
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
I now know who I will
break!
© J. Michael McGarr, 2017
Managed source
© J. Michael McGarr, 2017
eliminate reduce
consumer pain
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
© J. Michael McGarr, 2017
Shorten adoption curve
© J. Michael McGarr, 2017
Give me a
version that
works!© J. Michael McGarr, 2017
Distributed refactoring
© J. Michael McGarr, 2017
Gradle Lint
© J. Michael McGarr, 2017
How do I
Change my API
across hundreds of
repos?
© J. Michael McGarr, 2017
1. Find all repositories
2. Find all API calls
3. Refactor all API calls
4. Issue pull requests en masse
© J. Michael McGarr, 2017
experimental
© J. Michael McGarr, 2017
Publisher feedback
Managed source
Distributed refactoring
© J. Michael McGarr, 2017
"Eventually consistent distributed monorepo"
-- Avi Bryant (Stripe)
© J. Michael McGarr, 2017
Will this work?
© J. Michael McGarr, 2017
Thank you
Mike McGarr
@SonOfGarr
mmcgarr@netflix.com
© J. Michael McGarr, 2017

Dependencies, distributed code and engineering velocity