Dependencies,
Distributed code
and engineering velocity
Mike McGarr
@SonOfGarr
Ā© J. Michael McGarr, 2017
I have a problem...
Ā© J. Michael McGarr, 2017
...that impacts every engineer at Netflix.
Ā© J. Michael McGarr, 2017
Affects you too.
Ā© J. Michael McGarr, 2017
I don't know the solution.
Ā© J. Michael McGarr, 2017
direction
Ā© J. Michael McGarr, 2017
So what is this problem?
Ā© J. Michael McGarr, 2017
SHARED
CODEĀ© J. Michael McGarr, 2017
cloud microservices
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
client libraries
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
transitive dependencies
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
consuming libraries
should be
easy© J. Michael McGarr, 2017
software isn't static
Ā© J. Michael McGarr, 2017
semantic versions
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
(*) 3.1.13
3.1.14
3.1.15
3.1.16
3.2.0
3.2.1
3.2.2
3.2.3
4.0.0-RC1
Ā© J. Michael McGarr, 2017
Which version is good?
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
(*) 3.1.13 āŒ
3.1.14 āŒ
3.1.15 āŒ
3.1.16 āœ”
3.2.0 āŒ
3.2.1 āŒ
3.2.2 āœ”
3.2.3 āŒ
4.0.0-RC1 ā“
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Semantic versioning is insufficient
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Which version of guava
will you get?
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Pinning versions is
technical debt
Ā© J. Michael McGarr, 2017
Just give me a
version that
works!Ā© J. Michael McGarr, 2017
consuming libraries
is actually
hard© J. Michael McGarr, 2017
Well, at least
publishing
is the easy, right?
Ā© J. Michael McGarr, 2017
Need to change foobar-client:3.2.2
→ Bug fix? 3.2.3
→ Behavior change? 3.2.0
→ API change? 4.0.0
Ā© J. Michael McGarr, 2017
Who will I break?
Ā© J. Michael McGarr, 2017
Who is consuming
my library?
Ā© J. Michael McGarr, 2017
Who is using this
API
that I want to change?
Ā© J. Michael McGarr, 2017
Publishers lack
organization-wide
visibility
Ā© J. Michael McGarr, 2017
There are
solutions out
there...
Ā© J. Michael McGarr, 2017
Option #1:
Just deal with pain
Ā© J. Michael McGarr, 2017
Option #2:
Share nothing
Ā© J. Michael McGarr, 2017
Option #2:
Share nothing little
Ā© J. Michael McGarr, 2017
Option #3:
Monorepo
Ā© J. Michael McGarr, 2017
→ all code in a single repository
→ source integration
→ head is production
→ no versions
→ atomic API changes
→ lock step deployments
Ā© J. Michael McGarr, 2017
constraints
Ā© J. Michael McGarr, 2017
monorepo depends on gates
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Our approach...
Ā© J. Michael McGarr, 2017
Can we get the benefits
of a monorepo,
without the constraints?
Ā© J. Michael McGarr, 2017
Publisher feedback
Managed source
Distributed refactoring
Ā© J. Michael McGarr, 2017
Publisher feedback
Ā© J. Michael McGarr, 2017
Astrid
Niagara
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
I now know who I will
break!
Ā© J. Michael McGarr, 2017
Managed source
Ā© J. Michael McGarr, 2017
eliminate reduce
consumer pain
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Ā© J. Michael McGarr, 2017
Shorten adoption curve
Ā© J. Michael McGarr, 2017
Give me a
version that
works!Ā© J. Michael McGarr, 2017
Distributed refactoring
Ā© J. Michael McGarr, 2017
Gradle Lint
Ā© J. Michael McGarr, 2017
How do I
Change my API
across hundreds of
repos?
Ā© J. Michael McGarr, 2017
1. Find all repositories
2. Find all API calls
3. Refactor all API calls
4. Issue pull requests en masse
Ā© J. Michael McGarr, 2017
experimental
Ā© J. Michael McGarr, 2017
Publisher feedback
Managed source
Distributed refactoring
Ā© J. Michael McGarr, 2017
"Eventually consistent distributed monorepo"
-- Avi Bryant (Stripe)
Ā© J. Michael McGarr, 2017
Will this work?
Ā© J. Michael McGarr, 2017
Thank you
Mike McGarr
@SonOfGarr
mmcgarr@netflix.com
Ā© J. Michael McGarr, 2017

Dependencies, distributed code and engineering velocity