Scaling Communication
via Continuous Integration
Th e Fi n di ngs Th us Fa r!

LB Denker
Anthropologist of
Developer Culture
@ Etsy

Anthropologist?!?
... only sort of ...

Methods of an
Anthropologist
Participant
Observation
Interviews
Surveys

Developer Culture
“The most important
component of The Etsy Way
is culture and that is as
difficult to teach as it is
important”
— Chad Dickerson, CEO

Acquiring Culture
“... humans acquire culture through the
learning processes of enculturation and
socialization ...”

Developer Culture
“Always Be Pushing”
Developer Happiness
We trust people

“Anthropology demands the open-mindedness
with which one must look and listen, record in
astonishment and that which one would not
have been able to guess.”
— Margaret Mead

Deployinating All the Pushes
One Button (stage)
Logging
Monitoring
No Child Safety Locks

Anyone Can Deploy!
Board Members Included

20 Minutes or Less
The Power of Continuous Deployment

Continuous What?!?
Continuous... Unceremoniously... Initiated By...
Push New Code
Deployment Anyone
To Production
Apply Quality
Integration Anyone
Process
Release New
Delivery Product
Feature

Continuous Deployment

Continuous Integration

Continuous Delivery ???

Continuous Delivery ???

Continuous Delivery ???

Communication Theory
1. The Basic Elements of Communication
Source Destination
Sender Message
Channel Feedback
Receiver
2. Goal-Oriented Communication

Goals
“Always Be Pushing”
You Shouldn’t Push on Red
You Should Write Clean Tests
You Should Write Clean Code
Trust Everyone To Do The Right Thing

“Always Be Pushing”
Instilling Confidence
Th e Fi rs t In te r v ie ws

Confidence
Continuous Deployment is...
Scary
A Safety Net
Depends on Your Point of View

Branching was
Too Costly

Always Ship Trunk
Branch in Code, not
in Version Control
Write Tools to Work
with Trunk
Do not Write Tools to
Work with Branches
Write Transparent
Tools

Quality Process was
Coupled to Pipeline

‘Try’ Before You Commit
Ability to Run Tests in
CI Environment
Actually Continuous
Integration!
Community Enforced!
No Excuses!

Confidence Boost!
The Impact of Knowing that Changes Pass the
Tests Before Getting in the Pipeline

Why ‘Try’?
No more clogging the
pipeline
CI in Pipeline was
Final Sanity Check

Goal
Achieved
“Always Be Pushing”

You Should Not
Push on Red
De-Noising the Communication Channel
Pa rt ic ip a n t Ob se r v ati on s over IRC

Where We Started From...
Number of
Total Time Flakiness Support
Tests
~30 Practically
Legacy Unknown ~50%
Minutes None
~8
Platform ~1200 ~20% Small Core
Minutes

A Bunch of Noise
Need to Know How Much is Being Covered
Need to Know What is Being Tested
Need to Be Able to Detect Test Runner Errors
Need to Have Actionable Test Results
(Ownership)

De-Noise
Number of
Total Time Flakiness Support
Tests
~30 Practically
Legacy Unknown ~50%
Minutes None
>8
Platform >1200 ~5% Small Core
Minutes

Goal
Achieved
You Should Not
Push on Red

You Should
Write Clean Tests
Writing Simple Tests
Isn’t Always So Simple
Alter n ati ve Te ach ing Met h od s

How To Improve Test Quality
Teach Them!
But People Need to
Want to Learn

Prove It!
Number of Average
Test Time
Tests Time per Test
Platform Tests 7:01 2332 0.18 seconds
0.00337
Unit Tests 0:10 2432
seconds
DBUnit Tests 0:32 70 0.09 seconds
December 2010

Test Classification
PHPUnit @group
PHPUnit XML Configuration + Some Pake
Split By Point of Error
ie. cache, database, network, sleep, time, etc.
Flaky Test Silo

Goal
Achieved
You Should
Write Clean Tests

Too Much
To Read
... and to
Communicate

How Do Communicate This?
E-Mail? Nope!
Wiki? Nope!
Blog? Nope!
Simplify Jenkins UI?
YES!

Silent Green
Taking a Chapter Out of xUnit Principles

Jenkins Plugins
Master Project Plugin
Triggering User Plugin
Deployinator Plugin
* All Available on github.com/etsy

You Should
Write Clean Code
Not Everyone
Can Read (and Follow) a Coding Standard
En a bl ing C ul tu re In fl ue n ce

Clean Code
Start with Syntax
Move to Style
Then to Design Level

PHP Lint
Run Locally
Run on Pre-Commit
Run on CI

Using CodeSniffer
Draft Coding Style Guidelines
Experiment With Various Sniffs
Discover Actual Coding Syntax Tendencies
Decide on Changes to Coding Style Guidelines
Create a Strategy on How to Get to Green

Work in
Progress
You Should
Write Clean Code

Getting to Green
Send ‘Blame’ E-Mails
Use a Staging Job in Try
Do a Fix-It Competition

Trust Everyone
To Do The Right Thing
Balancing Enforcement with Freedom
Fu tu re Wor k

Continuously Growing
a Continuously Evolving Culture

Recently Done
Code Reviews, Second Pair of Eyes
PHPUnit --strict Standards
PHPUnit Testing Standards
Log All the Errors E_ALL | E_STRICT

More To Do
Entire Coding Standard in CI
Mess Detector Standards
More PHPUnit Testing Standards
...

Developer Culture
“Always Be Pushing”
Developer Happiness
We trust people

Company
etsy.com
About Our Craft
codeascraft.com
More From Me
elblinkin.info