At Etsy, we continuously deploy code. New features and bug fixes can go from development to production in 20 minutes. We have maintained this low barrier to deployment over the past year despite nearly doubling the number of people that can deploy. Communication has a huge impact on the velocity of development and moving forward. We explicitly communicate over IRC, but we also evolve communication patterns through several other vectors, and one of those vectors is out Continuous Integration environment. In this talk I will not only discuss how communication patterns at Etsy have evolved with our growing development team, but I will also elaborate on how team size, role composition, motivation, culture, and available tools sets can impact communication patterns and subsequently throughput.

1. Scaling Communication
via Continuous Integration
Th e Fi n di ngs Th us Fa r!

2. LB Denker
Anthropologist of
Developer Culture
@ Etsy

3. Anthropologist?!?
... only sort of ...

4. Methods of an
Anthropologist
Participant
Observation
Interviews
Surveys

5. Developer Culture
“The most important
component of The Etsy Way
is culture and that is as
difficult to teach as it is
important”
— Chad Dickerson, CEO

6. Acquiring Culture
“... humans acquire culture through the
learning processes of enculturation and
socialization ...”

7. Developer Culture
“Always Be Pushing”
Developer Happiness
We trust people

8. “Anthropology demands the open-mindedness
with which one must look and listen, record in
astonishment and that which one would not
have been able to guess.”
— Margaret Mead

9. Deployinating All the Pushes
One Button (stage)
Logging
Monitoring
No Child Safety Locks

10. Anyone Can Deploy!
Board Members Included

11. 20 Minutes or Less
The Power of Continuous Deployment

12. Continuous What?!?
Continuous... Unceremoniously... Initiated By...
Push New Code
Deployment Anyone
To Production
Apply Quality
Integration Anyone
Process
Release New
Delivery Product
Feature

13. Continuous Deployment

14. Continuous Integration

15. Continuous Delivery ???

16. Continuous Delivery ???

17. Continuous Delivery ???

18. Communication Theory
1. The Basic Elements of Communication
Source Destination
Sender Message
Channel Feedback
Receiver
2. Goal-Oriented Communication

19. Goals
“Always Be Pushing”
You Shouldn’t Push on Red
You Should Write Clean Tests
You Should Write Clean Code
Trust Everyone To Do The Right Thing

20. “Always Be Pushing”
Instilling Confidence
Th e Fi rs t In te r v ie ws

21. Confidence
Continuous Deployment is...
Scary
A Safety Net
Depends on Your Point of View

22. Branching was
Too Costly

23. Always Ship Trunk
Branch in Code, not
in Version Control
Write Tools to Work
with Trunk
Do not Write Tools to
Work with Branches
Write Transparent
Tools

24. Quality Process was
Coupled to Pipeline

25. ‘Try’ Before You Commit
Ability to Run Tests in
CI Environment
Actually Continuous
Integration!
Community Enforced!
No Excuses!

26. Confidence Boost!
The Impact of Knowing that Changes Pass the
Tests Before Getting in the Pipeline

27. Why ‘Try’?
No more clogging the
pipeline
CI in Pipeline was
Final Sanity Check

28. Goal
Achieved
“Always Be Pushing”

29. You Should Not
Push on Red
De-Noising the Communication Channel
Pa rt ic ip a n t Ob se r v ati on s over IRC

30. Where We Started From...
Number of
Total Time Flakiness Support
Tests
~30 Practically
Legacy Unknown ~50%
Minutes None
~8
Platform ~1200 ~20% Small Core
Minutes

31. A Bunch of Noise
Need to Know How Much is Being Covered
Need to Know What is Being Tested
Need to Be Able to Detect Test Runner Errors
Need to Have Actionable Test Results
(Ownership)

32. De-Noise
Number of
Total Time Flakiness Support
Tests
~30 Practically
Legacy Unknown ~50%
Minutes None
>8
Platform >1200 ~5% Small Core
Minutes

33. Goal
Achieved
You Should Not
Push on Red

34. You Should
Write Clean Tests
Writing Simple Tests
Isn’t Always So Simple
Alter n ati ve Te ach ing Met h od s

35. How To Improve Test Quality
Teach Them!
But People Need to
Want to Learn

36. Prove It!
Number of Average
Test Time
Tests Time per Test
Platform Tests 7:01 2332 0.18 seconds
0.00337
Unit Tests 0:10 2432
seconds
DBUnit Tests 0:32 70 0.09 seconds
December 2010

37. Test Classification
PHPUnit @group
PHPUnit XML Configuration + Some Pake
Split By Point of Error
ie. cache, database, network, sleep, time, etc.
Flaky Test Silo

38. Goal
Achieved
You Should
Write Clean Tests

39. Too Much
To Read
... and to
Communicate

40. How Do Communicate This?
E-Mail? Nope!
Wiki? Nope!
Blog? Nope!
Simplify Jenkins UI?
YES!

41. Silent Green
Taking a Chapter Out of xUnit Principles

42. Jenkins Plugins
Master Project Plugin
Triggering User Plugin
Deployinator Plugin
* All Available on github.com/etsy

43. You Should
Write Clean Code
Not Everyone
Can Read (and Follow) a Coding Standard
En a bl ing C ul tu re In fl ue n ce

44. Clean Code
Start with Syntax
Move to Style
Then to Design Level

45. PHP Lint
Run Locally
Run on Pre-Commit
Run on CI

46. Using CodeSniffer
Draft Coding Style Guidelines
Experiment With Various Sniffs
Discover Actual Coding Syntax Tendencies
Decide on Changes to Coding Style Guidelines
Create a Strategy on How to Get to Green

47. Work in
Progress
You Should
Write Clean Code

48. Getting to Green
Send ‘Blame’ E-Mails
Use a Staging Job in Try
Do a Fix-It Competition

49. Trust Everyone
To Do The Right Thing
Balancing Enforcement with Freedom
Fu tu re Wor k

50. Continuously Growing
a Continuously Evolving Culture

51. Recently Done
Code Reviews, Second Pair of Eyes
PHPUnit --strict Standards
PHPUnit Testing Standards
Log All the Errors E_ALL | E_STRICT

52. More To Do
Entire Coding Standard in CI
Mess Detector Standards
More PHPUnit Testing Standards
...

53. Developer Culture
“Always Be Pushing”
Developer Happiness
We trust people

54. Company
etsy.com
About Our Craft
codeascraft.com
More From Me
elblinkin.info