‹#›
Sylvain Wallez - @bluxte
DevFest Toulouse, 2017-09-28
From50to270:Scalingadistributed
engineeringteam
Who'sthisguy?
Software Engineer - Elastic Cloud
Previously tech lead, CTO, architect, trainer...

...at OVH, Actoboard, Sigfox, Scoop.it, Joost, Anyware
Member of the Apache Software Foundation since 2003
Whoarewe?
Kibana
Elasticsearch
Beats Logstash
Security
Alerting
Monitoring
Reporting
X-Pack
Graph
Elastic Cloud
Whoarewe?
Whoarewe?
600elasticians
Whoarewe?
600elasticians
32countries
Whoarewe?
600elasticians
270engineers
32countries
Whoarewe?
600elasticians
270engineers
32countries
100+milliondownloads
Thefounders:2012
Shay Uri Simon Steven
Nextfewhires:
Nextfewhires:CFO??
6
DistributedBy
Design
‹#›
Adistributedsystemisasoftwaresystemin
whichcomponentslocatedonnetwork
computerscommunicateandcoordinatetheir
actionsbypassingmessages
Elasticsearchthesoftware,
wasdistributedfromday1
Whatdoesadistributedsoftware
anddistributed,opensource
companyhaveincommon?
Lots
‹#›
Distributedsystems:
communication
11
Asynchronous
Communications
Email:Filtersincluded
Timezone friendly
Email:Filtersincluded
How we reached a decision
Emailtransparency
15
Someemailbecomehistory
16
Reply-allisfun!
17
Emailsbecomet-shirts
18
Loveisintheair
18
Loveisintheair
18
Loveisintheair
GitHuballthethings
20
@mention someone
directly on
GH issue instead of
discussing
in Slack
21
GitHubfornon-techstuff
Synchronous
Communications
23
Join any room
Technical discussions in public
Public communication will educate everyone
Post a summary to email or GitHub
24
25
Meetings
25
Dev Standups
25
Meetings
25
25
Dev Standups
Collapses
25
Meetings
25
25 50
Demo DayDev Standups
Collapses
25
Meetings
25
25
Table Group
50 70
Demo DayDev Standups
Collapses
25
Meetings
25
25
Table Group
50 70
Demo DayDev Standups
Collapses Collapses
7025
Meetings
25
25
Table Group Black Belt
Session
50 70 140
Demo DayDev Standups
Collapses Collapses
7025
Meetings
25
25
Table Group Black Belt
Session
Cross Product/

Per Product
Sync
50 70 140 200
Demo DayDev Standups
Collapses Collapses
7025
Meetings
26
Meetingsareopen:devcal
27
Zoomvideoconferencing
28
Persistentzoomsession
AlwaysOnZoom:hopon,hopoff
29
Recordmeetingsforposterity
It'snotjustvirtual
Engineeringall-hands:2xayear
Engineeringall-hands:2timesayear
Vision, long term plans
Talks from CEO, VPs and investors
2 days team time
1 day cross product
1 day fun activities
Engineeringall-hands:meetingswithaview!
Engineeringall-hands:funday!
35
dev process
Agile, Waterfall
36
Consensus based
‹#›
GitHubtotrackthings
‹#›
FunwithGitHublabels
39
Jira-ifyingGitHubwithWaffle.io
40
PullRequests:PeerReview
41
LGTMs == ACKs
Releases
43
Before:Independentreleases
Timebasedreleases
45
ReleaseProcess:Timebased
x.y.z —> major.minor.patches
Release:Trainanalogy
ReleaseProcess:Timebased
Merge when ready and scalable
Otherwise catch the next train
No pressure on devs for pushing features
48
Teamdynamic
How’stheorgstructure
Flat org
Self organising teams
Highly available
50
Hirethebesttalentfromanywhere
Globallydistributed
Anodehasmanyroles
Engineers arenodes
Engineers areevangelists
Engineers writeblogs
Engineers dosupport
Engineers writedocumentation
Engineers haveautonomy
54
node
discovery
node
looking for other
nodes in elastic
55
Hiring
“how many golf balls fit in an airplane ?”
Hiring
“how many golf balls fit in an airplane ?”
Hiring
No puzzle questions
Each team has different interview styles
Technical and team fit for distributed
environment
Nodesareindependent
59
0
10
20
30
40
50
0-3 3-6 9-12 12-15 15-18 18+
Averageexperience
OnBoarding:Mentoring
Day 1
High touch mentoring
Mentor != Manager
OnBoarding:X-School
Week-long orientation in Mountain View, CA
Meet team leads, product team
Big Picture: Understand business functions
62
Takeaways
Keytakeaways
Transparency & openness
Keytakeaways
Trust by default
Keytakeaways
Don’t build processes around mistakes
Keytakeaways
Progress over perfection
Inclosing…
They say it won’t work at 50, 100, 500..
Questions?
Moreresources
Initially presented as OSCon, Austin: https://conferences.oreilly.com/oscon/oscon-tx/public/schedule/detail/56992
https://www.elastic.co/elasticon/conf/2016/sf/inside-elastic-engineering
https://www.elastic.co/elasticon/conf/2017/sf/on-distributed-systems-and-distributed-teams
https://zachholman.com/posts/how-github-works/
Fonts and slide design were inspired by Zach Holman: https://zachholman.com/posts/slide-design-for-developers/

Elastic - From 50 to 270, how to scale a distributed engineering team