DevOps is not about tools, but good ones definitely make facilitate the ability to run fast.

1. JUST EAT: embracing DevOps
Or: How we make a Windows-based ecommerce platform work
(with AWS)
@petemounce & @justeat_tech

2. JUST EAT: Who are we?
● In business since 2001 in DK, 2005 in UK
● Tech team is ~50 people in UK, ~20 people in Ukraine
● Cloud native in AWS
○ Except for the bits that aren’t (yet)
● Very predictable load
● ~1000 orders/minute at peak in UK
● We’re recruiting!
○ http://tech.just-eat.com/jobs/
○ http://tech.just-eat.com/jobs/senior-software-engineer-
platform-services/
○ Lots of other roles

3. JUST EAT: Who are we?
Oh, yeah - we do online takeaway.
We’re an extra sales channel for our restaurant partners.
We do the online part.
Challenging!
We make this work.

4. What are we?
We do high-volume ecommerce.
Windows platform.
Most production code is C#, .NET 4 or 4.5.
Most automation is ruby 1.9.x. Some powershell.
Ongoing legacy transformation; no big rewrites.
Splitting up a monolithic system into SOA/APIs, incrementally.

5. Tech culture
“You ship it, you operate it”
Each team owns their own features, infrastructure-up.
Minimise dependencies between teams.
Each team has autonomy to work on what they want within
some constraints.
Rules:
● don’t break backwards compatibility
● use what you want - but operate it yourself
● other teams must be able to launch & verify your stuff in
their environments

6. But how?
Table-stakes for this to work (well):
1. Persistent group chat
2. Real-time monitoring
3. Real-time alerting
4. Centralised logging
Make it easier to debug in production without a debugger.

7. Persistent group chat
We use HipChat.
You could use IRC / Campfire / Hangouts.
● Persistent - jump in, read up
● Searchable history
● Integrate other tools to it
● hubot for fun and profit
○ @jebot trg pd emergency with msg “we’re out of champagne in the
office fridge”

8. Real-time monitoring
Microsoft’s SCOM requires an AD
Publish OS-level performance counters with perftap - windows
analogue of collectd we found and customised
Receive metrics into statsd
Visualise time-series data with graphite
○ 10s granularity retained for 13 months
○ AWS’ CloudWatch gives you 1min / 2 weeks
Addictive!

9. Real-time alerting
This is the 21st century; emailing someone their server is down
doesn’t cut it.
seyren runs our checks.
Publishes to
● HipChat
● PagerDuty
● SMS
● statsd event metrics (coming soon, hopefully)

10. Centralised logging
Windows doesn’t have syslog.
Out of the box EventLog isn’t quite it.
Publish logs via nxlog agent.
Receive logs into logstash cluster.
Filter, transform and enrich into elasticsearch cluster.
Query, visualise and dashboard via kibana.

11. Without these things, operating a distributed system on
Windows is hard.
Windows at scale assumes that you have an Active Directory.
We don’t.
● No Windows network load-balancing.
● No centrally trusted authentication.
● No central monitoring (SCOM) to harvest performance
counters.
● No easy remote command execution (WinRM wants an AD,
too)
● Other stuff; these are the highlights.

12. The most important things
● Culture
● Principles that everyone lives by
● Devolve autonomy down to people on the ground
● (Tools)

13. Did we mention we’re hiring?
We’re pragmatic.
We’re successful.
We support each other.
We use sharp tools that we pick ourselves based on merit.
Join us!
○ http://tech.just-eat.com/jobs/
○ http://tech.just-eat.com/jobs/senior-software-engineer-
platform-services/
○ Lots of other roles

14. Any questions?