JUST EAT: embracing DevOps
Or: How we make a Windows-based ecommerce platform work
(with AWS)
@petemounce & @justeat_tech
JUST EAT: Who are we?
● In business since 2001 in DK, 2005 in UK
● Tech team is ~50 people in UK, ~20 people in Ukraine
● ...
JUST EAT: Who are we?
Oh, yeah - we do online takeaway.
We’re an extra sales channel for our restaurant partners.
We do th...
What are we?
We do high-volume ecommerce.
Windows platform.
Most production code is C#, .NET 4 or 4.5.
Most automation is ...
Tech culture
“You ship it, you operate it”
Each team owns their own features, infrastructure-up.
Minimise dependencies bet...
But how?
Table-stakes for this to work (well):
1. Persistent group chat
2. Real-time monitoring
3. Real-time alerting
4. C...
Persistent group chat
We use HipChat.
You could use IRC / Campfire / Hangouts.
● Persistent - jump in, read up
● Searchabl...
Real-time monitoring
Microsoft’s SCOM requires an AD
Publish OS-level performance counters with perftap - windows
analogue...
Real-time alerting
This is the 21st century; emailing someone their server is down
doesn’t cut it.
seyren runs our checks....
Centralised logging
Windows doesn’t have syslog.
Out of the box EventLog isn’t quite it.
Publish logs via nxlog agent.
Rec...
Without these things, operating a distributed system on
Windows is hard.
Windows at scale assumes that you have an Active ...
The most important things
● Culture
● Principles that everyone lives by
● Devolve autonomy down to people on the ground
● ...
Did we mention we’re hiring?
We’re pragmatic.
We’re successful.
We support each other.
We use sharp tools that we pick our...
Any questions?
Upcoming SlideShare
Loading in...5
×

JUST EAT: Tools we use to enable our culture

518

Published on

DevOps is not about tools, but good ones definitely make facilitate the ability to run fast.

Published in: Engineering, Technology
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
518
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

JUST EAT: Tools we use to enable our culture

  1. 1. JUST EAT: embracing DevOps Or: How we make a Windows-based ecommerce platform work (with AWS) @petemounce & @justeat_tech
  2. 2. JUST EAT: Who are we? ● In business since 2001 in DK, 2005 in UK ● Tech team is ~50 people in UK, ~20 people in Ukraine ● Cloud native in AWS ○ Except for the bits that aren’t (yet) ● Very predictable load ● ~1000 orders/minute at peak in UK ● We’re recruiting! ○ http://tech.just-eat.com/jobs/ ○ http://tech.just-eat.com/jobs/senior-software-engineer- platform-services/ ○ Lots of other roles
  3. 3. JUST EAT: Who are we? Oh, yeah - we do online takeaway. We’re an extra sales channel for our restaurant partners. We do the online part. Challenging! We make this work.
  4. 4. What are we? We do high-volume ecommerce. Windows platform. Most production code is C#, .NET 4 or 4.5. Most automation is ruby 1.9.x. Some powershell. Ongoing legacy transformation; no big rewrites. Splitting up a monolithic system into SOA/APIs, incrementally.
  5. 5. Tech culture “You ship it, you operate it” Each team owns their own features, infrastructure-up. Minimise dependencies between teams. Each team has autonomy to work on what they want within some constraints. Rules: ● don’t break backwards compatibility ● use what you want - but operate it yourself ● other teams must be able to launch & verify your stuff in their environments
  6. 6. But how? Table-stakes for this to work (well): 1. Persistent group chat 2. Real-time monitoring 3. Real-time alerting 4. Centralised logging Make it easier to debug in production without a debugger.
  7. 7. Persistent group chat We use HipChat. You could use IRC / Campfire / Hangouts. ● Persistent - jump in, read up ● Searchable history ● Integrate other tools to it ● hubot for fun and profit ○ @jebot trg pd emergency with msg “we’re out of champagne in the office fridge”
  8. 8. Real-time monitoring Microsoft’s SCOM requires an AD Publish OS-level performance counters with perftap - windows analogue of collectd we found and customised Receive metrics into statsd Visualise time-series data with graphite ○ 10s granularity retained for 13 months ○ AWS’ CloudWatch gives you 1min / 2 weeks Addictive!
  9. 9. Real-time alerting This is the 21st century; emailing someone their server is down doesn’t cut it. seyren runs our checks. Publishes to ● HipChat ● PagerDuty ● SMS ● statsd event metrics (coming soon, hopefully)
  10. 10. Centralised logging Windows doesn’t have syslog. Out of the box EventLog isn’t quite it. Publish logs via nxlog agent. Receive logs into logstash cluster. Filter, transform and enrich into elasticsearch cluster. Query, visualise and dashboard via kibana.
  11. 11. Without these things, operating a distributed system on Windows is hard. Windows at scale assumes that you have an Active Directory. We don’t. ● No Windows network load-balancing. ● No centrally trusted authentication. ● No central monitoring (SCOM) to harvest performance counters. ● No easy remote command execution (WinRM wants an AD, too) ● Other stuff; these are the highlights.
  12. 12. The most important things ● Culture ● Principles that everyone lives by ● Devolve autonomy down to people on the ground ● (Tools)
  13. 13. Did we mention we’re hiring? We’re pragmatic. We’re successful. We support each other. We use sharp tools that we pick ourselves based on merit. Join us! ○ http://tech.just-eat.com/jobs/ ○ http://tech.just-eat.com/jobs/senior-software-engineer- platform-services/ ○ Lots of other roles
  14. 14. Any questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×