UPDATE: This slide deck is superseded by http://www.slideshare.net/samsalisbury/mesos-at-opentable-talk-20151111 From talk I did at the London Mesos user group, October 23rd 014.

1. MESOS @
Feedback/questions @samsalisbury

2. WHO AM I?
• Engineer at OpenTable for 1 year
• New to Mesos
• Mostly working in Go
• @samsalisbury on twitter/github

3. Diners Restaurants

4. Many Few

5. ! +Canada
"
+Mexico
# $

7. A HUGE ASP.NET APP
Prior to 2012

8. VM
A HUGE ASP.NET APP
Prior to 2012

9. DATACENTRE
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
Prior to 2012

10. DATACENTRE
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
Prior to 2012

11. DATACENTRE
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
APP
VM
A HUGE
ASP.NET
PriorA tPoP 2012

12. DATACENTRE
“Fortnightly” VM
big VM
bang deployments.
VM
A HUGE
A HUGE
A HUGE
Tech ASP.constrained NET
ASP.NET
ASP.NET
APP
APP
to single APP
platform.
One VM
defect blocks VM
all progress.
VM
A HUGE
A HUGE
A HUGE
Difficult ASP.NET
to experiment ASP.NET
with ASP.NET
APP
APP
APP
features.
VM
A HUGE
ASP.NET
APP
Not much fun.
VM
A HUGE
ASP.NET
APP

13. DATACENTRE
“Fortnightly” VM
VM
release cycle.
VM
A HUGE
A HUGE
A HUGE
ASP.Big NET
bang ASP.deployments.
NET
ASP.NET
APP
APP
APP
One VM
defect blocks VM
all progress.
VM
A HUGE
A HUGE
A HUGE
Difficult ASP.NET
to experiment ASP.NET
with ASP.NET
APP
APP
APP
features.
VM
A HUGE
ASP.NET
APP
Not much fun.
VM
A HUGE
ASP.NET
APP
Early 2012

14. SOA all the things!

15. CODE BASE
A HUGE ASP.NET APP
Prior to 2012

16. CODE BASE
EXTERNAL
API SEARCH REVIEWS
EMAILS WHITE
LABEL
REST.
PROFILES
RESERV-ATION
API
PHOTO
SERVICE REST. A P I
PERSON
API
HOME
PAGE WIDGETS
CONTENT
F RONT SERVICE DOOR SVC
MENU AP I
FEEDBACK
AVAI LAB-I
LEGACY
FORM
L I T Y A P I
API
Now

17. CODE BASE
Release EXTERNAL
whenever API SEARCH a feature REVIEWS
is ready.
Use whatever EMAILS tech WHITE
REST.
LABEL
is most PROFILES
appropriate.
RESERV-ATION
PHOTO
No one API
defect SERVICE blocks all REST. progress.
A P I
PERSON
HOME
Easy API
to experiment PAGE with WIDGETS
features.
CONTENT
F RONT SERVICE DOOR MENU AP I
Much more SVC
fun!
FEEDBACK
AVAI LAB-I
LEGACY
FORM
L I T Y A P I
API
Now

18. CODE BASE
EXTERNAL
API SEARCH REVIEWS
EMAILS WHITE
LABEL
REST.
PROFILES
RESERV-ATION
API
PHOTO
SERVICE REST. A P I
PERSON
API
However.
HOME
PAGE WIDGETS
CONTENT
F RONT SERVICE DOOR SVC
MENU AP I
FEEDBACK
AVAI LAB-I
LEGACY
FORM
L I T Y A P I
API
Now

19. DATACENTRE
VM
EXTERNAL
API
VM
SEARCH
VM
REVIEWS
VM
EMAILS
VM
RESERVAT-ION
API
VM
VM
WHITE
LABEL
VM
REST.
PROFILES
VM
PHOTO
SERVICE
VM
REST. A P I
VM VM

20. DATACENTRE
VM
EXTERNAL
API
VM
SEARCH
VM
REVIEWS
VM
EMAILS
VM
RESERVAT-ION
API
VM
VM
WHITE
LABEL
VM
REST.
PROFILES
VM
PHOTO
SERVICE
VM
REST. A P I
VM VM

21. DATACENTRE
VM
SEARCH
VVMM VVMM
VVMM VM
REVIEWS
VM
EMAILS
VVMM VVMM
RESERVAT-ION
API
VVMM VVMM
WHITE
LABEL
VVMM VVMM
REST.
PROFILES
VVMM VVMM
PHOTO
SERVICE
VM
REST. A P I
VM
EXTERNAL
API

22. DATACENTRE
VM
SEARCH
VVMM VVMM
VVMM VM
REVIEWS
VM
EMAILS
VVMM VVMM
RESERVAT-ION
API
VVMM VVMM
WHITE
LABEL
VVMM VVMM
REST.
PROFILES
VVMM VVMM
PHOTO
SERVICE
VM
REST. A P I
VM
EXTERNAL
API 1,231
servers, and rising
(source: Puppetboard)

23. DATACENTRE
VM
SEARCH
VVMM VVMM
VVMM VM
REVIEWS
VM
EMAILS
VVMM VVMM
RESERVAT-ION
API
VVMM VVMM
WHITE
LABEL
VVMM VVMM
REST.
PROFILES
VVMM VVMM
PHOTO
SERVICE
VM
REST. A P I
VM
EXTERNAL
API 1,231
servers, and rising
(source: Puppetboard)
This is expensive.

24. DATACENTRE
VM
SEARCH
VVMM VVMM
VVMM VM
REVIEWS
VM
EMAILS
VVMM VVMM
RESERVAT-ION
API
VVMM VVMM
WHITE
LABEL
VVMM VVMM
REST.
PROFILES
VVMM VVMM
PHOTO
SERVICE
VM
REST. A P I
VM
EXTERNAL
API 1,231
servers, and rising
(source: Puppetboard)
This is expensive.
And hard to manage.

25. DATACENTRE
VM
SEARCH
VVMM VVMM
VVMM VM
REVIEWS
VM
EMAILS
VVMM VVMM
RESERVAT-ION
API
VVMM VVMM
WHITE
LABEL
VVMM VVMM
REST.
PROFILES
VVMM VVMM
PHOTO
SERVICE
VM
REST. A P I
VM
EXTERNAL
API
WDWDN?

26. Puppetise all the things! everything.
Mid 2013

27. Puppetise everything.
Mid 2013

28. Puppetise everything.
This helped a lot.
Mid 2013

29. Puppetise everything.
This helped a lot.
However…
Mid 2013

30. Puppetise everything.
Infrastructure still very complex.
And expensive.

31. Dn dn dnnnnnn.

32. Early 2014

33. AWS
US Prod EU Prod QA
Now

34. MARATHON
CHRONOS
SPARK
SINGULARITY

35. MARATHON
DISCOVERY OAUTH
SERVER
RESTAURANT
REPORTING
AVAI LABI L ITY
COMPONENTS NEW BLOG MENU API
…
https://github.com/mesosphere/marathon

36. CHRONOS
UPDATING
PERSISTENT CACHES
COORDINATING
LARGE
INTERDEPENDENT
DATA PROCESSING
JOBS
https://github.com/airbnb/chronos

38. SPARK
DATA SCIENCE MAGIC
http://spark.apache.org

39. SINGULARITY
EVERYTHING?
https://github.com/HubSpot/Singularity

40. Mesos seems really cool so far.

41. Of course, now we have a new set of problems.

42. Where is my app running?!

43. Where is my app running?!
There are 2 basic approaches to this problem.

44. Service discovery.

45. DISCOVERY
SERVER
(USES ETCD)
APP 1
APP 2
APP 3
Announce
Announce
Announce
State
State
State
Data
Data
Data

46. Fixed addresses?

47. Enter Mesoshub…
https://github.com/opentable/mesoshub

48. Enter Mesoshub…
A web UI over HAProxy.
Integrated with Marathon.
https://github.com/opentable/mesoshub

49. MESOSHUB
(USES
HAPROXY)
APP 1
APP 2
APP 3
Data
Data
Data
http://app1.ot.com
http://app2.ot.com
http://app3.ot.com
Data
Data
Data

50. Enter Mesoshub…
Mesoshub is open source.
We would love contributions!
https://github.com/opentable/mesoshub

51. Use both together for maximum flexibility!

52. The future…

53. + = ?
Configuring Mesos clusters with Puppet?

54. Deployment?
Orchestration?
Stateful apps?

55. That’s all, folks!