Mesos + Singularity:Mesos + Singularity:
PaaS automation for mortalsPaaS automation for mortals
Gregory Chomatas @gchomata...
120 meters: My shortest travel to a Conference120 meters: My shortest travel to a Conference
Thales of Miletus - 624 BC
Those who can, do, the others philosophise...
Really?
Miletus
Optionality* with MesosOptionality* with Mesos
Invest in a Mesos-powered PaaS and keep
doing what you love most; building ...
Blogging
SEO
Social Media
CMS
Lead Management
Landing Pages
Calls-to-Action
Marketing Automation
Email
Analytics
CRM Sync
The underlying culture & structureThe underlying culture & structure
12-factor apps
.net monolith to microservices
Small, ...
High productivityHigh productivity
~100 engineers
800+ components that can be
updated/scaled independently
QA: ~400 small ...
1. Develop locally
2. Provision QA aws instance
3. Deploy via local Python script
4. Provision PROD aws instance
5. Deploy...
Statically partitioning the cluster is inefficientStatically partitioning the cluster is inefficient
The cost of flexibility & asynchronicityThe cost of flexibility & asynchronicity
High operational overhead
Poor utilisatio...
Redress the balance with a Mesos-based PaaSRedress the balance with a Mesos-based PaaS
Abstract away machines
Homogenous e...
To Boldly go...to Singularity
Singularity: do more with a single schedulerSingularity: do more with a single scheduler
Great UI & HTTP API
Native Docker...
Singularity: do even more...Singularity: do even more...
Security / artifact signature
verification
Agent & Rack maintenanc...
The PaaS StackThe PaaS Stack
BUILD DEPLOY RUN
Jenkins Orion Singularity
The Build / Deploy cycleThe Build / Deploy cycle
buildpack runner
S3
The Deployer - Dry runThe Deployer - Dry run
The Deployer - DeployingThe Deployer - Deploying
SingularitySingularity
Deployable viewDeployable view
SingularitySingularity Task viewTask view
Singularity fSingularity file tailingile tailing
SingularitySingularity Health Checks & ResourcesHealth Checks & Resources
SingularitySingularity All Deployables viewAll Deployables view
SingularitySingularity
Cluster StatusCluster Status
SingularitySingularity
Cluster Maintenance viewCluster Maintenance view
Migration to Mesos - TimelineMigration to Mesos - Timeline
Manual Server ProvisioningManual Server Provisioning
Server provisioning UI usageServer provisioning UI usage
1800+ deployables1800+ deployables
~300 deploys / day~300 deploys / day
10 minutes from git10 minutes from git
push to pro...
I want persistent IMAP connections to 200k+I want persistent IMAP connections to 200k+
inboxes (Jul 2015)inboxes (Jul 2015)
Stateful Services
Single Process services
Hard coded stationary hosts
Cgroups memory isolation
User resistance
Migration I...
All eggs in one basket
Mesos / Framework issues (pingback
port)
Failures (Zookeeper, Mesos, Singularity)
Cluster Maintenan...
Phased rollout of new Kernel, Instance
types
Rolling upgrade of instance basic
SW with puppet vars
MaintenanceMaintenance
...
Optionality with Mesos@HubSpotOptionality with Mesos@HubSpot
Singularity
Ghidorah - Load Balancers in Mesos
Massive Builds...
The sweet spotThe sweet spot
Source: Mark Leslie (http://firstround.com/review/The-Arc-of-Company-Life-and-How-to-Prolong-I...
Invest early in a deploy & build infrastructureInvest early in a deploy & build infrastructure
Dedicate 1-2 engineers to e...
HubSpot Blog: How We Built Our Stack For Shipping at Scale​
Blazar: An out-of-this world build system!
Baragon: Load Balan...
Mesos + Singularity: PaaS automation & Sustainable Development Velocity for mortals
Upcoming SlideShare
Loading in …5
×

Mesos + Singularity: PaaS automation & Sustainable Development Velocity for mortals

588 views

Published on

Singularity is an open source, battle-tested framework for Mesos Clusters. It combines distributed job scheduling and lifecycle automation of resilient microservices. It is ideal for building PaaS infrastructures that can help high growth companies to sustain development velocity by significantly reducing the operational complexity and deployment friction. In this presentation, we share our experiences in developing the HubSpot PaaS / Deploy pipeline and in moving all our QA and production environment to Mesos. Our key goals are: a) to demonstrate that Mesos clusters provide a mature, safe and proven solution for deploying highly available, complex software architectures comprised of thousands of components, b) to prove that Mesos clusters are not only for the big boys and invite medium size companies to follow HubSpot example and start reaping the benefits of Mesos.

Published in: Software
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
588
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
17
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Mesos + Singularity: PaaS automation & Sustainable Development Velocity for mortals

  1. 1. Mesos + Singularity:Mesos + Singularity: PaaS automation for mortalsPaaS automation for mortals Gregory Chomatas @gchomatas PaaS team
  2. 2. 120 meters: My shortest travel to a Conference120 meters: My shortest travel to a Conference
  3. 3. Thales of Miletus - 624 BC Those who can, do, the others philosophise... Really? Miletus
  4. 4. Optionality* with MesosOptionality* with Mesos Invest in a Mesos-powered PaaS and keep doing what you love most; building your product * Optionality is the property of asymmetric upside (preferably unlimited) w ith corresponding limited downside (preferably tiny)
  5. 5. Blogging SEO Social Media CMS Lead Management Landing Pages Calls-to-Action Marketing Automation Email Analytics CRM Sync
  6. 6. The underlying culture & structureThe underlying culture & structure 12-factor apps .net monolith to microservices Small, autonomous teams with end-to-end ownership - no ops
  7. 7. High productivityHigh productivity ~100 engineers 800+ components that can be updated/scaled independently QA: ~400 small to medium AWS machines PROD: ~750 medium to large AWS machines Source: Martin Fowler http://martinfowler.com/bliki/MicroservicePremium.html
  8. 8. 1. Develop locally 2. Provision QA aws instance 3. Deploy via local Python script 4. Provision PROD aws instance 5. Deploy via local Python script 6. Repeat 4 & 5 to scale … 10. Repeat 4&5 at 4am to replace hw But for how long...But for how long...
  9. 9. Statically partitioning the cluster is inefficientStatically partitioning the cluster is inefficient
  10. 10. The cost of flexibility & asynchronicityThe cost of flexibility & asynchronicity High operational overhead Poor utilisation & elasticity Higher rate of failures
  11. 11. Redress the balance with a Mesos-based PaaSRedress the balance with a Mesos-based PaaS Abstract away machines Homogenous environment Scale out in seconds Centralized deployables registry Sept 2013: Our First Mesos Cluster
  12. 12. To Boldly go...to Singularity
  13. 13. Singularity: do more with a single schedulerSingularity: do more with a single scheduler Great UI & HTTP API Native Docker Support Health Checks Load Balancing API Log Maintenance Oct 2013: Start building Singularity
  14. 14. Singularity: do even more...Singularity: do even more... Security / artifact signature verification Agent & Rack maintenance Webhooks Auto-rollback Email Notifications Executor cleanup Singularity Components
  15. 15. The PaaS StackThe PaaS Stack BUILD DEPLOY RUN Jenkins Orion Singularity
  16. 16. The Build / Deploy cycleThe Build / Deploy cycle buildpack runner S3
  17. 17. The Deployer - Dry runThe Deployer - Dry run
  18. 18. The Deployer - DeployingThe Deployer - Deploying
  19. 19. SingularitySingularity Deployable viewDeployable view
  20. 20. SingularitySingularity Task viewTask view
  21. 21. Singularity fSingularity file tailingile tailing
  22. 22. SingularitySingularity Health Checks & ResourcesHealth Checks & Resources
  23. 23. SingularitySingularity All Deployables viewAll Deployables view
  24. 24. SingularitySingularity Cluster StatusCluster Status
  25. 25. SingularitySingularity Cluster Maintenance viewCluster Maintenance view
  26. 26. Migration to Mesos - TimelineMigration to Mesos - Timeline
  27. 27. Manual Server ProvisioningManual Server Provisioning
  28. 28. Server provisioning UI usageServer provisioning UI usage
  29. 29. 1800+ deployables1800+ deployables ~300 deploys / day~300 deploys / day 10 minutes from git10 minutes from git push to productionpush to production
  30. 30. I want persistent IMAP connections to 200k+I want persistent IMAP connections to 200k+ inboxes (Jul 2015)inboxes (Jul 2015)
  31. 31. Stateful Services Single Process services Hard coded stationary hosts Cgroups memory isolation User resistance Migration IssuesMigration Issues
  32. 32. All eggs in one basket Mesos / Framework issues (pingback port) Failures (Zookeeper, Mesos, Singularity) Cluster Maintenance Missing features Operational IssuesOperational Issues
  33. 33. Phased rollout of new Kernel, Instance types Rolling upgrade of instance basic SW with puppet vars MaintenanceMaintenance Rolling upgrade of master/agent process with ansible Local testing on docker cluster Roll out at infra-QA then product-QA and last to Production cluster Deploy tools deploy themselves but maintain command line alternative with fabric
  34. 34. Optionality with Mesos@HubSpotOptionality with Mesos@HubSpot Singularity Ghidorah - Load Balancers in Mesos Massive Builds in Mesos Baragon - Tasks Load Balancer Manager Mesos Spark Cluster
  35. 35. The sweet spotThe sweet spot Source: Mark Leslie (http://firstround.com/review/The-Arc-of-Company-Life-and-How-to-Prolong-It/)
  36. 36. Invest early in a deploy & build infrastructureInvest early in a deploy & build infrastructure Dedicate 1-2 engineers to experiment onDedicate 1-2 engineers to experiment on a Mesos powered PaaSa Mesos powered PaaS Try Singularity today!Try Singularity today! github.com/HubSpot/Singularity
  37. 37. HubSpot Blog: How We Built Our Stack For Shipping at Scale​ Blazar: An out-of-this world build system! Baragon: Load Balancer API Useful linksUseful links

×