Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mesos HTTP API

751 views

Published on

This presentation was part of the talk "Mesos HTTP API" by Vinod Kone (Technical Lead for Mesos at Twitter) and Isabel Jimenez (Software Engineer at Mesosphere) at MesosCon 2015

Abstract
======
The previous Mesos API provided language bindings (a.k.a. drivers) for frameworks to interact with Mesos. Framework developers don’t like dealing with dependencies in native libraries, and with Mesos’ new HTTP API, they no longer have to! In this session, (Vinod Kone, Technical Lead for Mesos at Twitter and Isabel Jimenez, Software Engineer at Mesosphere) will provide a comprehensive walk-through of recent advancements with the Mesos API, explaining the design rationale and highlighting specific improvements that simplify writing frameworks to Mesos. Attendees will see real examples of interacting with the new Mesos API, and get an update on frameworks that have already implemented the HTTP API. Attendees will also learn about two types of endpoints supported by HTTP API -- /events and /call -- and best practices for using both.

Try Mesosphere for Free: https://mesosphere.com/try

Published in: Technology
  • Be the first to comment

Mesos HTTP API

  1. 1. M E S O S H T T P A P I @ v i n o d k o n e @ i j i m e n e
  2. 2. Mesos 1.0 IS COMING
  3. 3. M E S O S A P I S S c h e d u l e r M a s t e r S l a v e E x e c u t o r Scheduler API Executor API Internal API Operator API
  4. 4. D E P E N D E N C E O N N A T I V E L I B R A RY S c h e d u l e r M a s t e rJ a v a L i b r a r y N a t i v e L i b r a r y mesos.jar libmesos.so Hard to debug Not portable
  5. 5. N O N - S TA N D A R D F R A M E W O R K A P I POST /master/mesos.internal.LaunchTasksMessage HTTP/1.1 User-Agent: libprocess/scheduler-1234-23-23342@127.0.0.1:8081 Libprocess-From: scheduler-1234-23-23342@127.0.0.1:8081 Connection: Keep-Alive Host: Transfer-Encoding: chunked
  6. 6. N E T W O R K I N G C O N S T R A I N T S S c h e d u l e r M a s t e r Firewall Containers :( Network partitions :(
  7. 7. L A C K O F A P I V E R S I O N I N G GET /metrics/snapshot Version GET /state.json “version” : 0.23.0 JSON
  8. 8. P I TA F O R M E S O S D E V S Lot of boiler plate to add new calls/events Forced upgrade dependencies
  9. 9. – H A R V E Y D E N T “You either die a hero… or live long enough to see yourself become the villain” replaced by a better API
  10. 10. 1 . 0 G O A L S Consistent APIs Versioning
  11. 11. N E W H T T P A P I • Standard HTTP 1.1 • Versioned ! • Well documented
  12. 12. N E W M E S O S A P I S Endpoint API Hosted by /api/v1/scheduler Scheduler API Master /api/v1/executor Executor API Slave /api/v1/internal Internal API Master /api/v1/admin Operator API Master / Slave
  13. 13. S C H E D U L E R H T T P A P I • Based on Calls and Events • Scheduler opens connections to the master • A persistent connection to receive events • One (or more) connection(s) to send calls
  14. 14. R A T I O N A L E Simplicity Off-the-shelf HTTP client libraries No native dependencies
  15. 15. R A T I O N A L E Upgradability Familiar to existing APIs killTask() Call.Kill
  16. 16. R A T I O N A L E Extensibility Easy to add support for new features
  17. 17. C A L L S Old API S U B S C R I B E start() T E A R D O W N stop() A C C E P T acceptOffers() D E C L I N E declineOffer() R E V I V E reviveOffers() K I L L killTask() S H U T D O W N * Shutdown executor * A C K N O W L E D G E acknowledgeStatusUpdate() R E C O N C I L E reconcileTasks() M E S S A G E sendFrameworkMessage() R E Q U E S T requestResources()
  18. 18. E V E N T S Old API S U B S C R I B E D registered() / reregistered() O F F E R S resourceOffers() R E S C I N D offerRescinded() U P D AT E statusUpdate() M E S S A G E frameworkMessage() FA I L U R E slaveLost() / executorLost() E R R O R error() H E A R T B E AT * Periodic heartbeats *
  19. 19. P R O T O C O L • Every call is a HTTP POST request • application/json or application/x-protobuf • SUBSCRIBE call results in a “200 OK” streaming response • Record-IO formatted events • chunked encoding • All successful non-SUBSCRIBE calls result in “202 Accepted”
  20. 20. S U B S C R I P T I O N R E Q U E S T POST /api/v1/scheduler HTTP/1.1 Host: masterhost:5050 Content-Type: application/json Accept: application/json Connection: close { “type” : “SUBSCRIBE”, “subscribe” : { “framework_info” : { “user” : “foo”, “name” : “Example HTTP Framework” }, “force” : true } }
  21. 21. S U B S C R I P T I O N R E S P O N S E HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked <Event Length> { “type” : “SUBSCRIBED”, “subscribed” : { “framework_id” : {“value”:“12220-3440-12532-2345”}, “heartbeat_interval_seconds” : 15 } } <more events>
  22. 22. K I L L POST /api/v1/scheduler HTTP/1.1 Host: masterhost:5050 Content-Type: application/son Accept: application/json Connection: close { “framework_id” : {“value” : “12220-3440-12532-2345”}, “type” : “KILL”, “kill” : { “task_id” : {“value” : “12220-3440-12532-my-task”}, “slave_id” : {“value” : “12220-3440-12532-S123345”} } } Response: HTTP/1.1 202 Accepted
  23. 23. D I S C O N N E C T I O N S & PA R T I T I O N S • Master tracks the persistent subscription connection • Reconnect within failover timeout • Subscribe.force : Failover • Periodic HEARTBEATs sent by master
  24. 24. V E R S I O N I N G Explicit Simple Avoid version explosion /api/v1/scheduler /api/v1/executor /api/v1/admin /api/v1/internal
  25. 25. A P I V E R S I O N V S R E L E A S E V E R S I O N • API version == Major release version • v1 API supported by 1.0.0, 1.4.0, 1.20.0 • vN API released in N-1 release version • vN API considered stable in the last N-1 release • Version bumping • Major/API version bumped for backwards incompatible changes (> yearly) • Minor version bumped regularly (4-8 weeks)
  26. 26. R O A D T O 1 . 0 v1 scheduler API v1 executor API 0.24.0 0.25.0 v1 operator API 1.0 v1 internal API
  27. 27. D E M O
  28. 28. R A T I O N A L E Simplicity
  29. 29. R A T I O N A L E Simplicity Upgradability
  30. 30. R A T I O N A L E Simplicity Upgradability Extensibility
  31. 31. A C K N O W L E D G E M E N T S
  32. 32. T H A N K Y O U MESOS-2288

×