Building a Great Web API - Evan Cooke - QCON 2011


Published on

This presentation explores how fast signup, a clear value proposition, efficient quick starts, concise documentation, easy authentication and debugability are common attributes of many successful web APIs. The Twilio API is used as an example of how a focus on developer experience helps drive API adoption.

Published in: Technology

Building a Great Web API - Evan Cooke - QCON 2011

  2. 2. One simple goalfor building great Web APIs...
  3. 3. 5Minutes
  4. 4. It should take no more than 5 minutes for adeveloper to perform a useful task using your API for the first time
  5. 5. What is a Web API?Definition for today:• HTTP/HTTPS endpoint• JSON/XML• Hosted API/Web service• Example: Amazon S3
  6. 6. Twilio Web service APIs to automate Voice and SMS phone communications Carriers Inbound Calls Voice Outbound Calls Mobile/Browser VoIP Send To/From Phone SMS Numbers Short CodesDeveloper Phone Dynamically Buy Numbers Phone NumbersEnd User
  7. 7. Why 5 Minutes?• Clear value proposition• Fast signup• Simple API• Efficient quickstarts• Concise accessible docs• Easy authentication (basic vs. oauth)• Debuggable
  8. 8. API Adoption FunnelHear about API Good APIs Try it Promote Adoption (tell friends) Build it Production Use Buy it Traction
  9. 9. API Adoption FunnelHear about API Try it 5-Minute Goal Build it Production Use Buy it Traction
  10. 10. 5-minute APIs API Design
  11. 11. Co un te rE 5-minute APIs xa API Design m ple
  12. 12. Media processing API• HTTP API to analyze large (MBs) media files• High tps ~100,000/day• POST data to the API You Media API• Analysis API synchronously returns analysis of media (could be minutes/hours later)
  13. 13. Media processing API• Control and data in the same request (100 MB every request)• Unclear error conditions. Can you resend request? You API• Synchronously wait for ??? response• No request history/billing information
  14. 14. Media processing API1 Original Request POST key=2hkh&mode=bob&filter=yeah Body is 100MB of binary data Problem Can we safely retry the request on a transient failure?
  15. 15. Media processing API2 Add a token allowing us to safely retry POST key=2hkh&mode=bob&filter=yeah&token=TK123 Body is 100MB of binary data Problem Exposed to transient failures if the processing takes minutes/hours?
  16. 16. Media processing API3 Add webhook url to asynchronously respond POST key=2hkh&mode=bob&filter=yeah&token=TK123&c Body is 100MB of binary data Problem Request is huge and fragile when data is included with control cmd?
  17. 17. Media processing API4 Async fetch media & move POST params to body POST Body key=2hkh mode=bob filter=yeah token=TK123 cbUrl= mediaUrl=
  18. 18. Media processing API5 Version API and make URL more RESTful POST Body key=2hkh mode=bob filter=yeah token=TK123 cbUrl= mediaUrl= Response URI
  19. 19. 5-minute APIs API Design
  20. 20. Idempotency• Idempotence is the property of certain operations that they can be applied multiple times without changing the result.• Request failures happen. Provide users a safe way to retry requests• Example:POST /BankAccount/Funds{‘value’: 1000, ‘token’: ‘TX123’}
  21. 21. Self-documentation• Sweat what and how you name. Naming is is the best documentation.• Example:GET /Users/ID123GET /Users/ID123/FriendsGET /Users/ID123/Photos
  22. 22. RESTfulness• Adherence to REST object model and verbs provides a clean way to expose business logic• Create POST /Users• Fetch GET /Users/ID123• Modify PUT/POST /Users/ID123• Remove DELETE /Users/ID123
  23. 23. Versioning• Do it. Remember you will be stuck supporting old API versions (indefinitely?)• Examples GET /api/1/blag GET /api/v1/blag GET /api/20101206/blag
  24. 24. Statefulness• When possible, offload the work of keeping state/history because it’s hard!• ExamplesPOST /CallsGET /Calls/CA123GET /Calls/CA123/Recordings/RE123
  25. 25. Predictability• The API should do what users expect• Examples <Play>music.mp3</Play> <Play>music.aiff</Play>
  26. 26. Responsiveness• When a response is available immediately, use a synchronous response. Otherwise, consider an asynchronous webhook rather then polling• Examples:POST /Object/TransformsBody cbUrl=
  27. 27. Key Ideas• Idempotency• Self-documentation• RESTfulness• Versioning• Statefulness• Predictability• Responsiveness
  28. 28. 5-minute APIs “Good”
  29. 29. What makes a good API? • Easy to Learn • Easy to use (even without documentation) • Hard to Misuse • Easy to read and maintain code that uses it • Sufficiently powerful to satisfy requirements • Easy to extend • Appropriate to audience How to Design a Good API and Why it Matters Joshua Bloch, Google
  30. 30. Human-centric
  31. 31. Saleforce auth Campaigns from Adword API
  32. 32. Dynamic phone numberprovisioning via Twilio API
  33. 33. SaleforceCRM data Saleforce app APIs GoogleMaps API Twilio Client API
  34. 34. Simplicity
  36. 36. Infrastructure• 100’s of prod hosts in continuous operation• 80+ service types running in prod - Python(Twisted/GEvent), PHP, Java• 50+ prod database servers - MySQL, Redis• Prod deployments several times/day across 7 engineering teams
  37. 37. Website Deployment Content Frequency(Risk) 4 buckets Website CodeLog Scale 1000x REST API Big DB 100x Schema 10x 1x CMS PHP/Ruby Python/Java SQL etc. etc.
  38. 38. Website DeploymentContent Processes Website Code REST API Big DB Schema One Click CI Tests CI Tests CI Tests One Click Human Sign-off Human Sign-off One Click Human Assisted Click
  39. 39. Cluster automation via boxconfig
  40. 40. Cluster automation via boxconfig• Build and deployment system - boot entire Twilio stack with one key press• Host configuration - versioned code & config• Host orchestration - load balancing• Monitoring and alerting - nagios• Multi-datacenter deployment & analytics
  41. 41. Cluster automation via boxconfig role role role Start Roles Fetch ProvisionS3 SVN/git Boxconfig Base (AMI) Vanilla Linux Host (cloud/colo)
  42. 42. twilio Evan Cooke @emcooke