Building Consistent RESTful APIs in a high-performance environment

Building Consistent RESTful APIs in a High- Performance Environment Yegor Borovikov, Software Architect Brandon Duncan, Director of Engineering LinkedIn Corporation http://blog.linkedin.com/

Topics We’ll Cover > Examples of RESTful APIs  What’s missing?  Variety versus Uniformity > Domain Model as Foundation  Provides Uniformity  Allows Flexibility > Examples of LinkedIn APIs > Using Incentives to Scale  Some LinkedIn production APIs metrics > Q&A 2

Examples of RESTful APIs What if you want to get a person’s profile? > Use one of these… http://social.yahooapis.com/v1/user/{guid}/profile http://api.linkedin.com/v1/people/{guid} http://www.orkut.com/social/rest/people/{guid}/@self <?xml version="1.0" encoding="UTF-8"?> <person> <id>111222</id> <first-name>Gertrude</first-name> <last-name>Stein</last-name> <headline>Author of Tender Buttons</headline> <connections total="76"> … </person> 3

Examples of RESTful APIs What’s Missing? > Ability to get exactly what you need (variety)  If you need more, may require multiple API calls (if they exist)  If you need less, resources are wasted > Consistency of responses (uniformity)  “Same” object returned by different APIs may have different structure  Once in production, hard to get consistent later 4

Examples of RESTful APIs Multiple Calls to Get What You Need > Want to get user’s friend’s profile? Do this…  http://social.yahooapis.com/v1/user/123/connections <connections yahoo:start="0" yahoo:count="1" yahoo:total="1"> <connection yahoo:uri="http://social.yahooapis.com/v1/user/123/connection/456? view=usercard"> <guid>456</guid> <contactId>4</contactId> </connection> </connections> 5

Examples of RESTful APIs Multiple Calls to Get What You Need > … then make second call to get friend’s profile: http://social.yahooapis.com/v1/user/456/profile <profile yahoo:uri="http://social.yahooapis.com/v1/user/456/profile"> <guid>456</guid> <birthdate>3/3</birthdate> <created>2008-08-4T17:13:56Z</created> ... </profile>  Latent, redundant data  Optimization requires stickiness 6

Typical Solution Variety versus Uniformity > Solution: introduce another call > Desire for variety of responses undermines uniformity of requests > Leads to RPC-like REST APIs > Many APIs + Great Documentation = Lots of Reading + Lack of Automation 7

Domain Model as Foundation Sample Domain Model /people : Person[] // collection of Person resources /id : string // primary key /name : string /email : string // unique key /photo : url /best-friend : Person /friends : Person[] /jobs : Job[] // collection of Job resources /company : Company /title : string /start-date : date /end-date : date … /companies : Company[] /name : string /ceo : Person … 8

Domain Model as Foundation Follow request URL to navigate through your model > To get a person’s profile: http://api.linkedin.com/v2/people/123 /people[/id=123] <person uri=“urn:linkedin:v2:people/123” key=“123”> /id <id>123</id> /name <name>Reid Hoffman</name> /email /photo <email>reid@linkedin.com</email> /best-friend <best-friend uri=“urn:linkedin:v2:people/456”/> /friends … /jobs /company </person> /title /start-date /end-date …  Conventional URL in request /companies /name  Default representation in response /ceo … 9

Domain Model as Foundation Fine-grained Request > What if you only need certain fields (e.g., name and photo)? http://api.linkedin.com/v2/people/123:(name,photo) <person> <name>Reid Hoffman</name> /people[/id=123] <photo>http://media.linkedin.com/photos/123.jpeg</photo> /id </person> /name /email /photo /best-friend /friends /jobs /company /title /start-date /end-date … 10

Domain Model as Foundation Fine-grained Request > To get names and photos of one’s friends and their best friends: …/v2/people/456/friends:(name,photo,best-friend: (name,photo)) /people[/id=456] /id <friends total=“66” start=“0”> /name <friend uri=“urn:linkedin:v2:people/123” key=“123”> /email /photo <name>Reid Hoffman</name> /best-friend <photo>http://media.linkedin.com/photos/123.jpeg</photo> /friends <best-friend uri=“urn:linkedin:v2:people/456” key=“456”> /123 /id <name>Brandon Duncan</name> /name <photo>http://media.linkedin.com/photos/456.jpeg</photo> /email /photo </best-friend> /best-friend </friend> /name <friend>…</friend> /photo /jobs </friends> … 11

Domain Model as Foundation Fine-grained Request > Allows client to construct custom calls > Better than digging for the closest matching API: http://social...com/v1/user/123/profile http://social...com/v1/user/123/profile/usercard http://social...com/v1/user/123/profile/tinyusercard > Allows optimization on the backend 12

Domain Model as Foundation Benefits > Provides a frame for both request and response semantics > Still allows for flexible syntax  Requests – path, query params, matrix params…  Responses – JSON, XML, POJOs, protobuff… > Helps to unify and automate many development tasks on both ends  Request / response creation, parsing, marshalling  Code (and documentation) generation  Discovery services 13

Examples of LinkedIn APIs HTTP GET - Read …/people/email=brandon@gmail.com/friends?sort=name …/people/123/friends;sort=name:(name,jobs;sort=start-date) …/people:(id,name,photo)?name=page&company=google …/people::(123,456) …/people::(123,456):(name,photo) 14

Examples of LinkedIn APIs HTTP PUT - Update > Set the user’s name: /people[/id=123] /id /name PUT http://api.linkedin.com/v2/people/123/name /email <name>Reid Hoffmann</name> /photo /best-friend … > Update the user’s profile - change name and best- friend and remove photo: PUT http://api.linkedin.com/v2/people/123 <person> /people[/id=123] <name>Reid Hoffman</name> /id /name <best-friend uri=“urn:linkedin:v2:people/999”/> /email <photo xsi:nil=“true”/> /photo </person> /best-friend … 15

Examples of LinkedIn APIs HTTP POST - Create > Add a friend /people[/id=123] /id POST http://api.linkedin.com/v2/people/123/friends /name /email <friend uri=“urn:linkedin:v2:people/888”/> /photo /best-friend 201 Created /friends /456 Location: http://api.linkedin.com/v2/people/123/friends/888 /888 … 16

Examples of LinkedIn APIs HTTP DELETE - Remove > Remove a friend DELETE http://api.linkedin.com/v2/people/123/friends/456 > Delete a company DELETE http://api.linkedin.com/v2/companies/exchange=NYSE&ticker=GM > Delete two companies DELETE http://api.linkedin.com/v2/companies::(664,665) 17

Use Standard Headers > Content-Type > Last-Modified > Accept > Vary > Authorization > Cache-Control > Content-MD5 > Location > Warning 18

Incentive System > Multiple ways to get at the same data > Partner can ask for exactly what they need > Associate cost with resources, system of accounting creates incentives for partners > Throttling by resource rather than API 19

Real-World Example Xobni Toolbar > Xobni makes ~20 million profile API calls per week > Default representation is ~2k on average > Using in-line filter brings average to ~1.5k  25% reduction in response size  ~11000 Mbits savings per day  11k Mbits out of LinkedIn datacenter  11k Mbits into Xobni datacenter  Saves both companies money 20

Yegor Borovikov yborovik@linkedin.com Brandon Duncan bduncan@linkedin.com blog.linkedin.com 21

http://blog.linkedin.com	4187
http://www.slideshare.net	107
http://cristofersonbueno.posterous.com	60
http://blogs.sun.com	40
url_unknown	8
http://ylone.net	8
http://blogs.oracle.com	7
http://www.nohuddleoffense.de	5
http://feeds.feedburner.com	4
https://cristofersonbueno.posterous.com	2
http://www.lifeyun.com	2
http://planets.sun.com	2
https://si0.twimg.com	1
http://google.com HTTP	1
http://85.114.139.198	1
http://paper.li	1
http://74.125.93.132	1
http://translate.googleusercontent.com	1
http://webcache.googleusercontent.com	1
http://feeds2.feedburner.com	1

Building Consistent RESTful APIs in a high-performance environment

by LinkedIn on Jul 06, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

20 Embeds 4,440

Statistics

Building Consistent RESTful APIs in a high-performance environment — Presentation Transcript