Constructing Web APIs with Rack, Sinatra and MongoDB

Constructing Web APIs with Rack, Sinatra andmongoDBoisín hurleyoi.sin@nis.io@oisin

web API (ecosystem)

web API (mobile access)

web API (revenue)

a good API is...focussed ‣ clear in its intent ‣ epitomizes good coding/behavioural practice ‣ has minimal sugar ‣ has a minimum of control surfaces

a good API is...evolvable ‣ your API will have consumers ‣ you don’t suddenly break the consumers, ever ‣ you control the API lifecycle, you control the expectations

a good web API is...responsive ‣ unchatty ‣ bandwidth sensitive ‣ latency savvy ‣ does paging where appropriate ‣ not unnecessarily fine-grained

a good web API is...resilient ‣ stable in the presence of badness ‣ traps flooding/overload ‣ adapts to surges ‣ makes good on shoddy requests, if possible ‣ authenticates, if appropriate

example application ‣ flavour of the month - location tracker! ‣ now that apple/google no longer do our work for us ‣ register a handset ‣ add a location ‘ping’ signal from handset to server https://github.com/oisin/plink

design (focussed) ‣ PUT a handset for registration ‣ POST location details ‣ DEL a handset when not in use ‣ focussed and short

design (evolvable) ‣ hit it with a hammer - put a version into URL - /api/v1.3/... ‣ in good company - google, twitter ‣ produce a compatibility statement ‣ what it means to minor/major level up ‣ enforce this in code

design (resilience) ‣ mongoDB for scaling ‣ write code to work around badness ‣ throttling of client activity with minimum call interval ‣ not using auth in this edition...

design (responsiveness) ‣ this API is very fine-grained, but not chatty ‣ we should queue to decouple POST response time from db ‣ but mongo is meant to be super-fast ‣ so maybe we get away with it in this edition :)

technologies (sinatra) ‣ web DSL ‣ low hassle whut? ‣ rack compatible http://www.sinatrarb.com/

technologies (rack) ‣ rack - a ruby webserver interface ‣ we’re going to use this for two things ‣ throttling for bad clients using a Rack middleware ‣ mounting multiple Sinatra apps with Rack::Builder (later on) http://rack.rubyforge.org/

technologies (mongodb) ‣ high performance ‣ non-relational ‣ horizontal scaling ‣ may give us resilience and responsiveness ‣ also nice client on MacOS :) http://www.mongodb.org http://mongohub.todayclose.com/

technologies (mongo_mapper) ‣ ORM for mongoDB ‣ a slight tincture of ActiveRecord : models, associations, dynamic finders ‣ embedded documents ‣ indices ‣ also, I like DataMapper and this is a little similar http://mongomapper.com/

mongoDB (deploys)

mongoDB is document-oriented ‣ collections contain documents, which can contain keys, arrays and other documents ‣ a document is like a JSON dictionary (in fact, it’s BSON) ‣ indices, yes, but no schema in the RDBMS sense - but you do plan!

mongoDB is a database ‣ foreign keys - can reference documents living in other collections ‣ indices - same as RDBMS - use in the same way ‣ datatypes - JSON basics plus some others including regex and code ‣ flexible querying with js, regex, kv matching ‣ but no JOINs all the same query

mongoDB can scale ‣ by relaxing some of the constraints of relational DBs, better horizontal scaling can be achieved ‣ replica sets for scaling reads ‣ replica sets & sharding for scaling writes ‣ map/reduce for batch processing of data (like GROUP BY)http://www.mongodb.org/display/DOCS/Replicationhttp://www.mongodb.org/display/DOCS/Sharding

cap/brewer’s theorem All nodes see all data at the same time Consistency Partition Availability Tolerance Node failures do not Only total network failure prevent operation will cause system to respond incorrectly Pick Any Two

consistency model (read) master slave http://blog.mongodb.org/post/475279604/on-distributed-consistency-part-1

mongoDB is performance oriented ‣ removes features that impede performance ‣ will not replace your SQL store ‣ good for this example app - because we want fast ‘write’ performance and scale (consistency not so much) ‣ GridFS - chunkifies and stores your files - neat!

code (API) ➊ ➋ ➍ ➌ config.ru https://github.com/oisin/plink

code (versioning) ➊ ➋ ➌ https://github.com/oisin/plink

code (mongo) ➊ ➋ ➌ ➍ ➎ https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink

code (mongo configure) ➊ ➋ https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink

mongo (console) ➊ ➋ ➌

code (mongo queries) ➊ where query ➋ & creation dynamic query ➌ ➍ deletion https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink

code (mongo embedded docs) ➊ ➋ ➌ ➍ ➎ https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink

mongo (capped collections) ‣ Fixed size, high performance LRU ‣ Maintains insertion order - great for logs/comments/etc ‣ not in use in this example application ‣ embedded documents - no cap on arrays ‣ putting location data in another collection - not sensible ‣ hacked it in the example app

code (throttling) ➋ ➊ ➌ custom throttle strategy https://github.com/datagraph/rack-throttle https://github.com/oisin/plink

deploy

deploy DE VE CR LO AC PE K R

deploy

deploy BU CR TT AC K

viewing the data ➋ ➊ another rack app

fast test (restclient) https://github.com/archiloque/rest-client

wraps (mongo) ‣ programming is straightforward with mongo_mapper ‣ works well with heroku ‣ haven’t done any work with sharding/replication ‣ complement RDBMS - e.g. for GridFS files storage, logs, profiles ‣ worthy of further study and experimentation

improvements (example) ‣ authentication using Rack::Warden ‣ queued invocations using delayed_job ‣ some eye candy for the tracking data ‣ suggestions welcome :-) http://github.com/oisin/plink

http://oisinh.wordpress.com	676
http://www.redditmedia.com	30
http://paper.li	6
url_unknown	4
http://twitter.com	3
https://twitter.com	2
http://translate.googleusercontent.com	2
http://a0.twimg.com	2

Constructing Web APIs with Rack, Sinatra and MongoDB

by Oisin Hurley on May 10, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

8 Embeds 725

Statistics

Constructing Web APIs with Rack, Sinatra and MongoDB — Presentation Transcript