Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Constructing Web APIs with Rack, Sinatra and MongoDB

22,212 views

Published on

Slides for my talk at Ruby Ireland on 10 May 11. Showing some of the capabilities of mongoDB, using it from a Sinatra applications and deploying it to Heroku and Cloud Foundry

Published in: Technology

Constructing Web APIs with Rack, Sinatra and MongoDB

  1. 1. Constructing Web APIs with Rack, Sinatra andmongoDBoisín hurleyoi.sin@nis.io@oisin
  2. 2. web API (ecosystem)
  3. 3. web API (mobile access)
  4. 4. web API (revenue)
  5. 5. a good API is...focussed ‣ clear in its intent ‣ epitomizes good coding/behavioural practice ‣ has minimal sugar ‣ has a minimum of control surfaces
  6. 6. a good API is...evolvable ‣ your API will have consumers ‣ you don’t suddenly break the consumers, ever ‣ you control the API lifecycle, you control the expectations
  7. 7. a good web API is...responsive ‣ unchatty ‣ bandwidth sensitive ‣ latency savvy ‣ does paging where appropriate ‣ not unnecessarily fine-grained
  8. 8. a good web API is...resilient ‣ stable in the presence of badness ‣ traps flooding/overload ‣ adapts to surges ‣ makes good on shoddy requests, if possible ‣ authenticates, if appropriate
  9. 9. example application ‣ flavour of the month - location tracker! ‣ now that apple/google no longer do our work for us ‣ register a handset ‣ add a location ‘ping’ signal from handset to server https://github.com/oisin/plink
  10. 10. design (focussed) ‣ PUT a handset for registration ‣ POST location details ‣ DEL a handset when not in use ‣ focussed and short
  11. 11. design (evolvable) ‣ hit it with a hammer - put a version into URL - /api/v1.3/... ‣ in good company - google, twitter ‣ produce a compatibility statement ‣ what it means to minor/major level up ‣ enforce this in code
  12. 12. design (resilience) ‣ mongoDB for scaling ‣ write code to work around badness ‣ throttling of client activity with minimum call interval ‣ not using auth in this edition...
  13. 13. design (responsiveness) ‣ this API is very fine-grained, but not chatty ‣ we should queue to decouple POST response time from db ‣ but mongo is meant to be super-fast ‣ so maybe we get away with it in this edition :)
  14. 14. technologies (sinatra) ‣ web DSL ‣ low hassle whut? ‣ rack compatible http://www.sinatrarb.com/
  15. 15. technologies (rack) ‣ rack - a ruby webserver interface ‣ we’re going to use this for two things ‣ throttling for bad clients using a Rack middleware ‣ mounting multiple Sinatra apps with Rack::Builder (later on) http://rack.rubyforge.org/
  16. 16. technologies (mongodb) ‣ high performance ‣ non-relational ‣ horizontal scaling ‣ may give us resilience and responsiveness ‣ also nice client on MacOS :) http://www.mongodb.org http://mongohub.todayclose.com/
  17. 17. technologies (mongo_mapper) ‣ ORM for mongoDB ‣ a slight tincture of ActiveRecord : models, associations, dynamic finders ‣ embedded documents ‣ indices ‣ also, I like DataMapper and this is a little similar http://mongomapper.com/
  18. 18. mongoDB (deploys)
  19. 19. mongoDB is document-oriented ‣ collections contain documents, which can contain keys, arrays and other documents ‣ a document is like a JSON dictionary (in fact, it’s BSON) ‣ indices, yes, but no schema in the RDBMS sense - but you do plan!
  20. 20. mongoDB is a database ‣ foreign keys - can reference documents living in other collections ‣ indices - same as RDBMS - use in the same way ‣ datatypes - JSON basics plus some others including regex and code ‣ flexible querying with js, regex, kv matching ‣ but no JOINs all the same query
  21. 21. mongoDB can scale ‣ by relaxing some of the constraints of relational DBs, better horizontal scaling can be achieved ‣ replica sets for scaling reads ‣ replica sets & sharding for scaling writes ‣ map/reduce for batch processing of data (like GROUP BY)http://www.mongodb.org/display/DOCS/Replicationhttp://www.mongodb.org/display/DOCS/Sharding
  22. 22. cap/brewer’s theorem All nodes see all data at the same time Consistency Partition Availability Tolerance Node failures do not Only total network failure prevent operation will cause system to respond incorrectly Pick Any Two
  23. 23. consistency model (read) master slave http://blog.mongodb.org/post/475279604/on-distributed-consistency-part-1
  24. 24. mongoDB is performance oriented ‣ removes features that impede performance ‣ will not replace your SQL store ‣ good for this example app - because we want fast ‘write’ performance and scale (consistency not so much) ‣ GridFS - chunkifies and stores your files - neat!
  25. 25. code (API) ➊ ➋ ➍ ➌ config.ru https://github.com/oisin/plink
  26. 26. code (versioning) ➊ ➋ ➌ https://github.com/oisin/plink
  27. 27. code (mongo) ➊ ➋ ➌ ➍ ➎ https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink
  28. 28. code (mongo configure) ➊ ➋ https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink
  29. 29. mongo (console) ➊ ➋ ➌
  30. 30. code (mongo queries) ➊ where query ➋ & creation dynamic query ➌ ➍ deletion https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink
  31. 31. code (mongo embedded docs) ➊ ➋ ➌ ➍ ➎ https://github.com/jnunemaker/mongomapper https://github.com/oisin/plink
  32. 32. mongo (capped collections) ‣ Fixed size, high performance LRU ‣ Maintains insertion order - great for logs/comments/etc ‣ not in use in this example application ‣ embedded documents - no cap on arrays ‣ putting location data in another collection - not sensible ‣ hacked it in the example app
  33. 33. code (throttling) ➋ ➊ ➌ custom throttle strategy https://github.com/datagraph/rack-throttle https://github.com/oisin/plink
  34. 34. deploy
  35. 35. deploy DE VE CR LO AC PE K R
  36. 36. deploy
  37. 37. deploy BU CR TT AC K
  38. 38. viewing the data ➋ ➊ another rack app
  39. 39. fast test (restclient) https://github.com/archiloque/rest-client
  40. 40. wraps (mongo) ‣ programming is straightforward with mongo_mapper ‣ works well with heroku ‣ haven’t done any work with sharding/replication ‣ complement RDBMS - e.g. for GridFS files storage, logs, profiles ‣ worthy of further study and experimentation
  41. 41. improvements (example) ‣ authentication using Rack::Warden ‣ queued invocations using delayed_job ‣ some eye candy for the tracking data ‣ suggestions welcome :-) http://github.com/oisin/plink

×