Your SlideShare is downloading. ×
Os Gottfrid
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Os Gottfrid

351
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
351
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. DBSlayer Introducing DBSlayer http://code.nytimes.com/projects/dbslayer Derek Gottfrid The New York Times
  • 2. Problem: Provide Web Apps Access to the Database
  • 3. Alternate Problem: Avoid evil horrible doctor show.
  • 4. Solution JSON + HTTP + MYSQL
  • 5. Simple Example select ci.* from City ci, Country c where c.name = 'United States' and c.code = ci.CountryCode order by Population desc
  • 6. HTTP Request http://localhost:9090/db?%7B%22SQL%22: %22select%20ci.*%20from%20City%20ci,% 20Country%20c%20where%20c.name%20 =%20'United%20States'%20and%20c.code %20=%20ci.CountryCode%20order%20by% 20Population%20desc%20limit%2010;%22 %7D
  • 7. HTTP Response HTTP/1.0 200 OK Date: Wed, 27 Jun 2007 20:55:59 GMT Server: dbslayer/server beta-9 Connection: Close Content-type: text/plain; charset=utf-8 Content-Length: 746
  • 8. JSON Payload {quot;RESULTquot; : {quot;TYPESquot; : [quot;MYSQL_TYPE_LONGquot; , quot;MYSQL_TYPE_STRINGquot; , quot;MYSQL_TYPE_STRINGquot; , quot;MYSQL_TYPE_STRINGquot; , quot;MYSQL_TYPE_LONGquot;] , quot;HEADERquot; : [quot;IDquot; , quot;Namequot; , quot;CountryCodequot; , quot;Districtquot; , quot;Populationquot;] , quot;ROWSquot; : [[3793 , quot;New Yorkquot; , quot;USAquot; , quot;New Yorkquot; , 8008278] , [3794 , quot;Los Angelesquot; , quot;USAquot; , quot;Californiaquot; , 3694820] , [3795 , quot;Chicagoquot; , quot;USAquot; , quot;Illinoisquot; , 2896016] , [3796 , quot;Houstonquot; , quot;USAquot; , quot;Texasquot; , 1953631] , [3797 , quot;Philadelphiaquot; , quot;USAquot; , quot;Pennsylvaniaquot; , 1517550] , [3798 , quot;Phoenixquot; , quot;USAquot; , quot;Arizonaquot; , 1321045] , [3799 , quot;San Diegoquot; , quot;USAquot; , quot;Californiaquot; , 1223400] , [3800 , quot;Dallasquot; , quot;USAquot; , quot;Texasquot; , 1188580] , [3801 , quot;San Antonioquot; , quot;USAquot; , quot;Texasquot; , 1144646] , [3802 , quot;Detroitquot; , quot;USAquot; , quot;Michiganquot; , 951270]]}}
  • 9. Python Sample Client import urllib,urllib2,cjson db = lambda q: cjson.decode( urllib2.urlopen( “http://localhost:9090/db? ”+urllib.quote('{“SQL ”:”%s”}' % q)).read())
  • 10. Other Solutions MySQLProxy ● SQLRelay ● mysqli ●
  • 11. Requirements Independent ● Scalable ● Flexible ● Simple ●
  • 12. Disclaimer Works for us.
  • 13. Problems Not Solved Back end scripts – ● bulk import/export Large Data Retrievals ● – cursors Native Binds / ● Prepared Statements Enterprise Buzzword ● Compliance A Sober Lindsay ●
  • 14. DBSlayer is the middle-man Connection Pooling ● Multi-threaded ● Round robin distribution of queries ● Automatic fail over to alternate database ● Easy configuration ● Statistics / Logging ● Tiny code base ●
  • 15. Design Considerations
  • 16. Flexiblity Loose coupling of front and middle ● easy upgrade – abstract location of db – makes re-config a – snap Language independent ● JSON easily shared across languages – Common place to add additional features ●
  • 17. Scalability Round robin across slaves ● Automatic fail over for slaves ● Easily add more slaves ● HTTP plays nice with load balancers ● JSON is very cachable ● Stateless is required for effective scaling ●
  • 18. Simplicity HTTP and JSON - everyone knows them ● Transparent protocol – capture, interpret ● Lots of tools to work with HTTP ● Works with your browser ● HTTP server base – easy to deploy new ● endpoints
  • 19. Browser Testing
  • 20. Stats
  • 21. Pretty Pictures
  • 22. View Source Built on top of MySQL C API and Apache ● Runtime Libraries (APR) Why APR? ● Platform agnostic – Memory Pools – Networking / Threading Abstractions – Proven –
  • 23. 3 Parts server – handles network, HTTP parsing, ● threading, logging db – handle mysql access, type mapping, ● connection failover util – json de/serializer ●
  • 24. Future / Your Chance improve HTTP caching layer ● ● complaince partition / shard ● more database support ● backends (APR new language ● dbd) bindings alternate serializer ● more stats / ● XML logging security ● scriptable filters ●
  • 25. Other (tighten up them ....)
  • 26. Another amazing talk... Rails Under The Knife Jacob Harris July 26th – 2:35pm Portland 255
  • 27. BoF Amazon S3/EC2 computing against large ● datasets – 7:30 F150 Future Newspapers – 8:30 E143-144 ●
  • 28. NYTIMES.COM open.nytimes.com – developers blog ● code.nytimes.com – code repository ●
  • 29. Free Stuff Visit the NYTimes Booth
  • 30. The End Questions: derek@nytimes.com