Services Oriented Architecture with PHP and MySQL

Slide 1: Services Oriented Architecture with PHP and MySQL Joe Stump, Lead Architect, Digg

Slide 2: Introductions I’m Joe Stump, Lead Architect for Digg.

Slide 3: Reduce HTTP Requests Bundle JavaScript and CSS Use sprites for images Reduce images / outside objects This talk doesn’t cover MySQL details. The point of this discussion is how to minimize MySQL’s impact on your company’s data layer.

Slide 4: Old habits die hard Data requests are sequential Data requests are blocking Tightly coupled Scaling is not abstracted

Slide 5: Services Oriented Architecture (SOA) Data is requested from a service Data requests are ran in parallel Data requests are asynchronous Data layer is loosely coupled Scalability is abstracted

Slide 6: What are my options? Running requests over HTTP New York Times’ DBSlayer Danga’s Gearman DIY

Slide 7: HTTP w/ PHP 1. Group requests for data at the top 2. Open a socket for each request 1. Sockets must be non-blocking 2. Make sure to TCP_NODELAY 3. Use __get() to block for results 4. See Services_Digg_Request

Slide 8: Grouping Data Requests

Slide 9: Opening the Sockets

Slide 10: Opening the Sockets

Slide 11: Returning Data

Slide 12: HTTP Conclusions HTTP is widely supported in all languages Easy to get up and running Lots of options for servers / tuning Lots of overhead in protocol Apache has a lot of overhead

Slide 13: DBSlayer Small HTTP daemon written in C Uses JSON for communications Connection pooling Load balancing and failover Tightly coupled to MySQL (for now) Tightly coupled to SQL No intelligence

Slide 14: Gearman Highly scalable queuing system Simple/Efficient binary protocol Jobs can return results (e.g. data) Sets of jobs are ran in parallel Queue can scale linearly PHP, Perl, Python, Ruby, C clients Poorly documented Not very “robust”

Slide 15: Do It Yourself Highly customized solutions (Flickr) Extremely efficient for custom cases Customize your protocols Requires more resources

Slide 16: What goes in the Service layer? Smart caching strategies Data mapping and distribution Intelligent grouping of data results Partitioning logic

Slide 17: DO WANT! Intelligently group data into endpoints Version your endpoints Bundle and group requests

Slide 18: EPIC FAIL! Don’t just send SQL over for translation Hundreds of teeny tiny endpoints Not running SOA requests in parallel

Slide 19: Resources Services_Digg http://pear.php.net/package/Services_Digg Danga’s Gearman http://www.danga.com/gearman/ Net_Gearman http://code.google.com/p/netgearman/ New York Timesʼ DBSlayer http://code.nytimes.com/projects/dbslayer

Slide 20: Questions?!

Slide 21: Contact/Flame Me Joe Stump joe@digg.com http://joestump.net