Do you queue?<br />By Kevin Schroeder<br />Technology Evangelist<br />Zend Technologies<br />
About Kevin<br /> Past: Programming/Sys Admin<br /> Current: Technology Evangelist/Author/Composer<br />         @kpschrad...
Could your PHP apps benefit from being able to process data or execute asynchronously?<br />Twtpoll results<br />
Performance<br />Execute logic apart from the main request (asynchronicity)<br />Scalability<br />The ability to handle no...
People often say that performance and scalability are separate things.<br />This is inaccurate (consider Google)<br />Perf...
DON’T!!<br />You are not Facebook<br />You probably won’t be<br />Don’t overcomplicate your problems by trying to be<br />...
Typical anatomy of a PHP Application<br />Bad for scala-bility!<br />Presentation<br />Application Control<br />Database A...
Good for Scalability<br />Presentation<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml...
Defined tasks<br />Loose coupling<br />Resource discovery<br />What helps make software scalable?<br />
The Golden Rule of Scalability<br />“It can probably wait”<br />
Pre-caching data<br />Data analysis<br />Pre-analysis (predicting where your users will go)<br />Data processing<br />Pre-...
A sledgehammer can hit a machine<br />Scalability and High Availability are yin and yang<br />A site that can’t keep runni...
Waste disk space<br />Control usage (don’t let users do anything they want)<br />Pre-calculate as much as possible<br />Ca...
<ul><li>Build a deployment mechanism with NO hardcoded values like directory or resource locations
Make as much as possible configurable/discoverable</li></ul>Decouple/Partition<br />Don’t tie everything (relationships an...
Use Cron /w PHP CLI (probably shouldn’t do this)<br />Use Gearman<br />Use home-grown (don’t do this)<br />Use pcntl_fork(...
Your only real options<br />Very cloud friendly<br />* I am not an expert on Gearman.  Corrections will be taken in the sp...
Schedule jobs in the future<br />Set recurring jobs<br />Execute immediately, as resources are available (my fav)<br />Uti...
Job Queue Architecture – Elastic Backend<br />Web Server /w JQ<br />Web Server<br />Users!<br />Web Server /w JQ<br />Web ...
Job Queue Architecture – Elastic Frontend<br />Web Server<br />Web Server /w JQ<br />Users!<br />Web Server<br />Web Serve...
Create a task-handling controller<br />Create an abstract task class<br />Understands the Job Queue<br />Self contained<br...
comzendjobqueueManager<br />Handles connecting to the queue and passing results back and forth<br />comzendjobqueueJobAbst...
Create job and set data<br />Execute job<br />Job passes itself to the queue manager<br />Manager serializes job<br />Mana...
Let’s write some code<br />(no I’m not copping out with slides.  We’re all told to show our work in grade skool)<br />
Follow us!<br />Zend Technologies<br />http://twitter.com/zend<br />http://twitter.com/kpschrade (me!)<br />
Upcoming SlideShare
Loading in …5
×

Do you queue (updated)

1,248 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,248
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Do you queue (updated)

  1. 1. Do you queue?<br />By Kevin Schroeder<br />Technology Evangelist<br />Zend Technologies<br />
  2. 2. About Kevin<br /> Past: Programming/Sys Admin<br /> Current: Technology Evangelist/Author/Composer<br /> @kpschrade<br />
  3. 3. Could your PHP apps benefit from being able to process data or execute asynchronously?<br />Twtpoll results<br />
  4. 4. Performance<br />Execute logic apart from the main request (asynchronicity)<br />Scalability<br />The ability to handle non-immediate logic as resources are available<br />Why would you want to queue?<br />
  5. 5. People often say that performance and scalability are separate things.<br />This is inaccurate (consider Google)<br />Performance is the speed by which a request is executed<br />Scalability is the ability of that request to maintain its performance as load/infrastructure increases<br />Performance & Scalability<br />
  6. 6. DON’T!!<br />You are not Facebook<br />You probably won’t be<br />Don’t overcomplicate your problems by trying to be<br />So how do you scale to Facebook size?<br />
  7. 7. Typical anatomy of a PHP Application<br />Bad for scala-bility!<br />Presentation<br />Application Control<br />Database Access<br />Business Logic<br />Presentation<br />Application Control<br />Business Logic<br />Presentation<br />| 7<br />
  8. 8. Good for Scalability<br />Presentation<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Application Control<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Business Logic<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Database Access<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />| 8<br /> | 6-Jun-11<br />
  9. 9. Defined tasks<br />Loose coupling<br />Resource discovery<br />What helps make software scalable?<br />
  10. 10. The Golden Rule of Scalability<br />“It can probably wait”<br />
  11. 11. Pre-caching data<br />Data analysis<br />Pre-analysis (predicting where your users will go)<br />Data processing<br />Pre-calculating (preparing data for the next request)<br />Data is “out of date” once it leaves the web server<br />Immediacy is seldom necessary<br />Asynchronous execution uses<br />
  12. 12. A sledgehammer can hit a machine<br />Scalability and High Availability are yin and yang<br />A site that can’t keep running is not scalable<br />A site that can’t scale will fail (if it gets really popular)<br />Machines can be added and removed at will<br />Not “cloudy” necessarily<br />No single point of failure<br />Data exists in at least two, preferably at least three, places<br />Characteristics<br />
  13. 13. Waste disk space<br />Control usage (don’t let users do anything they want)<br />Pre-calculate as much as possible<br />Calculate and cache/store<br />Don’t scan large amounts of data<br />Keep data processing off the front end servers<br />Don’t just cache<br />Don’t let it substitute for thought<br />Cache hit rates can be meaningless if you have hundreds of cache hits for a request<br />Considerations<br />
  14. 14. <ul><li>Build a deployment mechanism with NO hardcoded values like directory or resource locations
  15. 15. Make as much as possible configurable/discoverable</li></ul>Decouple/Partition<br />Don’t tie everything (relationships and such) into the database<br />Use queues/messaging<br />Stomp interfaces are really good for PHP – Can also use Java Bridge<br />Zend_Queue has several interfaces<br />Try to use stateless interfaces<br />polling is more scalable than idle connections; introduces lag<br />Considerations<br />
  16. 16. Use Cron /w PHP CLI (probably shouldn’t do this)<br />Use Gearman<br />Use home-grown (don’t do this)<br />Use pcntl_fork() (NEVER do this)<br />Use Zend Server Job Queue<br />Options<br />
  17. 17. Your only real options<br />Very cloud friendly<br />* I am not an expert on Gearman. Corrections will be taken in the spirit that they are given.<br />For obvious reasons, I will focus on Zend Server<br />
  18. 18. Schedule jobs in the future<br />Set recurring jobs<br />Execute immediately, as resources are available (my fav)<br />Utilize ZendJobQueue()<br />Using the Zend Server Job Queue<br />
  19. 19. Job Queue Architecture – Elastic Backend<br />Web Server /w JQ<br />Web Server<br />Users!<br />Web Server /w JQ<br />Web Server<br />Load Balancer<br />Web Server /w JQ<br />Web Server<br />Pros<br />Scale the backend as necessary<br />Default (easy) mechanism<br />Cons<br />Getting the job status requires using a DB<br />
  20. 20. Job Queue Architecture – Elastic Frontend<br />Web Server<br />Web Server /w JQ<br />Users!<br />Web Server<br />Web Server /w JQ<br />Load Balancer<br />Web Server<br />Web Server /w JQ<br />Pros<br />Easy to communicate with the Job Queue server handling the job<br />Cons<br />Requires you to build your own interface<br />
  21. 21. Create a task-handling controller<br />Create an abstract task class<br />Understands the Job Queue<br />Self contained<br />If Elastic Backend: connects to localhost<br />If Elastic Frontend: connects to load balancer (my preferred), load balanced JQ servers manage themselves<br />Execute the task, have it serialize itself and send it to send to the task handler<br />Kevin’s favorite way to implement it<br />
  22. 22. comzendjobqueueManager<br />Handles connecting to the queue and passing results back and forth<br />comzendjobqueueJobAbstract<br />Abstract class that a job would be based off of<br />comzendjobqueueResponse<br />The response from the manager when a job is queued. Contains the server name and job number<br />orgeschradejobsScandir<br />The actual job that scans the directory<br />orgeschradejobsScandirResult<br />An object that represents the data found<br />Classes involved in the demo<br />
  23. 23. Create job and set data<br />Execute job<br />Job passes itself to the queue manager<br />Manager serializes job<br />Manager uses HTTP call through a load balancer to queue the job<br />The queue on the other end returns the job id and server name<br />Job ID and server name is passed to the client<br />Client polls the manager to get a completed job<br />When the job is returned pass the serialized version of the executed job<br />Execution Flow<br />
  24. 24. Let’s write some code<br />(no I’m not copping out with slides. We’re all told to show our work in grade skool)<br />
  25. 25. Follow us!<br />Zend Technologies<br />http://twitter.com/zend<br />http://twitter.com/kpschrade (me!)<br />
  26. 26. Get this information and all the examples at eschrade.com…<br />

×