Task Scheduling and Asynchronous Processing Evolved. Zend Server Job Queue By Sam Hennessy – Zend Professional Services Consultant

Offline Processing Why? Perceived performance Managing load Any work that isn’t needed for giving feedback to the user should be done offline.

Why?

Perceived performance

Managing load

Offline Processing Examples Sending e-mails Processing orders Indexing new content Cleaning the database or file system Either periodically or after some user action Pulling an RSS feed every hour Checking exchange rates every day Generating periodical reports

Sending e-mails

Processing orders

Indexing new content

Cleaning the database or file system

Either periodically or after some user action

Pulling an RSS feed every hour

Checking exchange rates every day

Generating periodical reports

Blog Example Post comment Save in the DB Rebuild RSS feeds Send email notifications Display “Thank You” message

Blog Example Post comment Save in the DB Rebuild RSS feeds Send email notifications Display thank you message Queue Tasks Job Queue

Architectural Overview PHP + Job Queue Extension TCP or Unix Socket Job Queue Damon Persistent Storage (SQLite) HTTP Request Executing Server

Goals Of Zend Job Queues Ability to off-load PHP code execution Jobs should never be “lost” Ability to queue jobs from PHP API Ability to create recurring & deferred jobs Management GUI

Ability to off-load PHP code execution

Jobs should never be “lost”

Ability to queue jobs from PHP API

Ability to create recurring & deferred jobs

Management GUI

The ZendJobQueue class The ZendJobQueue class contains almost all the PHP API for Job Queue To perform most tasks, you will need to connect to a Job Queue server by instantiating a ZendJobQueue object: // Connect to the default JQ server $queue = new ZendJobQueue (); // Connect to any other JQ server $queue = new ZendJobQueue ( "tcp://1.2.3.4:5678" );

The ZendJobQueue class contains almost all the PHP API for Job Queue

To perform most tasks, you will need to connect to a Job Queue server by instantiating a ZendJobQueue object:

Creating Jobs All jobs are created using the ZendJobQueue::createHttpJob() method Passing a partial URL will create the job with $_SERVER['HTTP_HOST'] in the host nam e $queue = new ZendJobQueue (); $queue -> createHttpJob ( 'http://backend.local/jobs/somejob.php' ); $queue -> createHttpJob ( '/jobs/otherjob.php' );

All jobs are created using the ZendJobQueue::createHttpJob() method

Passing a partial URL will create the job with $_SERVER['HTTP_HOST'] in the host nam e

Passing Parameters Simple parameters can be passed as part of the query string Complex parameters can also be passed Pass any serializable data as the 2 nd parameter of createHttpJob() Data will be JSON-encoded – so anything which can be represented as JSON can be used $queue -> createHttpJob ( '/jobs/otherjob.php?foo=bar&uid=1792' );

Simple parameters can be passed as part of the query string

Complex parameters can also be passed

Pass any serializable data as the 2 nd parameter of createHttpJob()

Data will be JSON-encoded – so anything which can be represented as JSON can be used

Passing Parameters (example) $params = array( 'cart' => array( 'items' => array( array( 'id' => 324 , 'qty' => 1 , 'price' => 19.95 ), array( 'id' => 75 , 'qty' => 2 , 'price' => 14.95 , 'size' => 'XL' ) ), 'total' => 49.85 , 'coupon' => null , 'giftwrap' => true ), 'user' => $user ); $queue -> createHttpJob ( 'http://backend/jobs/checkout.php' , $params );

Accessing Parameters Inside the Job code, use the ZendJobQueue::getCurrentJobParams() static method to get the job's parameters: You can also json_decode() the raw POST data $params = ZendJobQueue :: getCurrentJobParams (); $params = json_decode ( file_get_contents ( 'php://input' ));

Inside the Job code, use the ZendJobQueue::getCurrentJobParams() static method to get the job's parameters:

You can also json_decode() the raw POST data

Additional Job Options The 3 rd parameter of createHttpJob is an associative array of parameters: name priority persistent predecessor http_headers schedule schedule_time

The 3 rd parameter of createHttpJob is an associative array of parameters:

name

priority

persistent

predecessor

http_headers

schedule

schedule_time

Creating Deferred Jobs You can set a job's (estimated) execution time by passing the schedule_time option: The job will not run before the specified time // Process the form at 3:00 am $runAt = date ( 'Y-m-d h:i:s' , strtotime ( '+1 day 3:00am' )); $options = array( 'schedule_time' => $runAt ); $queue -> createHttpJob ( 'http://backend/jobs/process.php' , $_POST , $options );

You can set a job's (estimated) execution time by passing the schedule_time option:

The job will not run before the specified time

Creating Recurring Jobs You can create a recurring job from API using the schedule option This option takes a cron-like expression that specifies scheduling // Run on Sunday, Monday, and Tuesday at midnight $jq -> createHttpJob ( 'http://localhost/jobs/feed/405' , null , array( 'schedule' => '0 0 * * 0,1,2' )); // Run every other day of the month at 2:30pm $jq -> createHttpJob ( 'http://localhost/jobs/feed/405' , null , array( 'schedule' => '30 14 */2 * *' ));

You can create a recurring job from API using the schedule option

This option takes a cron-like expression that specifies scheduling

Reporting Logical Failures A logical failure happens when something has programatically failed e.g. you were unable to send an e-mail or connect to a SOAP web service Unlike an execution failure which is a technical failure (e.g. 500 error from server) Logical failures need to be reported by the programmer: ZendJobQueue :: setCurrentJobStatus ( ZendJobQueue :: OK ); if (! doTheImportantStuff ()) { ZendJobQueue :: setCurrentJobStatus ( ZendJobQueue :: ERROR ); }

A logical failure happens when something has programatically failed

e.g. you were unable to send an e-mail or connect to a SOAP web service

Unlike an execution failure which is a technical failure (e.g. 500 error from server)

Logical failures need to be reported by the programmer:

Job HTTP Request POST /job.php HTTP/1.0 Host: localhost Content-Type: text/json Content-Length: 34 Connection: close Accept: */* USER_AGENT: Zend Server Job Queue/4.1 Cookie: SESS273df6b85bb9a37ced781f806219fc6a=7k6atppt8akccooobs26htsim7; ZDEDebuggerPresent=php,phtml,php3 {"var1":15,"nested":["a","b","c"]}

Job HTTP Response HTTP/1.1 200 OK Date: Thu, 10 Sep 2009 17:29:20 GMT Server: Apache/2.2.9 (Debian) PHP/5.2.10 X-Powered-By: PHP/5.2.10 ZendServer/4.0 Set-Cookie: ZDEDebuggerPresent=php,phtml,php3; path=/ X-Job-Queue-Status: 1 Something is wrong! Vary: Accept-Encoding Content-Length: 152 Connection: close Content-Type: text/html array(2) { ["var1"]=> int(15) ["nested"]=> array(3) { [0]=> string(1) "a" [1]=> string(1) "b" [2]=> string(1) "c" } }

Live Demo

Questions?

Thank You! Please come find me if you have any more questions

Please come find me if you have any more questions