Drupal 7 Queues

Drupal 7 Queues

• 1.
  Drupal 7 Queues
• 2.
  Drupal Queues Newto Drupal 7 First in first out data structure Used internally by Drupal Uses OO principles Fully customisable
• 3.
  Why Use Queues?Batch processing of large amounts of data/items Delay processing of complex calculations so system load is more stable Sequential processing of items Preventing API service black listing
• 4.
  Drupal 7 ImplementationsBatch API Cron Aggregator Module Update Module
• 5.
  Drupal Queues Foundin the file: /modules/system/system.queue.inc Classes are used to wrap queue functionality Reliable vs Non-reliable
• 6.
  Reliable Generally keptin database table Every item will be executed at least once Items exist over several requests If the request fails then queue remains intact
• 7.
  Non-reliable Generally keptin memory only All items might exist in a single request No guarantee that all queue items will be executed No guarantee that all items will be executed in order If request fails then queue might be lost
• 8.
  Drupal Queue Classes
• 9.
  DrupalQueue Single staticmethod get() DrupalQueue::get('my_queue'); Returns a queue object of a given name and a given type (reliable or non-reliable) Force reliable queue by passing TRUE as second parameter Default returned class is SystemQueue Essentially a queue object factory
• 10.
  SystemQueue Example ofa reliable queue class Implements DrupalReliableQueueInterface Uses the database table queue to store and retrieve the queue Items are 'leased' to ensure no two processes get the same queue item Default class for new queues
• 11.
  queue Table Createdat Drupal install
• 12.
  MemoryQueue Example ofan non-reliable queue class All queue items are stored in memory $queue parameter in class contains the queue Also implements item leasing
• 13.
  System Variables Threesystem variables are used by DrupalQueue to decide what sort of object to return 'queue_class_' . $name Default is NULL queue_default_class Default is SystemQueue queue_default_reliable_class Default is SystemQueue
• 14.
  Using Drupal Queues
• 15.
  Create A Queue$queue = DrupalQueue::get('my_queue', TRUE); $item = array( 'dataitem1' => 'something', 'int' => 123 ); $queue->createItem($item); echo $queue->numberOfItems(); // 1
• 16.
  Create A MemoryQueuevariable_set('queue_default_class', 'MemoryQueue'); $queue = DrupalQueue::get('my_queue'); $item = array( 'dataitem1' => 'something', 'int' => 123 ); $queue->createItem($item); echo $queue->numberOfItems(); // 1
• 17.
  Get Item FromQueue $queue = DrupalQueue::get('my_queue'); $got_item = $queue->claimItem(); echo $got_item->data['dataitem1'];
• 18.
  Retrieved Item StructurestdClass Object ( [data] => Array ( [dataitem1] => something [qwe] => 123 ) [item_id] => 89 )
• 19.
  Change Lease LengthCan pass the lease time (in seconds) to claimItem() $got_item = $queue->claimItem( 100 );
• 20.
  Release Or DeletereleaseItem() resets the lease time $queue->releaseItem($got_item); deleteItem() removes item from queue $queue->deleteItem($got_item);
• 21.
  Customizing Create customqueue class Create variable called “queue_class_” . $name Value is the class name of your queue class Call DrupalQueue::get() with the variable $name as a string
• 22.
  Customizing Reliable classMUST implement DrupalReliableQueueInterface Can also extend SystemQueue All Queue classes should at least implement DrupalQueueInterface
• 23.
  Using Custom Classesvariable_set('queue_class_mycustom', 'MyCustomQueueClass'); $queue = DrupalQueue::get('mycustom');
• 24.
  Stack Class Lastin first out implementation class Stack extends SystemQueue { public function claimItem($lease_time = 30) { while (TRUE) { $item = db_query_range('SELECT data, item_id FROM {queue} q WHERE expire = 0 AND name = :name ORDER BY created DESC ', 0, 1, array(':name' => $this->name))->fetchObject(); // ...
• 25.
  EventQueue Class Requiredas part of the NWDUG website Any items added would not be available for 45 minutes to give a period of grace after creating an event node class EventQueue extends SystemQueue { public function claimItem($lease_time = 30) { while (TRUE) { $item = db_query_range('SELECT data, item_id FROM {queue} q WHERE expire = 0 AND name = :name AND created >= UNIX_TIMESTAMP(DATE_ADD(NOW(), INTERVAL 2700 SECOND)) ORDER BY created ASC', 0, 1, array(':name' => $this->name))->fetchObject();
• 26.
  WatchdogSystemQueue Extends SystemQueueand creates a log every time anything is done Maintains SystemQueue functionality
• 27.
  WatchdogSystemQueue class WatchdogSystemQueueextends SystemQueue { public function __construct($name) { watchdog('queue', '%name queue created', array('%name' => $name)); parent::__construct($name); } public function createItem($data) { watchdog('queue', 'Item created : %data', array('%data' => print_r($data, TRUE))); parent::createItem($data); } public function claimItem($lease_time = 3600) { $return_value = parent::claimItem($lease_time); watchdog('queue', 'Item claimed %item (lease time = %lease)', array( '%item' => print_r($return_value, TRUE), '%lease' => $lease_time)); return $return_value; } //....
• 28.
  RandomMemoryQueue Extends theMemoryQueue class Items are added in order but are retrieved in random order
• 29.
  RandomMemoryQueue class RandomMemoryQueueextends MemoryQueue { public function claimItem($lease_time = 30) { $available_items = array(); // Extract the remining available items foreach ($this->queue as $key => $item) { if ($item->expire == 0) { $available_items[] = $item; } } // Randomly select one (if available) if (count($available_items) > 0) { $queue_length = count($this->queue); $rand_item = rand(0, $queue_length - 1); $item = $available_items[$rand_item]; $item->expire = time() + $lease_time; return $item; } return FALSE; } }
• 30.
  Creating And DestroyingDrupalQueueInterface has two methods available for creating and destroying the queue createQueue() Should be called within an install hook deleteQueue() Should be called within an uninstall hook
• 31.
  Tips Unless youreally need to rewrite the entire class it is best to extend SystemQueue or MemoryQueue For open source projects try to keep the same retrieved item structure as the system queues No checks for unique items in default system queues
• 32.
  Resources Drupal QueuesAPI http://api.drupal.org/api/drupal/modules--system--system.queue.inc/group/queue/7 http://bit.ly/kuphnc Queue UI Module http://drupal.org/project/queue_ui Source code is well documented /modules/system/system.queue.inc Full write up of this talk on #! code http://www.hashbangcode.com/
• 33.
  Questions? $queue =DrupalQueue::get('questions'); $question = $queue->claimItem(); echo $question->data['question'];
• 34.
  Blog http:///www.norton42.org.uk/ Twitter@philipnorton42 #! code http://www.hashbangcode.com/ #! code on Twitter @hashbangcode Philip Norton