Gearman is an open-source application framework that allows for the distribution of work across multiple machines. It uses a queueing system to dispatch jobs to workers to be processed asynchronously. Clients submit jobs to the Gearman server which handles distributing jobs to available worker processes to execute the requested tasks. This allows for scalable and fault-tolerant processing of jobs in parallel across a pool of machines.

1. Gearman 'The manager' ” since it dispatches jobs to be done, but does not do anything useful itself.”

2. Presentation done using info from... http://www.slideshare.net/pcdinh/gearman-and-asynchronous-processing-in-php-applications-6135047 http://assets.en.oreilly.com/1/event/45/The%20Gearman%20Cookbook%20Presentation.pdf http://www.gearman.org http://nz.php.net/manual/en/book.gearman.php others...

3. Scalable Solutions.. More Hardware Caching Precalculated Data Load Balancing Multi-tier application Job Queue

4. History Created by Danga Interactive. Some company that developed Memcache. Original implementation in perl (2005). 2008 rewriteen in C by Brian Aker PHP Extension by James Luedke

5. Used by Digg: 45+ Servers, 400K jobs/day Yahoo: 60+ servers, 6M jobs/day And many others..

6. Installing Compiling: tar xzf gearmand-X.Y.tar.gz cd gearmand-X.Y ./configure make make install Starting server: $ gearmand -d Pecl extension: tar xzf gearman-X.Y.tgz cd gearman-X.Y phpize ./configure make make install To add to the php.ini: extension="gearman.so"

7. Terminology Client: Create jobs to be run and send them to a job server. Worker: Run jobs given from the job server. Job Server: Handle the job queue form clients to workers.

8. “ A massively distributed, massively fault tolerant fork mechanism.” - Joe Stump, SimpleGeo Gearman is...

9. Open Source. Simple & Fast. Multi-language. Flexible application design. Load Balancing. No single point of failure. Features

10. Client Worker Job Server Job Server Client Client Client Worker Worker Worker

11. Memory Memcached Mysql/Drizzle PostgreSQL SQLite Tokio Cabinet Queue Options

12. Foreground (synchronus) Or Background (asynchronus)

13. Fishpond _Controller_Front::getResource('gearman') ->getGearmanClient() ->doBackground("updateCompetitorPrice", $this->_barcode); ->do("updateCompetitorPrice", $this->_barcode); GearmanClient::do() - Run a single task and return a result GearmanClient::doLow() - Run a single low priority task GearmanClient::doBackground() - Run a task in the background GearmanClient::doHighBackground() - Run a high priority task in the background GearmanClient::doLowBackground() - Run a low priority task in the background Gearman Client

14. Scatter / Gather. Map / Reduce. Asynchronus Queues. Pipeline Processing. Strategies

15. Scatter / Gather Price Calculation Image Resize Recomendations Product Detail Client

16. $client = Fishpond_Controller_Front::getResource('gearman') ->getGearmanClient(); //adding gearman tasks $client->addTask("getProductDetail", $barcode); $client->addTask("getPrice", $barcode); $client->addTask("resizeImage", serialize($barcode,100,100)); $client->addTask("getRecomendations", $barcode); //callbacks to know when this finish $client->setCompleteCallback(array($this, "complete")); //runing tasks $client->runTasks(); /** * Callback when task is complete * */ public function complete($task) { $data = $task->data(); }

17. GearmanClient::addTaskHigh() - Add a high priority task to run in parallel GearmanClient::addTaskLow() - Add a low priority task to run in parallel GearmanClient::addTaskBackground() - Add a background task to be run in parallel GearmanClient::addTaskHighBackground() - Add a high priority background task to be run in parallel GearmanClient::addTaskLowBackground() - Add a low priority background task to be run in parallel GearmanClient::runTasks() - Run a list of tasks in parallel Task Methods

18. GearmanClient::setDataCallback() - Callback function when there is a data packet for a task GearmanClient::setCompleteCallback() - Set a function to be called on task completion GearmanClient::setCreatedCallback() - Set a callback for when a task is queued. GearmanClient::setExceptionCallback() - Set a callback for worker exceptions. GearmanClient::setFailCallback() - Set callback for job failure. GearmanClient::setStatusCallback() - Set a callback for collecting task status. GearmanClient::setWarningCallback() - Set a callback for worker warnings. GearmanClient::setWorkloadCallback() - Set a callback for accepting incremental data updates Client Callback

19. Concurrent tasks with different workers. All tasks run in the time for longest running. Must have enough workers available. Scatter / Gather

20. Map/Reduce Client Task T Task T-0 Task T-3 Task T-2 Task T-1 Task T-00 Task T-02 Task T-01

21. Asynchronous Queues No everyting need inmediate procesing.. Competitor pricing. Emails. Whole price engine. Loging. Etc. Example: $gearmanClient = Fishpond_Controller_Front::getResource('gearman')->getGearmanClient(); $gearmanClient->doBackground("updateCompetitorPrice", $this->_barcode);

22. Loging <VirtualHost *:80> ServerName example.com DocumentRoot /var/www/ CustomLog “| gearman -n -f looger” common (client) </VirtualHost>

23. Pipeline Procesing Client Task T Output Worker Operation 3 Worker Operation 2 Worker Operation 1

24. Questions ?