Your SlideShare is downloading. ×
Gearmanpresentation 110308165409-phpapp01
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Gearmanpresentation 110308165409-phpapp01


Published on



Published in: Education, Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Gearman The manager ”since it dispatches jobs to be done,but does not do anything useful itself.”
  • 2. Presentation done using info from...
  • 3. Scalable Solutions..  More Hardware  Caching  Precalculated Data  Load Balancing  Multi-tier application  Job Queue
  • 4. History Created by Danga Interactive. Some company that developed Memcache. Original implementation in perl (2005). 2008 rewriteen in C by Brian Aker PHP Extension by James Luedke
  • 5. Used byDigg: 45+ Servers, 400K jobs/dayYahoo: 60+ servers, 6M jobs/dayAnd many others..
  • 6. InstallingCompiling:tar xzf gearmand-X.Y.tar.gzcd gearmand-X.Y./configuremakemake installStarting server:$ gearmand -dPecl extension:tar xzf gearman-X.Y.tgzcd gearman-X.Yphpize./configuremakemake installTo add to the php.ini:extension=""
  • 7. Terminology Client: Create jobs to be run and send them to a job server. Worker: Run jobs given from the job server. Job Server: Handle the job queue form clients to workers.
  • 8. Gearman is...“A massively distributed,massively fault tolerantfork mechanism.”- Joe Stump, SimpleGeo
  • 9. Features Open Source. Simple & Fast. Multi-language. Flexible application design. Load Balancing. No single point of failure.
  • 10. Client Client Client Client Job Server Job ServerWorker Worker Worker Worker
  • 11. Queue Options  Memory  Memcached  Mysql/Drizzle  PostgreSQL  SQLite  Tokio Cabinet
  • 12. Foreground(synchronus) Or Background(asynchronus)
  • 13. Gearman Client Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient() ->doBackground("updateCompetitorPrice", $this->_barcode); ->do("updateCompetitorPrice", $this->_barcode); GearmanClient::do() - Run a single task and return a result GearmanClient::doLow() - Run a single low priority task GearmanClient::doBackground() - Run a task in the background GearmanClient::doHighBackground() - Run a high priority task in the background GearmanClient::doLowBackground() - Run a low priority task in the background
  • 14. Strategies Scatter / Gather. Map / Reduce. Asynchronus Queues. Pipeline Processing.
  • 15. Scatter / Gather ClientProduct Detail Recomendations Price Calculation Image Resize
  • 16. $client = Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient(); //adding gearman tasks $client->addTask("getProductDetail", $barcode); $client->addTask("getPrice", $barcode); $client->addTask("resizeImage", serialize($barcode,100,100)); $client->addTask("getRecomendations", $barcode); //callbacks to know when this finish $client->setCompleteCallback(array($this, "complete")); //runing tasks $client->runTasks();/** * Callback when task is complete * */public function complete($task){ $data = $task->data();}
  • 17. Task Methods GearmanClient::addTaskHigh() - Add a high priority task to run in parallel GearmanClient::addTaskLow() - Add a low priority task to run in parallel GearmanClient::addTaskBackground() - Add a background task to be run in parallel GearmanClient::addTaskHighBackground() - Add a high priority background task to be run in parallel GearmanClient::addTaskLowBackground() - Add a low priority background task to be run in parallel GearmanClient::runTasks() - Run a list of tasks in parallel
  • 18. Client CallbackGearmanClient::setDataCallback() - Callback function when there is a data packet for a taskGearmanClient::setCompleteCallback() - Set a function to be called on task completionGearmanClient::setCreatedCallback() - Set a callback for when a task is queued.GearmanClient::setExceptionCallback() - Set a callback for worker exceptions.GearmanClient::setFailCallback() - Set callback for job failure.GearmanClient::setStatusCallback() - Set a callback for collecting task status.GearmanClient::setWarningCallback() - Set a callback for worker warnings.GearmanClient::setWorkloadCallback() - Set a callback for accepting incremental data updates
  • 19. Scatter / Gather Concurrent tasks with different workers. All tasks run in the time for longest running. Must have enough workers available.
  • 20. Map/Reduce Client Task TTask T-0 Task T-1 Task T-2 Task T-3Task T-00 Task T-01 Task T-02
  • 21. Asynchronous Queues No everyting need inmediate procesing..  Competitor pricing.  Emails.  Whole price engine.  Loging.  Etc.Example:$gearmanClient = Fishpond_Controller_Front::getResource(gearman)->getGearmanClient();$gearmanClient->doBackground("updateCompetitorPrice", $this->_barcode);
  • 22. Loging<VirtualHost *:80> ServerName DocumentRoot /var/www/ CustomLog “| gearman -n -f looger” common (client)</VirtualHost>
  • 23. Pipeline Procesing Client Task T Worker Worker WorkerOperation 1 Operation 2 Operation 3 Output
  • 24. Questions ?