Gearman         The manager ”since it dispatches jobs to be done,but does not do anything useful itself.”
Presentation done using info from...http://www.slideshare.net/pcdinh/gearman-and-asynchronous-processing-in-php-applicatio...
Scalable Solutions..            More Hardware            Caching            Precalculated Data            Load Balanci...
History    Created by Danga Interactive.    Some company that developed Memcache.    Original implementation in perl (2...
Used byDigg: 45+ Servers, 400K jobs/dayYahoo: 60+ servers, 6M jobs/dayAnd many others..
InstallingCompiling:tar xzf gearmand-X.Y.tar.gzcd gearmand-X.Y./configuremakemake installStarting server:$ gearmand -dPecl...
Terminology    Client: Create jobs to be run and send them to a job server.    Worker: Run jobs given from the job serve...
Gearman is...“A massively distributed,massively fault tolerantfork mechanism.”- Joe Stump, SimpleGeo
Features    Open Source.    Simple & Fast.    Multi-language.    Flexible application design.    Load Balancing.    ...
Client                Client   Client                 Client         Job Server                      Job ServerWorker     ...
Queue Options      Memory      Memcached      Mysql/Drizzle      PostgreSQL      SQLite      Tokio Cabinet
Foreground(synchronus)     Or Background(asynchronus)
Gearman Client       Fishpond_Controller_Front::getResource(gearman)               ->getGearmanClient()              ->doB...
Strategies    Scatter / Gather.    Map / Reduce.    Asynchronus Queues.    Pipeline Processing.
Scatter / Gather                                 ClientProduct Detail                                           Recomendat...
$client = Fishpond_Controller_Front::getResource(gearman)      ->getGearmanClient();    //adding gearman tasks    $client-...
Task Methods    GearmanClient::addTaskHigh() - Add a high priority task to run in parallel    GearmanClient::addTaskLow(...
Client CallbackGearmanClient::setDataCallback() - Callback function when there is a data packet for a taskGearmanClient::s...
Scatter / Gather    Concurrent tasks with different workers.    All tasks run in the time for longest running.    Must ...
Map/Reduce                    Client                    Task TTask T-0     Task T-1        Task T-2      Task T-3Task T-00...
Asynchronous Queues                   No everyting need inmediate procesing..                                        Comp...
Loging<VirtualHost *:80> ServerName example.com DocumentRoot /var/www/ CustomLog “| gearman -n -f looger” common   (client...
Pipeline Procesing                    Client                    Task T Worker             Worker        WorkerOperation 1 ...
Questions ?
Upcoming SlideShare
Loading in...5
×

Gearmanpresentation 110308165409-phpapp01

711

Published on

gearman

Published in: Education, Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
711
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Gearmanpresentation 110308165409-phpapp01

  1. 1. Gearman The manager ”since it dispatches jobs to be done,but does not do anything useful itself.”
  2. 2. Presentation done using info from...http://www.slideshare.net/pcdinh/gearman-and-asynchronous-processing-in-php-applications-6135047http://assets.en.oreilly.com/1/event/45/The%20Gearman%20Cookbook%20Presentation.pdfhttp://www.gearman.orghttp://nz.php.net/manual/en/book.gearman.phpothers...
  3. 3. Scalable Solutions..  More Hardware  Caching  Precalculated Data  Load Balancing  Multi-tier application  Job Queue
  4. 4. History Created by Danga Interactive. Some company that developed Memcache. Original implementation in perl (2005). 2008 rewriteen in C by Brian Aker PHP Extension by James Luedke
  5. 5. Used byDigg: 45+ Servers, 400K jobs/dayYahoo: 60+ servers, 6M jobs/dayAnd many others..
  6. 6. InstallingCompiling:tar xzf gearmand-X.Y.tar.gzcd gearmand-X.Y./configuremakemake installStarting server:$ gearmand -dPecl extension:tar xzf gearman-X.Y.tgzcd gearman-X.Yphpize./configuremakemake installTo add to the php.ini:extension="gearman.so"
  7. 7. Terminology Client: Create jobs to be run and send them to a job server. Worker: Run jobs given from the job server. Job Server: Handle the job queue form clients to workers.
  8. 8. Gearman is...“A massively distributed,massively fault tolerantfork mechanism.”- Joe Stump, SimpleGeo
  9. 9. Features Open Source. Simple & Fast. Multi-language. Flexible application design. Load Balancing. No single point of failure.
  10. 10. Client Client Client Client Job Server Job ServerWorker Worker Worker Worker
  11. 11. Queue Options  Memory  Memcached  Mysql/Drizzle  PostgreSQL  SQLite  Tokio Cabinet
  12. 12. Foreground(synchronus) Or Background(asynchronus)
  13. 13. Gearman Client Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient() ->doBackground("updateCompetitorPrice", $this->_barcode); ->do("updateCompetitorPrice", $this->_barcode); GearmanClient::do() - Run a single task and return a result GearmanClient::doLow() - Run a single low priority task GearmanClient::doBackground() - Run a task in the background GearmanClient::doHighBackground() - Run a high priority task in the background GearmanClient::doLowBackground() - Run a low priority task in the background
  14. 14. Strategies Scatter / Gather. Map / Reduce. Asynchronus Queues. Pipeline Processing.
  15. 15. Scatter / Gather ClientProduct Detail Recomendations Price Calculation Image Resize
  16. 16. $client = Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient(); //adding gearman tasks $client->addTask("getProductDetail", $barcode); $client->addTask("getPrice", $barcode); $client->addTask("resizeImage", serialize($barcode,100,100)); $client->addTask("getRecomendations", $barcode); //callbacks to know when this finish $client->setCompleteCallback(array($this, "complete")); //runing tasks $client->runTasks();/** * Callback when task is complete * */public function complete($task){ $data = $task->data();}
  17. 17. Task Methods GearmanClient::addTaskHigh() - Add a high priority task to run in parallel GearmanClient::addTaskLow() - Add a low priority task to run in parallel GearmanClient::addTaskBackground() - Add a background task to be run in parallel GearmanClient::addTaskHighBackground() - Add a high priority background task to be run in parallel GearmanClient::addTaskLowBackground() - Add a low priority background task to be run in parallel GearmanClient::runTasks() - Run a list of tasks in parallel
  18. 18. Client CallbackGearmanClient::setDataCallback() - Callback function when there is a data packet for a taskGearmanClient::setCompleteCallback() - Set a function to be called on task completionGearmanClient::setCreatedCallback() - Set a callback for when a task is queued.GearmanClient::setExceptionCallback() - Set a callback for worker exceptions.GearmanClient::setFailCallback() - Set callback for job failure.GearmanClient::setStatusCallback() - Set a callback for collecting task status.GearmanClient::setWarningCallback() - Set a callback for worker warnings.GearmanClient::setWorkloadCallback() - Set a callback for accepting incremental data updates
  19. 19. Scatter / Gather Concurrent tasks with different workers. All tasks run in the time for longest running. Must have enough workers available.
  20. 20. Map/Reduce Client Task TTask T-0 Task T-1 Task T-2 Task T-3Task T-00 Task T-01 Task T-02
  21. 21. Asynchronous Queues No everyting need inmediate procesing..  Competitor pricing.  Emails.  Whole price engine.  Loging.  Etc.Example:$gearmanClient = Fishpond_Controller_Front::getResource(gearman)->getGearmanClient();$gearmanClient->doBackground("updateCompetitorPrice", $this->_barcode);
  22. 22. Loging<VirtualHost *:80> ServerName example.com DocumentRoot /var/www/ CustomLog “| gearman -n -f looger” common (client)</VirtualHost>
  23. 23. Pipeline Procesing Client Task T Worker Worker WorkerOperation 1 Operation 2 Operation 3 Output
  24. 24. Questions ?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×