Gearmanpresentation 110308165409-phpapp01
Upcoming SlideShare
Loading in...5
×
 

Gearmanpresentation 110308165409-phpapp01

on

  • 793 views

gearman

gearman

Statistics

Views

Total Views
793
Views on SlideShare
793
Embed Views
0

Actions

Likes
1
Downloads
9
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Gearmanpresentation 110308165409-phpapp01 Gearmanpresentation 110308165409-phpapp01 Presentation Transcript

  • Gearman The manager ”since it dispatches jobs to be done,but does not do anything useful itself.”
  • Presentation done using info from...http://www.slideshare.net/pcdinh/gearman-and-asynchronous-processing-in-php-applications-6135047http://assets.en.oreilly.com/1/event/45/The%20Gearman%20Cookbook%20Presentation.pdfhttp://www.gearman.orghttp://nz.php.net/manual/en/book.gearman.phpothers...
  • Scalable Solutions..  More Hardware  Caching  Precalculated Data  Load Balancing  Multi-tier application  Job Queue
  • History Created by Danga Interactive. Some company that developed Memcache. Original implementation in perl (2005). 2008 rewriteen in C by Brian Aker PHP Extension by James Luedke
  • Used byDigg: 45+ Servers, 400K jobs/dayYahoo: 60+ servers, 6M jobs/dayAnd many others..
  • InstallingCompiling:tar xzf gearmand-X.Y.tar.gzcd gearmand-X.Y./configuremakemake installStarting server:$ gearmand -dPecl extension:tar xzf gearman-X.Y.tgzcd gearman-X.Yphpize./configuremakemake installTo add to the php.ini:extension="gearman.so"
  • Terminology Client: Create jobs to be run and send them to a job server. Worker: Run jobs given from the job server. Job Server: Handle the job queue form clients to workers.
  • Gearman is...“A massively distributed,massively fault tolerantfork mechanism.”- Joe Stump, SimpleGeo
  • Features Open Source. Simple & Fast. Multi-language. Flexible application design. Load Balancing. No single point of failure.
  • Client Client Client Client Job Server Job ServerWorker Worker Worker Worker
  • Queue Options  Memory  Memcached  Mysql/Drizzle  PostgreSQL  SQLite  Tokio Cabinet
  • Foreground(synchronus) Or Background(asynchronus)
  • Gearman Client Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient() ->doBackground("updateCompetitorPrice", $this->_barcode); ->do("updateCompetitorPrice", $this->_barcode); GearmanClient::do() - Run a single task and return a result GearmanClient::doLow() - Run a single low priority task GearmanClient::doBackground() - Run a task in the background GearmanClient::doHighBackground() - Run a high priority task in the background GearmanClient::doLowBackground() - Run a low priority task in the background
  • Strategies Scatter / Gather. Map / Reduce. Asynchronus Queues. Pipeline Processing.
  • Scatter / Gather ClientProduct Detail Recomendations Price Calculation Image Resize
  • $client = Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient(); //adding gearman tasks $client->addTask("getProductDetail", $barcode); $client->addTask("getPrice", $barcode); $client->addTask("resizeImage", serialize($barcode,100,100)); $client->addTask("getRecomendations", $barcode); //callbacks to know when this finish $client->setCompleteCallback(array($this, "complete")); //runing tasks $client->runTasks();/** * Callback when task is complete * */public function complete($task){ $data = $task->data();}
  • Task Methods GearmanClient::addTaskHigh() - Add a high priority task to run in parallel GearmanClient::addTaskLow() - Add a low priority task to run in parallel GearmanClient::addTaskBackground() - Add a background task to be run in parallel GearmanClient::addTaskHighBackground() - Add a high priority background task to be run in parallel GearmanClient::addTaskLowBackground() - Add a low priority background task to be run in parallel GearmanClient::runTasks() - Run a list of tasks in parallel
  • Client CallbackGearmanClient::setDataCallback() - Callback function when there is a data packet for a taskGearmanClient::setCompleteCallback() - Set a function to be called on task completionGearmanClient::setCreatedCallback() - Set a callback for when a task is queued.GearmanClient::setExceptionCallback() - Set a callback for worker exceptions.GearmanClient::setFailCallback() - Set callback for job failure.GearmanClient::setStatusCallback() - Set a callback for collecting task status.GearmanClient::setWarningCallback() - Set a callback for worker warnings.GearmanClient::setWorkloadCallback() - Set a callback for accepting incremental data updates
  • Scatter / Gather Concurrent tasks with different workers. All tasks run in the time for longest running. Must have enough workers available.
  • Map/Reduce Client Task TTask T-0 Task T-1 Task T-2 Task T-3Task T-00 Task T-01 Task T-02
  • Asynchronous Queues No everyting need inmediate procesing..  Competitor pricing.  Emails.  Whole price engine.  Loging.  Etc.Example:$gearmanClient = Fishpond_Controller_Front::getResource(gearman)->getGearmanClient();$gearmanClient->doBackground("updateCompetitorPrice", $this->_barcode);
  • Loging<VirtualHost *:80> ServerName example.com DocumentRoot /var/www/ CustomLog “| gearman -n -f looger” common (client)</VirtualHost>
  • Pipeline Procesing Client Task T Worker Worker WorkerOperation 1 Operation 2 Operation 3 Output
  • Questions ?