Gearmanpresentation 110308165409-phpapp01
Upcoming SlideShare
Loading in...5

Gearmanpresentation 110308165409-phpapp01







Total Views
Slideshare-icon Views on SlideShare
Embed Views



0 Embeds 0

No embeds


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Gearmanpresentation 110308165409-phpapp01 Gearmanpresentation 110308165409-phpapp01 Presentation Transcript

    • Gearman The manager ”since it dispatches jobs to be done,but does not do anything useful itself.”
    • Presentation done using info from...
    • Scalable Solutions..  More Hardware  Caching  Precalculated Data  Load Balancing  Multi-tier application  Job Queue
    • History Created by Danga Interactive. Some company that developed Memcache. Original implementation in perl (2005). 2008 rewriteen in C by Brian Aker PHP Extension by James Luedke
    • Used byDigg: 45+ Servers, 400K jobs/dayYahoo: 60+ servers, 6M jobs/dayAnd many others..
    • InstallingCompiling:tar xzf gearmand-X.Y.tar.gzcd gearmand-X.Y./configuremakemake installStarting server:$ gearmand -dPecl extension:tar xzf gearman-X.Y.tgzcd gearman-X.Yphpize./configuremakemake installTo add to the php.ini:extension=""
    • Terminology Client: Create jobs to be run and send them to a job server. Worker: Run jobs given from the job server. Job Server: Handle the job queue form clients to workers.
    • Gearman is...“A massively distributed,massively fault tolerantfork mechanism.”- Joe Stump, SimpleGeo
    • Features Open Source. Simple & Fast. Multi-language. Flexible application design. Load Balancing. No single point of failure.
    • Client Client Client Client Job Server Job ServerWorker Worker Worker Worker
    • Queue Options  Memory  Memcached  Mysql/Drizzle  PostgreSQL  SQLite  Tokio Cabinet
    • Foreground(synchronus) Or Background(asynchronus)
    • Gearman Client Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient() ->doBackground("updateCompetitorPrice", $this->_barcode); ->do("updateCompetitorPrice", $this->_barcode); GearmanClient::do() - Run a single task and return a result GearmanClient::doLow() - Run a single low priority task GearmanClient::doBackground() - Run a task in the background GearmanClient::doHighBackground() - Run a high priority task in the background GearmanClient::doLowBackground() - Run a low priority task in the background
    • Strategies Scatter / Gather. Map / Reduce. Asynchronus Queues. Pipeline Processing.
    • Scatter / Gather ClientProduct Detail Recomendations Price Calculation Image Resize
    • $client = Fishpond_Controller_Front::getResource(gearman) ->getGearmanClient(); //adding gearman tasks $client->addTask("getProductDetail", $barcode); $client->addTask("getPrice", $barcode); $client->addTask("resizeImage", serialize($barcode,100,100)); $client->addTask("getRecomendations", $barcode); //callbacks to know when this finish $client->setCompleteCallback(array($this, "complete")); //runing tasks $client->runTasks();/** * Callback when task is complete * */public function complete($task){ $data = $task->data();}
    • Task Methods GearmanClient::addTaskHigh() - Add a high priority task to run in parallel GearmanClient::addTaskLow() - Add a low priority task to run in parallel GearmanClient::addTaskBackground() - Add a background task to be run in parallel GearmanClient::addTaskHighBackground() - Add a high priority background task to be run in parallel GearmanClient::addTaskLowBackground() - Add a low priority background task to be run in parallel GearmanClient::runTasks() - Run a list of tasks in parallel
    • Client CallbackGearmanClient::setDataCallback() - Callback function when there is a data packet for a taskGearmanClient::setCompleteCallback() - Set a function to be called on task completionGearmanClient::setCreatedCallback() - Set a callback for when a task is queued.GearmanClient::setExceptionCallback() - Set a callback for worker exceptions.GearmanClient::setFailCallback() - Set callback for job failure.GearmanClient::setStatusCallback() - Set a callback for collecting task status.GearmanClient::setWarningCallback() - Set a callback for worker warnings.GearmanClient::setWorkloadCallback() - Set a callback for accepting incremental data updates
    • Scatter / Gather Concurrent tasks with different workers. All tasks run in the time for longest running. Must have enough workers available.
    • Map/Reduce Client Task TTask T-0 Task T-1 Task T-2 Task T-3Task T-00 Task T-01 Task T-02
    • Asynchronous Queues No everyting need inmediate procesing..  Competitor pricing.  Emails.  Whole price engine.  Loging.  Etc.Example:$gearmanClient = Fishpond_Controller_Front::getResource(gearman)->getGearmanClient();$gearmanClient->doBackground("updateCompetitorPrice", $this->_barcode);
    • Loging<VirtualHost *:80> ServerName DocumentRoot /var/www/ CustomLog “| gearman -n -f looger” common (client)</VirtualHost>
    • Pipeline Procesing Client Task T Worker Worker WorkerOperation 1 Operation 2 Operation 3 Output
    • Questions ?