Gearman
Upcoming SlideShare
Loading in...5
×
 

Gearman

on

  • 2,230 views

How Gearman works and how we use it at dealnews.com.

How Gearman works and how we use it at dealnews.com.

Statistics

Views

Total Views
2,230
Slideshare-icon Views on SlideShare
2,229
Embed Views
1

Actions

Likes
7
Downloads
52
Comments
1

1 Embed 1

https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Gearman Gearman Presentation Transcript

  • The MagicalWorld of Gearman Brian Moon dealnews.com http://brian.moonspot.net/ @brianlmoon
  • Basic FeaturesUse Cases How It Works
  • “The way I like to think of Gearman is as a massivelydistributed, massively fault tolerant fork mechanism.” - Joe Stump
  • The Basics• Clients need jobs done• Workers can do jobs• Gearmand coordinates the work
  • Gearmand http://www.flickr.com/photos/andrefromont/4896802557
  • GearmandDaemon that manages the work.Does not do any work.Accepts a job id and a binary payload fromclients.Workers keep connections open at alltimes. http://www.flickr.com/photos/andrefromont/4896802557
  • Client http://www.flickr.com/photos/pitadel/4951801589
  • ClientClients connect to Gearmand and ask forwork to be done.The client can fire and forget or wait on aresponse.Multiple jobs can be done asynchronouslyby workers for one client. http://www.flickr.com/photos/pitadel/4951801589
  • Workers http://www.flickr.com/photos/nathaninsandiego/5972599772
  • WorkersDaemonized codeA single worker can do just one job or cando many jobs.Does not have to be written using thesame language as the worker. http://www.flickr.com/photos/nathaninsandiego/5972599772
  • Key Features• Background jobs• De-duplication of jobs• Multiple jobs per client• High, normal and low priority• Work will be resubmitted if not completed
  • Background Jobs• Clients can fire and forget work to be done• Well suited for data marshalling• Minimal ability to track the status
  • De-duplication• Clients provide a unique job id• If more than one client provides the same job id, work is done once• Not a cache, once the job is done, the id is gone. The work will be done again.
  • Priority• High, Normal and Low priority options.• New items are inserted at the end of the queue based on priority• Priority is per job type, not global
  • Worker Selection• Uses the “game show method”• Workers that do multiple jobs will more likely get jobs “higher” in their list• Can appear to be clearing out one queue over another, but not really a design choice
  • Operational Visibility• Gearmand can report status about jobs and workers• It is only a view of current status, not historical• Use outside tools to graph what work was done when
  • Marshalling Data
  • Memcached Main Main Main Database Database Database Web Web Web WebServers Web Servers Web Servers Web Servers Web Servers Servers Servers Servers
  • Memcached Main Main Main Database Database Database Web Web Web WebServers Web Servers Web Servers Web Servers Web Servers Servers Servers Servers This is so 2005!
  • Main Main Optimized Database Database CRO Database or In N Main Main Proc ess Main Database Database Database Web Web Web WebServers Web Servers Web Servers Web Servers Web Servers Servers Servers Servers
  • Main Main Optimized Database Database CRO Database or In N Main Main Proc ess Main Database Database Database Web Web Web WebServers Web Servers Web Servers Web Servers Web Servers Servers Servers Servers This is so 2009!
  • Main Main Optimized Database Database Database Main Main Main Web Web Database Database Web Web DatabaseServers Web Servers Web Servers Web Servers Web Servers Servers Servers Servers Gearman Gearman Gearman Gearman Workers Gearman Workers Gearman Workers Gearman Workers Gearman Workers Workers Workers Workers Gearmand Backend Events
  • Why Gearman• Rid us of database spikes• Changes “feel” realtime• In the case of an issue, changes can queue up and happen when things are stable• Changes can happen asynchronously
  • SMTP Replacement• Large daily newsletter at 3PM• Email alerts go out on demand to thousands of readers as deals are published• Bottleneck was from double queuing in the mail queue• SMTP Server was a single point of failure
  • Web Web Cron Web Web Cron CronServers Web Servers Cron Jobs Web Servers Web Backend Jobs Servers Web Servers Jobs Jobs Servers Servers Events Servers Gearmand Gearman Gearman Gearman Gearman Gearman Gearman Gearman Gearman Workers Gearman Workers Workers Gearman Workers Gearman Workers Gearman Gearman Workers Gearman Workers Gearman Workers SMTP Workers Gearman Workers SMTP Workers Workers Workers Workers Workers Server Workers Server
  • Logging
  • Logging Options• Disk - reliable unless load is high. Can’t be queried easily in real time.• MySQL - Can make complex queries against it. Under high load, data can be lost• Other - (Spread, Scribe, etc.) New daemons to manage, learn, scale, etc.
  • Logging via Gearman• Frontend can fire and forget log data, returning immediately to the application• Log data is queued• Workers can process the logs in any number of ways• Log data can be stored any number of ways
  • Web Web Web WebServers Web Writing Log Data Servers Web Servers Web Servers Web Servers Servers Servers Servers Gearmand Gearman Gearman Gearman Gearman Gearman Gearman Gearman Gearman Workers Gearman Workers Workers Gearman Workers Gearman Workers Gearman Gearman Workers Gearman Workers Gearman Workers MySQL Workers Gearman Workers MySQL Workers Workers Workers Workers Workers Server Workers Server
  • Web Web Querying Log Data Web WebServers Web Servers Web Servers Web Servers (Map Reduce “ish”) Backend Servers Servers Servers App Gearmand Gearman Gearman Gearman Gearman Gearman Gearman Gearman Gearman Workers Gearman Workers Workers Gearman Workers Gearman Workers Gearman Gearman Workers Gearman Workers Gearman Workers MySQL Workers Gearman Workers MySQL Workers Workers Workers Workers Workers Server Workers Server
  • Request Funneling
  • Normalizing URIshttp://dealnews.com/?ref=google_10-corporate&s_kwcid=%7Bifcontent%3AContentNetwork%7D%7Bifsearch%3A%7Bkeyword%7D%7D%7C%7Bcreative%7D&WT.term=newdeals&WT.campaign=1799&WT.source=google&WT.medium=cpc&WT.content=606053200&cshift_ck=1880996632cs606053200&WT.srch=1http://dealnews.com/?sort=categoryhttp://dealnews.com/?view=large
  • Normalizing URIs http://dealnews.com/
  • Normalizing URIs• Define what parameters a request needs • sort • view • region • date • start• Throw out the rest• Sort what you need• Build the real URL
  • Normalizing URIs • http://dealnews.com/ • http://dealnews.com/?sort=category • http://dealnews.com/?ref=foobar • http://dealnews.com/?region=nyc All become:http://dealnews.com/?sort=category&view=large&region=nyc (assuming the user is in New York)
  • Why normalize/funnel?• We can now cache the data for this request and know it is the same data even if the original URI is different. (cache reuse)• We can fetch the content only once for all requests coming in for the content via request funneling.
  • Why normalize/funnel?• 72 Unique URIs for the front page in 3 minute spike. There were only 6 possible real versions. (normalizing)• Thousands of syndication requests hit the app servers between 10:43 and 10:45. There were only 86 unique URIs. (funneling)
  • Request Funneling Proxy ServerApache Apache Apache Apache Apache Child Child Child Child Child http://dealnews.com/?sort=category&view=large&region=nyc Gearmand Gearman Web Worker Server
  • What does a worker do?• Builds a new URI from the input data• Makes an HTTP request to an app server• If cacheable, stores the data in the cache (important!)• Returns the data (page) to the proxy (via Gearmand)
  • The Magical World of Gearman Brian Moon dealnews.com http://brian.moonspot.net/ @brianlmoon More Information: http://gearman.org/ Need to run PHP workers?https://github.com/brianlmoon/GearmanManager