Jared Mooring
Platform Engineer
Former lead developer
• BigPond GameArena
• BigPond Kids
• BigPond Prepaid
• Ausgamers
What is distribution?
It seems to be a ‘buzz word’ at the moment
'A distributed system consists of multiple autonomous
c...
SCENARIO
– PHOTO GALLERY
1. User uploads photo through
form
2. Photo get resized – thumb,
medium, original
3. Web server r...
Issues?
Web server is meant to server HTTP requests
– Why should it do the heavy lifting?
– Be kind to your presentation ...
Alternative - Distribute
Job
Worker
Job
Server
Job
Request
ApacheUser
Upload
Image
Process
Request
Request to
resize image...
Message Queue Systems
Store data as a ‘message’ for processing
– Serialized objects
– Text
Allows different applications...
Introducing
‘Gearman provides a generic application framework to
farm out work to other machines or processes that are
be...
Basics
Gearman Layer
Server Manages jobs / Multiple instances
Client Makes request to job server – it wants work done!
Wor...
Example Time
http://gearman.org
–Source / Repositories
API available through PECL
–http://pecl.php.net/package/gearman
–http://php.ne...
How are Yahoo!7 using it?
Platform for distributing management of content
– Developers are able to build distributed plug...
Who else is using it?
ALL DONE! THANK YOU
Twitter: @jadzor
Forward Pass: http://fwdpass.it/bN4Q4v
MammothVPS Coupon:
–http://www.mammothvps.c...
Introduction to Distributed PHP - Sydney PHP Users Group October 2010
Introduction to Distributed PHP - Sydney PHP Users Group October 2010
Upcoming SlideShare
Loading in …5
×

Introduction to Distributed PHP - Sydney PHP Users Group October 2010

1,774 views
1,705 views

Published on

1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
1,774
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
10
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • Being a good developer is more than writing code
    Be aware of your environment
    What OS are you developing for?
    Are you using a cluster?
    Where are your servers? AU/EU/US?
    Do you have servers for specific tasks?
    Web Servers
    Mail Servers
    Tools Servers
  • Use your web server to serve HTTP requests
    Offload (distribute) the tasks that aren’t related to serving HTTP requests
    Move the image processing
    Send a request to resize images
    Worker picks up request, resizes image
    Writes image to disk
    Returns paths to images

  • Client does:
* Connect to gearmand.
* submit's a job for a particular func.

    Gearmand does:
* Acks the job, finds all *sleeping workers* related to the function.
* Sends them all a 'noop' command to wake them up.

    Worker does:
* Urk, I'm awake now.
* Worker asks for jobs.
* If jobs, do work.
* If no jobs, sends command 'pre_sleep' to all gearmand's, etc.
  • Introduction to Distributed PHP - Sydney PHP Users Group October 2010

    1. 1. Jared Mooring Platform Engineer Former lead developer • BigPond GameArena • BigPond Kids • BigPond Prepaid • Ausgamers
    2. 2. What is distribution? It seems to be a ‘buzz word’ at the moment 'A distributed system consists of multiple autonomous computers that communicate through a computer network. The computers interact with each other in order to achieve a common goal' – wikipedia Share the work, sharing is caring!
    3. 3. SCENARIO – PHOTO GALLERY 1. User uploads photo through form 2. Photo get resized – thumb, medium, original 3. Web server resizes images 4. Stores new sizes on disk 5. Display resized photos
    4. 4. Issues? Web server is meant to server HTTP requests – Why should it do the heavy lifting? – Be kind to your presentation server(s) CPU spent on resizing images
    5. 5. Alternative - Distribute Job Worker Job Server Job Request ApacheUser Upload Image Process Request Request to resize image Receive request Idle worker Idle Worker Idle Worker Resize image Wait for other connections
    6. 6. Message Queue Systems Store data as a ‘message’ for processing – Serialized objects – Text Allows different applications to interact – Includes different operating systems – Different languages Message Persistence – RAM: Memcache – DB: mySQL, SQLite, PostgreSQL
    7. 7. Introducing ‘Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work’ - http://gearman.org
    8. 8. Basics Gearman Layer Server Manages jobs / Multiple instances Client Makes request to job server – it wants work done! Worker Performs job / Tells server what it can do / Distributed
    9. 9. Example Time
    10. 10. http://gearman.org –Source / Repositories API available through PECL –http://pecl.php.net/package/gearman –http://php.net/manual/en/book.gearman.php Drivers available for other languages –Perl –Python –Java –.NET
    11. 11. How are Yahoo!7 using it? Platform for distributing management of content – Developers are able to build distributed plugins – Individual business units Building feeds RSS/MRSS Video transcoding – Allows us to add new nodes easily – Distributed managed transcoding
    12. 12. Who else is using it?
    13. 13. ALL DONE! THANK YOU Twitter: @jadzor Forward Pass: http://fwdpass.it/bN4Q4v MammothVPS Coupon: –http://www.mammothvps.com.au –SYDPHP

    ×