Thoughts on Events + A brief intro to a practicable events SDK – Gearman  [email_address]
Events vs. Threading <ul><li>A historical quarrel </li></ul><ul><li>Actually individual things </li></ul><ul><ul><li>A Des...
Threads <ul><li>Grew from OS and evolved into user level  </li></ul><ul><li>Managing concurrency </li></ul><ul><li>Multipl...
Callbacks don’t work with locks <ul><li>Deadlocks: </li></ul><ul><ul><li>Self block </li></ul></ul><ul><ul><ul><li>Spinnin...
What’s “wrong” with Threads? <ul><li>Painful for many cases,  </li></ul><ul><ul><li>Data structure and complexity, OS sche...
Events <ul><li>A process consists of the processing of a series of events.  </li></ul><ul><ul><li>A single event being pro...
Events (cont.) <ul><li>Dispatcher and Handlers need to be efficient </li></ul><ul><li>External event sequence is not under...
Good Practices for Events <ul><li>Do All Jobs in as events </li></ul><ul><ul><li>Micro kernel </li></ul></ul><ul><ul><li>O...
To be or not? <ul><li>Use threads only when there is a payoff </li></ul><ul><li>Killer indicator for threads? </li></ul><u...
Farm out jobs with Gearman! <ul><li>Originally developed with Perl and rewritten in C </li></ul>
Farm out jobs with Gearman! <ul><li>Diverse API supports, PHP, Perl, C, Ruby, Java, Python, C# </li></ul><ul><li>Reusing S...
Working model Both synchronous and asynchronous  (event silent db persistency)
Deployment model <ul><li>Easy to extend with new boxes/CPU/Core# </li></ul><ul><li>Need to refine routine/balancing </li><...
More about Gearman <ul><li>Easy to deploy </li></ul><ul><li>A stateful protocol </li></ul><ul><ul><li>Seek gearman.org for...
Example-1 <ul><li>Shell command dispatch </li></ul><ul><li>Client </li></ul><ul><ul><li>gearman -h 10.64.13.168 -p7003  -f...
Example-2 <ul><li>Perl Handler and Worker </li></ul><ul><ul><li>Original document needs refines </li></ul></ul><ul><li>Cli...
Example-2 (cont.) <ul><li>Worker </li></ul><ul><li>Job Server remains </li></ul>
Example-3 <ul><li>Monitoring the server </li></ul><ul><li>Text cmd interaction </li></ul>
A Map/Reduce App – Log Processing <ul><li>Collection </li></ul><ul><ul><li>Tail –f access_log|gearman –n –f logger </li></...
FIN <ul><li>Q/A </li></ul><ul><li>More thoughts on events dispatching </li></ul><ul><ul><li>Routing, Debug, Metric, Tuning...
Upcoming SlideShare
Loading in …5
×

A Practical Event Driven Model

873 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
873
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • http://docs.sun.com/app/docs/doc/819-7051/6n919hpc2?l= zh&amp;a =view
  • http://www.vpnall.com/2009/05/gearman- 分布式远程过程处理框架 /
  • A Practical Event Driven Model

    1. 1. Thoughts on Events + A brief intro to a practicable events SDK – Gearman [email_address]
    2. 2. Events vs. Threading <ul><li>A historical quarrel </li></ul><ul><li>Actually individual things </li></ul><ul><ul><li>A Design model vs. An execution concept </li></ul></ul><ul><li>Killer App for threading </li></ul><ul><ul><li>Concurrency is requested </li></ul></ul><ul><ul><li>Overlap IO </li></ul></ul><ul><ul><li>Scientific computing </li></ul></ul>
    3. 3. Threads <ul><li>Grew from OS and evolved into user level </li></ul><ul><li>Managing concurrency </li></ul><ul><li>Multiple independent execution streams </li></ul><ul><li>Share states </li></ul><ul><li>Pre-emptive scheduling </li></ul><ul><li>Synchronization needed </li></ul><ul><ul><li>Computable or blocked at any instant </li></ul></ul>
    4. 4. Callbacks don’t work with locks <ul><li>Deadlocks: </li></ul><ul><ul><li>Self block </li></ul></ul><ul><ul><ul><li>Spinning on an obtained lock </li></ul></ul></ul><ul><ul><li>Recursive block </li></ul></ul><ul><ul><ul><li>Never call external functions which possibly depends on current module on any tours </li></ul></ul></ul><ul><ul><li>Cross block </li></ul></ul><ul><ul><ul><li>Typical dead lock </li></ul></ul></ul><ul><ul><li>Not yield between to attempts </li></ul></ul><ul><ul><ul><li>CPU scheduling </li></ul></ul></ul><ul><ul><ul><li>Can other threads have slices between two attempts from the same thread </li></ul></ul></ul><ul><li>Th1: PolChk()  ScanCall() </li></ul><ul><li>Th2: ScanCall()  Encrypt()  PolChk() </li></ul>
    5. 5. What’s “wrong” with Threads? <ul><li>Painful for many cases, </li></ul><ul><ul><li>Data structure and complexity, OS scheduling </li></ul></ul><ul><ul><li>PC? Mac? Embedded? </li></ul></ul><ul><ul><li>Debugging (spot reproducing?) </li></ul></ul><ul><ul><li>Tool (Static/run-time inspection, Locklint?) </li></ul></ul><ul><li>Prototyping, trial, fast iteration and delivery? </li></ul>
    6. 6. Events <ul><li>A process consists of the processing of a series of events. </li></ul><ul><ul><li>A single event being processed at any instant </li></ul></ul><ul><ul><li>Mathematically equivalent with threads </li></ul></ul><ul><ul><li>When interactions are requested, things are different </li></ul></ul><ul><ul><li>Each event is treated as an independent transaction </li></ul></ul><ul><ul><li>Pre-emption is not permitted </li></ul></ul><ul><ul><ul><li>FIFO ( buffer, priority on queue) </li></ul></ul></ul><ul><ul><li>Synchronization not needed explicitly </li></ul></ul><ul><ul><li>A simple and single access model to data </li></ul></ul>
    7. 7. Events (cont.) <ul><li>Dispatcher and Handlers need to be efficient </li></ul><ul><li>External event sequence is not under control </li></ul><ul><li>Dispatch routine </li></ul>
    8. 8. Good Practices for Events <ul><li>Do All Jobs in as events </li></ul><ul><ul><li>Micro kernel </li></ul></ul><ul><ul><li>One handler for each inputs (Really? A DO example?) </li></ul></ul><ul><li>Do not inhibit events </li></ul><ul><li>Stateful design </li></ul><ul><ul><li>Traditional methodology </li></ul></ul><ul><ul><li>Tools, debug, inspection, Fast iteration </li></ul></ul><ul><ul><li>Easy to test with varies means </li></ul></ul>
    9. 9. To be or not? <ul><li>Use threads only when there is a payoff </li></ul><ul><li>Killer indicator for threads? </li></ul><ul><ul><li>CPU saturation (used to) </li></ul></ul><ul><ul><li>concurrency is requested explicitly </li></ul></ul><ul><li>Which indicating an simple event model? </li></ul><ul><ul><li>Varied porting environments? </li></ul></ul><ul><ul><li>A stateful prototype for fast verification </li></ul></ul><ul><ul><li>Thread unsafe lib/algorithm </li></ul></ul><ul><li>Employ right model at exact layer and with exact benefits expectation </li></ul>
    10. 10. Farm out jobs with Gearman! <ul><li>Originally developed with Perl and rewritten in C </li></ul>
    11. 11. Farm out jobs with Gearman! <ul><li>Diverse API supports, PHP, Perl, C, Ruby, Java, Python, C# </li></ul><ul><li>Reusing Supports (Refactory on existing lib/process wrapper/shell/script) </li></ul><ul><li>DB persistency (mysql, postgreSQL) </li></ul><ul><li>Queue/Timing supports </li></ul>
    12. 12. Working model Both synchronous and asynchronous (event silent db persistency)
    13. 13. Deployment model <ul><li>Easy to extend with new boxes/CPU/Core# </li></ul><ul><li>Need to refine routine/balancing </li></ul><ul><li>Failover supports </li></ul><ul><li>Tuning workers/Job-servers run-time </li></ul>
    14. 14. More about Gearman <ul><li>Easy to deploy </li></ul><ul><li>A stateful protocol </li></ul><ul><ul><li>Seek gearman.org for protocol specificaitons </li></ul></ul><ul><li>Workers could serve as child process by gearman server </li></ul><ul><li>Easy to extend with customized module </li></ul><ul><li>Wrapping existing logics </li></ul><ul><li>Fasten Integration external job handlers </li></ul>
    15. 15. Example-1 <ul><li>Shell command dispatch </li></ul><ul><li>Client </li></ul><ul><ul><li>gearman -h 10.64.13.168 -p7003 -f wc < /welcome/gearmand.log </li></ul></ul><ul><li>Worker </li></ul><ul><ul><li>gearman -h 10.64.13.168 -p7003 -w -f wc -- wc -l </li></ul></ul><ul><li>Job-server </li></ul>
    16. 16. Example-2 <ul><li>Perl Handler and Worker </li></ul><ul><ul><li>Original document needs refines </li></ul></ul><ul><li>Client </li></ul>
    17. 17. Example-2 (cont.) <ul><li>Worker </li></ul><ul><li>Job Server remains </li></ul>
    18. 18. Example-3 <ul><li>Monitoring the server </li></ul><ul><li>Text cmd interaction </li></ul>
    19. 19. A Map/Reduce App – Log Processing <ul><li>Collection </li></ul><ul><ul><li>Tail –f access_log|gearman –n –f logger </li></ul></ul><ul><ul><li>Gearman Apache log module </li></ul></ul><ul><li>Processing </li></ul><ul><ul><li>Paralel grep </li></ul></ul><ul><ul><li>Data mining </li></ul></ul>
    20. 20. FIN <ul><li>Q/A </li></ul><ul><li>More thoughts on events dispatching </li></ul><ul><ul><li>Routing, Debug, Metric, Tuning </li></ul></ul><ul><li>Introducing events in some level </li></ul><ul><ul><li>UI </li></ul></ul><ul><ul><li>Log Processing </li></ul></ul><ul><ul><li>Routine Jobs (Dummy Tasks) </li></ul></ul><ul><ul><li>Out Sourcing Tasks (Asynch/Synch) </li></ul></ul>

    ×