How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins

  • 2,535 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
2,535
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
14
Comments
2
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. YAPC::Asia 2009 Tokyo 2009/9/10 Gosuke Miyashita
  • 2. Speaker
    • Gosuke Miyashita ( mizzy )
    • Technical manager at paperboy&co.
    • Have most kids in Japan Perl world?
      • 3 kids.Even with Yappo.
      • 4 th kid will be born in November.
  • 3.  
  • 4.  
  • 5. Agenda
    • Asynchronous processing with event driven programming
    • Asynchronous processing with Danga::Socket
    • Overview of Perlbal asynchronous processing
    • How to write asynchronous Perlbal plugins
    • Summary
  • 6.  
  • 7. Event driven programming
    • A programming paradigm in which the flow of the program is determined by events
    • Counter paradigm of flow-based programming
  • 8. Events
    • I/O
      • On I/O read ready or write ready
    • Timers
      • On time passed
    • Signal
      • On getting a signal
    • Child process
      • On child process exit
  • 9. Main loop
    • Also called “event loop”
    • On each loop, check that events are occurred or not, and process the events.
    • After process the events, return to the loop and repeat the loop till next events occur
    • This is the flow of asynchronous processing
  • 10. Eaxmple of main loop Process timers Wait I/O events Process I/O events Post main loop process
  • 11.  
  • 12. Events supported by Danga::Socket
    • I/O
    • Timers
  • 13. Process I/O events
    • Danga::Socket supports 3 I/O event notification facility
      • kqueue
      • epoll
      • poll
    • Select adequate facility automatically
  • 14. Add I/O watcher with Danga::Socket
    • use base ‘Danga::Socket’;
    • sub new {
    • # pass file descripter
    • $self->SUPER::new( $fd );
    • }
    • sub event_read {
    • # process when $fd is read ready
    • }
    • sub event_write {
    • # procdess when $fd is write ready
    • }
  • 15. Add I/O wathcer (pattern 2)
    • Danga::Socket->AddOtherFds(
    • $fd => sub {
    • # process when $fd is read ready
    • },
    • );
  • 16. Add I/O watcher (pattern 3)
    • Danga::Socket::Callback->new(
    • handle => $fd ,
    • on_read_ready => sub {
    • # process on read ready
    • },
    • on_write_ready => sub {
    • # process on write ready
    • },
    • );
  • 17. Add timer watcher with Danga::Socket
    • Danga::Socket->AddTimer(
    • 10,
    • sub {
    • # process after 10 seconds
    • },
    • }
  • 18. Start main loop with Danga::Socket
    • Danga::Socket->EventLoop();
  • 19. Main loop(again) Process Timers Wait I/O events Process I/O events Post main loop process
  • 20.  
  • 21. Perlbal mechanism(as reverse proxy) BackendHTTP ClientProxy TCPListener Client Server ClientProxy Client BackendHTTP Danga::Socket based objects
  • 22. Perlbal with a plugin (on start_proxy_request hook) BackendHTTP ClientProxy TCPListener Client Server Plugin::Hoge Target of this session
  • 23.  
  • 24. Hooks
    • Perlbal has many plugin hooks
    • Explain with start_proxy_request hook in this session
  • 25. In case of a synchronous plugin ClientProxy TCPListener Client Server Plugin::Sync Client
  • 26. In case of an asynchronous plugin ClientProxy TCPListener Client Server Plugin::Async Client ClientProxy
  • 27. How to write asynchronous plugins?
    • Write the main process based on Danga::Socket
      • process within the main loop of Danga::Socket
      • Create Danga::Socket based class
      • Or use Danga::Socket::Callback
      • Or use Danga::Socket::AddTimer
      • Also other libraries need to be non-blocking
        • You can use Gearman::Client::Async with blocking processes
  • 28. How to write asynchronous plugins?
    • When finish asynchronous process, go to next phase
      • Call back function
      • Also need to fix Perlbal itself
  • 29. Main process of a plugin
    • package My::Drizzle ;
    • use base ‘Danga::Socket’ ;
    • use Net::Drizzle ':constants' ;
    • sub new {
    • $self->SUPER::new($fh);
    • $self->watch_read(1);
    • }
    • sub event_read {
    • # Check db request status and call back when finished
    • }
  • 30. Attention
    • You can also use Danga::Socket::Callback
    • You cannot use AddOtherFds()
      • File descriptors added by Add AddOtherFds() are only evaluated at the beginning of the main loop
      • So if already in the main loop, AddOtherFds() are meaningless.
    • You can call epoll_ctl, EV_SET and so on directly, but no portability
  • 31. plugin register
    • package Perlbal::Plugin::AsyncDb ;
    • sub register {
    • my ( $class, $svc ) = @_;
    • $svc->register_hook(
    • 'Async' => 'start_proxy_request',
    • &request_db ,
    • );
    • return 1;
    • }
  • 32. Call asynchronous process
    • sub request_db {
    • my Perlbal::ClientProxy $client = shift ;
    • My::Drizzle->new(
    • callback => sub {
    • # call back
    • },
    • );
    • return 1; # very important!
    • }
  • 33. In case of “return 0” BackendHTTP ClientProxy TCPListener Client Server Plugin::Async Go to next step even if Plugin::Async is not finished
  • 34. “ return 1” & “call back” BackendHTTP ClientProxy TCPListener Client Plugin::Async Stop the process and go to main loop Server return 1 call back
  • 35. Call back function
    • If plugin process finished, ClientProxy must go to the next process(call BackendHTTP)
    • Stopped at the following code of handle_request() :
    • return if $svc->run_hook(
    • 'start_proxy_request', $self
    • );
    • ClientProxy must start processing from the next of this code
  • 36. Call back function
    • my Perlbal::ClientProxy
    • $client = shift ;
    • My::Drizzle->new(
    • callback => sub {
    • $client->{async_complete} = 1;
    • $client->handle_request;
    • },
    • );
  • 37. Patch of ClientProxy::handle_request
    • - return if $svc->run_hook(
    • - 'start_proxy_request', $self
    • - );
    • + unless ( $self->{async_complete} ) {
    • + return if $svc->run_hook(
    • + 'start_proxy_request', $self
    • + );
    • + }
    • + $self->{async_complete} = 0;
  • 38.  
  • 39. Points of asynchronous Perlbal plugins
    • Process within the Danga::Socket main loop
    • return 1 when plugin called
    • Restart ClientProxy process by a call back function when plugin finished
    • Also need to fix Perlbal itself
    • Be careful with order of plugins
      • Following plugins on a same hook are ignored when after a plugin returns 1
  • 40.