How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins

YAPC::Asia 2009 Tokyo 2009/9/10 Gosuke Miyashita

Speaker
Gosuke Miyashita （ mizzy ）
Technical manager at paperboy&co.
Have most kids in Japan Perl world?
3 kids.Even with Yappo.
4 th kid will be born in November.

Agenda
Asynchronous processing with event driven programming
Asynchronous processing with Danga::Socket
Overview of Perlbal asynchronous processing
How to write asynchronous Perlbal plugins
Summary

Event driven programming
A programming paradigm in which the flow of the program is determined by events
Counter paradigm of flow-based programming

Events
I/O
On I/O read ready or write ready
Timers
On time passed
Signal
On getting a signal
Child process
On child process exit

Main loop
Also called “event loop”
On each loop, check that events are occurred or not, and process the events.
After process the events, return to the loop and repeat the loop till next events occur
This is the flow of asynchronous processing

Eaxmple of main loop Process timers Wait I/O events Process I/O events Post main loop process

Events supported by Danga::Socket
I/O
Timers

Process I/O events
Danga::Socket supports 3 I/O event notification facility
kqueue
epoll
poll
Select adequate facility automatically

Add I/O watcher with Danga::Socket
use base ‘Danga::Socket’;
sub new {
# pass file descripter
$self->SUPER::new( $fd );
}
sub event_read {
# process when $fd is read ready
}
sub event_write {
# procdess when $fd is write ready
}

Add I/O wathcer (pattern 2)
Danga::Socket->AddOtherFds(
$fd => sub {
# process when $fd is read ready
},
);

Add I/O watcher (pattern 3)
Danga::Socket::Callback->new(
handle => $fd ,
on_read_ready => sub {
# process on read ready
},
on_write_ready => sub {
# process on write ready
},
);

Add timer watcher with Danga::Socket
Danga::Socket->AddTimer(
10,
sub {
# process after 10 seconds
},
}

Start main loop with Danga::Socket
Danga::Socket->EventLoop();

Main loop(again) Process Timers Wait I/O events Process I/O events Post main loop process

Perlbal mechanism(as reverse proxy) BackendHTTP ClientProxy TCPListener Client Server ClientProxy Client BackendHTTP Danga::Socket based objects

Perlbal with a plugin (on start_proxy_request hook) BackendHTTP ClientProxy TCPListener Client Server Plugin::Hoge Target of this session

Hooks
Perlbal has many plugin hooks
Explain with start_proxy_request hook in this session

In case of a synchronous plugin ClientProxy TCPListener Client Server Plugin::Sync Client

In case of an asynchronous plugin ClientProxy TCPListener Client Server Plugin::Async Client ClientProxy

How to write asynchronous plugins?
Write the main process based on Danga::Socket
process within the main loop of Danga::Socket
Create Danga::Socket based class
Or use Danga::Socket::Callback
Or use Danga::Socket::AddTimer
Also other libraries need to be non-blocking
You can use Gearman::Client::Async with blocking processes

How to write asynchronous plugins?
When finish asynchronous process, go to next phase
Call back function
Also need to fix Perlbal itself

Main process of a plugin
package My::Drizzle ;
use base ‘Danga::Socket’ ;
use Net::Drizzle ':constants' ;
sub new {
$self->SUPER::new($fh);
$self->watch_read(1);
}
sub event_read {
# Check db request status and call back when finished
}

Attention
You can also use Danga::Socket::Callback
You cannot use AddOtherFds()
File descriptors added by Add AddOtherFds() are only evaluated at the beginning of the main loop
So if already in the main loop, AddOtherFds() are meaningless.
You can call epoll_ctl, EV_SET and so on directly, but no portability

plugin register
package Perlbal::Plugin::AsyncDb ;
sub register {
my ( $class, $svc ) = @_;
$svc->register_hook(
'Async' => 'start_proxy_request',
&request_db ,
);
return 1;
}

Call asynchronous process
sub request_db {
my Perlbal::ClientProxy $client = shift ;
My::Drizzle->new(
callback => sub {
# call back
},
);
return 1; # very important!
}

In case of “return 0” BackendHTTP ClientProxy TCPListener Client Server Plugin::Async Go to next step even if Plugin::Async is not finished

“ return 1” & “call back” BackendHTTP ClientProxy TCPListener Client Plugin::Async Stop the process and go to main loop Server return 1 call back

Call back function
If plugin process finished, ClientProxy must go to the next process(call BackendHTTP)
Stopped at the following code of handle_request() :
return if $svc->run_hook(
'start_proxy_request', $self
);
ClientProxy must start processing from the next of this code

Call back function
my Perlbal::ClientProxy
$client = shift ;
My::Drizzle->new(
callback => sub {
$client->{async_complete} = 1;
$client->handle_request;
},
);

Patch of ClientProxy::handle_request
- return if $svc->run_hook(
- 'start_proxy_request', $self
- );
+ unless ( $self->{async_complete} ) {
+ return if $svc->run_hook(
+ 'start_proxy_request', $self
+ );
+ }
+ $self->{async_complete} = 0;

Points of asynchronous Perlbal plugins
Process within the Danga::Socket main loop
return 1 when plugin called
Restart ClientProxy process by a call back function when plugin finished
Also need to fix Perlbal itself
Be careful with order of plugins
Following plugins on a same hook are ignored when after a plugin returns 1

http://coderwall.com	88
http://www.slideshare.net	11
http://logpi.jp	5

How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins

by Gosuke Miyashita on Sep 10, 2009

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 104

Statistics

How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins Presentation Transcript