How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins
Upcoming SlideShare
Loading in...5
×
 

How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins

on

  • 4,044 views

 

Statistics

Views

Total Views
4,044
Views on SlideShare
3,940
Embed Views
104

Actions

Likes
3
Downloads
14
Comments
2

3 Embeds 104

http://coderwall.com 88
http://www.slideshare.net 11
http://logpi.jp 5

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins Presentation Transcript

  • YAPC::Asia 2009 Tokyo 2009/9/10 Gosuke Miyashita
  • Speaker
    • Gosuke Miyashita ( mizzy )
    • Technical manager at paperboy&co.
    • Have most kids in Japan Perl world?
      • 3 kids.Even with Yappo.
      • 4 th kid will be born in November.
  •  
  •  
  • Agenda
    • Asynchronous processing with event driven programming
    • Asynchronous processing with Danga::Socket
    • Overview of Perlbal asynchronous processing
    • How to write asynchronous Perlbal plugins
    • Summary
  •  
  • Event driven programming
    • A programming paradigm in which the flow of the program is determined by events
    • Counter paradigm of flow-based programming
  • Events
    • I/O
      • On I/O read ready or write ready
    • Timers
      • On time passed
    • Signal
      • On getting a signal
    • Child process
      • On child process exit
  • Main loop
    • Also called “event loop”
    • On each loop, check that events are occurred or not, and process the events.
    • After process the events, return to the loop and repeat the loop till next events occur
    • This is the flow of asynchronous processing
  • Eaxmple of main loop Process timers Wait I/O events Process I/O events Post main loop process
  •  
  • Events supported by Danga::Socket
    • I/O
    • Timers
  • Process I/O events
    • Danga::Socket supports 3 I/O event notification facility
      • kqueue
      • epoll
      • poll
    • Select adequate facility automatically
  • Add I/O watcher with Danga::Socket
    • use base ‘Danga::Socket’;
    • sub new {
    • # pass file descripter
    • $self->SUPER::new( $fd );
    • }
    • sub event_read {
    • # process when $fd is read ready
    • }
    • sub event_write {
    • # procdess when $fd is write ready
    • }
  • Add I/O wathcer (pattern 2)
    • Danga::Socket->AddOtherFds(
    • $fd => sub {
    • # process when $fd is read ready
    • },
    • );
  • Add I/O watcher (pattern 3)
    • Danga::Socket::Callback->new(
    • handle => $fd ,
    • on_read_ready => sub {
    • # process on read ready
    • },
    • on_write_ready => sub {
    • # process on write ready
    • },
    • );
  • Add timer watcher with Danga::Socket
    • Danga::Socket->AddTimer(
    • 10,
    • sub {
    • # process after 10 seconds
    • },
    • }
  • Start main loop with Danga::Socket
    • Danga::Socket->EventLoop();
  • Main loop(again) Process Timers Wait I/O events Process I/O events Post main loop process
  •  
  • Perlbal mechanism(as reverse proxy) BackendHTTP ClientProxy TCPListener Client Server ClientProxy Client BackendHTTP Danga::Socket based objects
  • Perlbal with a plugin (on start_proxy_request hook) BackendHTTP ClientProxy TCPListener Client Server Plugin::Hoge Target of this session
  •  
  • Hooks
    • Perlbal has many plugin hooks
    • Explain with start_proxy_request hook in this session
  • In case of a synchronous plugin ClientProxy TCPListener Client Server Plugin::Sync Client
  • In case of an asynchronous plugin ClientProxy TCPListener Client Server Plugin::Async Client ClientProxy
  • How to write asynchronous plugins?
    • Write the main process based on Danga::Socket
      • process within the main loop of Danga::Socket
      • Create Danga::Socket based class
      • Or use Danga::Socket::Callback
      • Or use Danga::Socket::AddTimer
      • Also other libraries need to be non-blocking
        • You can use Gearman::Client::Async with blocking processes
  • How to write asynchronous plugins?
    • When finish asynchronous process, go to next phase
      • Call back function
      • Also need to fix Perlbal itself
  • Main process of a plugin
    • package My::Drizzle ;
    • use base ‘Danga::Socket’ ;
    • use Net::Drizzle ':constants' ;
    • sub new {
    • $self->SUPER::new($fh);
    • $self->watch_read(1);
    • }
    • sub event_read {
    • # Check db request status and call back when finished
    • }
  • Attention
    • You can also use Danga::Socket::Callback
    • You cannot use AddOtherFds()
      • File descriptors added by Add AddOtherFds() are only evaluated at the beginning of the main loop
      • So if already in the main loop, AddOtherFds() are meaningless.
    • You can call epoll_ctl, EV_SET and so on directly, but no portability
  • plugin register
    • package Perlbal::Plugin::AsyncDb ;
    • sub register {
    • my ( $class, $svc ) = @_;
    • $svc->register_hook(
    • 'Async' => 'start_proxy_request',
    • &request_db ,
    • );
    • return 1;
    • }
  • Call asynchronous process
    • sub request_db {
    • my Perlbal::ClientProxy $client = shift ;
    • My::Drizzle->new(
    • callback => sub {
    • # call back
    • },
    • );
    • return 1; # very important!
    • }
  • In case of “return 0” BackendHTTP ClientProxy TCPListener Client Server Plugin::Async Go to next step even if Plugin::Async is not finished
  • “ return 1” & “call back” BackendHTTP ClientProxy TCPListener Client Plugin::Async Stop the process and go to main loop Server return 1 call back
  • Call back function
    • If plugin process finished, ClientProxy must go to the next process(call BackendHTTP)
    • Stopped at the following code of handle_request() :
    • return if $svc->run_hook(
    • 'start_proxy_request', $self
    • );
    • ClientProxy must start processing from the next of this code
  • Call back function
    • my Perlbal::ClientProxy
    • $client = shift ;
    • My::Drizzle->new(
    • callback => sub {
    • $client->{async_complete} = 1;
    • $client->handle_request;
    • },
    • );
  • Patch of ClientProxy::handle_request
    • - return if $svc->run_hook(
    • - 'start_proxy_request', $self
    • - );
    • + unless ( $self->{async_complete} ) {
    • + return if $svc->run_hook(
    • + 'start_proxy_request', $self
    • + );
    • + }
    • + $self->{async_complete} = 0;
  •  
  • Points of asynchronous Perlbal plugins
    • Process within the Danga::Socket main loop
    • return 1 when plugin called
    • Restart ClientProxy process by a call back function when plugin finished
    • Also need to fix Perlbal itself
    • Be careful with order of plugins
      • Following plugins on a same hook are ignored when after a plugin returns 1
  •