• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins
 

How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins

on

  • 3,945 views

 

Statistics

Views

Total Views
3,945
Views on SlideShare
3,841
Embed Views
104

Actions

Likes
3
Downloads
14
Comments
2

3 Embeds 104

http://coderwall.com 88
http://www.slideshare.net 11
http://logpi.jp 5

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins Presentation Transcript

    • YAPC::Asia 2009 Tokyo 2009/9/10 Gosuke Miyashita
    • Speaker
      • Gosuke Miyashita ( mizzy )
      • Technical manager at paperboy&co.
      • Have most kids in Japan Perl world?
        • 3 kids.Even with Yappo.
        • 4 th kid will be born in November.
    •  
    •  
    • Agenda
      • Asynchronous processing with event driven programming
      • Asynchronous processing with Danga::Socket
      • Overview of Perlbal asynchronous processing
      • How to write asynchronous Perlbal plugins
      • Summary
    •  
    • Event driven programming
      • A programming paradigm in which the flow of the program is determined by events
      • Counter paradigm of flow-based programming
    • Events
      • I/O
        • On I/O read ready or write ready
      • Timers
        • On time passed
      • Signal
        • On getting a signal
      • Child process
        • On child process exit
    • Main loop
      • Also called “event loop”
      • On each loop, check that events are occurred or not, and process the events.
      • After process the events, return to the loop and repeat the loop till next events occur
      • This is the flow of asynchronous processing
    • Eaxmple of main loop Process timers Wait I/O events Process I/O events Post main loop process
    •  
    • Events supported by Danga::Socket
      • I/O
      • Timers
    • Process I/O events
      • Danga::Socket supports 3 I/O event notification facility
        • kqueue
        • epoll
        • poll
      • Select adequate facility automatically
    • Add I/O watcher with Danga::Socket
      • use base ‘Danga::Socket’;
      • sub new {
      • # pass file descripter
      • $self->SUPER::new( $fd );
      • }
      • sub event_read {
      • # process when $fd is read ready
      • }
      • sub event_write {
      • # procdess when $fd is write ready
      • }
    • Add I/O wathcer (pattern 2)
      • Danga::Socket->AddOtherFds(
      • $fd => sub {
      • # process when $fd is read ready
      • },
      • );
    • Add I/O watcher (pattern 3)
      • Danga::Socket::Callback->new(
      • handle => $fd ,
      • on_read_ready => sub {
      • # process on read ready
      • },
      • on_write_ready => sub {
      • # process on write ready
      • },
      • );
    • Add timer watcher with Danga::Socket
      • Danga::Socket->AddTimer(
      • 10,
      • sub {
      • # process after 10 seconds
      • },
      • }
    • Start main loop with Danga::Socket
      • Danga::Socket->EventLoop();
    • Main loop(again) Process Timers Wait I/O events Process I/O events Post main loop process
    •  
    • Perlbal mechanism(as reverse proxy) BackendHTTP ClientProxy TCPListener Client Server ClientProxy Client BackendHTTP Danga::Socket based objects
    • Perlbal with a plugin (on start_proxy_request hook) BackendHTTP ClientProxy TCPListener Client Server Plugin::Hoge Target of this session
    •  
    • Hooks
      • Perlbal has many plugin hooks
      • Explain with start_proxy_request hook in this session
    • In case of a synchronous plugin ClientProxy TCPListener Client Server Plugin::Sync Client
    • In case of an asynchronous plugin ClientProxy TCPListener Client Server Plugin::Async Client ClientProxy
    • How to write asynchronous plugins?
      • Write the main process based on Danga::Socket
        • process within the main loop of Danga::Socket
        • Create Danga::Socket based class
        • Or use Danga::Socket::Callback
        • Or use Danga::Socket::AddTimer
        • Also other libraries need to be non-blocking
          • You can use Gearman::Client::Async with blocking processes
    • How to write asynchronous plugins?
      • When finish asynchronous process, go to next phase
        • Call back function
        • Also need to fix Perlbal itself
    • Main process of a plugin
      • package My::Drizzle ;
      • use base ‘Danga::Socket’ ;
      • use Net::Drizzle ':constants' ;
      • sub new {
      • $self->SUPER::new($fh);
      • $self->watch_read(1);
      • }
      • sub event_read {
      • # Check db request status and call back when finished
      • }
    • Attention
      • You can also use Danga::Socket::Callback
      • You cannot use AddOtherFds()
        • File descriptors added by Add AddOtherFds() are only evaluated at the beginning of the main loop
        • So if already in the main loop, AddOtherFds() are meaningless.
      • You can call epoll_ctl, EV_SET and so on directly, but no portability
    • plugin register
      • package Perlbal::Plugin::AsyncDb ;
      • sub register {
      • my ( $class, $svc ) = @_;
      • $svc->register_hook(
      • 'Async' => 'start_proxy_request',
      • &request_db ,
      • );
      • return 1;
      • }
    • Call asynchronous process
      • sub request_db {
      • my Perlbal::ClientProxy $client = shift ;
      • My::Drizzle->new(
      • callback => sub {
      • # call back
      • },
      • );
      • return 1; # very important!
      • }
    • In case of “return 0” BackendHTTP ClientProxy TCPListener Client Server Plugin::Async Go to next step even if Plugin::Async is not finished
    • “ return 1” & “call back” BackendHTTP ClientProxy TCPListener Client Plugin::Async Stop the process and go to main loop Server return 1 call back
    • Call back function
      • If plugin process finished, ClientProxy must go to the next process(call BackendHTTP)
      • Stopped at the following code of handle_request() :
      • return if $svc->run_hook(
      • 'start_proxy_request', $self
      • );
      • ClientProxy must start processing from the next of this code
    • Call back function
      • my Perlbal::ClientProxy
      • $client = shift ;
      • My::Drizzle->new(
      • callback => sub {
      • $client->{async_complete} = 1;
      • $client->handle_request;
      • },
      • );
    • Patch of ClientProxy::handle_request
      • - return if $svc->run_hook(
      • - 'start_proxy_request', $self
      • - );
      • + unless ( $self->{async_complete} ) {
      • + return if $svc->run_hook(
      • + 'start_proxy_request', $self
      • + );
      • + }
      • + $self->{async_complete} = 0;
    •  
    • Points of asynchronous Perlbal plugins
      • Process within the Danga::Socket main loop
      • return 1 when plugin called
      • Restart ClientProxy process by a call back function when plugin finished
      • Also need to fix Perlbal itself
      • Be careful with order of plugins
        • Following plugins on a same hook are ignored when after a plugin returns 1
    •