Lazy Data Using Perl

Like this? Share it with your network

Share

Lazy Data Using Perl

  • 1,427 views
Uploaded on

Your data's lifecycle is an important part of making it usable. Lazy data is loaded once, when it is needed, and does not need to be checked every time it is used. This talk describes using......

Your data's lifecycle is an important part of making it usable. Lazy data is loaded once, when it is needed, and does not need to be checked every time it is used. This talk describes using Trampoline objects, subroutines and state variables to initialize the data structure only once.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,427
On Slideshare
1,427
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Virtuous Lazyness For Your Data Doing it once andknowing youve done it. Steve LembarkWorkhorse Computing lembark@wrkhors.com
  • 2. Data wants to be free, it also wants to be very expensive.● Data you never use is free; accessing gets expensive.● Managing datas cost requires controlling its lifecycle.● Mis­managing the lifecycle causes problems: ● Caching table lookups in forked httpd crashes your database. ● Pre­processing messages files kills your startup times. ● Reading XML configuration data you never use wastes most of  your testing budget. ● Your tests fail because of a connection failure to a database  server that you dont use in the test.● Fixing these problems requires using lazy data.
  • 3. False Lazyness● We have all see the two most common data  management stratagies: ● You load everything at startup to avoid checking it every  time it is used. ● You check everything every time before using it to avoid  loading it up front.● Both approaches ignore important knowledge: ● You know when the data is needed. ● You know that the data was loaded.
  • 4. One alternative: Scalar Cache● $cache ||= read_cache_values; ● Seems nice: You know that $cache is populated. ● This requires dereferencing a hashref throughout the  program, which is expensive. ● What youd rather do is just use $cache{ $foobar }  without having to check it every time.● Even checking if( ! %cache ) is expensive – arrays  are much cheaper to check.
  • 5. True Lazyness: Do Something Once● Truly Lazy data means loading you data when you  need it and knowing that it is loaded.● Perl gives us – of course – more than one way: ● Trampoline objects and subroutines. ● “state” variables, introduced in v5.10.● Trampolines are flyweight objects – data structures or  subroutines that transform themselves when used.● State variables are assigned only once, at runtime, the  first time they are used.
  • 6. Follow The Bouncing Object● Object::Trampoline – flyweight data you dont know  isnt there.● These delay calling the “real” constructor until the  object is actually used. ● Your constructor gets called before the first method call. ● At that point it can cache, parse, or compute whatever it  needs.● Spreads the cost of loading data set each over the  lifetime of a process.
  • 7. Example: Delay Expensive XML● Use an initializer to read and parse the XML.● The Object::Trampoline calls your constructor once  to transform the object the first time you call a  method.● Parsing the XML is pushed off until you actually use  the data.● Requires using a hashref – and possibly methods –  to access the data.
  • 8. package XML::Message;use Object::Trampoline;...sub new{ Object::Trampline->install( XML::Message, $path );}sub install{ my $error = &construct; $error->initialize( @_ )}sub initialize{ my ( $err, $path ) = @_; %$err = %{ XMLin $path => @lots_of_args }; $err}# calling translate bounces the trampoline exactly oncesub translate { … }
  • 9. Tramploline Subroutine● Similar to a trampoline object: A portion of the code  runs once and replaces itself.● One­shot code initializes the cache, which can now  be a simple hash.● Symbol::qualify_to_ref make this painless in Perl. ● An anonymous sub manages the cache. ● A named subroutine loads the cache, replaces itself with  the manager, and re­dispatches to the manager.
  • 10. The Simplest Versionmy %foo_cache = (); ● Minimal code my $handler= sub includes:{ my $foo = shift; ● Closed­over cache, %foo_cache{ $foo } Or die “Bogus foo: $foo unknown” ● Subref to permenant }; cache manager,sub do_something{ ● Initial subroutine. %cache = initialize_foo_cache; ● The initial subroutine  my $ref = qualify_to_ref do_something; initializes, installs,  *$ref = $handler; and re­dispatches. goto &$handler}
  • 11. BEGIN Blocks Are CleanerBEGIN{ my $name = foo_handler; ● The block isolates  my $ref = qualify_to_ref $name; my %cache = (); cache, ref, and  my $handler sub variables;  = sub { allows re­cycling  %cache{ $_[0] } or die ... }; the ref. *$ref = sub ● This is also rather  { %cache = init_the_cache; amenable to  *$ref = $handler; installation by  module. goto &$handler }}
  • 12. Sub::Trampolinesub install_trampoline{ my ( $name, $init, $mgr ) = @_; ● Aside from the actual  my $caller = caller; assignment, init code is  my $ref = qualify_to_ref $name, $caller; identical. *$ref ● Simply pass the name,  = sub { manager, and init  $init->(); assignment. &$ref = $mgr; ● The module can call  goto &$mgr } $init, replace itself.} ● Caller defines the  cache and handler.
  • 13. Using Sub::Trampolineuse Sub::Trampoline;my %cache1 = ();my @cache2 = ();my $cache1_mgr= sub { $cache1{ $_[0] } or croak "Unknown $_[0]" };my $cache2_mgr= sub { first { $_ eq $_[0] } @cache2 } };my $init_cache1 = sub { %cache1 = select_from_hell }sub init_cache2 { @cache2 = XMLin $nasty_messy_xml_struct }install_trampoline( subone => $init_cache1, $cache1_mgr );install_trampoline( known => &init_cache2, $cache2_mgr );my $value1 = subone $key1;if( known $value ){ … }else{ carp “Unknown: $value” }
  • 14. True One­Shot: Empty $handler● If you want to run something exactly once but dont  know where it might be called initially: my $manager = sub(){};● You can also substitute a trampoline object with a  constructor that does the work and no methods.● Calling the object once constructs it, after which the  classes constructor can stub itself.● Useful for sharing the cache variable: the init  populates it once and stubs itself to do nothing more.
  • 15. Cycling The Cache● There are times when you want to purge and re­ initialize the cache● A trampoline object with populate and use subs that  flip­flop can handle this easily.● Re­assigning a trampoline re­initializes the cache: $cache = Object::Trampoline->init_cache( $class => @argz ) if $age > $time_max;
  • 16. v5.10 Introduced “state” Variables● Scoped like a lexical.● Assinged once at runtime.● Maintain value within a single lexical context  throughout the program.● Assign the cache or assign a flag variable with the  side effect of populating the cache.● Currently supports only scalars.
  • 17. Obvious case: Assign the cache.● Assign the cache at runtime: sub cache_mangler { state $cache = init_cache; … }● $cache will be assigned at runtime, the first time  cache_mangler is called.● The value will be retained between calls.● Catch: $cache is only available within  cache_mangler, not outside of it.
  • 18. Initialize a Shared Cacheuse v5.10; ● Subs may want to share my %cache = (); a cache.sub init ● $y and $z are assigned { %k or %k = …; at most once per } executison when foo or sub foo { state $y = init; … } bar are called.sub bar { state $z = init; … }… ● The sanity check in init  only needs to be handled my $foo = foo bletch;my $bar = bar blort; at most per subroutine.
  • 19. Summary● True lazyness includes managing data.● Preloading it all or testing it at each step are not lazy.● Object::Trampoline provides one way.● Trampoline subroutines offer another approach.● v5.10 introduced state variables which provide a few  ways to initialize something once.