SlideShare a Scribd company logo
1 of 19
Download to read offline
Virtuous Lazyness
  For Your Data

  Doing it once and
knowing you've done it.

  Steve Lembark
Workhorse Computing
 lembark@wrkhors.com
Data wants to be free,
    it also wants to be very expensive.
●   Data you never use is free; accessing gets expensive.
●   Managing data's cost requires controlling its lifecycle.
●   Mis­managing the lifecycle causes problems:
    ●   Caching table lookups in forked httpd crashes your database.
    ●   Pre­processing messages files kills your startup times.
    ●   Reading XML configuration data you never use wastes most of 
        your testing budget.
    ●   Your tests fail because of a connection failure to a database 
        server that you don't use in the test.
●   Fixing these problems requires using lazy data.
False Lazyness
●   We have all see the two most common data 
    management stratagies:
    ●   You load everything at startup to avoid checking it every 
        time it is used.
    ●   You check everything every time before using it to avoid 
        loading it up front.
●   Both approaches ignore important knowledge:
    ●   You know when the data is needed.
    ●   You know that the data was loaded.
One alternative: Scalar Cache
●   $cache ||= read_cache_values;
    ●   Seems nice: You know that $cache is populated.
    ●   This requires dereferencing a hashref throughout the 
        program, which is expensive. 
●   What you'd rather do is just use $cache{ $foobar } 
    without having to check it every time.
●   Even checking if( ! %cache ) is expensive – arrays 
    are much cheaper to check.
True Lazyness:
                Do Something Once
●   Truly Lazy data means loading you data when you 
    need it and knowing that it is loaded.
●   Perl gives us – of course – more than one way:
    ●   Trampoline objects and subroutines.
    ●   “state” variables, introduced in v5.10.
●   Trampolines are flyweight objects – data structures or 
    subroutines that transform themselves when used.
●   State variables are assigned only once, at runtime, the 
    first time they are used.
Follow The Bouncing Object
●   Object::Trampoline – flyweight data you don't know 
    isn't there.
●   These delay calling the “real” constructor until the 
    object is actually used.
    ●   Your constructor gets called before the first method call.
    ●   At that point it can cache, parse, or compute whatever it 
        needs.
●   Spreads the cost of loading data set each over the 
    lifetime of a process.
Example: Delay Expensive XML
●   Use an initializer to read and parse the XML.
●   The Object::Trampoline calls your constructor once 
    to transform the object the first time you call a 
    method.
●   Parsing the XML is pushed off until you actually use 
    the data.
●   Requires using a hashref – and possibly methods – 
    to access the data.
package XML::Message;
use Object::Trampoline;
...

sub new
{
    Object::Trampline->install( 'XML::Message', $path );
}

sub install
{
    my $error = &construct;

    $error->initialize( @_ )
}

sub initialize
{
    my ( $err, $path ) = @_;

    %$err    = %{ XMLin $path => @lots_of_args };

    $err
}

# calling translate bounces the trampoline exactly once

sub translate { … }
Tramploline Subroutine
●   Similar to a trampoline object: A portion of the code 
    runs once and replaces itself.
●   One­shot code initializes the cache, which can now 
    be a simple hash.
●   Symbol::qualify_to_ref make this painless in Perl.
    ●   An anonymous sub manages the cache.
    ●   A named subroutine loads the cache, replaces itself with 
        the manager, and re­dispatches to the manager.
The Simplest Version
my %foo_cache = ();
                                          ●   Minimal code 
my $handler
= sub                                         includes:
{
   my $foo = shift;                           ●   Closed­over cache,
     %foo_cache{ $foo }
     Or die “Bogus foo: '$foo' unknown”
                                              ●   Subref to permenant 
};                                                cache manager,
sub do_something
{
                                              ●   Initial subroutine.
   %cache = initialize_foo_cache;
                                          ●   The initial subroutine 
     my $ref
     = qualify_to_ref 'do_something';         initializes, installs, 
     *$ref   = $handler;
                                              and re­dispatches.
     goto &$handler
}
BEGIN Blocks Are Cleaner
BEGIN
{
   my $name = 'foo_handler';           ●   The block isolates 
   my $ref   = qualify_to_ref $name;
   my %cache = ();                         cache, ref, and 
    my $handler                            sub variables; 
    = sub
    {                                      allows re­cycling 
       %cache{ $_[0] } or die ...
    };                                     the ref.
    *$ref
    = sub
                                       ●   This is also rather 
    {
       %cache   = init_the_cache;
                                           amenable to 
        *$ref   = $handler;
                                           installation by 
                                           module.
        goto &$handler
    }
}
Sub::Trampoline
sub install_trampoline
{
   my ( $name, $init, $mgr ) = @_;
                                       ●   Aside from the actual 
    my $caller = caller;
                                           assignment, init code is 
    my $ref
    = qualify_to_ref $name, $caller;
                                           identical.
    *$ref                              ●   Simply pass the name, 
    = sub
    {                                      manager, and init 
       $init->();
                                           assignment.
        &$ref   = $mgr;
                                       ●   The module can call 
        goto &$mgr
    }                                      $init, replace itself.
}
                                       ●   Caller defines the 
                                           cache and handler.
Using Sub::Trampoline
use Sub::Trampoline;

my %cache1   = ();
my @cache2   = ();

my $cache1_mgr
= sub { $cache1{ $_[0] } or croak "Unknown '$_[0]'" };

my $cache2_mgr
= sub { first { $_ eq $_[0] } @cache2 } };

my $init_cache1 = sub { %cache1 = select_from_hell }
sub init_cache2 { @cache2 = XMLin $nasty_messy_xml_struct }

install_trampoline( subone     => $init_cache1, $cache1_mgr );
install_trampoline( known      => &init_cache2, $cache2_mgr );

my $value1   = subone $key1;

if( known $value )
{ … }
else
{ carp “Unknown: '$value'” }
True One­Shot: Empty $handler
●   If you want to run something exactly once but don't 
    know where it might be called initially:
      my $manager = sub(){};
●   You can also substitute a trampoline object with a 
    constructor that does the work and no methods.
●   Calling the object once constructs it, after which the 
    classes constructor can stub itself.
●   Useful for sharing the cache variable: the init 
    populates it once and stubs itself to do nothing more.
Cycling The Cache
●   There are times when you want to purge and re­
    initialize the cache
●   A trampoline object with populate and use subs that 
    flip­flop can handle this easily.
●   Re­assigning a trampoline re­initializes the cache:

    $cache
    = Object::Trampoline->init_cache( $class => @argz )
    if $age > $time_max;
v5.10 Introduced “state” Variables
●   Scoped like a lexical.
●   Assinged once at runtime.
●   Maintain value within a single lexical context 
    throughout the program.
●   Assign the cache or assign a flag variable with the 
    side effect of populating the cache.
●   Currently supports only scalars.
Obvious case: Assign the cache.
●   Assign the cache at runtime:
    sub cache_mangler { state $cache = init_cache; … }
●   $cache will be assigned at runtime, the first time 
    cache_mangler is called.
●   The value will be retained between calls.
●   Catch: $cache is only available within 
    cache_mangler, not outside of it.
Initialize a Shared Cache
use v5.10;                       ●   Subs may want to share 
my %cache    = ();                   a cache.
sub init                         ●   $y and $z are assigned 
{
   %k or %k = …;                     at most once per 
}
                                     executison when foo or 
sub foo { state $y = init; … }       bar are called.
sub bar { state $z = init; … }

…
                                 ●   The sanity check in init 
                                     only needs to be handled 
my $foo = foo 'bletch';
my $bar = bar 'blort';               at most per subroutine.
Summary
●   True lazyness includes managing data.
●   Preloading it all or testing it at each step are not lazy.
●   Object::Trampoline provides one way.
●   Trampoline subroutines offer another approach.
●   v5.10 introduced state variables which provide a few 
    ways to initialize something once.

More Related Content

What's hot

PHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolvePHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolveXSolve
 
Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)Fabien Potencier
 
SPL: The Missing Link in Development
SPL: The Missing Link in DevelopmentSPL: The Missing Link in Development
SPL: The Missing Link in Developmentjsmith92
 
Speed up your developments with Symfony2
Speed up your developments with Symfony2Speed up your developments with Symfony2
Speed up your developments with Symfony2Hugo Hamon
 
Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)Kang-min Liu
 
Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)brian d foy
 
Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015Lin Yo-An
 
Looping the Loop with SPL Iterators
Looping the Loop with SPL IteratorsLooping the Loop with SPL Iterators
Looping the Loop with SPL IteratorsMark Baker
 
PHP Performance Trivia
PHP Performance TriviaPHP Performance Trivia
PHP Performance TriviaNikita Popov
 
Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...Puppet
 
Learning Perl 6
Learning Perl 6 Learning Perl 6
Learning Perl 6 brian d foy
 
Zend Certification Preparation Tutorial
Zend Certification Preparation TutorialZend Certification Preparation Tutorial
Zend Certification Preparation TutorialLorna Mitchell
 
Rich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 ApplicationRich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 ApplicationKirill Chebunin
 
Introduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 TokyoIntroduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 TokyoMasahiro Nagano
 
What's New in Perl? v5.10 - v5.16
What's New in Perl?  v5.10 - v5.16What's New in Perl?  v5.10 - v5.16
What's New in Perl? v5.10 - v5.16Ricardo Signes
 
Object Calisthenics Applied to PHP
Object Calisthenics Applied to PHPObject Calisthenics Applied to PHP
Object Calisthenics Applied to PHPGuilherme Blanco
 

What's hot (20)

PHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolvePHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolve
 
Php & my sql
Php & my sqlPhp & my sql
Php & my sql
 
Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)
 
SPL: The Missing Link in Development
SPL: The Missing Link in DevelopmentSPL: The Missing Link in Development
SPL: The Missing Link in Development
 
Speed up your developments with Symfony2
Speed up your developments with Symfony2Speed up your developments with Symfony2
Speed up your developments with Symfony2
 
Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)
 
Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)
 
Web 4 | Core JavaScript
Web 4 | Core JavaScriptWeb 4 | Core JavaScript
Web 4 | Core JavaScript
 
Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015
 
Looping the Loop with SPL Iterators
Looping the Loop with SPL IteratorsLooping the Loop with SPL Iterators
Looping the Loop with SPL Iterators
 
PHP Performance Trivia
PHP Performance TriviaPHP Performance Trivia
PHP Performance Trivia
 
Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...
 
Web 8 | Introduction to PHP
Web 8 | Introduction to PHPWeb 8 | Introduction to PHP
Web 8 | Introduction to PHP
 
Learning Perl 6
Learning Perl 6 Learning Perl 6
Learning Perl 6
 
Zend Certification Preparation Tutorial
Zend Certification Preparation TutorialZend Certification Preparation Tutorial
Zend Certification Preparation Tutorial
 
Rich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 ApplicationRich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 Application
 
Introduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 TokyoIntroduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 Tokyo
 
PerlScripting
PerlScriptingPerlScripting
PerlScripting
 
What's New in Perl? v5.10 - v5.16
What's New in Perl?  v5.10 - v5.16What's New in Perl?  v5.10 - v5.16
What's New in Perl? v5.10 - v5.16
 
Object Calisthenics Applied to PHP
Object Calisthenics Applied to PHPObject Calisthenics Applied to PHP
Object Calisthenics Applied to PHP
 

Similar to Lazy Data Using Perl

Object::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your CodeObject::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your CodeWorkhorse Computing
 
Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4Jeff Carouth
 
Auto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK NodesAuto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK Nodesnihiliad
 
Lithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate FrameworksLithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate FrameworksNate Abele
 
Drupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency InjectionDrupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency InjectionPhilip Norton
 
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)arcware
 
PHP 5.3 Overview
PHP 5.3 OverviewPHP 5.3 Overview
PHP 5.3 Overviewjsmith92
 
Introducing PHP Latest Updates
Introducing PHP Latest UpdatesIntroducing PHP Latest Updates
Introducing PHP Latest UpdatesIftekhar Eather
 
Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)andrewnacin
 
The promise of asynchronous php
The promise of asynchronous phpThe promise of asynchronous php
The promise of asynchronous phpWim Godden
 
How Kris Writes Symfony Apps
How Kris Writes Symfony AppsHow Kris Writes Symfony Apps
How Kris Writes Symfony AppsKris Wallsmith
 
WordPress plugin #3
WordPress plugin #3WordPress plugin #3
WordPress plugin #3giwoolee
 
Dependency injection in Drupal 8
Dependency injection in Drupal 8Dependency injection in Drupal 8
Dependency injection in Drupal 8Alexei Gorobets
 
Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.Workhorse Computing
 
Perl web frameworks
Perl web frameworksPerl web frameworks
Perl web frameworksdiego_k
 
Curscatalyst
CurscatalystCurscatalyst
CurscatalystKar Juan
 

Similar to Lazy Data Using Perl (20)

Object::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your CodeObject::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your Code
 
Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4
 
Subroutines
SubroutinesSubroutines
Subroutines
 
Mojolicious
MojoliciousMojolicious
Mojolicious
 
Auto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK NodesAuto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK Nodes
 
Lithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate FrameworksLithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate Frameworks
 
Drupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency InjectionDrupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency Injection
 
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
 
PHP 5.3 Overview
PHP 5.3 OverviewPHP 5.3 Overview
PHP 5.3 Overview
 
Magic methods
Magic methodsMagic methods
Magic methods
 
Introducing PHP Latest Updates
Introducing PHP Latest UpdatesIntroducing PHP Latest Updates
Introducing PHP Latest Updates
 
Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)
 
The promise of asynchronous php
The promise of asynchronous phpThe promise of asynchronous php
The promise of asynchronous php
 
How Kris Writes Symfony Apps
How Kris Writes Symfony AppsHow Kris Writes Symfony Apps
How Kris Writes Symfony Apps
 
WordPress plugin #3
WordPress plugin #3WordPress plugin #3
WordPress plugin #3
 
Dependency injection in Drupal 8
Dependency injection in Drupal 8Dependency injection in Drupal 8
Dependency injection in Drupal 8
 
Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.
 
Doctrine in FLOW3
Doctrine in FLOW3Doctrine in FLOW3
Doctrine in FLOW3
 
Perl web frameworks
Perl web frameworksPerl web frameworks
Perl web frameworks
 
Curscatalyst
CurscatalystCurscatalyst
Curscatalyst
 

More from Workhorse Computing

Wheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility ModulesWheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility ModulesWorkhorse Computing
 
Paranormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add UpParanormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add UpWorkhorse Computing
 
The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.Workhorse Computing
 
Generating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in PosgresqlGenerating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in PosgresqlWorkhorse Computing
 
Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!Workhorse Computing
 
BSDM with BASH: Command Interpolation
BSDM with BASH: Command InterpolationBSDM with BASH: Command Interpolation
BSDM with BASH: Command InterpolationWorkhorse Computing
 
BASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic InterpolationBASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic InterpolationWorkhorse Computing
 
The W-curve and its application.
The W-curve and its application.The W-curve and its application.
The W-curve and its application.Workhorse Computing
 
Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.Workhorse Computing
 
Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.Workhorse Computing
 
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6Workhorse Computing
 

More from Workhorse Computing (20)

Wheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility ModulesWheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility Modules
 
mro-every.pdf
mro-every.pdfmro-every.pdf
mro-every.pdf
 
Paranormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add UpParanormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add Up
 
The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.
 
Unit Testing Lots of Perl
Unit Testing Lots of PerlUnit Testing Lots of Perl
Unit Testing Lots of Perl
 
Generating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in PosgresqlGenerating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in Posgresql
 
Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!
 
BSDM with BASH: Command Interpolation
BSDM with BASH: Command InterpolationBSDM with BASH: Command Interpolation
BSDM with BASH: Command Interpolation
 
Findbin libs
Findbin libsFindbin libs
Findbin libs
 
Memory Manglement in Raku
Memory Manglement in RakuMemory Manglement in Raku
Memory Manglement in Raku
 
BASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic InterpolationBASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic Interpolation
 
Effective Benchmarks
Effective BenchmarksEffective Benchmarks
Effective Benchmarks
 
Metadata-driven Testing
Metadata-driven TestingMetadata-driven Testing
Metadata-driven Testing
 
The W-curve and its application.
The W-curve and its application.The W-curve and its application.
The W-curve and its application.
 
Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.
 
Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.
 
Smoking docker
Smoking dockerSmoking docker
Smoking docker
 
Getting Testy With Perl6
Getting Testy With Perl6Getting Testy With Perl6
Getting Testy With Perl6
 
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
 
Neatly folding-a-tree
Neatly folding-a-treeNeatly folding-a-tree
Neatly folding-a-tree
 

Recently uploaded

Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Karmanjay Verma
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationBuild Intuit
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...Karmanjay Verma
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?Rustici Software
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 

Recently uploaded (20)

Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientation
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 

Lazy Data Using Perl

  • 1. Virtuous Lazyness For Your Data Doing it once and knowing you've done it. Steve Lembark Workhorse Computing lembark@wrkhors.com
  • 2. Data wants to be free, it also wants to be very expensive. ● Data you never use is free; accessing gets expensive. ● Managing data's cost requires controlling its lifecycle. ● Mis­managing the lifecycle causes problems: ● Caching table lookups in forked httpd crashes your database. ● Pre­processing messages files kills your startup times. ● Reading XML configuration data you never use wastes most of  your testing budget. ● Your tests fail because of a connection failure to a database  server that you don't use in the test. ● Fixing these problems requires using lazy data.
  • 3. False Lazyness ● We have all see the two most common data  management stratagies: ● You load everything at startup to avoid checking it every  time it is used. ● You check everything every time before using it to avoid  loading it up front. ● Both approaches ignore important knowledge: ● You know when the data is needed. ● You know that the data was loaded.
  • 4. One alternative: Scalar Cache ● $cache ||= read_cache_values; ● Seems nice: You know that $cache is populated. ● This requires dereferencing a hashref throughout the  program, which is expensive.  ● What you'd rather do is just use $cache{ $foobar }  without having to check it every time. ● Even checking if( ! %cache ) is expensive – arrays  are much cheaper to check.
  • 5. True Lazyness: Do Something Once ● Truly Lazy data means loading you data when you  need it and knowing that it is loaded. ● Perl gives us – of course – more than one way: ● Trampoline objects and subroutines. ● “state” variables, introduced in v5.10. ● Trampolines are flyweight objects – data structures or  subroutines that transform themselves when used. ● State variables are assigned only once, at runtime, the  first time they are used.
  • 6. Follow The Bouncing Object ● Object::Trampoline – flyweight data you don't know  isn't there. ● These delay calling the “real” constructor until the  object is actually used. ● Your constructor gets called before the first method call. ● At that point it can cache, parse, or compute whatever it  needs. ● Spreads the cost of loading data set each over the  lifetime of a process.
  • 7. Example: Delay Expensive XML ● Use an initializer to read and parse the XML. ● The Object::Trampoline calls your constructor once  to transform the object the first time you call a  method. ● Parsing the XML is pushed off until you actually use  the data. ● Requires using a hashref – and possibly methods –  to access the data.
  • 8. package XML::Message; use Object::Trampoline; ... sub new { Object::Trampline->install( 'XML::Message', $path ); } sub install { my $error = &construct; $error->initialize( @_ ) } sub initialize { my ( $err, $path ) = @_; %$err = %{ XMLin $path => @lots_of_args }; $err } # calling translate bounces the trampoline exactly once sub translate { … }
  • 9. Tramploline Subroutine ● Similar to a trampoline object: A portion of the code  runs once and replaces itself. ● One­shot code initializes the cache, which can now  be a simple hash. ● Symbol::qualify_to_ref make this painless in Perl. ● An anonymous sub manages the cache. ● A named subroutine loads the cache, replaces itself with  the manager, and re­dispatches to the manager.
  • 10. The Simplest Version my %foo_cache = (); ● Minimal code  my $handler = sub includes: { my $foo = shift; ● Closed­over cache, %foo_cache{ $foo } Or die “Bogus foo: '$foo' unknown” ● Subref to permenant  }; cache manager, sub do_something { ● Initial subroutine. %cache = initialize_foo_cache; ● The initial subroutine  my $ref = qualify_to_ref 'do_something'; initializes, installs,  *$ref = $handler; and re­dispatches. goto &$handler }
  • 11. BEGIN Blocks Are Cleaner BEGIN { my $name = 'foo_handler'; ● The block isolates  my $ref = qualify_to_ref $name; my %cache = (); cache, ref, and  my $handler sub variables;  = sub { allows re­cycling  %cache{ $_[0] } or die ... }; the ref. *$ref = sub ● This is also rather  { %cache = init_the_cache; amenable to  *$ref = $handler; installation by  module. goto &$handler } }
  • 12. Sub::Trampoline sub install_trampoline { my ( $name, $init, $mgr ) = @_; ● Aside from the actual  my $caller = caller; assignment, init code is  my $ref = qualify_to_ref $name, $caller; identical. *$ref ● Simply pass the name,  = sub { manager, and init  $init->(); assignment. &$ref = $mgr; ● The module can call  goto &$mgr } $init, replace itself. } ● Caller defines the  cache and handler.
  • 13. Using Sub::Trampoline use Sub::Trampoline; my %cache1 = (); my @cache2 = (); my $cache1_mgr = sub { $cache1{ $_[0] } or croak "Unknown '$_[0]'" }; my $cache2_mgr = sub { first { $_ eq $_[0] } @cache2 } }; my $init_cache1 = sub { %cache1 = select_from_hell } sub init_cache2 { @cache2 = XMLin $nasty_messy_xml_struct } install_trampoline( subone => $init_cache1, $cache1_mgr ); install_trampoline( known => &init_cache2, $cache2_mgr ); my $value1 = subone $key1; if( known $value ) { … } else { carp “Unknown: '$value'” }
  • 14. True One­Shot: Empty $handler ● If you want to run something exactly once but don't  know where it might be called initially: my $manager = sub(){}; ● You can also substitute a trampoline object with a  constructor that does the work and no methods. ● Calling the object once constructs it, after which the  classes constructor can stub itself. ● Useful for sharing the cache variable: the init  populates it once and stubs itself to do nothing more.
  • 15. Cycling The Cache ● There are times when you want to purge and re­ initialize the cache ● A trampoline object with populate and use subs that  flip­flop can handle this easily. ● Re­assigning a trampoline re­initializes the cache: $cache = Object::Trampoline->init_cache( $class => @argz ) if $age > $time_max;
  • 16. v5.10 Introduced “state” Variables ● Scoped like a lexical. ● Assinged once at runtime. ● Maintain value within a single lexical context  throughout the program. ● Assign the cache or assign a flag variable with the  side effect of populating the cache. ● Currently supports only scalars.
  • 17. Obvious case: Assign the cache. ● Assign the cache at runtime: sub cache_mangler { state $cache = init_cache; … } ● $cache will be assigned at runtime, the first time  cache_mangler is called. ● The value will be retained between calls. ● Catch: $cache is only available within  cache_mangler, not outside of it.
  • 18. Initialize a Shared Cache use v5.10; ● Subs may want to share  my %cache = (); a cache. sub init ● $y and $z are assigned  { %k or %k = …; at most once per  } executison when foo or  sub foo { state $y = init; … } bar are called. sub bar { state $z = init; … } … ● The sanity check in init  only needs to be handled  my $foo = foo 'bletch'; my $bar = bar 'blort'; at most per subroutine.
  • 19. Summary ● True lazyness includes managing data. ● Preloading it all or testing it at each step are not lazy. ● Object::Trampoline provides one way. ● Trampoline subroutines offer another approach. ● v5.10 introduced state variables which provide a few  ways to initialize something once.