PL/Perl New Features
    in PostgreSQL 9.0


     Tim Bunce - June 2010
     Creative Commons BY-NC-SA 3.0
PL/Perl Changes

   ‣ USER
   ‣ INTERNAL
   ‣ DBA
   ‣ NYTPROF
PL/Perl Changes

   ‣ USER
   ‣ INTERNAL
   ‣ DBA
   ‣ NYTPROF
New Builtins
quote_...

 quote_literal( "foo"            ) ==> "'foo'"
 quote_literal( "don't "carp"" ) ==> "'don''t "carp"'"
 quote_li...
quote_...(undef)


 quote_literal( undef   ) ==> undef


quote_nullable( undef   ) ==> "NULL"


   quote_ident( undef   ) ...
{en,de}code_bytea
encode_bytea( "foo"            ) ==> "x666f6f"
decode_bytea( "x666f6f"      ) ==> "foo"
decode_bytea( "1...
looks_like_number

looks_like_number( 1           ) ==> 1
looks_like_number( 0           ) ==> 1
looks_like_number( "+7.2e...
encode_array_*

encode_array_literal( ["foo","bar"] )
   ==> "{"foo", "bar"}"


encode_array_constructor( ["foo","bar"] )
...
encode_array_*


encode_array_literal(       "foo"   ) ==> "foo"
encode_array_constructor(   "foo"   ) ==> "'foo'"


encod...
Trusted require/use
• require/use work for already loaded modules
use strict;            # old way: BEGIN { strict->import...
CONTEXT: ...

• PL/Perl tracks the context of log messages
  - before:
    WARNING: ...some warning from perl code...


  ...
DO '...' LANGUAGE plperl;

• Arbitrary chunks of perl code can be executed
    directly from psql, or client apps, via DO
...
Other Changes
• Using $a and $b in sort blocks now works!
• eval { ... } and eval "..."
  - can now be used in plperl
• EN...
PL/Perl Changes

   ‣ USER
   ‣ INTERNAL
   ‣ DBA
   ‣ NYTPROF
INTERNAL

• The Safe module is no longer used for plperl
  - Improved security and reduced call overheads
  - Upgrade to l...
PL/Perl Changes

   ‣ USER
   ‣ INTERNAL
   ‣ DBA
   ‣ NYTPROF
New plperl.* Config
• Specify perl code to run during initialization:

  plperl.on_init            = '...perl...'
  plperl....
~ Timeline ~
▾   Perl interpreter created on demand
    ▿ Options from PERL5OPT env var are processed
    ▿ PL/Perl suppor...
plperl.on_init

• Handy to set global perl configuation
  plperl.on_init='use lib qw(/myapp); use ...;'
  plperl.on_init='r...
PL/Perl Best Practice

•   Include explicit use statements in functions
•   Don't assume modules have been pre-loaded
•   ...
plperl.on_plperl_init

• Originally intended for things like
  -   PGOPTIONS="-c plperl.on_plperl_init='...'"

  - to enab...
PL/Perl Changes

   ‣ USER
   ‣ INTERNAL
   ‣ DBA
   ‣ NYTPROF
Devel::NYTProf
Perl Source Code Profiler


PostgreSQL::PLPerl::NYTProf
Enabling NYTProf
• Via postgres.conf:
  plperl.on_init='use PostgreSQL::PLPerl::NYTProf'



• Via environment variable:
  ...
Reporting from NYTProf

• Writes per-backend data files:
  $PGDATA/nytprof.out.$pid



• To generate a report:
  nytprofhtm...
~ Demo ~
                Screencast: http://timbunce.blip.tv
Video: http://www.fosslc.org/drupal/content/plperl-new-featur...
Questions?

Tim.Bunce@pobox.com
http://blog.timbunce.org
 @timbunce on twitter
http://xkcd.com/519/
Upcoming SlideShare
Loading in...5
×

PL/Perl - New Features in PostgreSQL 9.0

4,764

Published on

Slides from my talk at PGcon 2010 in Ottawa.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
4,764
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

PL/Perl - New Features in PostgreSQL 9.0

  1. 1. PL/Perl New Features in PostgreSQL 9.0 Tim Bunce - June 2010 Creative Commons BY-NC-SA 3.0
  2. 2. PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF
  3. 3. PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF
  4. 4. New Builtins
  5. 5. quote_... quote_literal( "foo" ) ==> "'foo'" quote_literal( "don't "carp"" ) ==> "'don''t "carp"'" quote_literal( "" ) ==> "''" quote_nullable( "foo" ) ==> "'foo'" quote_nullable( "don't "carp"" ) ==> "'don''t "carp"'" quote_nullable( "" ) ==> "''" quote_ident( "foo" ) ==> "foo" quote_ident( "don't "carp"" ) ==> ""don't ""carp"""" quote_ident( "" ) ==> """"
  6. 6. quote_...(undef) quote_literal( undef ) ==> undef quote_nullable( undef ) ==> "NULL" quote_ident( undef ) ==> """" (warn)
  7. 7. {en,de}code_bytea encode_bytea( "foo" ) ==> "x666f6f" decode_bytea( "x666f6f" ) ==> "foo" decode_bytea( "146157157") ==> "foo" encode_bytea( "x{263a}" ) ==> "xe298ba" UTF8 decode_bytea( "xe298ba" ) ==> "342230272" Not UTF8 encode_bytea( "" ) ==> "x" decode_bytea( "x" ) ==> "" decode_bytea( "" ) ==> "" encode_bytea( undef ) ==> "x" (warn) decode_bytea( undef ) ==> "" (warn)
  8. 8. looks_like_number looks_like_number( 1 ) ==> 1 looks_like_number( 0 ) ==> 1 looks_like_number( "+7.2e-9" ) ==> 1 looks_like_number( "foo" ) ==> 0 looks_like_number( "" ) ==> 0 looks_like_number( undef ) ==> undef looks_like_number( " 4 " ) ==> 1 looks_like_number( "5plus" ) ==> 0 (but '5plus'+0=5)
  9. 9. encode_array_* encode_array_literal( ["foo","bar"] ) ==> "{"foo", "bar"}" encode_array_constructor( ["foo","bar"] ) ==> "ARRAY['foo', 'bar']" encode_array_literal( [1,[2,[undef]]] ) ==> "{"1", {"2", {NULL}}}" encode_array_constructor( [1,[2,[undef]]] ) ==> "ARRAY['1', ARRAY['2', ARRAY[NULL]]]"
  10. 10. encode_array_* encode_array_literal( "foo" ) ==> "foo" encode_array_constructor( "foo" ) ==> "'foo'" encode_array_literal( undef ) ==> undef encode_array_constructor( undef ) ==> "NULL"
  11. 11. Trusted require/use • require/use work for already loaded modules use strict; # old way: BEGIN { strict->import(); } • extra pre-loaded modules use warnings; use Carp; use feature qw(say); # for perl 5.10 or later use utf8; # if server_encoding is utf8
  12. 12. CONTEXT: ... • PL/Perl tracks the context of log messages - before: WARNING: ...some warning from perl code... - now: WARNING: ...some warning from perl code... CONTEXT: PL/Perl function "..."
  13. 13. DO '...' LANGUAGE plperl; • Arbitrary chunks of perl code can be executed directly from psql, or client apps, via DO • No need to create and run a stored procedure each time. DO $$ spi_exec("... $_ ...") for 'a'..'z'; $$ language plperl;
  14. 14. Other Changes • Using $a and $b in sort blocks now works! • eval { ... } and eval "..." - can now be used in plperl • END blocks are now run at end of session - they can't (currently) access the database • Warnings from perl are now WARNINGs - they used to be NOTICE
  15. 15. PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF
  16. 16. INTERNAL • The Safe module is no longer used for plperl - Improved security and reduced call overheads - Upgrade to latest security patch! • Validates return values are in server encoding - ERROR: invalid byte sequence for encoding • Internal code refactoring and cleanup
  17. 17. PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF
  18. 18. New plperl.* Config • Specify perl code to run during initialization: plperl.on_init = '...perl...' plperl.on_plperlu_init = '...perl...' plperl.on_plperl_init = '...perl...' • Can only be set by superuser or postgres.conf • Code can't access the database
  19. 19. ~ Timeline ~ ▾ Perl interpreter created on demand ▿ Options from PERL5OPT env var are processed ▿ PL/Perl support code bootstraps ▿ plperl.on_init code runs Above may happen in postmaster at startup if plperl is loaded via shared_preload_libraries ▾ Interpreter is specialised for plperl (or plperlu) ▿ Modules loaded: strict, warnings, features, Carp ▿ Perl operators are restricted (require, open etc.) ▿ DynaLoader package is deleted ▿ plperl.on_plperl_init code runs ▿ Database access is enabled
  20. 20. plperl.on_init • Handy to set global perl configuation plperl.on_init='use lib qw(/myapp); use ...;' plperl.on_init='require "plperloninit.pl";' • SECURITY RISK! Only load modules you're happy for plperl code to use! Also check any other modules loaded as dependencies! Use Devel::TraceLoad to see what's actually loaded: PERL5OPT='-MDevel::Trace=summary' pg_ctl ...
  21. 21. PL/Perl Best Practice • Include explicit use statements in functions • Don't assume modules have been pre-loaded • For plperlu that'll actually load if needed • For plperl it'll check that module was loaded - so you'll get an immediate clear failure if not - for example on a replica with old postgres.conf file
  22. 22. plperl.on_plperl_init • Originally intended for things like - PGOPTIONS="-c plperl.on_plperl_init='...'" - to enable debug or profiling for a session • But... • Can only be set by superuser or postgres.conf - sadly, due to SECURITY DEFINER risks. - You shouldn't write SECURITY DEFINER functions in plperl if untrusted users can use plperl!
  23. 23. PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF
  24. 24. Devel::NYTProf Perl Source Code Profiler PostgreSQL::PLPerl::NYTProf
  25. 25. Enabling NYTProf • Via postgres.conf: plperl.on_init='use PostgreSQL::PLPerl::NYTProf' • Via environment variable: PERL5OPT='-MPostgreSQL::PLPerl::NYTProf' pg_ctl ... • Immediately active. To enable on demand: NYTPROF=start=no PERL5OPT=... pg_ctl ... DO 'DB::enable_profile' LANGUAGE plperl;
  26. 26. Reporting from NYTProf • Writes per-backend data files: $PGDATA/nytprof.out.$pid • To generate a report: nytprofhtml --file=$PGDATA/nytprof.out.4321 --open
  27. 27. ~ Demo ~ Screencast: http://timbunce.blip.tv Video: http://www.fosslc.org/drupal/content/plperl-new-features-90
  28. 28. Questions? Tim.Bunce@pobox.com http://blog.timbunce.org @timbunce on twitter
  29. 29. http://xkcd.com/519/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×