PL/Perl - New Features in PostgreSQL 9.0 201012
Upcoming SlideShare
Loading in...5
×
 

PL/Perl - New Features in PostgreSQL 9.0 201012

on

  • 5,347 views

Slides of my talk given at PGDay.EU in Stuttgart Germany in December 2010.

Slides of my talk given at PGDay.EU in Stuttgart Germany in December 2010.

Statistics

Views

Total Views
5,347
Views on SlideShare
5,347
Embed Views
0

Actions

Likes
0
Downloads
9
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

PL/Perl - New Features in PostgreSQL 9.0 201012 PL/Perl - New Features in PostgreSQL 9.0 201012 Presentation Transcript

  • PL/Perl New Features in PostgreSQL 9.0 Tim Bunce - Dec 2010 Creative Commons BY-NC-SA 3.0
  • PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF ‣ FUTURESome features are in recent 8.x.y due to back-porting of security changes
  • PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF ‣ FUTURE
  • New Builtins
  • quote_... quote_literal( "foo" ) ==> "foo" quote_literal( "dont "carp"" ) ==> "dont "carp"" quote_literal( "" ) ==> ""quote_nullable( "foo" ) ==> "foo"quote_nullable( "dont "carp"" ) ==> "dont "carp""quote_nullable( "" ) ==> ""quote_ident( "foo" ) ==> "foo"quote_ident( "dont "carp"" ) ==> ""dont ""carp""""quote_ident( "" ) ==> """"
  • quote_...(undef) quote_literal( undef ) ==> undefquote_nullable( undef ) ==> "NULL" quote_ident( undef ) ==> """" (warn)
  • {en,de}code_byteaencode_bytea( "foo" ) ==> "x666f6f"decode_bytea( "x666f6f" ) ==> "foo"decode_bytea( "146157157") ==> "foo"encode_bytea( "x{263a}" ) ==> "xe298ba" UTF8decode_bytea( "xe298ba" ) ==> "342230272" Not UTF8encode_bytea( "" ) ==> "x"decode_bytea( "x" ) ==> ""decode_bytea( "" ) ==> ""encode_bytea( undef ) ==> "x" (warn)decode_bytea( undef ) ==> "" (warn)
  • looks_like_numberlooks_like_number( 1 ) ==> 1looks_like_number( 0 ) ==> 1looks_like_number( "+7.2e-9" ) ==> 1looks_like_number( "foo" ) ==> 0looks_like_number( "" ) ==> 0looks_like_number( undef ) ==> undeflooks_like_number( " 4 " ) ==> 1looks_like_number( "5plus" ) ==> 0 (but 5plus+0=5)
  • encode_array_*encode_array_literal( ["foo","bar"] ) ==> "{"foo", "bar"}"encode_array_constructor( ["foo","bar"] ) ==> "ARRAY[foo, bar]"encode_array_literal( [1,[2,[undef]]] ) ==> "{"1", {"2", {NULL}}}"encode_array_constructor( [1,[2,[undef]]] ) ==> "ARRAY[1, ARRAY[2, ARRAY[NULL]]]"
  • encode_array_*encode_array_literal( "foo" ) ==> "foo"encode_array_constructor( "foo" ) ==> "foo"encode_array_literal( undef ) ==> undefencode_array_constructor( undef ) ==> "NULL"
  • Trusted require/use• require/use work for already loaded modulesuse strict; # old way: BEGIN { strict->import(); }• extra pre-loaded modulesuse warnings;use Carp;use feature qw(say); # for perl 5.10 or lateruse utf8; # if server_encoding is utf8
  • CONTEXT: ...• PL/Perl tracks the context of log messages - before: WARNING: ...some warning from perl code... - now: WARNING: ...some warning from perl code... CONTEXT: PL/Perl function "..." - Thanks to Alexey Klyukin.
  • DO ... LANGUAGE ...;• Arbitrary chunks of code can be executed directly from psql, or client apps, via DO - Thanks to Petr Jelinek, Joshua Tolley, Hannu Valtonen• No need to create and run a stored procedure each time: DO $$ spi_exec("... $_ ...") for a..z; $$ language plperl;
  • Other Changes• Using $a and $b in sort blocks now works!• eval { ... } and eval "..." now work!• END blocks are now run at end of session - they cant (currently) access the database• Warnings from perl are now WARNINGs - they used to be NOTICE
  • PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF ‣ FUTURE
  • INTERNAL• The Safe module is no longer used for plperl - Now faster, simpler, and more secure• Validates return values are in server encoding - ERROR: invalid byte sequence for encoding - Thanks to Andrew Dunstan• Internal code refactoring and cleanup
  • PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF ‣ FUTURE
  • New plperl.* Config• Specify perl code to run during initialization: plperl.on_init = ...perl code... plperl.on_plperlu_init = ...perl code... plperl.on_plperl_init = ...perl code...• Can only be set by superuser or postgres.conf• Code cant access the database
  • Interpreter Lifecycle
  • Birth1. Perl interpreter created2. Options from PERL5OPT env var are processed3. PL/Perl support bootstrap code is executed4. plperl.on_init code runs (unrestricted)Above steps may happen in postmaster process at startup,if plperl is loaded via shared_preload_libraries.Otherwise they happen at first use.No access to database.
  • Adolescence6. Interpreter is specialised for plperl (if that’s used first) • Modules loaded: strict, warnings, features, Carp • Unsafe perl ops are restricted (require, open etc.) • DynaLoader package is deleted • plperl.on_plperl_init code runs (restricted)7. Database access is enabled8. Perl interpreter is made available for use9. Executes whatever action called it into existence
  • Siblings• If plplerlu code is run later - then a new interpreter is created - similarly if plperlu is run first and plperl run later• If plplerl with a different security context is run - then a new interpreter is created for the ROLE - That’s a recent security fix: http://wiki.postgresql.org/wiki/20101005securityreleaseNote impact on shared_preload_libraries in these cases
  • Death• Finally, when the session ends: - Access to the database is disabled - END blocks, if any, are run (if exiting cleanly)
  • plperl.on_init• Handy to set global perl configuation plperl.on_init=use lib qw(/myapp); use ...; plperl.on_init=require "plperloninit.pl"; Effectively defines ‘approved’ modules for plperl• SECURITY RISK! Only load modules youre happy for plperl code to use. Also check any other modules loaded as dependencies! Use Devel::TraceLoad to see whats actually loaded: PERL5OPT=-MDevel::Trace=summary pg_ctl ...
  • PL/Perl Best Practice• Include explicit use statements in functions For plperlu thatll actually load if needed For plperl itll check that module is loaded - so youll get an immediate clear failure if not - (e.g., on a replica with old postgres.conf file)
  • plperl.on_plperl_init• Originally intended for things like - PGOPTIONS="-c plperl.on_plperl_init=..." - to enable debug or profiling for a session• But...• Can only be set by superuser or postgres.conf - due to SECURITY DEFINER risk at the time - that’s now been patched (20101005, CVE-2010-3433) - so this restriction may be removed in future
  • PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF ‣ FUTURE
  • Devel::NYTProfPerl Source Code ProfilerPostgreSQL::PLPerl::NYTProf
  • Enabling NYTProf• Via postgres.conf: plperl.on_init=use PostgreSQL::PLPerl::NYTProf• Via environment variable: PERL5OPT=-MPostgreSQL::PLPerl::NYTProf pg_ctl ...• Is immediately active for all connections.• To enable on demand for one connection: NYTPROF=start=no PERL5OPT=... pg_ctl ... DO DB::enable_profile LANGUAGE plperl;
  • Reporting from NYTProf• Writes per-backend data files: $PGDATA/nytprof.out.$pid• To generate a report: nytprofhtml --file=$PGDATA/nytprof.out.4321 --open
  • ~ Demo ~ Of plperl.on_init in postgresql.conf And use of PostgreSQL::PLPerl::NYTProf Screencast: http://timbunce.blip.tv/file/3691795/Video: http://www.fosslc.org/drupal/content/plperl-new-features-90
  • PL/Perl Changes ‣ USER ‣ INTERNAL ‣ DBA ‣ NYTPROF ‣ FUTURE
  • Future Possibilities• Optimize trigger handling overheads• Remove overheads of threaded perls• $array_ref = decode_array_literal(‘{...}’)• $hash_ref = decode_hstore_literal(‘x=>42’)• Array params as array refs (with overloading)• Rewrite encode_array_literal in C.
  • Questions?Tim.Bunce@pobox.comhttp://blog.timbunce.org @timbunce on twitter
  • http://xkcd.com/519/