PostgreSQL Gems:
Running Perl and Ruby Inside the Database


David Fetter http://fetter.org/
Copyright 2007
All Rights Res...
PostgreSQL
Procedural Languages
PL/PgSQL
PL/Perl(U)
PL/Ruby(U)
PL/Scheme(U)
PL/Java(U)
PL/Parrot(U)?
The Trust Issue
Untrusted
   Can open pipes, filehandles
   Superuser
Trusted
   Can't
   Regular user
Perl


The Onion We Know and Love
Tools and Techniques

http://www.cpan.org
More Tools and Techniques


%_SHARED
Debugging Basics

warn
die
elog
More Advanced Debugging

use YAML;
...
warn Dump($complex_thing);
Debuggers

Debugging is twice as   hard as writing
the code in the first   place. Therefore,
if you write the code   as cl...
Ruby


A Nifty Gem
Tools and Techniques

http://rubygems.org/
LOLCODE!

           IM IN UR DATABASE
           READING UR ROWS

http://pgfoundry.org/projects/pllolcode/
A Custom Data Type

Bar Codes:

* UPC
* EAN
* GTIN
* Oy vey!

So you have to ask yourself a question...
How to do it

Luckily, they are all GTINs

And they have a checksum
How (not!) to do it in PL/Perl
CREATE OR REPLACE FUNCTION is_gtin(bigint) RETURNS
BOOLEAN
LANGUAGE plperl
AS $$
  my $i = ...
How to do it in PL/Ruby
CREATE OR REPLACE FUNCTION isa_gtin(bigint)
RETURNS BOOLEAN
LANGUAGE plruby
AS $$
  chars = args[0...
How to do it in SQL :)
CREATE OR REPLACE FUNCTION is_gtin(bigint)
RETURNS BOOLEAN
LANGUAGE sql
STRICT IMMUTABLE
AS $$
   S...
Creating The GTIN Data Type


CREATE TYPE gtin AS BIGINT
    CHECK ( is_gtin(VALUE) );
A More Complex Data Type


CREATE TYPE email AS TEXT
    CHECK ( is_email(VALUE) );
Defining A Function

CREATE OR REPLACE FUNCTION
is_email(text)
RETURNS BOOLEAN
LANGUAGE plperlu
AS $$
...
$$;
Perl Regex for Email
        (?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn
        015()]|[^x80-xff])*))*))*(?:...
Better Perl (1/3)


use strict;
use warnings;
Better Perl (2/3)
use Email::Valid;
my $address = shift;
my $checks = {
   -address => $address,
   -mxcheck => 1,
   -tld...
Better Perl (3/3)

if (defined Email::Valid->address( %$checks )) {
    return 'true'
}
warn quot;address failed $Email::Va...
The Email Domain in Action
oscon_2007=> SELECT ‘david@fetter.org’::email;
       email
------------------
 david@fetter.or...
Perl on Both Sides
#!/usr/bin/env perl

use strict;
use warnings;

use DBI;
my $dbh = DBI->connect(‘dbi:Pg:dbname=wow’,’us...
A Usual Program (1/3)

use Proc::ProcessTable;
my $t = new Proc::ProcessTable;
my $p = shift ( @{ $t->table });
my @fields...
Still Usual (2/3)
foreach my $f ($t->fields) {
    if ($p->{$f} =~ m/-?d*.?d+/) {
        push @fields, quot;$f FLOAT8quot...
Still Usual (3/3)

print <<CREATE_TYPE;
CREATE TYPE process_table_type AS (
    @{[ join(quot;,n    quot;, @fields) ]}
);
...
Down the Rabbit Hole

$ psql oscon_2007
Welcome to psql 8.2.4, the PostgreSQL interactive
terminal.
...
oscon_2007=> set m...
The PL/PerlU Function
CREATE OR REPLACE FUNCTION get_ps ()
RETURNS SETOF process_table_type
LANGUAGE plperlu AS
$$
    use...
A Query

SELECT *
FROM
  pg_class quot;c1quot;,
  pg_class quot;c2quot;,
  pg_class quot;c3quot;;
What is happening?!?
SELECT a.current_query, MAX(p.size)
FROM
     pg_stat_activity AS quot;aquot;
JOIN
     get_ps() AS q...
Aha!
       current_query    |   size
------------------------+-----------
SELECT *                |
FROM                 ...
Thanks!
Copyright David Fetter 2007
All Rights Reserved
http://fetter.org/
Upcoming SlideShare
Loading in...5
×

Os Fetterupdated

682

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
682
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Os Fetterupdated

  1. 1. PostgreSQL Gems: Running Perl and Ruby Inside the Database David Fetter http://fetter.org/ Copyright 2007 All Rights Reserved
  2. 2. PostgreSQL
  3. 3. Procedural Languages PL/PgSQL PL/Perl(U) PL/Ruby(U) PL/Scheme(U) PL/Java(U) PL/Parrot(U)?
  4. 4. The Trust Issue Untrusted Can open pipes, filehandles Superuser Trusted Can't Regular user
  5. 5. Perl The Onion We Know and Love
  6. 6. Tools and Techniques http://www.cpan.org
  7. 7. More Tools and Techniques %_SHARED
  8. 8. Debugging Basics warn die elog
  9. 9. More Advanced Debugging use YAML; ... warn Dump($complex_thing);
  10. 10. Debuggers Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. Kernighan
  11. 11. Ruby A Nifty Gem
  12. 12. Tools and Techniques http://rubygems.org/
  13. 13. LOLCODE! IM IN UR DATABASE READING UR ROWS http://pgfoundry.org/projects/pllolcode/
  14. 14. A Custom Data Type Bar Codes: * UPC * EAN * GTIN * Oy vey! So you have to ask yourself a question...
  15. 15. How to do it Luckily, they are all GTINs And they have a checksum
  16. 16. How (not!) to do it in PL/Perl CREATE OR REPLACE FUNCTION is_gtin(bigint) RETURNS BOOLEAN LANGUAGE plperl AS $$ my $i = 0; my $total = 0; for my $c (reverse split //, shift) { $i++; $total += $i % 2 ? $c : $c * 3; } return $total % 10 ? 'false' : 'true'; $$;
  17. 17. How to do it in PL/Ruby CREATE OR REPLACE FUNCTION isa_gtin(bigint) RETURNS BOOLEAN LANGUAGE plruby AS $$ chars = args[0].to_s.split('').reverse total = 0 chars.each_index { |i| total += i % 2 == 0 ? chars[i].to_i : chars[i].to_i * 3 } return !!(total % 10 == 0) $$;
  18. 18. How to do it in SQL :) CREATE OR REPLACE FUNCTION is_gtin(bigint) RETURNS BOOLEAN LANGUAGE sql STRICT IMMUTABLE AS $$ SELECT ( sum(dgt) % 10 ) = 0 FROM ( SELECT substring($1 from idx for 1)::smallint AS dgt FROM (SELECT generate_series(length($1), 1, -2) as idx) AS foo UNION ALL SELECT substring($1 from idx for 1)::smallint * 3 AS dgt FROM (SELECT generate_series(length($1) -1, 1, -2) as idx) AS foo ) AS bar; $$;
  19. 19. Creating The GTIN Data Type CREATE TYPE gtin AS BIGINT CHECK ( is_gtin(VALUE) );
  20. 20. A More Complex Data Type CREATE TYPE email AS TEXT CHECK ( is_email(VALUE) );
  21. 21. Defining A Function CREATE OR REPLACE FUNCTION is_email(text) RETURNS BOOLEAN LANGUAGE plperlu AS $$ ... $$;
  22. 22. Perl Regex for Email (?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn 015()]|[^x80-xff])*))*))*(?:(?:[^(040)<>@,;:quot;.[]000-037x80- xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xff])|quot;(?:[^x80-xffn015quot; ]|[^x80-xff])*quot;)(?:(?:[040t]|((?:[^x80-xffn015()]|[^x80-xf f]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*.(?:[040t]|((?:[ ^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80- xff])*))*))*(?:[^(040)<>@,;:quot;.[]000-037x80-xff]+(?![^(040)<>@,; :quot;.[]000-037x80-xff])|quot;(?:[^x80-xffn015quot;]|[^x80-xff])*quot;)) *(?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xff n015()]|[^x80-xff])*))*))*@(?:[040t]|((?:[^x80-xffn015()]| [^x80-xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*(?:[^(04 0)<>@,;:quot;.[]000-037x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80- xff])|[(?:[^x80-xffn015[]]|[^x80-xff])*])(?:(?:[040t]|((? :[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80 -xff])*))*))*.(?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|( (?:[^x80-xffn015()]|[^x80-xff])*))*))*(?:[^(040)<>@,;:quot;.[] 000-037x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xff])|[(?:[^ x80-xffn015[]]|[^x80-xff])*]))*|(?:[^(040)<>@,;:quot;.[]000-0 37x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xff])|quot;(?:[^x80-xf fn015quot;]|[^x80-xff])*quot;)(?:[^()<>@,;:quot;.[]x80-xff000-010012-03 7]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()]| [^x80-xff])*))*)|quot;(?:[^x80-xffn015quot;]|[^x80-xff])*quot;)*<(?:[04 0t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()] |[^x80-xff])*))*))*(?:@(?:[040t]|((?:[^x80-xffn015()]|[^x 80-xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*(?:[^(040)<>@ ,;:quot;.[]000-037x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xff] )|[(?:[^x80-xffn015[]]|[^x80-xff])*])(?:(?:[040t]|((?:[^ x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80-xff ])*))*))*.(?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^ x80-xffn015()]|[^x80-xff])*))*))*(?:[^(040)<>@,;:quot;.[]000- 037x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xff])|[(?:[^x80- xffn015[]]|[^x80-xff])*]))*(?:(?:[040t]|((?:[^x80-xffn01 5()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*,(? :[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn0 15()]|[^x80-xff])*))*))*@(?:[040t]|((?:[^x80-xffn015()]|[^ x80-xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*(?:[^(040)< >@,;:quot;.[]000-037x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xf f])|[(?:[^x80-xffn015[]]|[^x80-xff])*])(?:(?:[040t]|((?:[^ x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80-x ff])*))*))*.(?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?: [^x80-xffn015()]|[^x80-xff])*))*))*(?:[^(040)<>@,;:quot;.[]00 0-037x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xff])|[(?:[^x8 0-xffn015[]]|[^x80-xff])*]))*)*:(?:[040t]|((?:[^x80-xffn 015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*) ?(?:[^(040)<>@,;:quot;.[]000-037x80-xff]+(?![^(040)<>@,;:quot;.[]000 -037x80-xff])|quot;(?:[^x80-xffn015quot;]|[^x80-xff])*quot;)(?:(?:[040t] |((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()]|[ ^x80-xff])*))*))*.(?:[040t]|((?:[^x80-xffn015()]|[^x80-xf f]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*(?:[^(040)<>@,;:quot;. []000-037x80-xff]+(?![^(040)<>@,;:quot;.[]000-037x80-xff])|quot;(?: [^x80-xffn015quot;]|[^x80-xff])*quot;))*(?:[040t]|((?:[^x80-xffn 015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*@ (?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn 015()]|[^x80-xff])*))*))*(?:[^(040)<>@,;:quot;.[]000-037x80-xff ]+(?![^(040)<>@,;:quot;.[]000-037x80-xff])|[(?:[^x80-xffn015[ ]]|[^x80-xff])*])(?:(?:[040t]|((?:[^x80-xffn015()]|[^x80- xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*.(?:[040t]|((? :[^x80-xffn015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80 -xff])*))*))*(?:[^(040)<>@,;:quot;.[]000-037x80-xff]+(?![^(040)<>@ ,;:quot;.[]000-037x80-xff])|[(?:[^x80-xffn015[]]|[^x80-xff ])*]))*(?:[040t]|((?:[^x80-xffn015()]|[^x80-xff]|((?:[^x8 0-xffn015()]|[^x80-xff])*))*))*>)(?:[040t]|((?:[^x80-xffn 015()]|[^x80-xff]|((?:[^x80-xffn015()]|[^x80-xff])*))*))*
  23. 23. Better Perl (1/3) use strict; use warnings;
  24. 24. Better Perl (2/3) use Email::Valid; my $address = shift; my $checks = { -address => $address, -mxcheck => 1, -tldcheck => 1, -rfc822 => 1, };
  25. 25. Better Perl (3/3) if (defined Email::Valid->address( %$checks )) { return 'true' } warn quot;address failed $Email::Valid::Details check.quot;; return 'false';
  26. 26. The Email Domain in Action oscon_2007=> SELECT ‘david@fetter.org’::email; email ------------------ david@fetter.org (1 row) oscon_2007=> SELECT ‘david@fetter.con’::email; ERROR: value for domain email violates check constraint quot;email_checkquot;
  27. 27. Perl on Both Sides #!/usr/bin/env perl use strict; use warnings; use DBI; my $dbh = DBI->connect(‘dbi:Pg:dbname=wow’,’user’,’secret’, {RaiseError => 1, AutoCommit => 0}); # And go on as usual
  28. 28. A Usual Program (1/3) use Proc::ProcessTable; my $t = new Proc::ProcessTable; my $p = shift ( @{ $t->table }); my @fields = ();
  29. 29. Still Usual (2/3) foreach my $f ($t->fields) { if ($p->{$f} =~ m/-?d*.?d+/) { push @fields, quot;$f FLOAT8quot;; } elsif ($p->{$f} =~ m/^-?d+$/) { push @fields, quot;$f INT8quot;; } else { push @fields, quot;$f TEXTquot;; } }
  30. 30. Still Usual (3/3) print <<CREATE_TYPE; CREATE TYPE process_table_type AS ( @{[ join(quot;,n quot;, @fields) ]} ); CREATE_TYPE
  31. 31. Down the Rabbit Hole $ psql oscon_2007 Welcome to psql 8.2.4, the PostgreSQL interactive terminal. ... oscon_2007=> set mptt `./make_process_table_type.pl` oscon_2007=> :mptt CREATE TYPE
  32. 32. The PL/PerlU Function CREATE OR REPLACE FUNCTION get_ps () RETURNS SETOF process_table_type LANGUAGE plperlu AS $$ use Proc::ProcessTable; my $proctab = new Proc::ProcessTable; return $proctab->table; $$;
  33. 33. A Query SELECT * FROM pg_class quot;c1quot;, pg_class quot;c2quot;, pg_class quot;c3quot;;
  34. 34. What is happening?!? SELECT a.current_query, MAX(p.size) FROM pg_stat_activity AS quot;aquot; JOIN get_ps() AS quot;pquot; ON (a.procpid = p.pid) GROUP BY a.current_query;
  35. 35. Aha! current_query | size ------------------------+----------- SELECT * | FROM | pg_class quot;c1quot;, | pg_class quot;c2quot;, | pg_class quot;c3quot;; | 195290000 (1 row)
  36. 36. Thanks! Copyright David Fetter 2007 All Rights Reserved http://fetter.org/

×