Sql abstract from_query

Sql abstract from_query

• 1.
  23.08.14 - Page1 Département Office SQL::Abstract::FromQuery Building DB requests from Web queries YAPC::EU::2014, Sofia laurent.dami@justice.ge.ch Etat de Genève, Pouvoir Judiciaire Département Office
• 2.
  Power search :the "widget" way Source: https://rt.cpan.org/Search/Build.html
• 3.
  Power search :the "parsed" way Name : Smi* Gender : M Salary : > 4000 Job : ! programmer, analyst Birth : BETWEEN 01.01.1970 AND 31.12.1990 Address: ! NULL
• 4.
  Agenda • Background :SQL::Abstract & SQL::Abstract::More • SQL::Abstract::FromQuery – purpose – API • internals – Regexp::Grammars – multiple inheritance
• 5.
  Background : SQL::Abstract SQL::Abstract::More TranslatingPerl datastructures into SQL
• 6.
  SQL::Abstract ('SQLA') :example my $sqla = SQL::Abstract->new; my ($sql, @bind) = $sqla->select( $table_name, [qw/col1 col2 col3/], { col1 => 'val1', col2 => {-in => [qw/foo bar/]}, col3 => {-between => [qw/2012-01-01 2012-07-01/]}, col4 => {">" => 999}, col5 => {-like => 'foo%'}, }, [qw/col1 col2/], ); # SELECT col1, col2, col3 FROM $table_name # WHERE col1 = ? AND col2 IN (?, ?) AND col3 … etc # ORDER BY col1, col2
• 7.
  Purpose • generate SQLfrom Perl data structures – direct from config or from user input – avoid painful string manipulations with regex / join / etc. • placeholders and bind values – avoid datatype distinctions (strings / numbers /dates / etc.) – avoid SQL injection   used by DBIx::Class, DBIx::DataModel, …
• 8.
  "Special operators" inSQLA • support for user-defined SQL extensions. For ex : – fulltext index – multivalued columns – case-insensitive – LIKE / GLOB / etc. • useful for DBMS-independent queries in apps – the "special operator" translates to DBMS-specific syntax { col1 => {-contains => [qw/YAPC Sofia/]}, col2 => {-anyval => [12, 34, 56]}, col3 => {-glob => 'foo*'}, }
• 9.
  SQL::Abstract::More ('SQLAM') • fullycompatible with SQLA • named parameters • more SQL constructs (group by, limit/offset, union/intersect, etc.) • more syntax (aliases, joins, ordering direction, etc.) my $sqla = SQL::Abstract::More->new; my ($sql, @bind) = $sqla->select( -columns => [qw/table1.col|ct1 table2.col|ct2 … /], -from => $table_name, -where => { col1 => 'val1', col2 => {-in => [qw/foo bar/]}, … }, -union => [-columns => …, -from => …, -where => …], -order_by => [qw/+col1 -col2/], );
• 10.
  23.08.14 - Page1 Département Office SQLA, SQLAM are for programmers  what about users ? SQL Perl datastructure SQLA, SQLAMHTTP query SQLA::FromQuery
• 11.
  HTTP query Name :Smi* Gender : M Salary : > 4000 Job : ! programmer, analyst Birth : BETWEEN 01.01.1970 AND 31.12.1990 Address: ! NULL
• 12.
  Expected SQLA structure { Name=> {-like => 'Smi%'}, Gender => 'M', Salary => {'>' => 4000}, Job => {-not_in => [qw/programmer analyst/]}, Birth => {-between => [qw/1970-01-01 1990-12-31/]}, Address => {'<>' => undef}, } SELECT * FROM people WHERE Name LIKE 'Smi%' AND Gender = 'M' AND Salary > 4000 AND Job NOT IN ('programmer', 'analyst') AND Birth BETWEEN 1970-01-01 AND 1990-12-31 AND Address IS NOT NULL
• 13.
  SYNOPSIS my $parser =SQL::Abstract::FromQuery->new(); my $criteria = $parser->parse($http_query); # $http_query is an object with a ->param() method # .. or it can be just a hashref my ($sql, @bind) = $sqla->select($table, @cols, $criteria); # or my ($sql, @bind) = $sqlam->select( -columns => @cols, -from => $table, -where => $criteria, );
• 14.
  Options my $parser =SQL::Abstract::FromQuery->new( # additional components (optional) -components => [qw/FR Oracle/], # grammar rules for some specific fields (optional) -fields => { standard => [qw/field1 field2 .../], bool => [qw/bool_field1/], ... } );
• 15.
  Components • FR – datesau format dd.mm.yyyy – mots-clés en français (VRAI, FAUX, ENTRE, NUL, etc.) – messages d'erreur en français • Oracle – queries on date and time • ["to_date(?, '$date_format')", $date] • ["to_date(?, '$time_format')", $date]  WHERE D_BIRTH > TO_DATE('DD.MM.YYYY', ?) • Contains – generate fulltext queries • … (user-defined)
• 16.
  Field types :which grammar to apply • Standard – a plain value (number, string, date or time). – a list of values, separated by ','. – a negated value or list of values; negation is expressed by ! or != or - or <> – a comparison operator <=, <, >=, > followed by a plain value – the special word NULL – BETWEEN val1 AND val2 – boolean values YES, NO, TRUE or FALSE • Restricted – string / bool / date / … (user-defined)
• 17.
  Internals
• 18.
  Regexp::Grammars use Regexp::Grammars; qr{ <grammar:SQL::Abstract::FromQuery> <rule: standard> <MATCH=between> | <MATCH=op_and_value> | <MATCH=values> <rule: values> <[value]>+ % , <token: compare> <= | < | >= | > <rule: value> <MATCH=null> | <MATCH=date> | <MATCH=time> | <MATCH=string> <rule: date> <day=(dd?)>.<month=(dd?)>.<year=(ddd?d?)> | <year=(ddd?d?)>-<month=(dd?)>-<day=(dd?)>
• 19.
  Autoactions hooked tothe grammar sub between { my ($self, $h) = @_; return {-between => [$h->{min}, $h->{max}]}; } sub values { my ($self, $h) = @_; my $n_values = @{$h->{value}}; return $n_values > 1 ? {-in => $h->{value}} : $h->{value}[0]; }
• 20.
  Grammar Inheritance <grammar: SQL::Abstract::FromQuery::FR> <extends:SQL::Abstract::FromQuery> <rule: null> NULL? <rule: between> <SQL::Abstract::FromQuery::between> | ENTRE (*COMMIT) (?: <min=value> ET <max=value> | <error:> ) <rule: bool> O(?:UI)? (?{ $MATCH = 1 }) | V(?:RAI)? (?{ $MATCH = 1 }) | NO?N? (?{ $MATCH = 0 }) | F(?:AUX|ALSE)? (?{ $MATCH = 0 }) | Y(?:ES)? (?{ $MATCH = 1 })
• 21.
  Rules inheritance package SQL::Abstract::FromQuery::Oracle; useparent 'SQL::Abstract::FromQuery'; use mro 'c3'; ¨sub date { my ($self, $h) = @_; my $date_format = $self->{date_format} || 'YYYY-MM-DD'; my $date = $self->next::method($h); return ["to_date(?, '$date_format')", $date]; }
• 22.
  23.08.14 - Page1 Département Office Conclusion
• 23.
  Features • flexible syntaxfor user input – plain value – list of values – comparison operators – patterns – special constructs (BETWEEN, MATCH, etc.) – Including SQLA "special ops" • internationalization • automate specific behaviours for some form fields – data conversion – inject special ops
• 24.
  Usage • See App::AutoCRUD  come to tomorrow's talk 