Your SlideShare is downloading. ×
0
Working with databases in Perl Tutorial for FPW::2011, Paris [email_address] Département Office
Overview <ul><li>intended audience : beginners </li></ul><ul><ul><li>in Perl </li></ul></ul><ul><ul><li>in Databases </li>...
Relational databases RDBMS  = Relational Database Management System
Relational model join on c3 filter Table (rows + columns) projection c1 c2 c3 1 foo 1 2 foo 2 3 bar 1 c3 c4 1 xx 2 yy c1 c...
Maybe you don't want a RDBMS <ul><li>Other solutions for persistency in Perl: </li></ul><ul><ul><ul><li>BerkeleyDB   : per...
Features of RDBMS <ul><li>Relational </li></ul><ul><li>Indexing </li></ul><ul><li>Concurrency </li></ul><ul><li>Distribute...
Choosing a RDBMS <ul><li>Sometimes there is no choice (enforced by context) ! </li></ul><ul><li>Criteria </li></ul><ul><ul...
Talking to a RDBMS <ul><li>SQL  : Standard Query Language.  Except that  </li></ul><ul><ul><li>the standard is hard to fin...
Writing SQL SQL is too low-level, I don't ever want to see it  SQL is the most important part of my application, I won't l...
Data Definition Language (DDL) <ul><li>CREATE TABLE  author ( </li></ul><ul><li>author_id INTEGER PRIMARY KEY, </li></ul><...
Data Manipulation Language (DML) <ul><li>SELECT  author_name, distribution_name </li></ul><ul><li>FROM author INNER JOIN d...
Best practice : placeholders <ul><li>SELECT  author_name, distribution_name </li></ul><ul><li>FROM author INNER JOIN distr...
Perl DBI Basics
Architecture Database DBD driver DBI Object-Relational Mapper Perl program TIOOWTDI There is only one way to do it TAMMMWT...
DBD Drivers <ul><ul><li>Databases </li></ul></ul><ul><ul><ul><li>Adabas DB2 DBMaker Empress Illustra Informix Ingres Inter...
When SomeExoticDB has no driver  <ul><li>Quotes from DBI::DBD : </li></ul><ul><ul><ul><li>&quot; The first rule for creati...
DBI API <ul><li>handles </li></ul><ul><ul><li>the whole package (DBI) </li></ul></ul><ul><ul><li>driver handle ($dh) </li>...
Connecting <ul><li>my $dbh  = DBI-> connect ($connection_string); </li></ul><ul><li>my $dbh  = DBI-> connect ($connection_...
Some dbh attributes <ul><li>AutoCommit   </li></ul><ul><ul><li>if true, every statement is immediately committed </li></ul...
Data retrieval <ul><li>my $sth  = $dbh-> prepare ($sql); </li></ul><ul><li>$sth-> execute ( @bind_values ); </li></ul><ul>...
Other ways of fetching <ul><li>single row </li></ul><ul><ul><ul><li>fetchrow_array </li></ul></ul></ul><ul><ul><ul><li>fet...
Advanced Perl DBI
Transactions <ul><li>$dbh->{ RaiseError } = 1; # errors will raise exceptions </li></ul><ul><li>eval { </li></ul><ul><li>$...
Efficiency <ul><li>my $sth = $dbh->prepare(<<''); </li></ul><ul><li>SELECT author_id, author_name, e_mail </li></ul><ul><l...
Metadata <ul><li>datasources </li></ul><ul><ul><li>my @sources = DBI-> data_sources ($driver); </li></ul></ul><ul><li>tabl...
Lost connection <ul><li>manual recover </li></ul><ul><ul><ul><li>if ($dbh->errstr =~ /broken connection/i) { … }  </li></u...
Datatypes <ul><li>NULL    undef </li></ul><ul><li>INTEGER, VARCHAR, DATE    perl scalar </li></ul><ul><ul><li>usually DW...
Large objects <ul><li>usually : just scalars in memory </li></ul><ul><li>when reading : control BLOB size </li></ul><ul><u...
Tracing / profiling <ul><li>$dbh->trace($trace_setting, $trace_where) </li></ul><ul><ul><li>0 - Trace disabled.  </li></ul...
Stored procedures <ul><li>my $sth = $dbh->prepare($db_specific_sql); </li></ul><ul><li># prepare params to be passed to th...
Object-Relational Mapping (ORM)
ORM Principle r1 r2 ... c1 c2 c3 ... c3 c4 +c1: String +c2: String +c3:  class2 r1  :  class1 RDBMS r2  :  class1 Applicat...
ORM: What for ? <ul><li>[catalyst list] On Thu, 2006-06-08, Steve  wrote: </li></ul><ul><li>Not intending to start any sor...
ORM useful for … <ul><li>dynamic SQL </li></ul><ul><ul><li>navigation between tables </li></ul></ul><ul><ul><li>generate c...
Impedance mismatch <ul><li>SELECT c1, c2 FROM table1 </li></ul><ul><ul><ul><li>   missing  c3 , so cannot navigate to  cl...
ORM Landscape <ul><li>Leader </li></ul><ul><ul><li>DBIx::Class (a.k.a. DBIC) </li></ul></ul><ul><li>Also discussed here </...
Model (UML) Artist CD Track 1 * 1 *
DBIx::Class Schema <ul><li>package MyDatabase::Main; </li></ul><ul><li>use base qw/DBIx::Class::Schema/; </li></ul><ul><li...
DBIx::Class usage <ul><li>my $schema = MyDatabase::Main </li></ul><ul><li>->connect('dbi:SQLite:db/example.db'); </li></ul...
DBIx::DataModel Schema <ul><li>package MyDatabase; </li></ul><ul><li>use DBIx::DataModel; </li></ul><ul><li>DBIx::DataMode...
DBIx::DataModel usage <ul><li>my $dbh = DBI->connect('dbi:SQLite:db/example.db'); </li></ul><ul><li>MyDatabase->dbh($dbh);...
Conclusion
Further info <ul><li>Database textbooks </li></ul><ul><li>DBI manual ( L<DBI >, L< DBI:.FAQ >, L< DBI::Profile >) </li></u...
Bonus slides
Names for primary / foreign keys <ul><li>primary : unique;  foreign  : same name </li></ul><ul><li>author.author_id    di...
Locks and isolation levels <ul><li>Locks on rows </li></ul><ul><ul><li>shared  </li></ul></ul><ul><ul><ul><li>other client...
Cursors <ul><li>my $sql  = &quot;SELECT * FROM SomeTable  FOR UPDATE &quot;;  </li></ul><ul><li>my $sth1 = $dbh->prepare($...
Modeling (UML) Author Distribution Module 1 * 1 * ►  depends on * * ►  contains
Terminology Author Distribution Module 1 * 1 * ►  depends on * * ►  contains multiplicity association name class associati...
Implementation author_id author_name e_mail 1 * 1 * * * Author distrib_id module_id Dependency distrib_id distrib_name d_r...
Upcoming SlideShare
Loading in...5
×

Working with databases in Perl

6,970

Published on

An overview of the main questions/design issues when starting to work with databases in Perl

- choosing a database
- matching DB datatypes to Perl datatypes
- DBI architecture (handles, drivers, etc.)
- steps of DBI interaction : prepare/execute/fetch
- ORM principles and difficulties, ORMs on CPAN
- a few examples with DBIx::DataModel
- performance issues

First given at YAPC::EU::2009 in Lisbon. Updated version given at FPW2011 in Paris and YAPC::EU::2011 in Riga

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,970
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
147
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Working with databases in Perl"

  1. 1. Working with databases in Perl Tutorial for FPW::2011, Paris [email_address] Département Office
  2. 2. Overview <ul><li>intended audience : beginners </li></ul><ul><ul><li>in Perl </li></ul></ul><ul><ul><li>in Databases </li></ul></ul><ul><li>main topics </li></ul><ul><ul><li>Relational databases </li></ul></ul><ul><ul><li>Perl DBI basics </li></ul></ul><ul><ul><li>Advanced Perl DBI </li></ul></ul><ul><ul><li>Object-Relational Mappings </li></ul></ul><ul><li>disclaimer </li></ul><ul><ul><li>didn't have personal exposure to everything mentioned in this tutorial </li></ul></ul>
  3. 3. Relational databases RDBMS = Relational Database Management System
  4. 4. Relational model join on c3 filter Table (rows + columns) projection c1 c2 c3 1 foo 1 2 foo 2 3 bar 1 c3 c4 1 xx 2 yy c1 c2 c3 c4 1 foo 1 xx 2 foo 2 yy 3 bar 1 xx
  5. 5. Maybe you don't want a RDBMS <ul><li>Other solutions for persistency in Perl: </li></ul><ul><ul><ul><li>BerkeleyDB : persistent hashes / arrays </li></ul></ul></ul><ul><ul><ul><li>Judy : persistent dynamic arrays / hashes </li></ul></ul></ul><ul><ul><ul><li>Redis : persistent arrays / hashes / sets / sorted sets </li></ul></ul></ul><ul><ul><ul><li>CouchDB : OO/hierarchical database </li></ul></ul></ul><ul><ul><ul><li>MongoDB : document-oriented database </li></ul></ul></ul><ul><ul><ul><li>KiokuDB : persistent objects, front-end to BerkeleyDB / CouchDB / etc. </li></ul></ul></ul><ul><ul><ul><li>Plain Old File (using for example File::Tabular ) </li></ul></ul></ul><ul><ul><ul><li>KinoSearch : bunch of fields with fulltext indexing </li></ul></ul></ul><ul><ul><ul><li>LDAP : directory </li></ul></ul></ul><ul><ul><ul><li>Net::Riak : buckets and keys </li></ul></ul></ul><ul><ul><li>See http://en.wikipedia.org/wiki/NoSQL </li></ul></ul>
  6. 6. Features of RDBMS <ul><li>Relational </li></ul><ul><li>Indexing </li></ul><ul><li>Concurrency </li></ul><ul><li>Distributed </li></ul><ul><li>Transactions (commit / rollback ) </li></ul><ul><li>Authorization </li></ul><ul><li>Triggers and stored procedures </li></ul><ul><li>Internationalization </li></ul><ul><li>Fulltext </li></ul><ul><li>… </li></ul>
  7. 7. Choosing a RDBMS <ul><li>Sometimes there is no choice (enforced by context) ! </li></ul><ul><li>Criteria </li></ul><ul><ul><li>cost, proprietary / open source </li></ul></ul><ul><ul><li>volume </li></ul></ul><ul><ul><li>features </li></ul></ul><ul><ul><li>resources (CPU, RAM, etc.) </li></ul></ul><ul><ul><li>ease of installation / deployment / maintenance </li></ul></ul><ul><ul><li>stored procedures </li></ul></ul><ul><li>Common choices (open source) </li></ul><ul><ul><li>SQLite (file-based) </li></ul></ul><ul><ul><li>mysql </li></ul></ul><ul><ul><li>Postgres </li></ul></ul><ul><ul><ul><li>Postgres can have server-side procedures in Perl ! </li></ul></ul></ul>
  8. 8. Talking to a RDBMS <ul><li>SQL : Standard Query Language. Except that </li></ul><ul><ul><li>the standard is hard to find (not publicly available) </li></ul></ul><ul><ul><li>vendors rarely implement the full standard </li></ul></ul><ul><ul><li>most vendors have non-standard extensions </li></ul></ul><ul><ul><li>it's not only about queries </li></ul></ul><ul><ul><ul><li>DML : Data Manipulation Language </li></ul></ul></ul><ul><ul><ul><li>DDL : Data Definition Language </li></ul></ul></ul>
  9. 9. Writing SQL SQL is too low-level, I don't ever want to see it SQL is the most important part of my application, I won't let anybody write it for me
  10. 10. Data Definition Language (DDL) <ul><li>CREATE TABLE author ( </li></ul><ul><li>author_id INTEGER PRIMARY KEY, </li></ul><ul><li>author_name VARCHAR(20), </li></ul><ul><li>e_mail VARCHAR(20), </li></ul><ul><li>… </li></ul><ul><li>); </li></ul><ul><li>CREATE/ALTER/DROP/RENAME </li></ul><ul><li>DATABASE </li></ul><ul><li>INDEX </li></ul><ul><li>VIEW </li></ul><ul><li>TRIGGER </li></ul>
  11. 11. Data Manipulation Language (DML) <ul><li>SELECT author_name, distribution_name </li></ul><ul><li>FROM author INNER JOIN distribution </li></ul><ul><li>ON author.author_id = distribution.author_id </li></ul><ul><li>WHERE distribution_name like 'DBD::%'; </li></ul><ul><li>INSERT INTO author ( author_id, author_name, e_mail ) </li></ul><ul><li>VALUES ( 123, 'JFOOBAR', 'john@foobar.com' ); </li></ul><ul><li>UPDATE author </li></ul><ul><li>SET e_mail = 'john@foobar.com' </li></ul><ul><li>WHERE author_id = 3456; </li></ul><ul><li>DELETE FROM author </li></ul><ul><li>WHERE author_id = 3456; </li></ul>
  12. 12. Best practice : placeholders <ul><li>SELECT author_name, distribution_name </li></ul><ul><li>FROM author INNER JOIN distribution </li></ul><ul><li>ON author.author_id = distribution.author_id </li></ul><ul><li>WHERE distribution_name like ? ; </li></ul><ul><li>INSERT INTO author ( author_id, author_name, e_mail ) </li></ul><ul><li>VALUES ( ? , ? , ? ); </li></ul><ul><li>UPDATE author </li></ul><ul><li>SET e_mail = ? </li></ul><ul><li>WHERE author_id = ? ; </li></ul><ul><li>DELETE FROM author </li></ul><ul><li>WHERE author_id = ? ; </li></ul><ul><li> no type distinction (int/string)  statements can be cached </li></ul><ul><li> avoid SQL injection problems </li></ul><ul><ul><li>SELECT * FROM foo </li></ul></ul><ul><ul><li>WHERE val = $x ; </li></ul></ul><ul><ul><li>$x eq '123; DROP TABLE foo' </li></ul></ul><ul><li>sometimes other syntax (for ex. $1, $2) </li></ul>
  13. 13. Perl DBI Basics
  14. 14. Architecture Database DBD driver DBI Object-Relational Mapper Perl program TIOOWTDI There is only one way to do it TAMMMWTDI There are many, many many ways to do it TIMTOWTDI There is more than one way to do it
  15. 15. DBD Drivers <ul><ul><li>Databases </li></ul></ul><ul><ul><ul><li>Adabas DB2 DBMaker Empress Illustra Informix Ingres InterBase MaxDB Mimer Oracle Ovrimos PO Pg PrimeBase QBase Redbase SQLAnywhere SQLite Solid Sqlflex Sybase Unify mSQL monetdb mysql </li></ul></ul></ul><ul><ul><li>Other kinds of data stores </li></ul></ul><ul><ul><ul><li>CSV DBM Excel File iPod LDAP </li></ul></ul></ul><ul><ul><li>Proxy, relay, etc </li></ul></ul><ul><ul><ul><li>ADO Gofer JDBC Multi Multiplex ODBC Proxy SQLRelay </li></ul></ul></ul><ul><ul><li>Fake, test </li></ul></ul><ul><ul><ul><li>NullP Mock RAM Sponge </li></ul></ul></ul>
  16. 16. When SomeExoticDB has no driver <ul><li>Quotes from DBI::DBD : </li></ul><ul><ul><ul><li>&quot; The first rule for creating a new database driver for the Perl DBI is very simple: DON'T! &quot; </li></ul></ul></ul><ul><ul><ul><li>&quot; The second rule for creating a new database driver for the Perl DBI is also very simple: Don't -- get someone else to do it for you! &quot; </li></ul></ul></ul><ul><li>nevertheless there is good advice/examples </li></ul><ul><ul><li>see DBI::DBD </li></ul></ul><ul><li>Other solution : forward to other drivers </li></ul><ul><ul><li>ODBC (even on Unix) </li></ul></ul><ul><ul><li>JDBC </li></ul></ul><ul><ul><li>SQLRelay </li></ul></ul>
  17. 17. DBI API <ul><li>handles </li></ul><ul><ul><li>the whole package (DBI) </li></ul></ul><ul><ul><li>driver handle ($dh) </li></ul></ul><ul><ul><li>database handle ($dbh) </li></ul></ul><ul><ul><li>statement handle ($sth) </li></ul></ul><ul><li>interacting with handles </li></ul><ul><ul><li>objet-oriented </li></ul></ul><ul><ul><ul><li>->connect(…), ->prepare(…), ->execute(...), … </li></ul></ul></ul><ul><ul><li>tied hash </li></ul></ul><ul><ul><ul><li>->{AutoCommit}, ->{NAME_lc}, ->{CursorName}, … </li></ul></ul></ul>
  18. 18. Connecting <ul><li>my $dbh = DBI-> connect ($connection_string); </li></ul><ul><li>my $dbh = DBI-> connect ($connection_string, </li></ul><ul><li> $user, </li></ul><ul><li> $password, </li></ul><ul><li> { %attributes } ); </li></ul><ul><li>my $dbh = DBI-> connect_cached ( @args ); </li></ul>
  19. 19. Some dbh attributes <ul><li>AutoCommit </li></ul><ul><ul><li>if true, every statement is immediately committed </li></ul></ul><ul><ul><li>if false, need to call </li></ul></ul><ul><ul><ul><li>$dbh->begin_work(); </li></ul></ul></ul><ul><ul><ul><li>… # inserts, updates, deletes </li></ul></ul></ul><ul><ul><ul><li>$dbh->commit(); </li></ul></ul></ul><ul><li>RaiseError </li></ul><ul><ul><li>like autodie for standard Perl functions : errors raise exceptions </li></ul></ul><ul><li>see also </li></ul><ul><ul><li>PrintError </li></ul></ul><ul><ul><li>HandleError </li></ul></ul><ul><ul><li>ShowErrorStatement </li></ul></ul><ul><li>and also </li></ul><ul><ul><li>LongReadLen </li></ul></ul><ul><ul><li>LongTrunkOK </li></ul></ul><ul><ul><li>RowCacheSize </li></ul></ul><ul><ul><li>… </li></ul></ul> hash API : attributes can be set dynamically [ local ] $dbh->{$attr_name} = $val <ul><li>peek at $dbh internals </li></ul><ul><li>DB<1> x $dbh  {} </li></ul><ul><li>DB< 2 > x tied %$dbh  {…} </li></ul>
  20. 20. Data retrieval <ul><li>my $sth = $dbh-> prepare ($sql); </li></ul><ul><li>$sth-> execute ( @bind_values ); </li></ul><ul><li>my @columns = @{$sth->{NAME}}; </li></ul><ul><li>while (my $row_aref = $sth-> fetch ) { </li></ul><ul><li>… </li></ul><ul><li>} </li></ul><ul><li># or </li></ul><ul><li>$dbh-> do ($sql); </li></ul><ul><li>see also : prepare_cached </li></ul>
  21. 21. Other ways of fetching <ul><li>single row </li></ul><ul><ul><ul><li>fetchrow_array </li></ul></ul></ul><ul><ul><ul><li>fetchrow_arrayref (a.k.a fetch) </li></ul></ul></ul><ul><ul><ul><li>fetchrow_hashref </li></ul></ul></ul><ul><li>lists of rows (with optional slicing) </li></ul><ul><ul><ul><li>fetchall_arrayref </li></ul></ul></ul><ul><ul><ul><li>fetchall_hashref </li></ul></ul></ul><ul><li>prepare, execute and fetch </li></ul><ul><ul><ul><li>selectall_arrayref </li></ul></ul></ul><ul><ul><ul><li>selectall_hashref </li></ul></ul></ul><ul><li>vertical slice </li></ul><ul><ul><ul><li>selectcol_arrayref </li></ul></ul></ul> little DBI support for cursors
  22. 22. Advanced Perl DBI
  23. 23. Transactions <ul><li>$dbh->{ RaiseError } = 1; # errors will raise exceptions </li></ul><ul><li>eval { </li></ul><ul><li>$dbh-> begin_work (); # will turn off AutoCommit </li></ul><ul><li>… # inserts, updates, deletes </li></ul><ul><li>$dbh-> commit (); </li></ul><ul><li>}; </li></ul><ul><li>if ($@) { </li></ul><ul><li>my $err = $@; </li></ul><ul><li>eval {$dbh-> rollback ()}; </li></ul><ul><li>my $rollback_result = $@ || &quot;SUCCESS&quot;; </li></ul><ul><li>die &quot;FAILED TRANSACTION : $err&quot; </li></ul><ul><li> . &quot;; ROLLBACK: $rollback_result&quot;; </li></ul><ul><li>} </li></ul><ul><li>encapsulated in DBIx::Transaction or ORMs </li></ul><ul><li>$schema-> transaction ( sub { …} ); </li></ul><ul><li>nested transactions : must keep track of transaction depth </li></ul><ul><li>savepoint / release : only in DBIx::Class </li></ul>
  24. 24. Efficiency <ul><li>my $sth = $dbh->prepare(<<''); </li></ul><ul><li>SELECT author_id, author_name, e_mail </li></ul><ul><li>FROM author </li></ul><ul><li>my ($id, $name, $e_mail); </li></ul><ul><li>$sth->execute; </li></ul><ul><li>$sth-> bind_columns ( ($id, $name, $e_mail)); </li></ul><ul><li>while ($sth->fetch) { </li></ul><ul><li>print &quot;author $id is $name at $e_mailn&quot;; </li></ul><ul><li>} </li></ul> avoids cost of allocating / deallocating Perl variables  don't store a reference and reuse it after another fetch
  25. 25. Metadata <ul><li>datasources </li></ul><ul><ul><li>my @sources = DBI-> data_sources ($driver); </li></ul></ul><ul><li>table_info </li></ul><ul><ul><li>my $sth = $dbh-> table_info (@search_criteria); </li></ul></ul><ul><ul><li>while (my $row = $sth->fetchrow_hashref) { </li></ul></ul><ul><ul><li>print &quot;$row->{TABLE_NAME} : $row->{TABLE_TYPE}n&quot;; </li></ul></ul><ul><ul><li>} </li></ul></ul><ul><li>others </li></ul><ul><ul><li>column_info() </li></ul></ul><ul><ul><li>primary_key_info() </li></ul></ul><ul><ul><li>foreign_key_info() </li></ul></ul> many drivers only have partial implementations
  26. 26. Lost connection <ul><li>manual recover </li></ul><ul><ul><ul><li>if ($dbh->errstr =~ /broken connection/i) { … } </li></ul></ul></ul><ul><li>DBIx::RetryOverDisconnects </li></ul><ul><ul><li>intercepts requests (prepare, execute, …) </li></ul></ul><ul><ul><li>filters errors </li></ul></ul><ul><ul><li>attemps to reconnect and restart the transaction </li></ul></ul><ul><li>some ORMs have their own layer for recovering connections </li></ul><ul><li>some drivers have their own mechanism </li></ul><ul><ul><ul><li>$dbh->{mysql_auto_reconnect} = 1; </li></ul></ul></ul>
  27. 27. Datatypes <ul><li>NULL  undef </li></ul><ul><li>INTEGER, VARCHAR, DATE  perl scalar </li></ul><ul><ul><li>usually DWIM works </li></ul></ul><ul><ul><li>if needed, can specify explicitly </li></ul></ul><ul><ul><ul><li>$sth->bind_param($col_num, $value, SQL_DATETIME); </li></ul></ul></ul><ul><li>BLOB  perl scalar </li></ul><ul><li>ARRAY (Postgres)  arrayref </li></ul>
  28. 28. Large objects <ul><li>usually : just scalars in memory </li></ul><ul><li>when reading : control BLOB size </li></ul><ul><ul><li>$dbh->{LongReadLen} = $max_bytes; </li></ul></ul><ul><ul><li>$dbh->{LongTrunkOK} = 1 </li></ul></ul><ul><li>when writing : can inform the driver </li></ul><ul><ul><li>$sth->bind_param($ix, $blob, SQL_BLOB); </li></ul></ul><ul><li>driver-specific stream API. Ex : </li></ul><ul><ul><li>Pg : pg_lo_open, pg_lo_write, pg_lo_lseek </li></ul></ul><ul><ul><li>Oracle : ora_lob_read(…), ora_lob_write(…), ora_lob_append(…) </li></ul></ul>
  29. 29. Tracing / profiling <ul><li>$dbh->trace($trace_setting, $trace_where) </li></ul><ul><ul><li>0 - Trace disabled. </li></ul></ul><ul><ul><li>1 - Trace top-level DBI method calls returning with results or errors. </li></ul></ul><ul><ul><li>2 - As above, adding tracing of top-level method entry with parameters. </li></ul></ul><ul><ul><li>3 - As above, adding some high-level information from the driver and some internal information from the DBI. </li></ul></ul><ul><li>$dbh->{Profile} = 2; # profile at the statement level </li></ul><ul><ul><li>many powerful options </li></ul></ul><ul><ul><li>see L< DBI::Profile > </li></ul></ul>
  30. 30. Stored procedures <ul><li>my $sth = $dbh->prepare($db_specific_sql); </li></ul><ul><li># prepare params to be passed to the called procedure </li></ul><ul><li>$sth-> bind_param (1, $val1); </li></ul><ul><li>$sth->bind_param(2, $val2); </li></ul><ul><li># prepare memory locations to receive the results </li></ul><ul><li>$sth-> bind_param_inout (3, $result1); </li></ul><ul><li>$sth->bind_param_inout(4, $result2); </li></ul><ul><li># execute the whole thing </li></ul><ul><li>$sth->execute; </li></ul>
  31. 31. Object-Relational Mapping (ORM)
  32. 32. ORM Principle r1 r2 ... c1 c2 c3 ... c3 c4 +c1: String +c2: String +c3: class2 r1 : class1 RDBMS r2 : class1 Application table1 table2
  33. 33. ORM: What for ? <ul><li>[catalyst list] On Thu, 2006-06-08, Steve wrote: </li></ul><ul><li>Not intending to start any sort of rancorous discussion, </li></ul><ul><li>but I was wondering whether someone could illuminate </li></ul><ul><li>me a little? </li></ul><ul><li>I'm comfortable with SQL, and with DBI. I write basic </li></ul><ul><li>SQL that runs just fine on all databases, or more </li></ul><ul><li>complex SQL when I want to target a single database </li></ul><ul><li>(ususally postgresql). </li></ul><ul><li>What value does an ORM add for a user like me? </li></ul>
  34. 34. ORM useful for … <ul><li>dynamic SQL </li></ul><ul><ul><li>navigation between tables </li></ul></ul><ul><ul><li>generate complex SQL queries from Perl datastructures </li></ul></ul><ul><ul><li>better than phrasebook or string concatenation </li></ul></ul><ul><li>automatic data conversions (inflation / deflation) </li></ul><ul><li>expansion of tree data structures coded in the relational model </li></ul><ul><li>transaction encapsulation </li></ul><ul><li>data validation </li></ul><ul><li>computed fields </li></ul><ul><li>caching </li></ul><ul><li>schema deployment </li></ul><ul><li>… </li></ul> See Also : http://lists.scsys.co.uk/pipermail/catalyst/2006-June/008059.html
  35. 35. Impedance mismatch <ul><li>SELECT c1, c2 FROM table1 </li></ul><ul><ul><ul><li> missing c3 , so cannot navigate to class2 </li></ul></ul></ul><ul><ul><ul><li>is it a valid instance of class1 ? </li></ul></ul></ul><ul><li>SELECT * FROM table1 LEFT JOIN table2 ON … </li></ul><ul><ul><ul><li> what to do with the c4 column ? </li></ul></ul></ul><ul><ul><ul><li>is it a valid instance of class1 ? </li></ul></ul></ul><ul><li>SELECT c1, c2, length(c2) AS l_c2 FROM table1 </li></ul><ul><ul><ul><li> no predeclared method in class1 for accessing l_c2 </li></ul></ul></ul>c1 c2 c3 c3 c4 +c1: String +c2: String +c3: class2 r1 : class1 RDBMS RAM table1 table2
  36. 36. ORM Landscape <ul><li>Leader </li></ul><ul><ul><li>DBIx::Class (a.k.a. DBIC) </li></ul></ul><ul><li>Also discussed here </li></ul><ul><ul><li>DBIx::DataModel </li></ul></ul><ul><li>Many others </li></ul><ul><ul><li>Rose::DB, Jifty::DBI, Fey::ORM, ORM, DBIx::ORM::Declarative, Tangram, Coat::Persistent, DBR, DBIx::Sunny, DBIx::Skinny, DBI::Easy, … </li></ul></ul>
  37. 37. Model (UML) Artist CD Track 1 * 1 *
  38. 38. DBIx::Class Schema <ul><li>package MyDatabase::Main; </li></ul><ul><li>use base qw/DBIx::Class::Schema/; </li></ul><ul><li>__PACKAGE__->load_namespaces; </li></ul><ul><li>package MyDatabase::Main::Result::Artist; </li></ul><ul><li>use base qw/DBIx::Class/; </li></ul><ul><li>__PACKAGE__->load_components(qw/PK::Auto Core/); </li></ul><ul><li>__PACKAGE__->table('artist'); </li></ul><ul><li>__PACKAGE__->add_columns(qw/ artistid name /); </li></ul><ul><li>__PACKAGE__->set_primary_key('artistid'); </li></ul><ul><li>__PACKAGE__->has_many('cds' => </li></ul><ul><li>'MyDatabase::Main::Result::Cd'); </li></ul><ul><li>package ... </li></ul><ul><li>... </li></ul>
  39. 39. DBIx::Class usage <ul><li>my $schema = MyDatabase::Main </li></ul><ul><li>->connect('dbi:SQLite:db/example.db'); </li></ul><ul><li>my @artists = (['Michael Jackson'], ['Eminem']); </li></ul><ul><li>$schema->populate('Artist', [ </li></ul><ul><li>[qw/name/], </li></ul><ul><li>@artists, </li></ul><ul><li>]); </li></ul><ul><li>my $rs = $schema->resultset('Track')->search( </li></ul><ul><li>{ </li></ul><ul><li>'cd.title' => $cdtitle </li></ul><ul><li>}, </li></ul><ul><li>{ </li></ul><ul><li>join => [qw/ cd /], </li></ul><ul><li>} </li></ul><ul><li>); </li></ul><ul><li>while (my $track = $rs->next) { </li></ul><ul><li>print $track->title . &quot;n&quot;; </li></ul><ul><li>} </li></ul>
  40. 40. DBIx::DataModel Schema <ul><li>package MyDatabase; </li></ul><ul><li>use DBIx::DataModel; </li></ul><ul><li>DBIx::DataModel->Schema(__PACKAGE__) </li></ul><ul><li>->Table(qw/Artist artist artistid/) </li></ul><ul><li>->Table(qw/CD cd cdid /) </li></ul><ul><li>->Table(qw/Track track trackid /) </li></ul><ul><li>->Association([qw/Artist artist 1 /], </li></ul><ul><li>[qw/CD cds 0..* /]) </li></ul><ul><li>->Composition([qw/CD cd 1 /], </li></ul><ul><li>[qw/Track tracks 1..* /]); </li></ul>
  41. 41. DBIx::DataModel usage <ul><li>my $dbh = DBI->connect('dbi:SQLite:db/example.db'); </li></ul><ul><li>MyDatabase->dbh($dbh); </li></ul><ul><li>my @artists = (['Michael Jackson'], ['Eminem']); </li></ul><ul><li>MyDatabase::Artist->insert(['name'], @artists); </li></ul><ul><li>my $statement = MyDatabase->join(qw/CD tracks/)->select( </li></ul><ul><li>-columns => [qw/track.title|trtitle …/], </li></ul><ul><li>-where => { 'cd.title' => $cdtitle }, </li></ul><ul><li>-resultAs => 'statement', # default : arrayref of rows </li></ul><ul><li>); </li></ul><ul><li>while (my $track = $statement->next) { </li></ul><ul><li>print &quot;$track->{trtitle}n&quot;; </li></ul><ul><li>} </li></ul>
  42. 42. Conclusion
  43. 43. Further info <ul><li>Database textbooks </li></ul><ul><li>DBI manual ( L<DBI >, L< DBI:.FAQ >, L< DBI::Profile >) </li></ul><ul><li>Book : &quot;Programming the DBI&quot; </li></ul><ul><li>Vendor's manuals </li></ul><ul><li>ORMs </li></ul><ul><ul><li>DBIx::Class::Manual </li></ul></ul><ul><ul><li>DBIx::DataModel </li></ul></ul> mastering databases requires a lot of reading !
  44. 44. Bonus slides
  45. 45. Names for primary / foreign keys <ul><li>primary : unique; foreign : same name </li></ul><ul><li>author.author_id  distribution.author_id </li></ul><ul><ul><ul><li>RDBMS knows how to perform joins ( &quot;NATURAL JOIN&quot; ) </li></ul></ul></ul><ul><li>primary : constant; foreign : unique based on table + column name </li></ul><ul><li>author.id  distribution.author_id </li></ul><ul><ul><ul><li>ORM knows how to perform joins (RoR ActiveRecord) </li></ul></ul></ul><ul><ul><ul><li>SELECT * FROM table1, table2 ….  which id ? </li></ul></ul></ul><ul><li>primary : constant; foreign : just table name </li></ul><ul><li>author.id  distribution.author </li></ul><ul><ul><ul><li>$a_distrib->author() : foreign key or related record ? </li></ul></ul></ul> columns for joins should always be indexed
  46. 46. Locks and isolation levels <ul><li>Locks on rows </li></ul><ul><ul><li>shared </li></ul></ul><ul><ul><ul><li>other clients can also get a shared lock </li></ul></ul></ul><ul><ul><ul><li>requests for exclusive lock must wait </li></ul></ul></ul><ul><ul><li>exclusive </li></ul></ul><ul><ul><ul><li>all other requests for locks must wait </li></ul></ul></ul><ul><li>Intention locks (on whole tables) </li></ul><ul><ul><li>Intent shared </li></ul></ul><ul><ul><li>Intent exclusive </li></ul></ul><ul><li>Isolation levels </li></ul><ul><ul><li>read-uncommitted </li></ul></ul><ul><ul><li>read-committed </li></ul></ul><ul><ul><li>repeatable-read </li></ul></ul><ul><ul><li>serializable </li></ul></ul>SELECT … FOR READ ONLY SELECT … FOR UPDATE SELECT … LOCK IN SHARE MODE LOCK TABLE(S) … READ/WRITE SET TRANSACTION ISOLATION LEVEL …
  47. 47. Cursors <ul><li>my $sql = &quot;SELECT * FROM SomeTable FOR UPDATE &quot;; </li></ul><ul><li>my $sth1 = $dbh->prepare($sql); </li></ul><ul><li>$sth1->execute(); </li></ul><ul><li>my $curr = &quot;WHERE CURRENT OF $sth1->{CursorName} &quot;; </li></ul><ul><li>while (my $row = $sth1->fetch) { </li></ul><ul><li>if (…) { </li></ul><ul><li> $dbh->do(&quot;D ELETE FROM SomeTable WHERE $curr&quot;); </li></ul><ul><li>} else { </li></ul><ul><li>my $sth2 = $dbh->prepare( </li></ul><ul><li>&quot;UPDATE SomeTable SET col = ? WHERE $curr&quot;); </li></ul><ul><li> $sth2->execute($new_val); </li></ul><ul><li>… </li></ul>
  48. 48. Modeling (UML) Author Distribution Module 1 * 1 * ► depends on * * ► contains
  49. 49. Terminology Author Distribution Module 1 * 1 * ► depends on * * ► contains multiplicity association name class association composition
  50. 50. Implementation author_id author_name e_mail 1 * 1 * * * Author distrib_id module_id Dependency distrib_id distrib_name d_release author_id Distribution module_id module_name distrib_id Module 1 1 link table for n-to-n association
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×