• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
DBIx::Class vs. DBix::DataModel
 

DBIx::Class vs. DBix::DataModel

on

  • 1,340 views

 

Statistics

Views

Total Views
1,340
Views on SlideShare
1,340
Embed Views
0

Actions

Likes
2
Downloads
20
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    DBIx::Class vs. DBix::DataModel DBIx::Class vs. DBix::DataModel Presentation Transcript

    • DBIx::Class Département Office vs.DBIx::DataModel FPW 2012, Strasbourg laurent.dami@justice.ge.ch Département Office 29.06.12 - Page 1
    • Agenda• Introduction• Schema definition• Data retrieval• Joins• Classes and methods• Resultset/Statement objects• Customization• Conclusion
    • Introduction
    • ORM layer Perl program Object-Relational Mapper DBI DBD driver Database
    • ORM useful for …• dynamic SQL generation – navigation between tables – generate complex SQL queries from Perl datastructures – better than phrasebook or string concatenation• automatic data conversions (inflation / deflation)• transaction encapsulation• data validation• computed fields• caching• expansion of tree data structures coded in the relational model• …
    • CPAN ORM Landscape• Discussed here – DBIx::Class (a.k.a. DBIC) – DBIx::DataModel (a.k.a. DBIDM) DISCLAIMER - Im not an expert of DBIC - Ill try to be neutral in comparisons, but … - Wont cover all topics• Many other – Rose::DB, Jifty::DBI, Fey::ORM, ORM, DBIx::ORM::Declarative, Tangram, Coat::Persistent, ORLite, DBR, DBIx::Sunny, DBIx::Skinny, DBI::Easy, …
    • A bit of 2005 history Class::DBI (1999) SQL::Abstract (2001) DBIx::DataModel private (feb.05) Class::DBI::Sweet (29.05.05) - SQL::Abstract Class::DBI::Sweet 0.02 (29.06.05) -prefetch joinsDBIx::Class 0.01 (08.08.05) - SQL::AbstractDBIx::Class 0.03 (19.09.05) DBIx::DataModel 0.10 (16.09.05) - prefetch joins
    • Some figures• DBIC • DBIDM – thousands of users – a couple of users – dozens of contributors – 1 contributor – 162 files – 31 files – 167 packages – 40 packages – 1042 subs/methods – 175 subs/methods – 16759 lines of Perl – 3098 lines of Perl – 1817 comment lines – 613 comment lines – 19589 lines of POD – 8146 lines of POD – 70 transitive dependencies – 54 transitive dependencies
    • Our example : CPAN model straight from CPAN::SQLite Author 1 * 1 contains ► 1..* Distribution Module * * depends_on ►
    • Département OfficeSchema definition 29.06.12 - Page 1
    • DBIC Schema class Département Officeuse utf8;package FPW12::DBIC;use strict;use warnings;use base DBIx::Class::Schema;__PACKAGE__->load_namespaces;1; 29.06.12 - Page 1
    • DBIC Dist table class Département Officeuse utf8;package FPW12::DBIC::Result::Dist;use strict; use warnings;use base DBIx::Class::Core;__PACKAGE__->table("dists");__PACKAGE__->add_columns( dist_id => { data_type => "integer", is_nullable => 0, is_auto_increment => 1 }, dist_vers => { data_type => "varchar", is_nullable => 1, size => 20 }, ... # other columns );__PACKAGE__->set_primary_key("dist_id");__PACKAGE__->belongs_to (auth, FPW12::DBIC::Result::Auth, auth_id);__PACKAGE__->has_many (mods, FPW12::DBIC::Result::Mod, dist_id);__PACKAGE__->has_many (depends, FPW12::DBIC::Result::Depends, dist_id);__PACKAGE__->many_to_many(prereq_mods => depends, mod);# idem for classes Auth, Mod, etc. 29.06.12 - Page 1
    • DBIDM centralized declarationsuse DBIx::DataModel;DBIx::DataModel->Schema("FPW12::DBIDM")# class table primary key# ===== ===== ===========->Table(qw/Auth auths auth_id /)->Table(qw/Dist dists dist_id /)->Table(qw/Mod mods mod_id /)->Table(qw/Depends depends dist_id mod_id/)# class role multipl. (optional join keys)# ===== ==== ======= ====================->Association([qw/Auth auth 1 auth_id /], [qw/Dist dists * auth_id /])->Composition([qw/Dist dist 1 /], [qw/Mod mods * /])->Association([qw/Dist dist 1 /], [qw/Depends depends * /])->Association([qw/Mod mod 1 /], [qw/Depends depends * /])->Association([qw/Dist used_in_dist * depends distrib /], [qw/Mod prereq_mods * depends mod /]);
    • Schema def. comparison Département Office• DBIC • DBIDM – one file for each class – centralized file – regular Perl classes – dynamic class creation – full column info – no column info (except pkey) – 1-way "relationships" – 2-ways "associations" with • belongs_to • multiplicities • has_many • role names • may_have • diff association/composition • …. – custom join conditions – custom column names • any operator • only equalities • foreign and self 29.06.12 - Page 1
    • Département OfficeData retrieval 29.06.12 - Page 1
    • DBIC: Search Département Officeuse FPW12::DBIC;my $schema = FPW12::DBIC->connect($data_source);my @dists = $schema->resultset(Dist)->search( {dist_name => {-like => DBIx%}}, {columns => [qw/dist_name dist_vers/]},);foreach my $dist (@dists) { printf "%s (%s)n", $dist->dist_name, $dist->dist_vers;} 29.06.12 - Page 1
    • DBIDM: Search Département Officeuse FPW12::DBIDM;use DBI;my $datasource = "dbi:SQLite:dbname=../cpandb.sql";my $dbh = DBI->connect($datasource, "", "", {RaiseError => 1, AutoCommit => 1});FPW12::DBIDM->dbh($dbh);my $dists = FPW12::DBIDM::Dist->select( -columns => [qw/dist_name dist_vers/], -where => {dist_name => {-like => DBIx%}},);foreach my $dist (@$dists) { print "$dist->{dist_name} ($dist->{dist_vers})n";} 29.06.12 - Page 1
    • Simple search comparison Département Office• DBIC • DBIDM – schema is an object – schema is a class (default) or an object – result is a list or a resultset object – result is an arrayref (default) – accessor methods for columns – hash entries for columns – uses SQL::Abstract – uses SQL::Abstract::More • mostly hidden • user can supply a custom $sqla obj 29.06.12 - Page 1
    • SQL::Abstract new() Département Office• special operators – ex: DBMS-independent fulltext search # where {field => {-contains => [qw/foo bar/]} my $sqla = SQL::Abstract->new(special_ops => [ {regex => qr/^contains(:?_all|_any)?$/i, handler => sub { my ($self, $field, $op, $arg) = @_; my $connector = ($op =~ /any$/) ? | : & ; my @vals = ref $arg ? @$arg : ($arg); @vals = map { split /s+/ } grep {$_} @vals; my $sql = sprintf "CONTAINS($field, %s) > 0", join $connector, @vals; return ($sql); # no @bind } ]); 29.06.12 - Page 1
    • DBIC: Find single record Département Officemy $auth1 = $schema->resultset(Auth) ->find(123);my $auth2 = $schema->resultset(Auth) ->find({cpanid => DAMI}); 29.06.12 - Page 1
    • DBIDM: Find single record Département Officemy $auth1 = FPW12::DBIDM::Auth->fetch(123);my $auth2 = FPW12::DBIDM::Auth->search( -where => {cpanid => DAMI}, -result_as => firstrow,); 29.06.12 - Page 1
    • DBIC: Search args Département Office• 1st arg : "where" criteria• 2nd arg : attributes – distinct – order_by – group_by, having – columns # list of columns to retrieve from main table – +columns # additional columns from joined tables or from functions – join # see later – prefetch # see later – for – page, rows, offset, etc. – cache 29.06.12 - Page 1
    • DBIDM: Select args my $result = $source->select( -columns => @columns,, -where => %where, -group_by => @groupings, -having => %criteria, -order_by => @order, -for => read only, -post_SQL => sub { … }, -pre_exec => sub { … }, -post_exec => sub { … }, -post_bless => sub { … }, -page_size => …, -page_index => …, -limit => …, -offset => …, -column_types => %types, -result_as => $result_type, );
    • DBIDM: Polymorphic result-result_as => – rows (default) : arrayref of row objects – firstrow : a single row object (or undef) – hashref : hashref keyed by primary keys – [hashref => @cols] : cascaded hashref – flat_arrayref : flattened values from each row – statement : a statement object (iterator) – fast_statement : statement reusing same memory – sth : DBI statement handle – sql : ($sql, @bind_values) – subquery : ["($sql)", @bind]  dont need method variants : select_hashref(), select_arrayref(), etc.
    • DBIDM: Fast statement• like a regular statement – but reuses the same memory location for each row – see DBI::bind_col() my $statement = $source->select( . . . , -result_as => fast_statement ); while (my $row = $statement->next) { . . . # DO THIS : print $row->{col1}, $row->{col2} # BUT DONT DO THIS : push @results, $row; }
    • Advanced search comparison• DBIC • DBIDM – find by any unique constraint – fetch by primary key only – "inner" vs "outer" columns – all columns equal citizens – result is context-dependent – polymorphic result – optional caching – no caching – statement-specific inflators – callbacks
    • Joins
    • Task : list distributions and their authors Author 1 * 1 contains ► 1..* Distribution Module * * depends_on ►
    • DBIC: Joinmy @dists = $schema->resultset(Dist)->search( {dist_name => {-like => DBIx%}}, {columns => [qw/dist_name dist_vers/], join => auth, +columns => [{fullname => auth.fullname}], },);foreach my $dist (@dists) { printf "%s (%s) by %sn", $dist->dist_name, $dist->dist_vers, $dist->get_column(fullname); # no accessor meth}
    • DBIC: Join with prefetchmy @dists = $schema->resultset(Dist)->search( {dist_name => {-like => DBIx%}}, {columns => [qw/dist_name dist_vers/], prefetch => auth, },);foreach my $dist (@dists) { printf "%s (%s) by %sn", $dist->dist_name, $dist->dist_vers, $dist->auth->fullname; # already in memory}
    • DBIDM: Joinmy $rows = FPW12::DBIDM->join(qw/Dist auth/)->select( -columns => [qw/dist_name dist_vers fullname/], -where => {dist_name => {-like => DBIx%}},);foreach my $row (@$rows) { print "$row->{dist_name} ($row->{dist_vers}) " . "by $row->{fullname}n";} Dist Auth DBIDM::Source::Join new class created on the fly ..::AutoJoin::…
    • Multiple joins• DBICjoin => [ { abc => { def => ghi } }, { jkl => mno }, pqr ]• DBIDM->join(qw/Root abc def ghi jkl mno pqr/)
    • Join from an existing record• DBICmy $rs = $auth->dists(undef, {join|prefetch => mods});• DBIDMmy $rows = $auth->join(qw/dists mods/)->select;
    • DBIC: left/inner joins• attribute when declaring relationships # in a Book class (where Author has_many Books) __PACKAGE__->belongs_to( author => My::DBIC::Schema::Author, author, { join_type => left } );• cannot change later (when invoking the relationship)
    • DBIDM: Left / inner joins->Association([qw/Author author 1 /], [qw/Distribution distribs 0..* /]) # default : LEFT OUTER JOIN->Composition([qw/Distribution distrib 1 /], [qw/Module modules 1..* /]); # default : INNER JOIN# but defaults can be overriddenMy::DB->join([qw/Author <=> distribs/)-> . . .My::DB->join([qw/Distribution => modules /)-> . . .
    • Join comparison• DBIC • DBIDM – rows belong to 1 table class only – rows inherit from all joined tables – joined columns are either – flattening of all columns • "side-products", • prefetched (all of them, no choice) – fixed join type – default join type, can be overridden
    • Speed
    • Select speed Département Office list mods join Auth dist mods (109349 rows) (113895 rows)DBI 0.43 secs 1.36 secsDBIC regular 11.09 secs 15.50 secsDBIC prefetch n.a. 146.29 secsDBIC hashref inflator 10.06 secs 14.17 secsDBIDM regular 4.00 secs 5.01 secsDBIDM 2.25 secs 3.28 secsfast_statement 29.06.12 - Page 1
    • Insert speed Département Office insert (72993 rows)DBI 5.8 secsDBIC regular 40.35 secsDBIC populate 5.6 secsDBIDM regular 32.81 secsDBIDM bulk 32.57 secs 29.06.12 - Page 1
    • Département OfficeClasses and methods 29.06.12 - Page 1
    • DBIC : methods of a rowDB<1> m $auth # 152 methods DB<4> x mro::get_linear_isa(ref $auth)_auth_id_accessor 0 ARRAY(0x13826c4)_cpanid_accessor 0 FPW12::DBIC::Result::Auth_email_accessor 1 DBIx::Class::Core_fullname_accessor 2 DBIx::Class::Relationship_result_source_instance_accessor 3 DBIx::Class::Relationship::Helpersadd_to_dists 4 DBIx::Class::Relationship::HasManyauth_id 5 DBIx::Class::Relationship::HasOnecpanid 6 DBIx::Class::Relationship::BelongsTodists 7 DBIx::Class::Relationship::ManyToManydists_rs 8 DBIx::Classemail 9 DBIx::Class::Componentisedfullname 10 Class::C3::Componentisedresult_source_instance 11 DBIx::Class::AccessorGroupvia DBIx::Class::Core -> DBIx::Class::Relationship 12 Class::Accessor::Grouped -> DBIx::Class::Relationship::Helpers -> 13 DBIx::Class::Relationship::Accessor DBIx::Class::Relationship::HasMany: has_many 14 DBIx::Class::Relationship::CascadeActionsvia DBIx::Class::Core -> DBIx::Class::Relationship -> DBIx::Class::Relationship::Helpers -> 15 DBIx::Class::Relationship::ProxyMethods DBIx::Class::Relationship::HasOne: 16 DBIx::Class::Relationship::Base _get_primary_key 17 DBIx::Class::InflateColumnvia DBIx::Class::Core -> DBIx::Class::Relationship 18 DBIx::Class::Row -> DBIx::Class::Relationship::Helpers -> 19 DBIx::Class::PK::Auto DBIx::Class::Relationship::HasOne: _has_one 20 DBIx::Class::PKvia DBIx::Class::Core -> DBIx::Class::Relationship -> DBIx::Class::Relationship::Helpers -> 21 DBIx::Class::ResultSourceProxy::Table DBIx::Class::Relationship::HasOne: 22 DBIx::Class::ResultSourceProxy _validate_has_one_conditionvia DBIx::Class::Core -> DBIx::Class::Relationship -> DBIx::Class::Relationship::Helpers -> DBIx::Class::Relationship::HasOne: has_one...
    • DBIDM : methods of a rowDB<1> m $auth # 36 methods DB<4> x mro::get_linear_isa(ref $auth)distsinsert_into_dists 0 ARRAY(0x145275c)metadm 0 FPW12::DBIDM::Authvia DBIx::DataModel::Source::Table: _get_last_insert_id 1 DBIx::DataModel::Source::Tablevia DBIx::DataModel::Source::Table: 2 DBIx::DataModel::Source _insert_subtreesvia DBIx::DataModel::Source::Table: _rawInsertvia DBIx::DataModel::Source::Table: _singleInsertvia DBIx::DataModel::Source::Table: _weed_out_subtreesvia DBIx::DataModel::Source::Table: deletevia DBIx::DataModel::Source::Table: has_invalid_columnsvia DBIx::DataModel::Source::Table: insertvia DBIx::DataModel::Source::Table: updatevia DBIx::DataModel::Source::Table -> DBIx::DataModel::Source: TO_JSONvia DBIx::DataModel::Source::Table -> DBIx::DataModel::Source: apply_column_handlervia DBIx::DataModel::Source::Table -> DBIx::DataModel::Source: auto_expandvia DBIx::DataModel::Source::Table -> DBIx::DataModel::Source: bless_from_DBvia DBIx::DataModel::Source::Table -> DBIx::DataModel::Source: expandvia DBIx::DataModel::Source::Table -> DBIx::DataModel::Source: fetch
    • DBIC classes Componentised AccessorGroupBelongsTo, HasMany, etc. BelongsTo, HasMany, etc. ClassRLHelpers RLBase Row RSProxy ResultSource RS::TableRelationship InflateColumn PK::Auto PK RSProxy::Table Schema Core ResultSet application classes My::DB My::DB::Result::Table_n My::DB::ResultSet::Table_n objects schema row resultset instantiation inheritance delegation
    • DBIDM classes SourceSchema Table Join Statement application classesMy::DB My::DB::Table_n My::DB::AutoJoin:: objectsschema row row statement
    • DBIDM Meta-ArchitectureMeta::Schema Meta::Association Meta::Path Meta::Source Meta::Type meta::assoc meta::path Meta::Table Meta::Join meta::typeSchema Table Join My::DB meta::schema My::DB::Table_n meta::table My::DB::Auto_join meta::join
    • DBIC introspection methods• Schema – sources(), source($source_name)• ResultSource – primary_key – columns(), column_info($column_name) – relationships(), relationship_info($relname)
    • Architecture comparison• DBIC • DBIDM – very complex class structure – classes quite close to DBI concepts – no distinction front/meta layser – front classes and Meta classes – methods for – methods for • CRUD • CRUD • navigation to related objs • navigation to related objs • introspection • access to meta • setting up columns • setting up relationships • …
    • Département OfficeResultset/Statement objects 29.06.12 - Page 1
    • Example task Département Office• list names of authors – of distribs starting with DBIx – and version number > 2 29.06.12 - Page 1
    • DBIC ResultSet chaining Département Office my $dists = $schema->resultset(Dist)->search( {dist_name => {-like => DBIx%}}, ); my $big_vers = $dists->search({dist_vers => { ">" => 2}}); my @auths = $big_vers->search_related(auth, undef, {distinct => 1, order_by => fullname}); say $_->fullname foreach @auths;# Magical join ! Magical "group by" ! SELECT auth.auth_id, auth.email, auth.fullname, auth.cpanid FROM dists me JOIN auths auth ON auth.auth_id = me.auth_id WHERE ( ( dist_vers > ? AND dist_name LIKE ? ) ) GROUP BY auth.auth_id, auth.email, auth.fullname, auth.cpanid ORDER BY fullname 29.06.12 - Page 1
    • DBIDM Statement lifecycle new() schema + source new refine() sqlize() bind()-post_SQL sqlized prepare() bind()-pre_exec prepared execute() bind()-post_exec executed execute() -post_bless next() / all() bind()column types applied blessed data row(s)
    • DBIDM Statement::refine()my $stmt = FPW12::DBIDM->join(qw/Dist auth/)->select( -where => {dist_name => {-like => DBIx%}}, -result_as => statement,);$stmt->refine( -columns => [-DISTINCT => fullname], -where => {dist_vers => { ">" => 0}}, -order_by => fullname,);say $_->{fullname} while $_ = $stmt->next;
    • DBIDM subquerymy $stmt = FPW12::DBIDM::Dist->select( -where => {dist_name => {-like => DBIx%}}, -result_as => statement,);my $subquery = $stmt->select( -columns => auth_id, -where => {dist_vers => { ">" => 2}}, -result_as => subquery,);my $auths = FPW12::DBIDM::Auth->select( -columns => fullname, -where => {auth_id => {-in => $subquery}},);say $_->{fullname} foreach @$rows;
    • Statement comparison• DBIC • DBIDM – powerful refinement constructs – limited refinement constructs • more "where" criteria – explicit control of status • navigation to related source • column restriction • aggregation operators (e.g. "count")
    • Département OfficeOther features 29.06.12 - Page 1
    • Transactions Département Office• DBIC • DBIDM$schema->txn_do($sub); $sch->do_transaction($sub); – can be nested – can be nested – can have intermediate savepoints – no intermediate savepoints 29.06.12 - Page 1
    • DBIC inflation/deflation__PACKAGE__->inflate_column(insert_time,{inflate => sub { DateTime::Format::Pg->parse_datetime(shift); }, deflate => sub { DateTime::Format::Pg->format_datetime(shift) }, });
    • DBIDM Types (inflate/deflate)# declare a TypeMy::DB->Type(Multivalue => from_DB => sub {$_[0] = [split /;/, $_[0]] }, to_DB => sub {$_[0] = join ";", @$_[0] },);# apply it to some columns in a tableMy::DB::Author->metadm->define_column_type( Multivalue => qw/hobbies languages/,);
    • Extending/customizing DBIC• Subclassing – result classes – resultset classes – storage – sqlmaker
    • Extending / customizing DBIDM• Schema hooks for – SQL dialects (join syntax, alias syntax, limit / offset, etc.) – last_insert_id• Ad hoc subclasses for – SQL::Abstract – Table – Join – Statements• Statement callbacks• Extending table classes – additional methods – redefining _singleInsert method
    • Not covered here• inserts, updates, deletes• Schema generator• Schema versioning• inflation/deflation
    • Département OfficeTHANK YOU FOR YOUR ATTENTION 29.06.12 - Page 1
    • Département OfficeBonus slides 29.06.12 - Page 1
    • Why hashref instead of OO accessors ?• Perl builtin rich API for hashes (keys, values, slices, string interpolation)• good for import / export in YAML/XML/JSON• easier to follow steps in Perl debugger• faster than OO accessor methods• visually clear distinction between lvalue / rvalue – my $val = $hashref->{column}; – $hashref->{column} = $val;• visually clear distinction between – $row->{column} / $row->remote_table()
    • SQL::Abstract::More : extensions• -columns => [qw/col1|alias1 max(col2)|alias2/] – SELECT col1 AS alias1, max(col2) AS alias2• -columns => [-DISTINCT => qw/col1 col2 col3/] – SELECT DISTINCT col1, col2, col3• -order_by => [qw/col1 +col2 –col3/] – SELECT … ORDER BY col1, col2 ASC, col3 DESC• -for => "update" || "read only" – SELECT … FOR UPDATE