Databases - Have it
                              your way
                                Frederick Cheung - kgb



                                fred@texperts.com
                            http://www.spacevatican.org


Friday, 4 December 2009                                   1
kgb

      • Operates a number of Directory Enquiry type
              products in several countries
      • Runs the 542542 ‘Ask Us Anything’ SMS service in
              the US
      • 542542 backend is handled by several Rails apps

Friday, 4 December 2009                                    2
Whats the difference between 1080i and 1080p?
                                   kgb

      • Operates a number of Directory Enquiry type
              products in several countries
      • Runs the 542542 ‘Ask Us Anything’ SMS service in
              the US
      • 542542 backend is handled by several Rails apps

Friday, 4 December 2009                                       2
Whats the difference between 1080i and 1080p?
                                   kgb
   What is the address for the sperm bank of Pittsburgh?

      • Operates a number of Directory Enquiry type
              products in several countries
      • Runs the 542542 ‘Ask Us Anything’ SMS service in
              the US
      • 542542 backend is handled by several Rails apps

Friday, 4 December 2009                                       2
Whats the difference between 1080i and 1080p?
                                   kgb
   What is the address for the sperm bank of Pittsburgh?

      • Operates a number of Directory Enquiry type
              products in several countries
   • Runs the Greyhound bus station in Rochester, NY?
  Where is
           the 542542 ‘Ask Us Anything’ SMS service in
              the US
      • 542542 backend is handled by several Rails apps

Friday, 4 December 2009                                       2
Whats the difference between 1080i and 1080p?
                                   kgb
   What is the address for the sperm bank of Pittsburgh?

      • Operates a number of Directory Enquiry type
              products in several countries
   • Runs the Greyhound bus station in Rochester, NY?
  Where is
           the 542542 ‘Ask Us Anything’ SMS service in
              the US
      • 542542 backend is handled by several Rails apps
                                    Is marijuana legal in Oregon?


Friday, 4 December 2009                                             2
Whats the difference between 1080i and 1080p?
                                   kgb
   What is the address for the sperm bank of Pittsburgh?

      • Operates a number of Directory Enquiry type
              products in several countries
   • Runs the Greyhound bus station in Rochester, NY?
  Where is
           the 542542 ‘Ask Us Anything’ SMS service in
              the US
      • 542542 backend is handled by several Rails apps
                                    Is marijuana legal in Oregon?

 What is the longest life span of a dog in recorded history?
Friday, 4 December 2009                                             2
ORMs

                    • Hide the gunk that map objects to database
                          storage
                    • Eliminate a lot of the repetitive code
                    • don’t forget about not relational storage
                          (eg CouchDB)



Friday, 4 December 2009                                            3
Active Record within
                                  Rails
                    • redirect_to @person
                    • form_for @person
                    • rake tasks for migrations
                    • automatically configured on app load

Friday, 4 December 2009                                     4
What ORM
                             agnosticism isn’t

                    • gsub(‘ActiveRecord’, ‘DataMapper’)
                    • Not trying to make all ORM libraries look
                          the same




Friday, 4 December 2009                                           5
What ORM
                             agnosticism isn’t

                    • gsub(‘ActiveRecord’, ‘DataMapper’)
                    • Not trying to make all ORM libraries look
                          the same




Friday, 4 December 2009                                           5
Then what is it?

                    • Document / codify interactions between
                          persistence layer and rest of Rails
                    • Grease the wheels
                    • Don’t make you feel like a second class
                          citizen
                    • Should be almost invisible to end user -
                          things should just work


Friday, 4 December 2009                                          6
Choose your ORM on its features
 without worrying about bumps in
            the road



Friday, 4 December 2009        7
Patterns
                    • Active Record: “An object that wraps a
                          row in a database table or view,
                          encapsulates the database access, and adds
                          domain logic on that data.” (Martin Fowler)
                    • Data Mapper: “A layer of Mappers hat
                          moves data between objects and a database
                          while keeping them independent of each
                          other and the mapper itself.” (Martin
                          Fowler)


Friday, 4 December 2009                                                 8
Some ORMs, side by
                                side

                           Active Record, DataMapper, Sequel




Friday, 4 December 2009                                        9
Similar features across
                           the board
                    • User.new(params[:user])
                    • Associations, scopes, migrations etc.
                    • Watch out for subtle differences!



Friday, 4 December 2009                                       10
Similar features across
                           the board
                    • User.new(params[:user])
                    • Associations, scopes, migrations etc.
                    • Watch out for subtle differences!
                    #Active Record
                    person.save! #=> raises if invalid
                    person.save #=> returns false if invalid

                    #Sequel
                    person.save #=> raises if invalid
                    person.save! #=> save ignoring validations



Friday, 4 December 2009                                          10
Active Record

                    • You all know it
                    • Does a lot of manipulation of SQL
                          fragments - very difficult to build an Active
                          Record adapter for non sql datasource
                    • SQL fragments make some things easy,
                          others awkward



Friday, 4 December 2009                                                  11
Active Record Evolution

                    • In the beginning conditions were strings:
                          composing sets of conditions: yuck!
                    • hash conditions for equality & ranges
                    • named_scope
                    • and ...

Friday, 4 December 2009                                           12
Arel - Relation Algebra
      • generates db queries
      • Destined to underpin future version Active Record
      • Operations all closed under composition
       posts = Table(:posts)

       posts.where(posts[:subject].eq('Bacon'))

       posts.join(comments).on(posts[:id].eq(comments[:post_id]))




Friday, 4 December 2009                                             13
DataMapper
    • Query objects: lazy loaded definition of a query
    • Easy to compose queries
    • Easy construction of complicated joins
    • Modular design
  all_posts = Post.all

  about_test = all_posts.all :subject.like => '%test%'

  with_comment_by_fred = about_test.all Post.comments.name => 'fred'



Friday, 4 December 2009                                                14
A DM using Model
              • mappy feel to it
              • explicit about attributes (versus checking schema)
                 class Post
                   include DataMapper::Resource
                   include DataMapper::Timestamps
                   property :id, Serial
                   property :subject, String, :nullable => false
                   property :body, Text, :nullable => false
                   property :created_at, DateTime, :nullable => false
                   property :updated_at, DateTime, :nullable => false

                   has n, :comments
                 end

Friday, 4 December 2009                                                 15
Laziness
                    • attributes can be lazy loaded
                    • loads lazy loaded attributes in one query
                          for all results in collection
                    • property groups when defining laziness
                    • same strategy for loading associations
           Post.get 1
           => #<Post @id=1 @subject="Hello world" @body=<not loaded> >




Friday, 4 December 2009                                                  16
Laziness
                    • attributes can be lazy loaded
                    • loads lazy loaded attributes in one query
                          for all results in collection
                    • property groups when defining laziness
                    • same strategy for loading associations
           Post.get 1
           => #<Post @id=1 @subject="Hello world" @body=<not loaded> >




Friday, 4 December 2009                                                  16
Sequel
       • Database toolkit            - A ruby DSL for databases
                  DB[:posts].filter { created_at > Date::today - 1}.all


       • ORM layer (Migrations, associations etc. - the usual)
       • Master/Slave databases, sharding
       • pure ruby DSL for conditions
       • Mosts things are Datasets
Friday, 4 December 2009                                                   17
Sequel Model
                          class Post < Sequel::Model
                            plugin :timestamps
                            one_to_many :comments
                          end



      • comments_dataset
      • loads of customisation for eager loads
      • inverse associations
      • flexible - join on non standard columns
Friday, 4 December 2009                                18
Very modular




Friday, 4 December 2009                  19
Very modular
     Some high level




Friday, 4 December 2009                  19
Very modular
     Some high level
                    Identity map
                    Lazy attributes
                    Callbacks
                    Single/Class Table inheritance




Friday, 4 December 2009                              19
Very modular
     Some high level
                    Identity map
                    Lazy attributes
                    Callbacks
                    Single/Class Table inheritance
      Some quite grungy



Friday, 4 December 2009                              19
Very modular
     Some high level
                    Identity map
                    Lazy attributes
                    Callbacks
                    Single/Class Table inheritance
      Some quite grungy
                    Boolean readers
                    Association proxies
Friday, 4 December 2009                              19
Monolithic vs Modular
                    • Active Record in theory modular, but no
                          easy way to choose
                    • DM explicitly modular: plugins implement
                          validations, migrations, ...
                    • Sequel::Model entirely plugin driven
                    • Modularity great when you know what
                          you’re doing, but ...
                    • Tyranny of choice?
Friday, 4 December 2009                                          20
Conditions
   #Active Record
   Post.all :conditions => ["subject like ?", "%bacon%"]
   Post.all :conditions => {:approved => true}
   Post.all :conditions => ["comments.name like ?", "%bacon%"],
             :joins => :comments, :select => "distinct posts.*"

   #DataMapper
   Post.all :subject.like => '%bacon%'
   Post.all Post.comments.name.like => '%fred%'

   #Sequel
   Post.filter {subject.like '%test%'}.all
   Post.select('distinct posts.*').inner_join(:comments, :post_id
   => :id).filter {comments__name.like '%test%'}.all




Friday, 4 December 2009                                             21
More Finders
     #Active Record
     Post.find :all, :include => :comments, :order => 'created_at
     desc'

     #DataMapper

     Post.all :order => [:created_at.desc]

     #Sequel

     Post.eager(:comments).order(:created_at.desc).all
     Post.eager_graph(:comments).order('posts.created_at desc').all




Friday, 4 December 2009                                               22
You can use these
                                 today
                    • You can already build apps with other data
                          stores
                    • You can use some of the niceties like
                          redirect_to @person
                    • You need to figure out dependencies
                          between Action Pack and Active Record
                    • Compatibility worries
Friday, 4 December 2009                                            23
Little niggles


                    • Action summary doesn’t count time in DM/
                          Sequel as database time
                    • Some manual setup
                    • have to define to_param


Friday, 4 December 2009                                          24
Code time!




Friday, 4 December 2009                25
That’s all folks!

                              fred@texperts.com
                          http://www.spacevatican.org




Friday, 4 December 2009                                 26

Databases -- Have it Your Way (Frederick Cheung)

  • 1.
    Databases - Haveit your way Frederick Cheung - kgb fred@texperts.com http://www.spacevatican.org Friday, 4 December 2009 1
  • 2.
    kgb • Operates a number of Directory Enquiry type products in several countries • Runs the 542542 ‘Ask Us Anything’ SMS service in the US • 542542 backend is handled by several Rails apps Friday, 4 December 2009 2
  • 3.
    Whats the differencebetween 1080i and 1080p? kgb • Operates a number of Directory Enquiry type products in several countries • Runs the 542542 ‘Ask Us Anything’ SMS service in the US • 542542 backend is handled by several Rails apps Friday, 4 December 2009 2
  • 4.
    Whats the differencebetween 1080i and 1080p? kgb What is the address for the sperm bank of Pittsburgh? • Operates a number of Directory Enquiry type products in several countries • Runs the 542542 ‘Ask Us Anything’ SMS service in the US • 542542 backend is handled by several Rails apps Friday, 4 December 2009 2
  • 5.
    Whats the differencebetween 1080i and 1080p? kgb What is the address for the sperm bank of Pittsburgh? • Operates a number of Directory Enquiry type products in several countries • Runs the Greyhound bus station in Rochester, NY? Where is the 542542 ‘Ask Us Anything’ SMS service in the US • 542542 backend is handled by several Rails apps Friday, 4 December 2009 2
  • 6.
    Whats the differencebetween 1080i and 1080p? kgb What is the address for the sperm bank of Pittsburgh? • Operates a number of Directory Enquiry type products in several countries • Runs the Greyhound bus station in Rochester, NY? Where is the 542542 ‘Ask Us Anything’ SMS service in the US • 542542 backend is handled by several Rails apps Is marijuana legal in Oregon? Friday, 4 December 2009 2
  • 7.
    Whats the differencebetween 1080i and 1080p? kgb What is the address for the sperm bank of Pittsburgh? • Operates a number of Directory Enquiry type products in several countries • Runs the Greyhound bus station in Rochester, NY? Where is the 542542 ‘Ask Us Anything’ SMS service in the US • 542542 backend is handled by several Rails apps Is marijuana legal in Oregon? What is the longest life span of a dog in recorded history? Friday, 4 December 2009 2
  • 8.
    ORMs • Hide the gunk that map objects to database storage • Eliminate a lot of the repetitive code • don’t forget about not relational storage (eg CouchDB) Friday, 4 December 2009 3
  • 9.
    Active Record within Rails • redirect_to @person • form_for @person • rake tasks for migrations • automatically configured on app load Friday, 4 December 2009 4
  • 10.
    What ORM agnosticism isn’t • gsub(‘ActiveRecord’, ‘DataMapper’) • Not trying to make all ORM libraries look the same Friday, 4 December 2009 5
  • 11.
    What ORM agnosticism isn’t • gsub(‘ActiveRecord’, ‘DataMapper’) • Not trying to make all ORM libraries look the same Friday, 4 December 2009 5
  • 12.
    Then what isit? • Document / codify interactions between persistence layer and rest of Rails • Grease the wheels • Don’t make you feel like a second class citizen • Should be almost invisible to end user - things should just work Friday, 4 December 2009 6
  • 13.
    Choose your ORMon its features without worrying about bumps in the road Friday, 4 December 2009 7
  • 14.
    Patterns • Active Record: “An object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data.” (Martin Fowler) • Data Mapper: “A layer of Mappers hat moves data between objects and a database while keeping them independent of each other and the mapper itself.” (Martin Fowler) Friday, 4 December 2009 8
  • 15.
    Some ORMs, sideby side Active Record, DataMapper, Sequel Friday, 4 December 2009 9
  • 16.
    Similar features across the board • User.new(params[:user]) • Associations, scopes, migrations etc. • Watch out for subtle differences! Friday, 4 December 2009 10
  • 17.
    Similar features across the board • User.new(params[:user]) • Associations, scopes, migrations etc. • Watch out for subtle differences! #Active Record person.save! #=> raises if invalid person.save #=> returns false if invalid #Sequel person.save #=> raises if invalid person.save! #=> save ignoring validations Friday, 4 December 2009 10
  • 18.
    Active Record • You all know it • Does a lot of manipulation of SQL fragments - very difficult to build an Active Record adapter for non sql datasource • SQL fragments make some things easy, others awkward Friday, 4 December 2009 11
  • 19.
    Active Record Evolution • In the beginning conditions were strings: composing sets of conditions: yuck! • hash conditions for equality & ranges • named_scope • and ... Friday, 4 December 2009 12
  • 20.
    Arel - RelationAlgebra • generates db queries • Destined to underpin future version Active Record • Operations all closed under composition posts = Table(:posts) posts.where(posts[:subject].eq('Bacon')) posts.join(comments).on(posts[:id].eq(comments[:post_id])) Friday, 4 December 2009 13
  • 21.
    DataMapper • Query objects: lazy loaded definition of a query • Easy to compose queries • Easy construction of complicated joins • Modular design all_posts = Post.all about_test = all_posts.all :subject.like => '%test%' with_comment_by_fred = about_test.all Post.comments.name => 'fred' Friday, 4 December 2009 14
  • 22.
    A DM usingModel • mappy feel to it • explicit about attributes (versus checking schema) class Post include DataMapper::Resource include DataMapper::Timestamps property :id, Serial property :subject, String, :nullable => false property :body, Text, :nullable => false property :created_at, DateTime, :nullable => false property :updated_at, DateTime, :nullable => false has n, :comments end Friday, 4 December 2009 15
  • 23.
    Laziness • attributes can be lazy loaded • loads lazy loaded attributes in one query for all results in collection • property groups when defining laziness • same strategy for loading associations Post.get 1 => #<Post @id=1 @subject="Hello world" @body=<not loaded> > Friday, 4 December 2009 16
  • 24.
    Laziness • attributes can be lazy loaded • loads lazy loaded attributes in one query for all results in collection • property groups when defining laziness • same strategy for loading associations Post.get 1 => #<Post @id=1 @subject="Hello world" @body=<not loaded> > Friday, 4 December 2009 16
  • 25.
    Sequel • Database toolkit - A ruby DSL for databases DB[:posts].filter { created_at > Date::today - 1}.all • ORM layer (Migrations, associations etc. - the usual) • Master/Slave databases, sharding • pure ruby DSL for conditions • Mosts things are Datasets Friday, 4 December 2009 17
  • 26.
    Sequel Model class Post < Sequel::Model plugin :timestamps one_to_many :comments end • comments_dataset • loads of customisation for eager loads • inverse associations • flexible - join on non standard columns Friday, 4 December 2009 18
  • 27.
    Very modular Friday, 4December 2009 19
  • 28.
    Very modular Some high level Friday, 4 December 2009 19
  • 29.
    Very modular Some high level Identity map Lazy attributes Callbacks Single/Class Table inheritance Friday, 4 December 2009 19
  • 30.
    Very modular Some high level Identity map Lazy attributes Callbacks Single/Class Table inheritance Some quite grungy Friday, 4 December 2009 19
  • 31.
    Very modular Some high level Identity map Lazy attributes Callbacks Single/Class Table inheritance Some quite grungy Boolean readers Association proxies Friday, 4 December 2009 19
  • 32.
    Monolithic vs Modular • Active Record in theory modular, but no easy way to choose • DM explicitly modular: plugins implement validations, migrations, ... • Sequel::Model entirely plugin driven • Modularity great when you know what you’re doing, but ... • Tyranny of choice? Friday, 4 December 2009 20
  • 33.
    Conditions #Active Record Post.all :conditions => ["subject like ?", "%bacon%"] Post.all :conditions => {:approved => true} Post.all :conditions => ["comments.name like ?", "%bacon%"], :joins => :comments, :select => "distinct posts.*" #DataMapper Post.all :subject.like => '%bacon%' Post.all Post.comments.name.like => '%fred%' #Sequel Post.filter {subject.like '%test%'}.all Post.select('distinct posts.*').inner_join(:comments, :post_id => :id).filter {comments__name.like '%test%'}.all Friday, 4 December 2009 21
  • 34.
    More Finders #Active Record Post.find :all, :include => :comments, :order => 'created_at desc' #DataMapper Post.all :order => [:created_at.desc] #Sequel Post.eager(:comments).order(:created_at.desc).all Post.eager_graph(:comments).order('posts.created_at desc').all Friday, 4 December 2009 22
  • 35.
    You can usethese today • You can already build apps with other data stores • You can use some of the niceties like redirect_to @person • You need to figure out dependencies between Action Pack and Active Record • Compatibility worries Friday, 4 December 2009 23
  • 36.
    Little niggles • Action summary doesn’t count time in DM/ Sequel as database time • Some manual setup • have to define to_param Friday, 4 December 2009 24
  • 37.
    Code time! Friday, 4December 2009 25
  • 38.
    That’s all folks! fred@texperts.com http://www.spacevatican.org Friday, 4 December 2009 26