Uploaded on

 

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
324
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • * Work mostly with backend data transformations from Transactional Database to Reporting datastore\n* When you have a child the blogging slows down\n* Talk quickly when I am nervous, let me know if to fast\n
  • * About the little ruby Code in the middle\n* Will not focus on either DB OR DM\n
  • \n
  • \n
  • \n
  • \n
  • * With the addition of the initialization mostly just CRUD operations\n* CRUD\n
  • \n
  • \n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • * Raw Access to connection through adapter\n* Persisted through entire instance, so if time out need to reconnect\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • * Association - subclass of DataMapper::Associations::*::Relationship\n* Field - subclass of DataMapper::Property\n
  • * Association - subclass of DataMapper::Associations::*::Relationship\n* Field - subclass of DataMapper::Property\n
  • * Association - subclass of DataMapper::Associations::*::Relationship\n* Field - subclass of DataMapper::Property\n
  • * Association - subclass of DataMapper::Associations::*::Relationship\n* Field - subclass of DataMapper::Property\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • * Only top level Operation\n* own your own for recursive solution\n
  • \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • * .field - the repository name\n* .name - the resource name\n* value IS UNMARSHALED \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • * info - used for logging what you are sending to the repository\n
  • \n
  • * The first part of the presentation is the 80 percent that makes the most difference\n* The last (more repository specific) part is the 20 percent\n
  • * Mongo specific example \n* But demonstrates the process of modifying a models\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • * Be kind log\n* at info log the query parameters that are being passed to repository\n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Custom DataMapper Adapters By Josh Moore
  • 2. About Me• Josh Moore• Optimis International• github.com/joshsmoore• twitter.com/codingforrent• www.codingforrent.com
  • 3. About this presentation• Not an introduction to DataMapper• Mostly Code• Why?
  • 4. Examplesclass Comment class Post include include DataMapper::Resource DataMapper::Resource property :id, Serial property :id, Serial property :body, String property :title, String belongs_to :post has n, :commentsend end
  • 5. Terms term meaningStorage Engine Whatever is persisting the data Resource A model field A property on a model Repository DataMapper term for the storage engine
  • 6. Preamblerequire dm-core Just do itmodule DataMapper Just do it module Adapters class TestAdapter < AbstractAdapter ... end Just do it const_added(:TestAdapter) endend
  • 7. Symbol Hashdef initialize(name, options)end
  • 8. def initialize(name, options) Just do it superend
  • 9. def initialize(name, options) superend
  • 10. Connect to yourdef initialize(name, options) adapter super @conn = Mongo::Connection.new( options[:host], options[:port]) @adapter = @conn[options[:database]]end
  • 11. def initialize(name, options) super @conn = Mongo::Connection.new( Sub class of: DataMapper::Property options[:host], options[:port]) @adapter = @conn[options[:database]] @field_naming_convention = Proc.new do |field| field.model.storage_name + _ + field.name.to_s end Return a Stringend
  • 12. def initialize(name, options) super @conn = Mongo::Connection.new( options[:host], options[:port]) @adapter = @conn[options[:database]] @field_naming_convention = Proc.new do |field| field.model.storage_name String (class.to_s) + _ + field.name.to_s end @resource_naming_convention = Proc.new do |resource| resource.downcase end Return a Stringend
  • 13. def initialize(name, options) super @conn = Mongo::Connection.new( options[:host], options[:port]) @adapter = @conn[options[:database]] @field_naming_convention = Proc.new do |field| field.model.storage_name + _ + field.name.to_s end @resource_naming_convention = Proc.new do |resource| resource.downcase endend
  • 14. Array of Resourcesdef create(resources)end Return number of resources created
  • 15. def create(resources) resources.collect do |resource| initialize_serial(resource, @adapter[resource.class.storage_name].find.count) fields = attributes_as_fields( resource.attributes(:property)) @adapter[resource.class.storage_name].insert(fields) end.sizeend
  • 16. def create(resources) resources.collect do |resource| initialize_serial(resource, @adapter[resource.class.storage_name].find.count) fields = attributes_as_fields( resource.attributes(:property)) @adapter[resource.class.storage_name].insert(fields) end.sizeend
  • 17. • Accepts: Hash • Key: Sub class of: DataMapper::Property • Value: non marshaled data • example: {<DataMapper::Property::String(title)> => "hasdf"}def create(resources) resources.collect do |resource| initialize_serial(resource, @adapter[resource.class.storage_name].find.count) fields = attributes_as_fields( resource.attributes(:property)) @adapter[resource.class.storage_name].insert(fields) end.size • Return: Hash • Key: @field_naming_convention resultend • Value: Marshaled data • Only values that are set •Example: {"post_title" => "hasdf"}
  • 18. def create(resources) resources.collect do |resource| initialize_serial(resource, @adapter[resource.class.storage_name].find.count) fields = attributes_as_fields( resource.attributes(:property)) @adapter[resource.class.storage_name].insert(fields) end.sizeend
  • 19. def create(resources) resources.collect do |resource| initialize_serial(resource, @adapter[resource.class.storage_name].find.count) fields = attributes_as_fields( resource.attributes(:property)) @adapter[resource.class.storage_name].insert(fields) end.sizeend Unless an Exception is raised the resource will be considered saved
  • 20. DataMapper::Querydef read(query)end Return a Array of Hashes key: field name value: unmarshed value {field_name => value}
  • 21. def read(query) conditions = parse_query_conditions(query) @adapter[query.model.storage_name].find( conditions)end
  • 22. Query Structure DataMapper::Query #conditions Operation #operands Set Operation Condition Array #subject Field Association
  • 23. Query Structure DataMapper::Query #conditions Operation #operands Set Operation Condition Array #subject Field Association:conditions => ["updated_at > ?", Time.now]
  • 24. Query Structure DataMapper::Query #conditions Operation #operands Set Operation Condition Array #subject Field Association :conditions => ["updated_at > ?", Time.now]:conditions => {:updated_at => {"$gte" => Time.now}}
  • 25. Query Structure DataMapper::Query #conditions Operation #operands Set Operation Condition Hash Array #subject Field Association :conditions => ["updated_at > ?", Time.now]:conditions => {:updated_at => {"$gte" => Time.now}}
  • 26. DataMapper::Querydef parse_query_conditions(query) mongo_conditions = {} Return a hash mongo_conditions
  • 27. query.conditions.operands.each do |condition| def parse_query_conditions(query) mongo_conditions = {} case condition.class.to_s when DataMapper::Query::Conditions::GreaterThanComparison mongo_conditions[condition.subject.field] = { "$gt" => condition.value} when DataMapper::Query::Conditions::LessThanComparison mongo_conditions[condition.subject.field] = { "$lt" => condition.value} else mongo_conditions[condition.subject.field] = condition.value end mongo_conditionsend end
  • 28. query.conditions.operands.each do |condition| def parse_query_conditions(query) mongo_conditions = {} case condition.class.to_s when DataMapper::Query::Conditions::GreaterThanComparison mongo_conditions[condition.subject.field] = { "$gt" => condition.value} when DataMapper::Query::Conditions::LessThanComparison mongo_conditions[condition.subject.field] = { "$lt" => condition.value} else mongo_conditions[condition.subject.field] = condition.value end mongo_conditionsend end
  • 29. query.conditions.operands.each do |condition| def parse_query_conditions(query) mongo_conditions = {} case condition.class.to_s when DataMapper::Query::Conditions::GreaterThanComparison mongo_conditions[condition.subject.field] = { "$gt" => condition.value} when DataMapper::Query::Conditions::LessThanComparison mongo_conditions[condition.subject.field] = { "$lt" => condition.value} else mongo_conditions[condition.subject.field] = condition.value end mongo_conditionsend end
  • 30. def parse_query_conditions(query) mongo_conditions = {} query.conditions.operands.each do |condition| case condition.class.to_s when DataMapper::Query::Conditions::GreaterThanComparison mongo_conditions[condition.subject.field] = { "$gt" => condition.value} when DataMapper::Query::Conditions::LessThanComparison mongo_conditions[condition.subject.field] = { "$lt" => condition.value} else mongo_conditions[condition.subject.field] = condition.value end end mongo_conditionsend
  • 31. Post.all(:comments => {:body => hi 2})
  • 32. conditions.operands.each do |condition| ... case condition.class.to_s when ...InclusionComparison if condition.subject.instance_of?DataMapper::Associations::OneToMany::Relationship pk = condition.subject.parent_key.first.field ck = condition.subject.child_key.first.name mongo_conditions[pk] = {"$in" => condition.value.collect {|r| r.send(ck)}} else...
  • 33. conditions.operands.each do |condition| ... case condition.class.to_s when ...InclusionComparison if condition.subject.instance_of?DataMapper::Associations::OneToMany::Relationship pk = condition.subject.parent_key.first.field ck = condition.subject.child_key.first.name mongo_conditions[pk] = {"$in" => condition.value.collect {|r| r.send(ck)}} else...
  • 34. conditions.operands.each do |condition| ... case condition.class.to_s Array of properties when ...InclusionComparison * property - subclass of DataMapper::Property if condition.subject.instance_of?DataMapper::Associations::OneToMany::Relationship pk = condition.subject.parent_key.first.field ck = condition.subject.child_key.first.name mongo_conditions[pk] = {"$in" => condition.value.collect {|r| r.send(ck)}} elseArray of properties * property - subclass of DataMapper::Property...
  • 35. conditions.operands.each do |condition| ... case condition.class.to_s when ...InclusionComparison if condition.subject.instance_of?DataMapper::Associations::OneToMany::Relationship Array of resources pk = condition.subject.parent_key.first.field * [#<Comment..>, #<Comment..>,...] ck = condition.subject.child_key.first.name mongo_conditions[pk] = {"$in" => condition.value.collect {|r| r.send(ck)}} else...
  • 36. conditions.operands.each do |condition| ... case condition.class.to_s when ...InclusionComparison if condition.subject.instance_of?DataMapper::Associations::OneToMany::Relationship pk = condition.subject.parent_key.first.field ck = condition.subject.child_key.first.name mongo_conditions[pk] = {"$in" => condition.value.collect {|r| r.send(ck)}} else...
  • 37. conditions.operands.each do |condition| ... case condition.class.to_s when ...InclusionComparison if condition.subject.instance_of?DataMapper::Associations::OneToMany::Relationship pk = condition.subject.parent_key.first.field ck = condition.subject.child_key.first.name mongo_conditions[pk] = {"$in" => condition.value.collect {|r| r.send(ck)}} else...
  • 38. Operation• DataMapper::Query::Conditions::AndOperation• DataMapper::Query::Conditions::OrOperation• DataMapper::Query::Conditions::NotOperation
  • 39. Condition• DataMapper::Query::Conditions::EqualToComparison• DataMapper::Query::Conditions::InclusionComparison• DataMapper::Query::Conditions::RegexpComparison• DataMapper::Query::Conditions::LikeComparison• DataMapper::Query::Conditions::GreaterThanComparison• DataMapper::Query::Conditions::LessThanComparison• DataMapper::Query::Conditions::GreaterThanOrEqualToComparison• DataMapper::Query::Conditions::LessThanOrEqualToComparison
  • 40. What is not covered• Nested Operands
  • 41. If your backed does not have a query language A Array of Hashes key: field name value: unmarshed value [{field_name => value}]def read(query) query.filter_records(records)end
  • 42. def delete(resources) conditions = parse_query_conditions(resources.query) record_count = read(resources.query).count @adapter[resources.storage_name].remove(conditions) record_countend
  • 43. DataMapper::Collectiondef delete(resources) conditions = parse_query_conditions(resources.query) record_count = read(resources.query).count @adapter[resources.storage_name].remove(conditions) record_countend Number of resources deleted (int)
  • 44. def delete(resources) conditions = parse_query_conditions(resources.query) record_count = read(resources.query).count @adapter[resources.storage_name].remove(conditions) record_countend Unless an Exception is raised the resource will be considered saved
  • 45. def update(changes, resources) conditions = parse_query_conditions(resources.query) @adapter[resources.storage_name].update(conditions, {"$set" => attributes_as_fields(changes)}, {:multi => true}) read(resources.query).countend
  • 46. Unmarshaled hash of changes {<DataMapper::Property::String(title)> => "hasdf"} DataMapper::Collectiondef update(changes, resources) conditions = parse_query_conditions(resources.query) @adapter[resources.storage_name].update(conditions, {"$set" => attributes_as_fields(changes)}, {:multi => true}) read(resources.query).countend Number of resources updated (int)
  • 47. def update(changes, resources) conditions = parse_query_conditions(resources.query) @adapter[resources.storage_name].update(conditions, {"$set" => attributes_as_fields(changes)}, {:multi => true}) read(resources.query).countend Unless an Exception is raised the resource will be considered saved
  • 48. Be Kind log• DataMapper.logger
  • 49. You can stop now
  • 50. 80/20 rule
  • 51. But, if you want to keep going...
  • 52. Post.create.key
  • 53. Post.create.key ["4e15cab882034dc7f7000002"]
  • 54. Post.create.key
  • 55. Just do itmodule DataMapper Just do it module Is module Mongo def is_mongo(options={}) property :_id, String, :key => true, :default => end end end Just do it Model.append_extensions(Is::Mongo)end
  • 56. class Post include DataMapper::Resource property :title, String has n, :comments is :mongoend
  • 57. class Post include DataMapper::Resource property :title, String has n, :comments is :mongoend Post.create.key [""]
  • 58. def create(resources) resources.collect do |resource| fields =attributes_as_fields(resource.attributes(:property)) fields.delete _id b_id = @adapter[resource.class.storage_name].insert fields resource._id = b_id.to_s if resource.instance_variables.include? @_key resource.send :remove_instance_variable, :@_key end end.sizeend Post.create.key
  • 59. def create(resources) resources.collect do |resource| fields =attributes_as_fields(resource.attributes(:property)) fields.delete _id b_id = @adapter[resource.class.storage_name].insert fields resource._id = b_id.to_s if resource.instance_variables.include? @_key resource.send :remove_instance_variable, :@_key end end.sizeend Post.create.key
  • 60. def create(resources) resources.collect do |resource| fields =attributes_as_fields(resource.attributes(:property)) fields.delete _id b_id = @adapter[resource.class.storage_name].insert fields resource._id = b_id.to_s if resource.instance_variables.include? @_key resource.send :remove_instance_variable, :@_key end end.sizeend Post.create.key
  • 61. def create(resources) resources.collect do |resource| fields =attributes_as_fields(resource.attributes(:property)) fields.delete _id b_id = @adapter[resource.class.storage_name].insert fields resource._id = b_id.to_s if resource.instance_variables.include? @_key resource.send :remove_instance_variable, :@_key end end.sizeend Post.create.key
  • 62. def create(resources) resources.collect do |resource| fields =attributes_as_fields(resource.attributes(:property)) fields.delete _id b_id = @adapter[resource.class.storage_name].insert fields resource._id = b_id.to_s if resource.instance_variables.include? @_key resource.send :remove_instance_variable, :@_key end end.sizeend Post.create.key
  • 63. def create(resources) resources.collect do |resource| fields =attributes_as_fields(resource.attributes(:property)) fields.delete _id b_id = @adapter[resource.class.storage_name].insert fields resource._id = b_id.to_s if resource.instance_variables.include? @_key resource.send :remove_instance_variable, :@_key end end.sizeend Post.create.key ["4e15cab882034dc7f7000002"]
  • 64. def create(resources) resources.collect do |resource| fields =attributes_as_fields(resource.attributes(:property)) fields.delete _id b_id = @adapter[resource.class.storage_name].insert fields resource._id = b_id.to_s if resource.instance_variables.include? @_key resource.send :remove_instance_variable, :@_key end end.sizeend Post.create.key
  • 65. Whats left• Aggregation• Native types• DataMapper.logger