Migrating legacy data

1,506 views
1,459 views

Published on

A way to migrate data into your rails app

Published in: Technology
3 Comments
1 Like
Statistics
Notes
No Downloads
Views
Total views
1,506
On SlideShare
0
From Embeds
0
Number of Embeds
45
Actions
Shares
0
Downloads
19
Comments
3
Likes
1
Embeds 0
No embeds

No notes for slide

Migrating legacy data

  1. 1. Migrating legacy data A way to migrate data into your rails app
  2. 2. About me Patrick Hüsler Freelance developer @ http://www.huesler- informatik.ch Apple enthusiast Surfing and Kung Fu
  3. 3. Legacy...
  4. 4. Us...
  5. 5. Considerations RDBMS Keys Philosophy Structure Naming
  6. 6. Tasks Connect? Retrieve? Integrate? Map? Validate?
  7. 7. Connect database adapter (hopefully) add it to database.yml set_table_name set_primary_key set_sequence_name define associations
  8. 8. Retrieve Create corresponding AR models Tell AR to connect to the legacy database Use AR’s features to adjust things like table name, primary key etc Set up associations
  9. 9. 1 class Legacy::User < ActiveRecord::Base 2 3 establish_connection("legacydb") 4 set_table_name :users 5 6 belongs_to :login, :class_name => "Legacy::Login" 7 end
  10. 10. 1 class Legacy::Base < ActiveRecord::Base 2 self.abstract_class = true 3 establish_connection("legacydb") 4 end 5 6 class Legacy::User < Legacy::Base 7 belongs_to :login, :class_name => "Legacy::Login" 8 end 9 10
  11. 11. Integrate Put models in their own namespace to avoid collisions Put them in their own folder
  12. 12. 1 class Legacy::Base < ActiveRecord::Base 2 self.abstract_class = true 3 establish_connection("legacydb") 4 end 5 6 class Legacy::User < Legacy::Base 7 belongs_to :login, :class_name => "Legacy::Login" 8 end 9 10
  13. 13. Map Let the models handle the mapping
  14. 14. 1 class Legacy::Country < Legacy::Base 2 3 has_many :regions, :class_name => "Legacy::Region" 4 5 def code 6 abbrev 7 end 8 end
  15. 15. Validate Just use Active Record Create finders and scopes to only migrate the data you want Use memoization to speed up retrieval (e.g. fetch only valid records)
  16. 16. 1 class Legacy::User < Legacy::Base 2 belongs_to :login, :class_name => "Legacy::Login" 3 4 named_scope :active , :conditions => ["status = 1"] 5 6 validates_uniqueness_of :email, :scope => :status 7 validates_presence_of :first_name 8 validates_associated :login 9 end
  17. 17. Migrate Use migrators/importers
  18. 18. 1 # General Importer 2 # Code removed for the sake of this example 3 class Migration::Importer 4 def initialize(_entity_to_migrate,_migrated_entity) 5 @entity_to_migrate = _entity_to_migrate 6 @migrated_entity = _migrated_entity 7 end 8 9 def migrate 10 @migrated_entity.attributes.symbolize_keys.keys.each do |attr| 11 if @entity_to_migrate.respond_to?(attr) 12 @migrated_entity.send("#{attr}=",@entity_to_migrate.send(attr)) 13 end 14 end 15 @migrated_entity 16 end 17 end
  19. 19. 1 # General Importer 2 # Lines removed for the sake of this example 3 class Migration::Importer 4 def self.migrate_all(source_class,entity_name = nil) 5 migrate_all_from_collection(source_class.send(:all),entity_name) 6 end 7 8 def self.migrate_all_from_collection(collection,entity_name = nil) 9 entity_name ||= collection.first.class.to_s.pluralize 10 collection.each do |entity| 11 result = self.new(entity).migrate 12 result.save! 13 end 14 end 15 end
  20. 20. 1 class Migration::UserImporter < Migration::Importer 2 def initialize(user) 3 super(user,User.new) 4 end 5 6 def self.migrate_all 7 migrate_all_from_collection(Legacy::User.valid) 8 end 9 end
  21. 21. Test Integrates seamlessly with your rails test suit A separate folder helps to organize it
  22. 22. 1 require File.dirname(__FILE__) + '/../../spec_helper' 2 3 describe Migration::UserImporter do 4 5 before(:all) do 6 # setup your test here 7 end 8 9 it "should map email address" do 10 @migrated_user.email.should equal(@user_to_migrate.email) 11 end 12 13 it "should map login to old login" do 14 @migrated_user.old_login.should == @user_to_migrate.login.login 15 end 16 17 it "should not have an empty salt" do 18 @migrated_user.salt.should_not be_blank 19 end 20 21 end
  23. 23. Automate Rake to the rescue Let it deal with dependencies
  24. 24. 1 namespace :legacy do 2 namespace :data do 3 desc "import data from legacy schema" 4 task :import => :environment do 5 Rake::Task['legacy:data:import:users'].invoke 6 end 7 namespace :import do 8 task :fields_of_study do 9 puts "Migrating Fields of study" 10 Migration::FieldOfStudyImporter.migrate_all 11 puts "Done migrating fields of study" 12 end 13 14 task :universities do 15 puts "Migrating universities" 16 Migration::UniversityImporter.migrate_all 17 puts "Done migrating universities" 18 end 19 20 task :users => [:fields_of_study,:universities] do 21 puts "Migrating users" 22 Migration::UserImporter.migrate_all 23 puts "Done migrating users" 24 end 25 end 26 end 27 end
  25. 25. Deploy Just put in deploy.rb and let capistrano handle the rest
  26. 26. 1 namespace :deploy do 2 desc "Migrate data" 3 task :migrate_legacy_data, :roles => :app do 4 # Actual one liner that would not fit 5 # on a presentation screen 6 command = "cd #{deploy_to}/current;" 7 command << "RAILS_ENV='production' " 8 command << "/usr/bin/rake legacy:data:import" 9 run command 10 end 11 end
  27. 27. Questions?
  28. 28. Contact info Available for hire http://www.huesler-informatik.ch patrick.huesler@gmail.com https://www.xing.com/profile/Patrick_Huesler http://twitter.com/phuesler http://github.com/phuesler

×