• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
 

Database Schema Evolution

on

  • 681 views

Streamline your database changes by versioning your database instances and your database schema, running database instances locally and implementing database changes using migration scripts based on ...

Streamline your database changes by versioning your database instances and your database schema, running database instances locally and implementing database changes using migration scripts based on database refactoring patterns.

Statistics

Views

Total Views
681
Views on SlideShare
650
Embed Views
31

Actions

Likes
3
Downloads
11
Comments
0

2 Embeds 31

http://www.zealake.com 29
http://plus.url.google.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Database Schema Evolution Database Schema Evolution Presentation Transcript

    • Database Schema Evolution Lars Thorup ZeaLake Software Consulting August, 2013
    • Who is Lars Thorup? ● Software developer/architect ● C#, JavaScript and C++ ● Test Driven Development ● Coach: Teaching agile and automated testing ● Advisor: Assesses software projects and companies ● Founder and CEO of BestBrains and ZeaLake
    • Elevator pitch ● Don't do this ● Do this instead $ importData Error: Invalid column name 'PartnerId' $ ...??? $ importData Error: Database should be version 47 but is 44 $ migrate Migrated database from version 44 to 47 $ importData Success!
    • Keeping database and app in sync
    • Demo $ SampleData.exe migrate 32 Latest version is 73 Current version is 30 Target version is 32 Migrating up to version 31 using 31_section_track_up.sql Migrating up to version 32 using 32_action_kind_up.sql $
    • Got problems? ● Tests fail because data was changed by someone else? ● Upgrades fail because some script was missing? ● Upgrades and downgrades can't be tested reliably? ● Creating new test and development environments is a hassle? ● Who made this schema change and why? ● Applications fail because code and database are not in sync?
    • We have solutions! ● Never share a development database ● Automate database schema deployment ● Version your database schema and database instances ● Base script, up and down scripts ● Check db version on app start ● Tools: FlyWay, LiquiBase, home grown ● Database refactorings ● Migration conflicts on commit ● Data builders
    • Never share a development database ● Run development database locally ● MySQL ● Use free versions of commercial database products where available ● Oracle XE ● SQL Server Express ● Use compatible open source database products if feasible ● PostgreSQL instead of Netezza
    • Automate database schema deployment ● Recreate database from scratch ● Development boxes and CI servers ● $ mvn dbschema ● Upgrade database to newer version ● Test and production servers ● $ migrate ● Rollback database to older version ● Production servers ● $ migrate down 44
    • Version your schema and instances ● Schema described by .sql files ● Keep those .sql files under version control ● Include a SCHEMA_INFO table in the schema ● Record the database version in SCHEMA_INFO
    • Base script, up and down scripts ● 1-base-up.sql ● If you are not starting from scratch ● Contains a dump of the current schema ● Including the SCHEMA_INFO table ● 51-customer-unique-{up,down}.sql ● up: migrates the schema from version 50 to version 51 ● down: migrates the schema from version 51 to version 50 ● Tip: establish a naming convention ● List all scripts in a central file ● migrations.txt
    • Up and down scripts ● 51-customer-unique-up.sql alter table Customer add constraint NameUnique unique nonclustered (Name) on [primary] ● 51-customer-unique-down.sql alter table Customer drop constraint NameUnique
    • Check db version on app start ● Application code var expectedVersion = Config.ExpectedDatabaseVersion; var actualVersion = session.GetDBVersion(); if (expectedVersion != actualVersion) { throw new ApplicationException(string.Format( "The database needs migration (its version should be {0} but is {1})", expectedVersion, actualVersion)); }
    • Tools: FlyWay ● Open-Source, Java-based ● Plain SQL scripts
    • Tools: LiquiBase ● Open-Source, Java-based ● XML or formatted SQL scripts
    • Database refactorings ● Catalog of database refactorings ● Split Table, Merge Column, etc ● 18-replace-roles-with-privileges-up.sql alter table Participation add Privileges int constraint DF_privileges default 1 not null update Participation set Privileges=1 update Participation set Privileges=3 where UserId in (select Id from User where Email='admin') drop table UserRole
    • Migration conflicts on commit ● Commit might conflict on migrations.txt ● Merge migrations.txt, keep both ● Renumber your migration ● Test ● Commit
    • Test data builder [Test] public void GetResponseMedia() { // given var stub = new StubBuilder { Questions = new [] { new QuestionBuilder { Name = "MEDIA" }, }, Participants = new[] { new ParticipantBuilder { Name = "Lars", Votes = new [] { new VoteBuilder { Question = "MEDIA", Responses = new ResponseBuilder(new byte [] {1, 2, 3}) }, }}, }, }.Build(); var voteController = new VoteController(stub.Session); // when var result = voteController.GetResponseMedia(vote.Id, true) as MediaResult; // then Assert.That(result.Download, Is.True); Assert.That(result.MediaLength, Is.EqualTo(3)); Assert.That(TfResponse.ReadAllBytes(result.MediaStream), Is.EqualTo(new byte[] {1, 2, 3})); }
    • Read more ● http://techportal.inviqa.com/2011/01/11/database-version- control/ ● http://databaserefactoring.com/ ● http://flywaydb.org/