Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Борис Трофимов. Continuous Database migration-это просто!


Published on

Published in: Software, Technology
  • Be the first to comment

Борис Трофимов. Continuous Database migration-это просто!

  1. 1. Continuous DB migration Boris Trofimov @b0ris_1
  2. 2. Agenda  Dealing with changes  Perfect solution or Continuous DB migration  Carbon5 Introduction  Maven-driven mode  Embedded mode  Pros/Cos and best practices
  3. 3. Dealing with Changes  Applications evolve. God, bless bugs, refactoring and business changes.  We love applications and configure CI, VCS in order to track source code/application version.  But what’s about DB?
  4. 4. Dealing with Schema Change  FACT: Some modifications might require changes in DB schema.
  5. 5. Dealing with Schema Change  FACT: Some modifications might require changes in DB schema.  Changes are simple and risky.
  6. 6. Dealing with Schema Change  FACT: Some modifications might require changes in DB schema.  Changes are simple and risky.  Simple changes: – add one more table – add index
  7. 7. Dealing with Schema Change  FACT: Some application modifications might require changes in DB schema.  Changes are simple and risky.  Simple changes: – add one more table – add index  Risky changes: – rename/add column – change foreign key – migrate data from one table/column to another – denormalization on a fly
  8. 8. Dealing with Schema Change  What’s about old apps upgrade and multiple schema changes? App v1 App v10
  9. 9. The Impact  Risks to lose data  Painful downtime  Risks to break application (see #1 and #2)  Resources, efforts and budget
  10. 10. Friday’s deployment in some teams
  11. 11. How people solve it  Deny Friday’s deployments  Manual deployment  Hibernate-like schema update + SQLexec  Self-developed tools based on version number approach
  12. 12. Perfect solution  Dedicated reusable framework  Every Feature request leads to separated SQL migration script.  Framework tracks which changes were not applied and applies them.  Configurable time to launch apply procedure  Prevent Double changes  Have Change Log  Simple integration to existent project
  13. 13. Continuous DB Migration approach min + automatic migration = Continuous DB Migration
  14. 14. Carbon V
  15. 15. Why Carbon V  DB-migration framework  Open Source project  Lightweight framework  Simple usage
  16. 16. How it works  Dedicated project folder for SQL migration scripts. This folder should be available when framework takes control.  Each SQL migration script is pure SQL/DDL file with migration SQL code for specific feature  Each Delta file should have name YYYYMMDDHHMMSS_<FEATURE>.sql  When framework take control it checks and applies only new changes.  C5 uses own JDBC connection.  Way to re-usue existent DataSources with connection params
  17. 17. No magic, just…  C5 creates own table “schema_version” inside your database  C5 controls this table by itself (no needs to create or update). sql_file_name date duration 201405170900_user_auth.sql 2014-05-17 13:00:00 8 sec 201404130400_award_4565.sql 2014-04-14 14:00:00 6 sec
  18. 18. When to initiate migration procedure?  Maven-driven approach  Embedded mode
  19. 19. Maven-driven mode
  20. 20. Maven-driven mode
  21. 21. Configure Maven project  Update POM definition  Configure Explicit DB connection  Can be configured depending on specific maven profiles (staging, production)
  22. 22. Add SQL migration files  Use Project/src/main/db/migrations directory
  23. 23. Launch it  performs migration $ mvn db-migration:migrate  resets migration $ mvn db-migration:reset  check if DB is up to date $ mvn db-migration:validate  create new feature $ mvn db-migration:new -Dname=new_feature
  24. 24. Use cases  Manual runs from developer machine  Part of Continuous Deivery process – Add just one more Maven action inside specific CI configuration before deployment action
  25. 25. Embedded mode
  26. 26. Embedded mode  Schema Check is constituent part of application.  SQL changes are built-in into the application  Any checks and possible DB migration is performed every time when application launches.  Dedicated bean to carry out this responsibility
  27. 27. Create application bean import com.carbonfive.db.migration.DataSourceMigrationManager; import com.carbonfive.db.migration.DriverManagerMigrationManager; import com.carbonfive.db.migration.MigrationManager; import com.carbonfive.db.migration.ResourceMigrationResolver; public class DBMigratorImpl { DataSource dataSource; ... public void init(){ DataSourceMigrationManager migrationManager = new DataSourceMigrationManager(dataSource); migrationManager.setMigrationResolver(new ResourceMigrationResolver("classpath:/db/migrations")); migrationManager.migrate(); } }
  28. 28. Inject the bean into application context  Seamless integrated with Spring DI  It is possible to initialize ResourceMigrationResolver DataSourceMigrationManager directly though Spring DI without any line of code  Two places where bean might be integrated inside Spring context  Before Persistent beans  After Persistent beans
  29. 29. Embedded vs maven driven  E: Consistent and grace application upgrade  E: No delays between deployment and schema upgrade  M: Simple integration with CI  M: External commands like validate & reset  M: Quick manual migrations (hotfixes for instance)
  30. 30. Carbon5 Benefits  Lightweight framework  Brilliant extensibility (thanks to reasonable design)  Support many DBMS in unofficial mode  Simple integration  Real continuous DB migration
  31. 31. Need more features?
  32. 32. Flyway framework  Rich support for different build tools  The same integration approach on Carbon V  Pluggable architecture  Java-based migrations (two options: through JDBC Connection & Spring JDBCTemplate)  Extended service table
  33. 33. Best practices  Feature driven development (each SQL feature change in own file)  Make sql files safe in order to prevent it from double execution  Keep database connection settings in single place (use shared DataSource)  Do not make structure changes to the DB outside of your framework
  34. 34. What’s about schemaless ?
  35. 35. NoSQL migration approach  No dedicated explicit migration procedure (solved on business level in case of needs)  No shutdown or downtime  Rolling update – Every domain entity has version number – your v2 application should handle the v1 db format – Write code to convert entity to v2 on a fly (repository level) – Write modified entities to v2 on demand  Some DBs like RavenDB are able to perform auto-migration
  36. 36. References    Martin Fowler Evolutionary Database Design  Refactoring Databases
  37. 37. Presentation in a nutshell  Keep in mind this problem  Do not invent wheels, use external frameworks/techniques  And do not be afraid of Friday’s deployment then.
  38. 38. Thank you! Q&A