Search

Agile Database Development with Liquibase

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    I’m your presenter.

    I’m a Java developer.

    I work with a lot of open source technologies.

    I haven’t written a lot of Java in the past year. Mostly these days I write Groovy.

    I participate in the local development community by serving on the boards of www.denveropensource.org and www.iasadenver.org.

    The August Technology Group is my consulting firm.

    As software developers and architects, our worlds are filled with data.

    Unfortunately, the data is not always in the condition we’d like. We try to do a good job structuring it, but the reality is that we often fail, and even when we succeed, the business changes enough so that even our successes are short-lived.

    Our tool sets and processes have developed to deal expertly with rapidly changing code.

    Our tool sets and processes have developed to deal expertly with rapidly changing code.

    The same tools and practices have not been applied to data.
    Leading to one of two problems...

    The same tools and practices have not been applied to data.
    Leading to one of two problems...

    Will it die? It is said of evolutionary systems that things that fail to adapt to change die out. If only this were the case with the enterprise database...

    Rather than going extinct, the production DBAs surround it like priests, carefully filling it with embalming fluid and wrapping it in linen strips. (And possibly putting some operations personnel in the tomb to serve the database in the afterlife.)

    Inability to change and overprotection lead us to this dread antipattern.

    There are long-lived Enterprise applications that rely on it, so it can’t just go away.

    It isn’t really alive either, because business needs are constantly changing, but the database can’t change with them.

    At least the production DBAs seem to think so!

    We’ll figure out how to manage changes to the database such that we can make them with confidence in a way that brings the best of developer tools to bear and is relatively friendly to DBA workflows. We’ll do this with a mindset and a tool.

    The fix is what Scott Ambler calls “evolutionary [or agile] database development.” This consists of five components.

    Martin sez...

    Martin sez...

    Scott sez.... Code refactorings are really only concerned about behavior. DBs have behavior (stored procs, triggers, etc.) but also information. The database must say the same thing in the same way after the refactoring.

    These are two mistakes here: one is thinking we’re smart enough to do all this designing correctly at the outset—we’re not. The other is thinking that the database’s business context is a static thing—it isn’t. There is no “right” up front, because requirements will change constantly.

    Which doesn’t mean we can’t do a couple of days of designing at first; we can. It makes sense to try to anticipate what we can and make big, hard-to-change commitments correctly. We always expect change, though.

    TDD adoption in software development is low enough, but it is virtually unheard of in database development. The tooling lags behind and the expertise is singularly rare.

    There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.

    There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.

    There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.

    There are ways to do it. The tools aren’t what they are for TDD of code, but there are options.

    Bring all the knowledge, practices, and advantages of software source control to the database. Simply control all those text files in SVN or Git like you normally would. This practice is old hat.

    Every database artifact goes in to the repository.

    Developers need a place to deploy refactorings when they’re trying to get their tests to pass. This must be a local database not used by any other team member or system.

    If we learn how to refactor databases from Ambler and Sandalage, Liquibase is the tool that makes it easy. It is an XML-based love poem to Scott Ambler.

    Liquibase is fundamentally a command-line tool written in Java. It uses JDBC to communicate with the database, and can coexist well in a non-Java shop. I can be invoked from popular open-source build tools and frameworks.

    Liquibase is fundamentally a command-line tool written in Java. It uses JDBC to communicate with the database, and can coexist well in a non-Java shop. I can be invoked from popular open-source build tools and frameworks.

    Liquibase is fundamentally a command-line tool written in Java. It uses JDBC to communicate with the database, and can coexist well in a non-Java shop. I can be invoked from popular open-source build tools and frameworks.

    Liquibase is fundamentally a command-line tool written in Java. It uses JDBC to communicate with the database, and can coexist well in a non-Java shop. I can be invoked from popular open-source build tools and frameworks.

    We’ll consider three aspects of Liquibase. How it stores the schema, how it interacts with the database, and how to use it in some real-world scenarios.

    We’ll consider three aspects of Liquibase. How it stores the schema, how it interacts with the database, and how to use it in some real-world scenarios.

    We’ll consider three aspects of Liquibase. How it stores the schema, how it interacts with the database, and how to use it in some real-world scenarios.

    Must we rewrite our SQL in XML? We must. It’s painful and unappetizing, but worth it! Also, there’s a Grails plugin called Autobase that lets us do it in nice Groovy Builder syntax, which is preferable. We won’t address Autobase in detail here, but it’s worth looking in to.

    Must we rewrite our SQL in XML? We must. It’s painful and unappetizing, but worth it! Also, there’s a Grails plugin called Autobase that lets us do it in nice Groovy Builder syntax, which is preferable. We won’t address Autobase in detail here, but it’s worth looking in to.

    Must we rewrite our SQL in XML? We must. It’s painful and unappetizing, but worth it! Also, there’s a Grails plugin called Autobase that lets us do it in nice Groovy Builder syntax, which is preferable. We won’t address Autobase in detail here, but it’s worth looking in to.

    The changelog is a script that builds your schema one DDL statement at a time. Each changeSet is converted into a dialect-specific SQL statement and is executed in the database, then marked as complete in a log table.

    Extracts the database’s metadata and generates a changelog. This is the first step is getting started on a database that already exists.

    Since Liquibase tracks the status of all changesets in the changelog, we’ll need to tell it that our newly extracted changelog is in sync with the actual database. This command usually shouldn’t be executed except as the second step getting started with the tool.

    Plays any new changeSets in the changeLog against the database. This is the statement you’ll use the most during evolutionary database development with Liquibase.

    Marks the database for a future rollback.

    Rolls back changesets to given tag.

    Supposed to document the differences between the database and the changelog. I’ve not had good luck with this so far against MySQL and SQLServer databases.

    Evolutionary Database Development is not controversial among software developers, because we are largely persuaded of the core propositions. Moreover, Liquibase seems like a reasonable tool to help. Now let’s talk about some practices to put in place in a brownfield effort.

    People are using it, it works, it has literally years of knowledge and business process encoded in it. It’s also ugly and outdated, and its schema seems to have been built by the DBA Club after school. But you’re replacing it, with Grails...!

    1 Favorite

    Agile Database Development with Liquibase - Presentation Transcript

    1. AGILE DATABASE DEVELOPMENT WITH Tim Berglund June, 2009
    2. TIM BERGLUND
    3. WE BELIEVE CODE CHANGES
    4. WE BELIEVE CODE CHANGES
    5. WE ACT LIKE THE DATABASE DOESN’T
    6. WE ACT LIKE THE DATABASE DOESN’T
    7. TERMINAL MALADAPTION
    8. EMBALMING
    9. WON’T DIE
    10. NOT REALLY ALIVE
    11. CURSES THE DBA WHO CHANGES IT
    12. WE WILL BREAK THE CURSE
    13. AGILE DATABASE DEVELOPMENT
    14. Refactoring Evolutionary Automated Data Testing Modeling Agile Database Development Source Sandboxes Control
    15. REFACTORING
    16. REFACTORING
    17. REFACTORING This term occasionally gets abused!
    18. REFACTORING A disciplined way to make small changes to your source code to improve its design, making it easier to work with.
    19. REFACTORING Not adding features!
    20. REFACTORING Not fixing bugs!
    21. REFACTORING Opportunistically improving design to accommodate change.
    22. DATABASE REFACTORING A simple change to a database schema that improves its design while retaining both its behavioral and informational semantics.
    23. EVOLUTIONARY DATA MODELING
    24. EVOLUTIONARY DATA MODELING NOT TRYING TO “GET IT RIGHT UP FRONT.”
    25. EVOLUTIONARY DATA MODELING MAYBE SOME BIG THOUGHTS AT FIRST
    26. EVOLUTIONARY DATA MODELING BUILD THE SIMPLEST THING THAT CAN POSSIBLY WORK...EVERY DAY
    27. DATABASE TESTS TDD IN THE DB
    28. DATABASE TESTS
    29. DATABASE TESTS STORED PROCEDURES
    30. DATABASE TESTS STORED PROCEDURES SQLUNIT
    31. DATABASE TESTS STORED PROCEDURES SQLUNIT DBUNIT
    32. DATABASE TESTS STORED PROCEDURES SQLUNIT DBUNIT CUSTOM HYBRIDS
    33. SOURCE CONTROL
    34. SOURCE CONTROL LIKE MOTHER TAUGHT YOU
    35. SOURCE CONTROL
    36. SOURCE CONTROL DDL EXTRACT/MIGRATION SCRIPTS REFERENCE DATA STORED PROCS/TRIGGERS
    37. SANDBOXES
    38. COMMAND-LINE TOOL
    39. COMMAND-LINE TOOL JAVA/JDBC
    40. COMMAND-LINE TOOL JAVA/JDBC CROSS-PLATFORM
    41. COMMAND-LINE TOOL JAVA/JDBC CROSS-PLATFORM ANT, MAVEN, SPRING, GRAILS
    42. CHANGELOG
    43. CHANGELOG COMMANDS
    44. CHANGELOG COMMANDS MIGRATION SCENARIOS
    45. CHANGELOG
    46. CHANGELOG SQL IN XML
    47. CHANGELOG SQL IN XML COHERENT GROUPS OF OPERATIONS
    48. CHANGELOG SQL IN XML COHERENT GROUPS OF OPERATIONS TRACKED IN DATABASE
    49. CHANGELOG <changeSet author="tlberglund (generated)" id="1236660747556-11"> <createTable tableName="scshoppingcart"> <column autoIncrement="true" name="id" type="INT"> <constraints nullable="false"/> </column> <column name="userid" type="INT"> <constraints nullable="false"/> </column> <column name="lastmodified" type="DATETIME"/> <column name="name" type="VARCHAR(50)"/> </createTable> </changeSet> <changeSet author="tlberglund" id="spot-1-1-2"> <renameTable oldTableName="scscrubrecordwarehouse" newTableName="scrub_record_log" /> </changeSet>
    50. LIQUIBASE COMMANDS
    51. LIQUIBASE COMMANDS generateChangeLog
    52. LIQUIBASE COMMANDS changeLogSync
    53. LIQUIBASE COMMANDS update
    54. LIQUIBASE COMMANDS tag
    55. LIQUIBASE COMMANDS rollback
    56. LIQUIBASE COMMANDS diff
    57. DEMO
    58. AND WE’RE BACK
    59. Refactoring Evolutionary Automated Data Testing Modeling Agile Database Development Source Sandboxes Control
    60. CULTIVATING THE BROWNFIELD
    61. LEGACY ENTERPRISE APP
    62. NEW GRAILS HOTNESS
    63. HARD CUTOVER
    64. CREATIVE ORM MAPPINGS
    65. WITHERING ON THE VINE
    66. BROWNFIELD DATA INTEGRATION
    67. BROWNFIELD DATA INTEGRATION TIME ZERO MIGRATION
    68. BROWNFIELD DATA INTEGRATION TIME ZERO MIGRATION
    69. BROWNFIELD TESTING
    70. BROWNFIELD TESTING INTEGRATION TEST DATA
    71. BROWNFIELD TESTING
    72. BROWNFIELD TESTING T0 MIGRATION SCRIPTS
    73. BROWNFIELD TESTING
    74. THE CURSE IS BROKEN
    75. THANK YOU! TIM BERGLUND AUGUST TECHNOLOGY GROUP, LLC http://www.augusttechgroup.com tim.berglund@augusttechgroup.com @tlberglund
    76. PHOTO CREDITS HAMMER SHATTERING GLASS: HTTP://WWW.FLICKR.COM/PHOTOS/WHISPERWOLF/3486270713 DODO: HTTP://WWW.INTERNATIONALDOVESOCIETY.COM/MISCSPECIES/DODO.HTM EMBALMING: HTTP://LIBRARY.THINKQUEST.ORG/C0116982/HTML%20PAGE%20FOLDER/ HMUMMIFICATION.HTM ATTACKING MUMMY: HTTP://WWW.FLICKR.COM/PHOTOS/ROONBABOON/292393932/ CREEPY MUMMY HEAD: HTTP://WWW.FLICKR.COM/PHOTOS/DR-INI/446311713/ MARTIN FOWLER: HTTP://WWW.FLICKR.COM/PHOTOS/PRAGDAVE/173640462/ SCOTT AMBLER: HTTP://WWW.AMBYSOFT.COM/SCOTTAMBLER.HTML BRENDEN FRASER: HTTP://SCRAPETV.COM/NEWS/NEWS%20PAGES/ENTERTAINMENT/UNIVERSAL-FIRE- ACCIDENTAL-NEW-MUMMY-MOVIE-DELIBERATE-SCRAPE-TV-THE-WORLD-ON-YOUR-SIDE.HTML SAND: HTTP://WWW.FLICKR.COM/PHOTOS/LEVIATHOR/207625319/ MOTHER AND CHILD: HTTP://WWW.FLICKR.COM/PHOTOS/PATRICK_Q/268149208/ BROWN FIELD: HTTP://WWW.FLICKR.COM/PHOTOS/ARTUR02/471094378/ JALOPY: HTTP://WWW.FLICKR.COM/PHOTOS/DRAGONFLEYE/3246153/ JALOPY ON RT 66: HTTP://WWW.FLICKR.COM/PHOTOS/EVOETSCH/2436023218/ PIMPMOBILE: HTTP://WWW.FLICKR.COM/PHOTOS/AIRGAP/1053594933/ JUNKED JALOPY: HTTP://WWW.FLICKR.COM/PHOTOS/JABOOBIE/61014658/ PAINTING CAR: HTTP://WWW.FLICKR.COM/PHOTOS/GARETHJMSAUNDERS/2066690016/

    + Tim BerglundTim Berglund, 4 months ago

    custom

    383 views, 1 favs, 0 embeds more stats

    Learn to apply well-understood agile development pr more

    More info about this document

    CC Attribution License

    Go to text version

    • Total Views 383
      • 383 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 1
    • Downloads 13
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    -->
    Search
    RSS Feed
    What's new?

    © 2009 SlideShare Inc. All Rights Reserved