Building an Automated Database Deployment Pipeline
Upcoming SlideShare
Loading in...5
×
 

Building an Automated Database Deployment Pipeline

on

  • 658 views

The pace of business accelerates fairly continuously and application development moves right with it. But we’re still trying to deploy databases the same way we did 10 years ago. This session ...

The pace of business accelerates fairly continuously and application development moves right with it. But we’re still trying to deploy databases the same way we did 10 years ago. This session addresses the need for changes in organizational structure, process and technology necessary to arrive at a nimble, fast, automatable and continuous database deployment process. We’ll use actual customer case studies to illustrate both the common methods and the unique context that led to a continuous delivery process that is best described as a pipeline. You will learn how to customize common practices and tool sets to build a database deployment pipeline unique to your environment in order to speed your own database delivery while still protecting your organization’s most valuable asset, it’s data.

Statistics

Views

Total Views
658
Views on SlideShare
367
Embed Views
291

Actions

Likes
0
Downloads
7
Comments
0

18 Embeds 291

http://petermorlion.blogspot.be 223
http://petermorlion.blogspot.com 23
http://petermorlion.blogspot.co.uk 10
http://news.google.com 5
http://petermorlion.blogspot.de 5
http://petermorlion.blogspot.in 4
https://twitter.com 3
http://petermorlion.blogspot.pt 3
http://petermorlion.blogspot.com.au 2
http://petermorlion.blogspot.fr 2
http://petermorlion.blogspot.fi 2
http://feedly.com 2
http://petermorlion.blogspot.com.es 2
http://petermorlion.blogspot.se 1
http://petermorlion.blogspot.com.tr 1
http://prlog.ru 1
http://petermorlion.blogspot.ca 1
http://petermorlion.blogspot.ch 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Building an Automated Database Deployment Pipeline Building an Automated Database Deployment Pipeline Presentation Transcript

  • BUILDING AN AUTOMATED DATABASE DEPLOYMENT PIPELINE Grant Fritchey Red Gate Software Continuous delivery for databases
  • Goals  Understand the technology and process requirements to work towards automation step- by-step in your release pipeline.  Learn about the organizational changes necessary to support process modifications.  Appreciate why these changes are necessary in support of modern development and deployment methodologies.
  • grant@scarydba.com www.scarydba.com @gfritchey www.linkedin.com/in/scarydba Grant Fritchey Product Evangelist, Red Gate Software
  • ALM – and where the database fits in Three core processes in Application Lifecycle Management: Governance Development Operations
  • Natural friction across pipeline Development → Operations
  • Natural friction across pipeline Development → Operations Operations → Development Why?
  • Development focus is on speed
  •  Agile  Scrum  Lean  Feature-driven Development  Iterative
  • Operations focus is on production protectionprotection
  •  Monitoring  Deployment  Integrity  Data Management
  • Databases as a bottleneck – historically Odd languages  SQL  Cubes  X-Query 3 reasons why databases have traditionally slowed down deployments: Data persistence  Data outlives applications  Data can’t be replaced DBA paranoia  Frankly… 1 2 3
  • A quick primer on continuous delivery Common misconception: it’s about automating to Production
  • A quick primer on continuous delivery Common misconception: it’s about automating to Production In fact – there’s a difference between three continuous approaches: Development Source control CI server Test Production Continuous integration Continuous deployment Continuous delivery
  • The goals of continuous delivery for databases  Faster feedback on changes made o Continuously integrate team changes o Automated testing o Releases rehearsed in testing environments before deployed to Production = repeatability, repeatability and a strong team process for changes
  • Focus on the pipeline Think of the Toyota production system…  Start with Lean o Focus on the customer, eliminate waste o Continuously Improve o Empower the team o Optimize the whole o Plan for change o Automate processes o Build quality in
  • SOURCE CONTROL CONTINUOUS INTEGRATION: FUNDAMENTALS CONTINUOUS INTEGRATION: ADVANCED AUTOMATED DEPLOYMENT Four key stages of the deployment pipeline
  • SOURCE CONTROL  Greenfields  Existing systems  Test data Four key stages of the deployment pipeline
  •  Automate builds as first step o Incremental o Complete  Build library of manual tests  Automate testsCONTINUOUS INTEGRATION: FUNDAMENTALS Four key stages of the deployment pipeline
  •  Select CI tooling  Team discipline  Customization of build steps  Automate the testingCONTINUOUS INTEGRATION: ADVANCED Four key stages of the deployment pipeline
  •  Testing  Automation  Safety o Backups o Rollback strategyAUTOMATED DEPLOYMENT Four key stages of the deployment pipeline
  • Create development environment for automation Empower Development  Environment should let them work at their speed Take part in Development  Instead of stopping bad deployments, stop bad development Automate your build process  Supply clean production data or supply good sample data Get started on writing tests  Build your library
  • Different types of testing for different stages of the pipeline: Development Integration Testing QA Pre-Production/ Staging Production Automate testing Unit tests Integration tests Automated tests Deployment validation Behaviour validation Other validations
  • B C D A Always Be Continuously Delivering Deliver
  • Every delivery is practice
  • Fail faster  But less frequently  Smooth the process
  • Testing, testing, testing  Select the right tests for each stage  Review and act on the results
  • Automate
  • DBAs must work with Devs… Yes, you must protect the data for the business, but that must be tempered with helping deliver functionality. Changes to philosophy
  • Devs must work with DBAs… Yes, you may know more about business needs, so educate rather than isolate. Changes to philosophy
  • Project Management must think of operations as part of development… Yes:  Deployment is part of development  Release 1.1 and on are part of development o And planning for 1.1 is not premature optimization  Data retention is part of development Changes to philosophy
  • Changes to workplace SOURCE CONTROL CONTINUOUS INTEGRATION: FUNDAMENTALS CONTINUOUS INTEGRATION: ADVANCED AUTOMATED DEPLOYMENT
  • Changes to workplace for continuous database delivery  Empower developers to do more  Bring DBAs into development teams  Management must buy-in  Management must get out of the way
  • How are other companies getting started?
  • Case Study: Boxon  Global packaging and labelling company  Based in Sweden; sub-division in China  Three business areas: profitable packing solutions; intelligent marking; customized big-bags solutions for bulk handling  One developer responsible for ASP.NET application with SQL Server backend, enabling customers to control consistent printing of labels wherever the print shop is located in the world
  • Boxon – Initial Process  Deploying to production was scary…  Database changes not version- controlled  2-hour window - at night - to complete deployments to production – difficult to find quiet period with a global customer base PAIN POINTS
  • Boxon – Improved Process  Deployments are less scary now…  Tickets are logged and prioritized in Unfuddle (unfuddle.com)  Ticket numbers are logged in SVN and TeamCity to track items  Now use SQL Source Control and Subversion to version DB changes o Source of authority on database build  TeamCity (CI build server) is triggered on check-in of changes o Fast response – changes checked in frequently  Notification if build fails  Build packaged into a Nuget file for deployment BUILDING A PIPELINE
  • Case Study: Move with Us  Risk management solution and sales and marketing channel provider for residential property businesses in the UK  Located near Cambridge, UK  Customers put a heavy load on databases, so updates need to be efficient
  • Move with Us – Initial Process  Database code often not checked in to VCS due to risk of conflicts – changes were communicated by email instead  Hot fixes sometimes done directly on integration server  1 day needed to release updates – a lot of work in building and testing  Problems with merging code led to a lot of firefighting PAIN POINTS
  • Move with Us – Improved Process  Each developer has a local copy of database  SQL Source Control and Subversion used to check in database changes to version control  TeamCity (CI build server) automatically compiles changes on check-in  Successful builds packaged into a Nuget file automatically o Fast getting here – multiple database changes checked in per day  Nuget package deployed to Test on demand  If problems detected, return to Dev to fix BUILDING A PIPELINE
  • Case Study: Practice Fusion  Web-based electronic health record (EHR) company  Founded in 2005  Based in San Francisco  Hosts over 50 million patient records  Used in all 50 states and by 150k+ physicians  3 DBA engineers; 1 data architect; 10+ developers
  • Practice Fusion – Initial Process  Hand-crafted SQL scripts - inconsistent  Database changes not always version controlled  Changes released by opening dozens of files in SSMS  Long-winded team comms if further changes needed  Smoke-testing deployments showed some objects had been deployed straight to Prod and weren’t in deployment script PAIN POINTS
  • Practice Fusion – Improved Process  SQL Source Control and Subversion to version DB changes o Source of authority on database build  Jenkins (CI build server) is triggered on check-in of changes o More time to develop; less time managing scripts  Build failed by Jenkins if problems detected o Fewer issues deploying to production  Jenkins, SQL Compare and SQL Data Compare used to deploy changes to environments  Custom scripts for replication BUILDING A PIPELINE
  • Summary  Eliminate or mitigate friction o Slow is smooth, smooth is fast  Adopt lean methodologies and focus  Four key stages o Source control o Build automation o Continuous Integration o Continous Deployment  A-B-C-D  Change your philosophy  Change your workplace
  • Goals  Understand the technology and process requirements to work towards automation step- by-step in your release pipeline.  Learn about the organizational changes necessary to support process modifications.  Appreciate why these changes are necessary in support of modern development and deployment methodologies.
  • Documentation and resources  Continuous Delivery by Jez Humble and David Farley (Addison Wesley)  The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win by Gene Kim, Kevin Behr and George Spafford (IT Revolution Press)  The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt and Jeff Cox (Gower Publishing Ltd.)  Agile Organization by the agile admin (theagileadmin.com) Further resources:  Database Delivery Learning program: www.red-gate.com/delivery o Patterns and practices on Simple-Talk o Tutorials in Red Gate training academy  www.youtube.com/user/RedGateVideos - for recorded seminars
  • Image sources Author Source Information Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work” Tableatny Wikimedia Commons Athlete at Starting block.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution 2.0 Generic license. Source on Wikimedia Commons: “BXP135671” n.raveender Wikimedia Commons Stop No Entry.jpg – Wikimedia Commons. This file is in the public domain. Source on Wikimedia Commons: “Own work.” Original uploader was Liface at en.wikipedia Wikimedia Commons Cross Country US.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Transferred from en.wikipedia; transfer was stated to be made by User:TFCforever.” Notes on license: Liface at the English language Wikipedia, the copyright holder of this work, hereby publishes it under the following license:GNU head Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. Subject to disclaimers. Horia Varlan Flickr Graduated cylinders and beaker filled with chemical compounds – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license. Henry Mühlpfordt Flickr CERN Atlas Control Room 2010-07-01 – Flickr. This file is licensed under the Creative Commons Attribution-ShareAlike 2.0 Generic license. Department for Business, Innovation and Skills Flickr Toyota’s new Auris – Flickr. This file is licensed under the Creative Commons Attribution-NoDerivs 2.0 Generic license. Qrodo Photos Flickr Sport Action – Fencing – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license. William Warby Flickr Gears – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.