Moving from application automation to true DevOps by including the database
•Download as PPTX, PDF•
1 like•262 views
As software teams face increasing pressure to speed up the delivery cycle, the DevOps approach is gaining momentum. But the database is commonly a bottleneck. This presentation will explore how you can include the database in continuous integration and automated release management. Sharing best practices from high-performing IT organisations, we will discuss how you can extend DevOps practices to the database; increasing productivity, agility and optimizing performance.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Moving from application automation to true DevOps by including the database
Similar to Moving from application automation to true DevOps by including the database (20)
More from Red Gate Software
More from Red Gate Software (20)
Recently uploaded
Recently uploaded (20)
Moving from application automation to true DevOps by including the database
- 2. Tom Austin Head of Pre Sales Engineering tom.austin@red-gate.com /in/tomowenaustin/
- 3. 230 Redgaters and counting 17 years old 202,000 customers 2m SQL Server Central and Simple Talk users 91% of the Fortune 100 use our tools 4m website visits each year 1058 product releases last year 68 User Groups sponsored so far this year
- 4. “DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.” Donovan Brown, Principal DevOps Program Manager, Microsoft
- 5. 46x more frequent deployments 440x faster lead times 96x faster recovery from failures High performers have: https://puppet.com/resources/whitepaper/state-of-devops-report
- 6. “The greatest challenge with integrating database changes into a DevOps process is synchronizing application and database changes, and overcoming different development approaches.” http://www.red-gate.com/solutions/database- devops/report
- 7. 80%
- 8. 80% increase the speed of delivery
- 9. 80% increase the speed of delivery both applications and databases
- 12. Deployments must preserve the existing data Static data vs production data DBAs aren’t always part of DevOps Databases can drift
- 13. Environments & Development Protecting & Preserving data Continuous Integration & Deployment
- 14. With good collaboration between teams, effective management of environments and use of best practices like version control or automated provisioning, your development practices are optimized. Your team is free to focus on process improvements. You’re at advanced level for Environments & Development
- 15. • Changes are made in development and promoted to higher environments • Developers check in changes as they are made • Version control is the source of truth • Dev and Ops team work together with a common goal Environments & Development
- 16. “We’re now deploying at a much higher frequency than before, and have doubled deployments to production” Thomas Johansson, AFA Försäkring
- 17. By automating your database deployment pipeline and testing your work with realistic data and server environments, you’ve minimized the risk of introducing bugs and defects further downstream. You can now focus on iterative improvements that will enable you to further increase frequency of deployments. You’re at advanced level for Continuous Integration & Deployment
- 18. • Builds are not left broken • CD of database changes takes place to Production • A release management tool is used for automation • Staging always matches Production Continuous Integration & Deployment
- 19. “ABSA have been able to implement a one-step process from development through to production. This frees up the equivalent of an extra developer.” Rynhardt van Rensburg, Systems Analyst, ABSA Bank
- 20. With a solid data management strategy in place, you can readily demonstrate compliance. Effective monitoring across environments enables performance to be correlated with changes so you can pinpoint the cause of issues and apply continuous improvements. You’re at advanced level for Protecting & Preserving data
- 21. • Dedicated performance monitoring is in place • A solid database backup and recovery plan is in place • A modified copy of production data is used • There is a clearly-articulated data strategy Protecting & Preserving data
- 22. “SQL Monitor has saved us a least two hours a day… probably more if you add how it’s enabled us to change the way we work.” Matt Haigh, Senior Database Administrator, Mamas & Papas
- 23. Complete Redgate’s Database DevOps Maturity Assessment
- 25. Contact us: devops@red-gate.com Discover Redgate’s full Database DevOps solution www.red-gate.com/solutions
Editor's Notes
- Worked with Redgate SQL tools since 2008 Helped customers across the globe From start-ups to FTSE/Fortune 100 Specializes in Database Lifecycle Management (DLM) Writing and delivering Database DevOps training in Europe and Asia
- Headquarters in Cambridge, UK Offices in Pasadena and Austin 17 years old Over 200k customers, 800k users 91% Fortune 100 use RG Tools 1058 product releases last year
- “DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.” VALUE to end users 3 core areas
- throughput: how frequently a team can deploy code and how fast it can move from committing code to deploying it stability: how quickly the system can recover from downtime and how many changes succeed, versus how many fail. This year’s report showed that high performers (those who adopt DevOps practices) do significantly better than lower-performing peers in terms of throughput and stability. When compared to the 2016 report, the gap between high and low performers narrowed for throughput ( based upon deployment frequency and lead time) and widened for stability ( measured by looking at mean time to recover and change failure rate). This is likely because low-performing teams work to increase speed but don’t invest enough in building quality into the process.-High performers however, understand that they don’t have to trade speed for stability or vice versa because by building quality, they get both.
- Database centric – 1000 respondents Free to download
- DevOps is about changing culture and improving collaboration between development and operations. But it is also about automating as many of the common jobs in delivering software; testing, compliance and security checks, software packaging and configuration management, as well as deployment. Here we see a typical automated deployment pipleine– version control, continuous integration/delivery, automated testing, automated deployment etc. Most people are already doing this for the application.
- But many are jeopardising their DevOps success by failing to include the database. Databases are more problematic – business critical data needs to be safely and correctly preserved, databases carry state that needs to be managed as part of rolling out new or updating existing software. In addition to this, there are specific challenges in the FS industry: – extremely complex systems, legacy databases, siloed departments etc Redgate have worked with a number of organisations to extend DevOps practices to the database, creating reliable, scalable and repeatable processes for building, testing and deploying database changes alongside your application code. We believe that the database should just fall in place alongside the application as we can see on the pipeline above.
- What about databases? Why are they so problematic? Because of the persistent data the our businesses our built on? Schema changes vs existing data You can’t simply drop and replace the DB because you’ll lose your data. You have to create some sort of upgrade script. Twice as much code normally means twice as many errors and if the state you want and your way to get there don’t match, which is right? What is your source of truth? Without a clear idea of your source of truth – how is DevOps even possible? Reference data vs production data But while the database has existing data, not all of it is ‘production’ data in that sense. What about country codes? Reference data, lookup data, static data, the data that makes your system work. This data needs to get deployed with your schema changes. And you also need to think about how to transfer data in the other direction for testing. How do you test the latest dev build with production (or-production-like) data? DBAs aren’t always part of DevOps The word DevOps refers to the problems associated with siloed Dev and Ops teams. Nowhere is this more apparent than in the land of the database? Is there anyone here who has never heard of any problems between Dev and DBA teams? But it goes further than that. With application source code we use source control, we invented distributed source control systems and we debate about the optimal branching strategies and strategies for implementing continuous integration. We’ve barely begun to have these conversations about databases. We barely have strategies for how to provision individual developers with their own sandboxes. Sometimes different developers work in different ways, some working off scripts and others working directly on the database. We need a better way of working together. And finally testing. Our problems with database source control and acquiring suitable test data make it hard for us to provision test database environments manually, let alone automatically. Database drift And all these problems result in this final point. In DevOps and Continuous Delivery we often talk about cycle time. How long would it take you to make a one line change, run it through your normal testing process and get it to production? If the cycle time for your database is measured in days, weeks or months (or years?) then when you hit a production issue you don’t have time to go back to your source code. The business is haemorrhaging money and the DBA will make a decision: What is more expensive, the cost of delaying the fix, or the risk of making that low risk change now and fixing the problem right away. It’s all well and good wagging your finger at people who choose to make hot-fixes directly on production, but until we sort out the cycle time problem, it won’t go away. Production drift is a symptom of poor DevOps or DLM strategy. And drift causes more problems: Environment inconsistencies undermine your tests and can cause failed deployments either because code clashes or because important fixes are accidentally rolled back. Drift and poor DevOps processes are a vicious circle that needs to be broken.
- Whether you’re exploring the advantages of DevOps or you’re already fully immersed in the journey, including the database brings additional advantages. Now we can help you to assess the maturity of your own processes and use that insight to benchmark you against your competition and make recommendations for how to move forward. We’ve split out Database DevOps Maturity into 3 main areas E&D, CI&D and P&PD
- Process of provisioning becomes effortless with closely tied DBAs and Devs working together to promote stability Development Environments are disposable All code is version controlled – including the database
- Wholly owned by Sweden's labor market parties, AFA Försäkring (AFA Insurance) provides a range of insurance policies and financial support to over four million employees in both the public and private sector. it manages assets of approximately SEK 200 billion (US$24 billion). Behind the scenes, five development teams manage around 70 servers and 400 databases on the Microsoft platform alone, and look after two online systems handling up to 130,000 logins per week. -Manual scripts -SQL Compare -Source Control - DLMAS
- Automated Deployment and testing Database Unit Tests Database specific coding standards checks Removed manual handover Maintain visibility and review opportunity Test Data Generation/Management – automated provisioning of test environments
- A wholly-owned subsidiary of the Barclays Africa Group, ABSA Bank offers a range of retail, business, corporate, and investment banking, as well as wealth management products and services, primarily in South Africa and Namibia. A team of 40 developers supports the bank's payment processing systems and handles 3,000 high-value transactions per day Dev took 2-3 hours per dev per week
- Auditability Compliance adherence is clear Pro active environment improvement rather than daily maintenance checks Tie change and performance together
- Mamas & Papas (M&P) is a UK-based retailer and manufacturer supplying strollers (prams and pushchairs), baby products, furniture, and maternity wear. It was established in Huddersfield in 1981 and has grown as a family business to become one of the top nursery brands. M&P has over 35 stores in the UK & Ireland with a growing number of franchise stores in the Middle East, Far East, North America, South America, and Taiwan. With a turnover of $133m, it employs nearly 1,000 employees in its head office and across its retail stores. Matt Haigh is the Senior Database Administrator with the infrastructure team. His team look after 65 SQL Servers across both the head office and the stores, which run Electronic Point of Sale (EPOS) systems.