Download to read offline
Uploaded byKellyn Pot'Vin-Gorman
The Last Frontier- Virtualization, Hybrid Management and the Cloud
This document discusses virtualization, hybrid management, and cloud computing. It begins with an introduction to virtualization and discusses trends showing increasing adoption of public cloud infrastructure and platforms. The document then explores how companies are migrating applications and data to the cloud using various approaches like backups, data migration tools, and virtualization. It argues that data virtualization provides benefits over traditional migration methods by reducing costs, network usage, and storage requirements when moving workloads to the cloud.
More Related Content
Similar to The Last Frontier- Virtualization, Hybrid Management and the Cloud
More from Kellyn Pot'Vin-Gorman
![[BDD 2025 - Mobile Development] Exploring Apple’s On-Device FoundationModels](https://cdn.slidesharecdn.com/ss_thumbnails/md-exploringappleson-devicefoundationmodels-251124030840-d690542c-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Mobile Development] Mobile Engineer and Software Engineer: Are we...](https://cdn.slidesharecdn.com/ss_thumbnails/md-mobileengineerandsoftwareengineerarewestillrelevantsidiqpermana-251127010650-55224ef1-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Mobile Development] Crafting Immersive UI with E2E and AGSL Shade...](https://cdn.slidesharecdn.com/ss_thumbnails/md-craftingimmersiveuiwithe2eandagslshaderveronicaputrianggraini-251124030840-0c677f44-thumbnail.jpg?width=640&height=640&fit=bounds)
The Last Frontier- Virtualization, Hybrid Management and the Cloud
- 1. The Next FrontierVirtualization,Hybrid Management and the Cloud Kellyn Pot’vin-Gorman | Technical Intelligence Manager for the Office of CTO, Delphix
- 2. Kellyn Pot’Vin-Gorman Technical IntelligenceManager for the Office of CTO, Delphix • Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase, PostgreSQL…..) • Oracle ACE Director, (Alumni), Oak Table • APEX Women in Technology Award, CTA • STEM education with Raspberry Pi and Python • Liaison for Denver SQL Server User Group • Rocky Mountain Oracle Training Days Conference Director and Board Director • Author, blogger, (http://dbakevlar.com)
- 3. Virtualization is thecreation of a virtual (rather than actual) version of something, such as an operating system, a server, a storage device, database, files or network resources. What is Virtualization?
- 4. Virtualization Adoption Trend . Source:Gartner Forecasts Public cloud infrastructure as a Service, (IaaS) hardware and software Spending from 2015 to 2026, by segment, (in billions of US dollars) 0 20 40 60 80 100 120 140 160 180 200 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 SaaS and PaaS Public Cloud with IaaS
- 6. Source: Right ScaleState of the Cloud 2016 0% 10% 20% 30% 40% 50% 60% Physical Machine Virtual Machine Private Cloud Public Cloud 2017 2018 2019 Migration Plans- 2017-19
- 7.
- 8. An overall trendin companies that includes autonomic computing, where the IT environment will be able to manage itself based on perceived activity, and resource shared computing, in which computer processing power is utilized and/or paid for only as needed. The usual goal of virtualization is to centralize administrative tasks while improving scalability and work loads. The Future of Virtualization
- 9. Welcome to theNew Skill Adoption List!
- 10. Savings!! • Cost • Resource •Storage • Time Well, that depends on…. • Price per Gb • Complexity • SLAs and the biggest savings….
- 11. DBAs Can beViewed as the Bottleneck 3/25/2017 11 | • We are the gatekeepers • Archaic tools • Large amounts of data and complex migration steps • Fear of loss of role
- 12. 1. Just copydata and applications into the cloud… Straightforward approach Inefficient, non-incremental for large environments Open-source “bcp” uses encryption, compresses, and multi-threads Archaic processes recommended by vendors 2. Start with backups to IaaS storage, then populate re-hosted applications by restoring from those backups Cloud backups are easy, known technology How Are Companies Migrating to the Cloud?
- 13. Cost Estimates forAzure https://azure.microsoft.com/en-us/pricing/details/storage/blobs/ Storage Capacity LRS ZRS First 1 TB / Month $0.024 per GB $0.03 per GB Next 49 TB (1 to 50 TB) / Month $0.0236 per GB $0.0295 per GB Next 450 TB (50 to 500 TB) / Month $0.0232 per GB $0.029 per GB Next 500 TB (500 to 1,000 TB) / Month $0.0228 per GB $0.0285 per GB
- 14.
- 15. Backups cloud service:from on- premises to the Cloud Database server Cloud Storage
- 16. Migration Complete…Not soFast… • What if you only want dev and test in the cloud. • Data is migrated, but this doesn’t count for ongoing data loads, application connectivity across the network. • To refresh will take considerable time to perform with traditional tools or cloning methods. • Rarely a consideration for the difference in cost structure for processing large amounts of data from on-premise to the cloud.
- 17.
- 18.
- 19. https://docs.microsoft.com/en-us/azure/sql-database/sql-database-cloud-migrate • Database mustbe MSSQL 2005 or higher, (easy) • Ensure that the database is compatible with Azure SQL DB, (correct any incompatible functions, etc.) • Must have identified all performance issues that will be impacted beforehand. • Ensure there is as little physical distance between the cloud data center and bacpac files to be used for migration. • Disable management jobs that will hinder migration processing. • Drop any objects or historical data that can impact migration time and can be performed post migration.
- 20. https://azure.microsoft.com/en-us/downloads/migration-assistant/?&wt.mc_id=AID559320_SEM_14UHplSj&gclid=CJWky8vB6tICFdG2wAod0SEJqQ • Performs assessmentof IIS environment and creates report of what can be migrated. • Creates report to document what will require moving and any issues identified. • Creates any websites and associated databases in preparation for the migration. • If non-compatible products are in use, (which is in case of 90% of environments) then those must be addressed manually. • The tool is free to start, but will cost as project proceeds in duration.
- 21. From on-premises tothe Cloud, then restore- Assumption Database server Cloud Storage Database server
- 22. Much Improved Option- 1.Data virtualization Easy, secure, revolutionary Optimized for faster agile development and testing Optimized for new costing structures in IaaS With a software appliance, no concerns about hardware or software outside of virtualization lock-in. Migrating from on-prem to IaaS with Virtualization
- 23. Storage costs Thin-provisioning storageavoidance Data transfer costs Far less data transferred during provisioning/refresh operations • As opposed to the volume of data transferred using traditional cloning techniques Network-attached storage data transfers offset somewhat • Most non-production usage modifies or accesses only a tiny proportion of the total database during lifetime
- 25. Data Virtualization: FromProd to Virtual Validated Sync Environment Source Environment SCSI/SSL
- 26. Data Virtualization: FromProd to Virtual Validated Sync Environment Source Environment SCSI/SSL Any Storage Create as many VDBs as needed!
- 27. Data Virtualization: Space Savings DelphixVirtualization EngineValidated Sync Target Dozen Virtual Database Copies, (VDBs)
- 28. Data Virtualization: Howis it possible? Delphix Virtualization Engine Virtual Database Any Storage
- 29. Data Virtualization: on-premise Environment SourceDB server SQL Server 2008-2016 1 TB Delphix Virtualization Engine 2 TB storage Target DB server SQL Server 1008-2016 No database storage SCCI/SSL Backups via SMB
- 30. Data virtualization: fromon-premises into the Cloud Source DB server SQL Server 2008-2016 2TB storage Delphix Virtualization Engine 2 TB storage Target DB server SQL Server 2008-2016 No database storage
- 31. Optimized forthe cloud in the first place…not after! Different cost structures Much smaller storage footprint, much less data-transfer How Does Data Virtualization Enhance this?
- 32. Traditional copydata management techniques Developed without concerns about infrastructure chargeback This corresponds to higher cost. IaaS vendors monitor storage and data transfers Help meet SLAs, garner profits It’s not just the data that exists in the end, so transformations can equal big money for cloud vendors. Different cost structures
- 33. Know Thy Enemy… •Tune SQL and Apps to perform efficiently as possible- natural life of database is growth, (in processes, resources, etc.) before migrating. • The less network latency, the better- network tracing to eliminate database blame is important. • Many of the same tools and data provides value- DMVs provide data internally to SQL Server. • Look at management tools such as Cloudmonix, (formerly AzureWatch) AppDynamics, Dynatrace, Zabbix or Logic Monitor.
- 34. For Non-ProductionSystems… Change the way you’ve always performed tasks. Performing common tasks the same way as previously might end up costing more. Secure Data All IaaS alternatives promote encryption for data in- flight and for data at-rest, but encryption may not be the right answer… Secondary Considerations
- 35. All IaaSsolutions provide encryption in-flight and encryption at-rest But encryption doesn’t protect data as much as it needs to be . Europe already requires data masking, not just data encryption for any confindential data, (GDPR): http://ec.europa.eu/justice/data-protection/article- 29/documentation/opinion- recommendation/files/2014/wp216_en.pdf Confidential data
- 36. Encryption isreversible data obfuscation, which is very different from masking data. • Data masking is non-reversible. It solves the issue at the data level. Is authentication and authorization in non-production in compliance with security goals? All organizations will soon need to review if critical data in non- production environments be accessible to developers, testers and users. Confidential data
- 37. Masking personally-identifiable,(PII, HIPPA, PCI, etc.) information renders it useless from a security standpoint Resolves both the technical and personal responsibility issue. The data can be masked before it moves to non-production, removing unnecessary risk. Why Masking is Part of the Answer
- 38.
- 39.
- 40. Data virtualization: transformationby masking SQL Server Validated Sync Environment Delphix Masking Engine Delphix Virtualization Engine 2TB storage SQL Server Target No database storage
- 41. Data virtualization: Maskingand then to the Cloud Delphix Virtualization Engine 2TB storage Delphix Masking Engine Delphix Virtualization Engine 2 TB storage SQL Server Target Storage Only for Masked Data SQL Server Validated Sync Environment SQL Server Target Storage Only for Masked Data
- 42. Replication ensuresthat ongoing data refreshes aren’t costly and are simple For data protection, for scalability, and for migration Virtualized data is already compressed at-rest, further compressed and encrypted in-flight, incrementally replicated to another engine Selective Replications to VDBs
- 43. Once datahas been virtualized on-premises… It can be used for simplifying patching and upgrading on- prem or in preparation for transforming to Azure SQL Database Securely mask data, which is essential in today’s cloud world Then... securely virtualized data can be safely replicated to IaaS Data Virtualization + Upgrades = $$
- 44. On-Prem, Masked, thenReplicated to IaaS Delphix Masking Engine Delphix Virtualization Engine 4 TB storage Delphix Virtualization Engine 4 TB storage SQL Server Validated Sync Environment SQL Server Target Storage Only for Masked Data SQL Server Target Storage Only for Masked Data SQL Server Target Storage Only for Masked Data
- 45. • Watch forChargebacks Data virtualization minimizes storage and data transfer and transformation costs. • Confidential Data Data masking integrated with data virtualization truly secures PCI, PII, HIPPA and other confidential data. • Optimize and tune BEFORE and AFTER migrating to the cloud. Summary: Solutions for Cloud Migrations using Virtualization
- 46. Want to tryit out, download the Delphix AWS Trial- Available for Oracle, MSSQL Soon! https://www.delphix.com/products/free-trial-request Twittter: @DBAKevlar Linked in: http://linkedin.com/in/kellynpotvin Blog: http://dbakevlar.com
Editor's Notes
- #4 We all are quite familiar with Virtualization in the form of hosts or servers, but there are a number of areas we hear less about- virtualized networks and an area I specialize in- virtualized databases, apps and flatfiles. They may have a similar term identified with them, but the technology behind each offering and each vendor can be slightly to very different.
- #5 Microsoft ( MSFT ) announced its earnings for Q1 FY17 on October 20th. (Fiscal years end with June.) The company posted a 3% year-over-year growth (5% in constant currency) in revenues to $20.453 billion. In our pre-earnings note , we noted that cloud services would boost revenues across productivity and business processes and intelligent cloud divisions. Commercial cloud annualized revenue run rate exceeded $13 billion, and the company is on course to achieve $20 billion annual run rate by fiscal year 2018. As a result, the stock was trading at all-time high on Thursday (after market hours) Read more: http://www.nasdaq.com/article/microsoft-earnings-cloud-adoption-takes-center-stage-as-revenue-improve-slightly-cm697401#ixzz4afv45v4t
- #6 OK, who’s heard these words: Disruptions, catalyst, epic change, visionary, paradigm shift, machine learning, monetization and pivot Hybrid CLOUD and CLOUD
- #7 Over 60% of companies stated they are on 2-6 clouds, between databases, apps and files. Your clouds will become very much like your smart phone provider.
- #8 Amazon and Azure own this race, but a price war could shake up the future of which cloud our data resides. Almost 70% of the market is owned by Amazon and Microsoft During Amazon S3 outage, it wasn’t Azure or Oracle that benefited, but Google, which increased its business over 12% in just two days.
- #10 Yeah, those mundane buzzwords of what you did 5-10 years ago is now going away…what you did just 2 years ago is going to transition. You may be a “cloud administrator” or a database architect without a single database platform and on multiple cloud platforms. More likely multi-platform and multi-cloud.
- #11 We’re continually pushed that it once was about savings, but no longer. Where RDS was there to save the day, we now know that price per gb, it often comes out to the same or more than it once was on-prem. When license costs are increased and no limit on usage, budgets can be stretched.
- #13 Bulk copy protocol or other archaic processes
- #14 Amazon Pricing : https://aws.amazon.com/ec2/pricing/ Sep 2016
- #15 Do any of you see the problem with the high level project steps? We commonly leave optimizing the environment until after we’ve migrated to the cloud.
- #16 Standard backup and recovery methods Replication Cloning Continual feed to keep up to date or refresh on regular basis, via archaic tools- bcp, log shipping or paid replication tools.
- #19 And if you choose wrong or use more resources than expected, you can experience severe performance issues.
- #20 This is for Azure migrations- the requirements
- #21 Create this report- what can be migrated and what can’t? 90% of environments won’t fulfill the requirements and won’t migrate with the cloud migration asst.
- #22 Once final tests are done- you are testing. Perform final migration, final sync to prod and downtime to switch from on-prem to cloud.
- #24 80-90% storage savings from traditional migration methods. Data In flight can be significant cost for many cloud vendors Network is the new bottleneck. You can avoid that with less copies- one golden copy- we call it the “validated sync environment”
- #32 By going to a single source, loading to a single source and maintaining a single source, a smaller footprint is attained. Cost savings in the way of less storage required results in even bigger savings.
- #33 Different cloud manufacturers have different pricing structures- verify what you are being charged for and make sure those costs aren’t in contrast with your environment. Many avoid RDS on Amazon- we don’t support it. For our Oracle customers, too limited.
- #34 Before you start, tune SQL instaead of after. Use network tools like Nagios network analyzer or. Solarwinds Network Performance Monitor, (NPM) Your performance data can assist you in identifying huge IO, CPU and remote resource work that should be minimized beforehand.
- #35 Data in flight can cost you and data processing that was normal on-prem, may need to be redesigned post cloud migration. Inspect pricing small print carefully and know you’re final decision on *what* choice in cloud and type of service will determine. Encryption is important for production. SQL 2016 dynamic data masking isn’t production ready- three steps and I had ‘un-masked data!
- #37 Or does it shift the problem toward authentication and authorization?
- #46 DMVs are your friend. Embrace them and know each databases pain points and how they will effect chargebacks once the database, application and wprkload is migrated to the cloud. What may not have been an issue while on-prem, may become an expensive cost if overlooked post-project.