Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Denver SQL Saturday The Next Frontier

152 views

Published on

Newly updated slides on the Cloud and Virtualization!

Published in: Technology
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Denver SQL Saturday The Next Frontier

  1. 1. The Next Frontier Opportunities With the Cloud and Virtualization Kellyn Pot’Vin-Gorman
  2. 2. #SQLSatATL Kellyn Pot’Vin-Gorman Technical Intelligence Manager for the Office of CTO, Delphix • Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase, PostgreSQL…..) • Oracle ACE Director, (Alumni), Oak Table • APEX Women in Technology Award, CTA • STEM education with Raspberry Pi and Python • Liaison for Denver SQL Server User Group • Rocky Mountain Oracle Training Days Conference Director and Board Director • Author, blogger, (http://dbakevlar.com)
  3. 3. #SQLSatATL  Copy Data Management: Virtualized sanity for the DBA Realist.  Provisioning: Patching, refreshing and if you ask me one more time!  Cloud: Cloudy with a chance of failures.  Security: Yo Developer- Is that the SA password taped to your monitor??
  4. 4. The Life of a DBA  Provision Databases  Refresh and provide data to reporting, testing and development  Secure database environments  Optimize data access  Collaborate to solve business challenges
  5. 5. #SQLSatATL What is Copy Data Management, (CDM) The management of all non-production databases.  Broad Term- Physical and virtual clones  Managed or unmanaged  Command line or User Interface, (or both)  Administrative, Infrastructure, security
  6. 6. #SQLSatATL  Storage costs  Thin-provisioning storage avoidance  Data transfer costs  Far less data transferred during provisioning/refresh operations  As opposed to the volume of data transferred using traditional cloning techniques  Simplifies Provisioning vs. archaic processes to copy data Why Use Virtualization for CDM?
  7. 7. #SQLSatATL 7 ▶▶▶ Virtualize and Deployed▶ ▶ ▶ Copies: 90%+ Repeated Data Storage Pool for Delphix QA DEV PATCH TEST PRODUCTION Database/App Tier 1 TB 1 TB 0.6 TB Read From Production Spin a VIRTUAL database up a patch test, without having to remove a current development or test one. TEST
  8. 8. #SQLSatATL Example Environment- 1TB
  9. 9. #SQLSatATL Data Virtualization: From Prod to Virtual Validated Sync Environment Source Environment SCSI/SSL
  10. 10. #SQLSatATL Data Virtualization: From Prod to Virtual Validated Sync Environment Source Environment SCSI/SSL Any Storage Create as many VDBs as needed!
  11. 11. #SQLSatATL Data Virtualization: Space Savings Delphix Virtualization EngineValidated Sync Target Dozen Virtual Database Copies, (VDBs)
  12. 12. #SQLSatATL Data Virtualization: How is it possible? Delphix Virtualization Engine Any Storage Virtual Database
  13. 13. #SQLSatATL • Using any storage and only fraction of space • Syncs with native or third-party SQL Server backups • Can maintain two weeks of data changes • Managed just like any SQL Server database • Users can instantly provision a read/write virtual copy of a database • Can be used for replication, mirroring, and change data capture (CDC). This is Data Version Control
  14. 14. #SQLSatATL Administration Ease
  15. 15. #SQLSatATL Bookmarking and Branching Release 1.2 Release 1.3 Branch 1.3 Branch 1.1
  16. 16. #SQLSatATL 16 Spin up a new VIRTUAL DB and SQL bin Files Night Time ETL/Maintenance Challenges Finance Dev Test QA ETL BI Reporting DBCC’s Scripted out to spin up new VDBs to run nightly jobs, maintenance, etc.
  17. 17. #SQLSatATL 17 Epiphany e·piph·a·ny əˈpifənē/ noun a (1) : a usually sudden manifestation or perception of the essential nature or meaning of something (2) : an intuitive grasp of reality through something (as an event) usually simple and striking (3) : an illuminating discovery, realization, or disclosure b : a revealing scene or moment
  18. 18. #SQLSatATL 18 Patching- Got the T-shirt
  19. 19. #SQLSatATL 19 How Do We Build Out? How Long? Finance HR CRM
  20. 20. #SQLSatATL 20
  21. 21. #SQLSatATL 21 Patching and Upgrading Databases  Each patch, would need to be applied to a development database, requiring outage to development teams and then tested before applying to test, UAT and then onto production.  This has to be performed to EACH environment, every SQL Server, each quarter. Finance HR CRM UAT Test DevProd
  22. 22. #SQLSatATL 22  Downtime for valuable resources.  DBAs working afterhours  Each database must have it done and…  The tedious task must be performed over and over again.  Little opportunity for advanced learning.  Each database may experience different bugs. Risks/Challenges Of This Approach
  23. 23. #SQLSatATL 23 Spin up a new VIRTUAL DB and SQL bin files and apply patch to it. Environment Virtualization, DB Style CRM Finance HR
  24. 24. #SQLSatATL 24 No need to keep the extra VDBs post patch to prod. After Testing, Apply to Production CRM Finance HR
  25. 25. #SQLSatATL 25 The Compressed Copies in the Delphix Engine are Upgraded! Environment Virtualization, DB Style HR Finance CRM
  26. 26. #SQLSatATL 26 In minutes, refresh and complete! Environment Virtualization, DB Style CRM Finance HR
  27. 27. #SQLSatATL 27
  28. 28. #SQLSatATL 28 • I didn’t have to take away a valuable resource’s database environment to test the patches. • I didn’t have to apply the patches to subsequent environments, as they are virtualized copies of the source, simply requiring a refresh from production, post final patch. • I save significant time that commonly has to be allocated to quarterly and annual maintenance for patching. • I apply the patch twice- once to test, once to production. I only need to refresh my environments after I’m done. • For releases, this can be “containerized”, simplifying release and if required, rollback. Patching and Upgrading with Virtualization
  29. 29. #SQLSatATL 29 The quickest way to complete a task is not having to do it.
  30. 30. #SQLSatATL 30 The CLI, (Command Line Interface) for CDM should be robust and able to incorporate into DevOps What if I’m Retro- Command Line? • The name of the VDB you want to create • The group in which to create the VDB • The Oracle database name • The Oracle database unique name • The Oracle database instance number • The Oracle database instance name • The source dSource or VDB from which you wish to provision • The SCN or timestamp of the point you want to provision from, (commands can be run to get the list of snapshots or timeflow ranges.)
  31. 31. #SQLSatATL 31 https://docs.delphix.com/display/DOCS43/CLI+Cookbook%3A+Provisioning+a+VDB+from+a+TimeFlow+Bookmark Choose Source Snapshot: >snapshot list database=<dbname> timeflow ”<dbname>" timeflowRanges; commit; Create A Virtual Database: >database provision; delphix database provision > defaults delphix database provision defaults > set container=<Vdbname> delphix database provision defaults > commit;
  32. 32. #SQLSatATL 32 What a CLI Scripted Refresh Looks Like > database > select <VDB name> > refresh > set timeflowPointParameters.type= (one of TimeflowPointBookmark, TimeflowPointBookmarkTag, TimeflowPointLocation, TimeflowPointSemantic, TimeflowPointTimestamp as appropriate) > set timeflowPointParameters.location= (the location, timestamp, or bookmark you wish to refresh to) > set timeflowPointParameters.timeflow= (the timeflow associated with location) > commit This can all be called via a Powershell script…
  33. 33. #SQLSatATL 33
  34. 34. #SQLSatATL 34 • Monstrous deployments, (20+) benefit from a scripted, CLI option. • Single or several VDB deployment. • When investigating deep level issues, using the CLI can make details easier to search through. • When looking for “just the facts”, then a graphical UI is beneficial for high level error messages. • Self-service has great benefits and make it simple for those without deep level skills to take advantage of virtualized environments for development and testing. • The CLI helps with DevOps automation and orchestration. When Do I USE the CLI or Graphical UI?
  35. 35. #SQLSatATL ProjectTimeline Optimize Migrate Initial Testing Requirements Cloud Projects
  36. 36. #SQLSatATL Just copy data and applications into the cloud… • Straightforward approach • Inefficient, non-incremental for large environments • Open-source “bcp” uses encryption, compresses, and multi-threads • Archaic processes recommended by vendors Start with backups to IaaS storage, then populate re-hosted applications by restoring from those backups • Cloud backups are easy, known technology How Are Companies Migrating to the Cloud?
  37. 37. #SQLSatATL Cost Estimates for Azure https://azure.microsoft.com/en- us/pricing/details/storage/blobs/ Storage Capacity LRS First 1 TB / Month $0.024 per GB Next 49 TB (1 to 50 TB) / Month $0.0236 per GB Next 450 TB (50 to 500 TB) / Month $0.0232 per GB Next 500 TB (500 to 1,000 TB) / Month $0.0228 per GB ZRS $0.03 per GB $0.0295 per GB $0.029 per GB $0.0285 per GB
  38. 38. #SQLSatATL Backups cloud service: from on- premises to the Cloud Database server Cloud Storage
  39. 39. #SQLSatATL Migration Complete…Not so Fast… • What if you only want dev and test in the cloud. • Data is migrated, but this doesn’t count for ongoing data loads, application connectivity across the network. • To refresh will take considerable time to perform with traditional tools or cloning methods. • Rarely a consideration for the difference in cost structure for processing large amounts of data from on-premise to the cloud.
  40. 40. #SQLSatATL https://blogs.msdn.microsoft.com/brunoterkaly/2013/09/26/how-to-migrate-an-on-premises-sql-server- 2012-database-to-windows-azure-sql-database/ What’s the right configuration??
  41. 41. #SQLSatATL Cloud Performance
  42. 42. #SQLSatATL https://docs.microsoft.com/en-us/azure/sql-database/sql-database-cloud-migrate • Database must be MSSQL 2005 or higher, (easy) • Ensure that the database is compatible with Azure SQL DB, (correct any incompatible functions, etc.) • Must have identified all performance issues that will be impacted beforehand. • Ensure there is as little physical distance between the cloud data center and bacpac files to be used for migration. • Disable management jobs that will hinder migration processing. • Drop any objects or historical data that can impact migration time and can be performed post migration.
  43. 43. #SQLSatATL • Performs assessment of IIS environment and creates report of what can be migrated. • Creates report to document what will require moving and any issues identified. • Creates any websites and associated databases in preparation for the migration. • If non-compatible products are in use, (which is in case of 90% of environments) then those must be addressed manually. • The tool is free to start, but will cost as project proceeds in duration. https://azure.microsoft.com/en-us/downloads/migration- assistant/?&wt.mc_id=AID559320_SEM_14UHplSj&gclid=CJWky8vB6tICFdG2wAod0SEJqQ
  44. 44. #SQLSatATL From on-premises to the Cloud, then restore- Assumption Database server Cloud Storage Database server
  45. 45. #SQLSatATL Much Improved Option- 1. Data virtualization Easy, secure, revolutionary Optimized for faster agile development and testing Optimized for new costing structures in IaaS With a software appliance, no concerns about hardware or software outside of virtualization lock-in. Migrating from on-prem to IaaS with Virtualization
  46. 46. #SQLSatATL Data Virtualization: on-premise Environment Source DB server SQL Server 2008-2016 1 TB Delphix Virtualization Engine 2 TB storage Target DB server SQL Server 1008-2016 No database storage SCCI/SSL Backups via SMB
  47. 47. #SQLSatATL Data virtualization: from on-premises into the Cloud Source DB server SQL Server 2008-2016 2TB storage Delphix Virtualization Engine 2 TB storage Target DB server SQL Server 2008-2016 No database storage
  48. 48. #SQLSatATL Optimized for the cloud in the first place…not after! Different cost structures Much smaller storage footprint, much less data-transfer How Does Data Virtualization Enhance this?
  49. 49. #SQLSatATL Traditional copy data management techniques Developed without concerns about infrastructure chargeback This corresponds to higher cost. IaaS vendors monitor storage and data transfers Help meet SLAs, garner profits It’s not just the data that exists in the end, so transformations can equal big money for cloud vendors. Different cost structures
  50. 50. #SQLSatATL Know Thy Enemy… • Tune SQL and Apps to perform efficiently as possible- natural life of database is growth, (in processes, resources, etc.) before migrating. • The less network latency, the better- network tracing to eliminate database blame is important. • Many of the same tools and data provides value- DMVs provide data internally to SQL Server. • Look at management tools such as Cloudmonix, (formerly AzureWatch) AppDynamics, Dynatrace, Zabbix or Logic Monitor.
  51. 51. #SQLSatATL For Non-Production Systems… Change the way you’ve always performed tasks. Performing common tasks the same way as previously might end up costing more. Secure Data All IaaS alternatives promote encryption for data in-flight and for data at-rest, but encryption may not be the right answer… Secondary Considerations
  52. 52. #SQLSatATL All IaaS solutions provide encryption in-flight and encryption at-rest But encryption doesn’t protect data as much as it needs to be . Europe already requires data masking, not just data encryption for any confindential data, (GDPR): http://ec.europa.eu/justice/data-protection/article- 29/documentation/opinion- recommendation/files/2014/wp216_en.pdf Confidential data
  53. 53. #SQLSatATL Encryption is reversible data obfuscation, which is very different from masking data. • Data masking is non-reversible. It solves the issue at the data level. Is authentication and authorization in non-production in compliance with security goals? All organizations will soon need to review if critical data in non- production environments be accessible to developers, testers and users. Confidential data
  54. 54. #SQLSatATL Masking personally-identifiable, (PII, HIPPA, PCI, etc.) information renders it useless from a security standpoint Resolves both the technical and personal responsibility issue. The data can be masked before it moves to non-production, removing unnecessary risk. Why Masking is Part of the Answer
  55. 55. #SQLSatATL Confidential data Production Non-Production Exposure
  56. 56. #SQLSatATL Data virtualization: transformation by masking SQL Server Validated Sync Environment Delphix Masking Engine Delphix Virtualization Engine 2TB storage SQL Server Target No database storage
  57. 57. #SQLSatATL Data virtualization: Masking and then to the Cloud Delphix Virtualization Engine 2 TB storage SQL Server Target Storage Only for Masked Data Delphix Virtualization Engine 2TB storage Delphix Masking Engine SQL Server Validated Sync Environment SQL Server Target Storage Only for Masked Data
  58. 58. #SQLSatATL Confidential data Production Non- Production Exposure Encryption Masking Solution
  59. 59. #SQLSatATL On-Prem, Masked, then Replicated to IaaS Delphix Virtualization Engine 4 TB storage SQL Server Validated Sync Environment SQL Server Target Storage Only for Masked Data SQL Server Target Storage Only for Masked Data Delphix Virtualization Engine 4 TB storage SQL Server Target Storage Only for Masked Data Delphix Masking Engine
  60. 60. #SQLSatATL Virtualization Makes Copy Data Management Simple Security with masking and encryption is best Cloud migrations are more successful when virtualized and planned accordingly. Patching and Maintenance can be done with less effort and resources.
  61. 61. #SQLSatATL Want to try it out, download the Delphix Azure Trial! https://www.delphix.com/products/free-trial- request Twittter: @DBAKevlar Linked in: http://linkedin.com/in/kellynpotvin Blog: http://dbakevlar.com
  62. 62. #SQLSatATL 62 Delphix with SQL Server- the Basics https://docs.delphix.com/docs/delphix-administration/sql-server-environments-and-data- sources/managing-sql-server-environments/overview-of-setting-up-sql-server- environments Delphix Upgrade Workflow: https://community.delphix.com/delphix/topics/tip-of-the-day- upgrading-a-sql-server-dsource Upgrading the Dsource after an Upgrade: https://docs.delphix.com/docs/delphix- administration/sql-server-environments-and-data-sources/virtualizing-databases-using- delphix-with-sql-server/managing-sql-server-dsources/additional-dsource- topics/upgrading-a-dsource-after-a-sql-server-upgrade Delphix in the Cloud https://www.delphix.com/solutions/cloud-migration-virtual-data References and Tips
  63. 63. #SQLSatATL

×