Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cloudy with a Chance of Databases

263 views

Published on

ECO 2017 Presentation on the Cloud and Database roadmap

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Cloudy with a Chance of Databases

  1. 1. © 2017 Delphix. All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Kellyn Pot’Vin-Gorman | Technical Intelligence Manager| November, 2017 Cloudy with a Chance of Databases
  2. 2. © 2017 Delphix. All Rights Reserved. Private and Confidential. Kellyn Pot’Vin-Gorman Technical Intelligence Manager, Delphix • Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase, Postgres…..) • Oracle ACE Director, (Alumni) • Oak Table Network • Microsoft Idera ACE • APEX Women in Technology Award, CTA 2014 • STEM education with Raspberry Pi and Python • President, Rocky Mtn Oracle User Group • Liaison, Denver SQL Server User Group • DevOps author, instructor and presenter. • Author, blogger, (http://dbakevlar.com)
  3. 3. © 2017 Delphix. All Rights Reserved. Private and Confidential.
  4. 4. © 2017 Delphix. All Rights Reserved. Private and Confidential.
  5. 5. © 2017 Delphix. All Rights Reserved. Private and Confidential. Cloud Trends • 85% of Enterprises have a multi-cloud strategy • 77% are hybrid cloud, (different than the 2017 cloud survey) • Workloads are being run in the cloud- • 41% in public clouds • 38% in private clouds • Enterprise Companies are choosing cloud • 65% want public cloud • 63% want private cloud solutions • 93% will be hybrid • State of the Cloud Survey, RightScale
  6. 6. © 2017 Delphix. All Rights Reserved. Private and Confidential. 6© 2016 Delphix Corporation
  7. 7. © 2017 Delphix. All Rights Reserved. Private and Confidential. Source: Right Scale State of the Cloud 2016 0% 10% 20% 30% 40% 50% 60% Physical Machine Virtual Machine Private Cloud Public Cloud 2017 2018 2019 Enterprise Company Cloud Migrations, 2017-2019
  8. 8. © 2017 Delphix. All Rights Reserved. Private and Confidential. 2017 Cloud Focus 0% 10% 20% 30% 40% 50% 60% Private Clouds Public Clouds Hybrid Clouds Cloud Investment
  9. 9. © 2017 Delphix. All Rights Reserved. Private and Confidential. Cloud Adoption and How It’s Changing 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Public Cloud Private Cloud Hybrid Cloud Any Cloud 2016 2017 2018 http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2017-state-cloud-survey
  10. 10. © 2017 Delphix. All Rights Reserved. Private and Confidential. Who’s Who  Azure’s market penetration, (Up from 20% to 34%)  AWS has remained consistent, (data before S3 outage)  Google was up 10-15%  Oracle is making headway in the SaaS arena, (Software as a Service) and with Oracle Cloud Infrastructure, (Bare Metal)  IBM? Archaic cloud of the Future?
  11. 11. © 2017 Delphix. All Rights Reserved. Private and Confidential. Cloud by Provider Cloud LTD, 2017 Trivia: Which Cloud Vendor benefitted most from the Amazon S3 outage? AWS Azure IBM Google Oracle Other
  12. 12. © 2017 Delphix. All Rights Reserved. Private and Confidential. The Cloud  AWS is now a $14 billion business - THIS YEAR.  Not just Oracle’s investment, but Microsoft, IBM and Google are all opening up new data centers are impressive rates.  Consider how much of the cloud is outside of our focus area, (desktop, applications and mobile)
  13. 13. © 2017 Delphix. All Rights Reserved. Private and Confidential. The Limitations?  Questions on Security  Network woes  Data Gravity  Which Cloud and…  Complex Cloud Management
  14. 14. © 2017 Delphix. All Rights Reserved. Private and Confidential.
  15. 15. © 2017 Delphix. All Rights Reserved. Private and Confidential. History of Databases
  16. 16. © 2017 Delphix. All Rights Reserved. Private and Confidential. Percentage of Database Types in 2017
  17. 17. © 2017 Delphix. All Rights Reserved. Private and Confidential. When you consider… An estimate of a 10% increase in data accessibility will result in more than $65 million in additional net income, you can see why data gravity is a big deal.
  18. 18. © 2017 Delphix. All Rights Reserved. Private and Confidential. How much data are we talking about? • More data has been created in just the last two years than the previous history of humanity • 1.7MB of new information created every second per human on the planet. • Introduction of big data often has same development pain points.
  19. 19. © 2017 Delphix. All Rights Reserved. Private and Confidential. Database Trend Impact
  20. 20. © 2017 Delphix. All Rights Reserved. Private and Confidential. Biggest Gains?
  21. 21. © 2017 Delphix. All Rights Reserved. Private and Confidential. What Has Lead to PostgreSQL Latest Success? • Lower cost of ownership • Advanced feature additions in PostgreSQL 10 • AmazonAurora support with PostgreSQL
  22. 22. © 2017 Delphix. All Rights Reserved. Private and Confidential. Oracle May Be Losing Ground, but It Has a Lot of Ground to Begin With…
  23. 23. © 2017 Delphix. All Rights Reserved. Private and Confidential.
  24. 24. © 2017 Delphix. All Rights Reserved. Private and Confidential. Savings!! • Stop with the savings conversation. • Become Educated on Tools that can Move the Education Needle • Learn Encryption and Masking • Become more fluent in both platform and DevOps So How, as DBAs, Do We Make it Easier?
  25. 25. © 2017 Delphix. All Rights Reserved. Private and Confidential. Sticker Shock  Estimates on Wasted Cloud Expenditures  30% of cloud is wasted – Perform Audits  When measured, it was closer to 45%  Best Tool to Impact Waste?  Containerize  Virtualize  Cloud vendors profit from vendor lock-in.
  26. 26. © 2017 Delphix. All Rights Reserved. Private and Confidential. How Are Companies Migrating to the Cloud? 1. Just copy data and applications into the cloud… • Straightforward approach • Inefficient, non-incremental for large environments • Insecure, physical file transfers • A truck, (yes, I’m looking at you, “snowmachine”) • Archaic processes recommended by vendors 2. Start with backups to IaaS storage, then populate re-hosted applications by restoring from those backups • Cloud backups are easy, known technology
  27. 27. © 2017 Delphix. All Rights Reserved. Private and Confidential. Standard Cloud Database Migration Database server Cloud Storage Database server
  28. 28. © 2017 Delphix. All Rights Reserved. Private and Confidential. Migration Complete…Not so Much… • Even if only development or test has been migrated to the cloud… • Data is migrated, but this doesn’t count for ongoing data loads, application connectivity across the network. • To refresh will take considerable time to perform. • Rarely a consideration for the difference in cost structure for processing large amounts of data from on- premise to the cloud.
  29. 29. © 2017 Delphix. All Rights Reserved. Private and Confidential. Network, the New Bottleneck • Few tools identify network latency in terms of percentage or cause. • Explore potential latency between cloud services and data transfer conditions on contracts. • Negotiate requirements before the agreement is signed and data is handed over. • Many organization are investing in a dedicated WAN for cloud environments and operations.
  30. 30. © 2017 Delphix. All Rights Reserved. Private and Confidential. AWS Demonstrates
  31. 31. © 2017 Delphix. All Rights Reserved. Private and Confidential. Jeremiah Wilton, https://www.slideshare.net/AmazonWebServices/dat205-new-launch-introduction-to-aws-database- migration-service Amazon Web Services
  32. 32. © 2017 Delphix. All Rights Reserved. Private and Confidential. • https://www.slideshare.net/AmazonWebServices/dat205-new-launch-introduction-to-aws-database-migration-service Cross Platform Migration from AWS
  33. 33. © 2017 Delphix. All Rights Reserved. Private and Confidential. • https://www.slideshare.net/AmazonWebServices/dat205-new-launch-introduction-to-aws-database-migration-service • Replicate data in on-premises databases to AWS • Replicate OLTP data to Amazon Redshift • Integrate tables from third-party software into your reporting or core OLTP systems • Hybrid cloud is a stepping stone in migration to AWS Amazon Cloud and AWS Choices
  34. 34. © 2017 Delphix. All Rights Reserved. Private and Confidential. • T2 pricing starts at $0.018 per Hour for T2.micro • C4 pricing starts at $0.154 per Hour for C4.large • 50GB GP2 storage included with T2 instances • 100GB GP2 storage included with C4 instances • Data transfer inbound and within AZ is free • Data transfer across AZs starts at $0.01 per GB https://aws.amazon.com/ec2/pricing/ AWS Pricing
  35. 35. © 2017 Delphix. All Rights Reserved. Private and Confidential. Azure SQL Database Migration
  36. 36. © 2017 Delphix. All Rights Reserved. Private and Confidential. Azure Pricing
  37. 37. © 2017 Delphix. All Rights Reserved. Private and Confidential. Oracle Cloud Pricing
  38. 38. © 2017 Delphix. All Rights Reserved. Private and Confidential. There’s a Reason for these…
  39. 39. © 2017 Delphix. All Rights Reserved. Private and Confidential. ProjectTimeline Optimize Migrate Initial Testing Requirements Cloud Projects
  40. 40. © 2017 Delphix. All Rights Reserved. Private and Confidential. What’s Missing? Over 90% of cloud projects don’t perform a preliminary identification of latency or data issues that will impact the project. No strategy to identify or optimize BEFORE migrating to the cloud.
  41. 41. © 2017 Delphix. All Rights Reserved. Private and Confidential. Secondary Considerations • For Non-Production Systems… • Change the way you’ve always performed tasks. • Performing common tasks the same way as previously might end up costing more. • Secure Data • All IaaS alternatives promote encryption for data in- flight and for data at-rest, but encryption may not be the right answer…
  42. 42. © 2017 Delphix. All Rights Reserved. Private and Confidential. Why is there a Sticker Shock with the Cloud? • If you don’t optimize for the cloud before you migrate.. Different cost structures will come back to bite you- • Much smaller storage footprint • Update data-transfers between systems/applications • Virtualize environments that have high percentage of duplicate data
  43. 43. © 2017 Delphix. All Rights Reserved. Private and Confidential. Cost Structures? • Traditional copy data management techniques • Developed without concerns about infrastructure chargeback • Nightly batchload ETLs • Traditional data pulls • Inefficient applications with thick database design • IaaS vendors monitor storage and data transfers • Help meet SLAs, garner profits • It’s not just the data that exists in the end, so transformations can equal big money for cloud vendors.
  44. 44. © 2017 Delphix. All Rights Reserved. Private and Confidential. Why Less Focus on Thin Provisioning Storage costs Thin-provisioning storage avoidance = $$$ Data transfer costs Far less data transferred during provisioning/refresh operations • As opposed to the volume of data transferred using traditional cloning techniques Network-attached storage data transfers offset somewhat • Most non-production usage modifies or accesses only a tiny proportion of the total database during lifetime
  45. 45. © 2017 Delphix. All Rights Reserved. Private and Confidential. Cloud and Performance KNOW THY ENEMY… • Tune SQL and Apps to perform efficiently as possible- natural life of database is growth, (in processes, resources, etc.) before migrating. • The less network latency, the better- network tracing to eliminate database blame is important. • Many of the same tools and data provides value- The AWR, (Automatic Workload Repository) is your friend! • Take advantage of cloud monitoring tools that can ease the learning curve as you migrate to the cloud.
  46. 46. © 2017 Delphix. All Rights Reserved. Private and Confidential. • https://www.channele2e.com/2016/01/21/top-50-cloud-monitoring-and-management-tools-which-are-best/ 1. Amazon CloudWatch 2. AppDynamics 3. Cloudmonix 4. Datadog 5. Dynatrace 6. Idera 7. Microsoft Cloud Monitoring 8. Nagios 9. New Relic 10.Solarwinds The Best Cloud Monitoring and Management Tools, (Alphabetically)
  47. 47. © 2017 Delphix. All Rights Reserved. Private and Confidential. Amazon Cloudwatch • This is a monitoring service for AWS cloud Collect and track metrics • Collect and monitor log files • Set alarms, and automatically react to changes
  48. 48. © 2017 Delphix. All Rights Reserved. Private and Confidential. AppDynamics https://docs.appdynamics.com/display/PRO14S/Architecture
  49. 49. © 2017 Delphix. All Rights Reserved. Private and Confidential. Cloudmonix, (formerly AzureWatch) • Supports Azure • Supports both Windows and secondary Linux with Azure • Agent is required on all hosts.
  50. 50. © 2017 Delphix. All Rights Reserved. Private and Confidential. Datadog • Strong DevOps Deployment • Docker, Jenkins compatible • Support AWS, Google, IBM • GUI can be overwhelming
  51. 51. © 2017 Delphix. All Rights Reserved. Private and Confidential. Dynatrace • Manages both Linux and Windows hosts • AWS, Azure, Cloud Foundry, OpenStack, OpenShift and more. • Hybrid management console
  52. 52. © 2017 Delphix. All Rights Reserved. Private and Confidential. • Designed for service providers, database professionals and IT professionals • Monitor and manage data center environments. • Acquired CopperEgg platform — now known as Uptime Cloud Monitor. Idera
  53. 53. © 2017 Delphix. All Rights Reserved. Private and Confidential. Microsoft Cloud Monitoring • Microsoft Centric cloud monitoring tools. • Migration assistant and IIS assistant
  54. 54. © 2017 Delphix. All Rights Reserved. Private and Confidential. Nagios • Has a network analyzer! • Cloud support for Amazon, EC2/S3 • Offers log and application performance monitoring • Still has open source features
  55. 55. © 2017 Delphix. All Rights Reserved. Private and Confidential. New Relic • Multi-tenant sourced • Monitoring • AWS, Azure and Google • Requires agent installation
  56. 56. © 2017 Delphix. All Rights Reserved. Private and Confidential. Solarwinds
  57. 57. © 2017 Delphix. All Rights Reserved. Private and Confidential.
  58. 58. © 2017 Delphix. All Rights Reserved. Private and Confidential. Confidential Data http://ec.europa.eu/justice/data-protection/article-29/documentation/opinion-recommendation/files/2014/wp216_en.pdf • GDPR, (General Data Protection Regulation States: • Must have lawful basis to post process data • Subject must have consented • Must have contract that needs data • Necessary and in order: • Compliance • Protect vital interests • In public interest • In legitimate interest of the owner/other party and doesn’t violate the freedom and rights of the subject.
  59. 59. © 2017 Delphix. All Rights Reserved. Private and Confidential. Confidential data Exposure Production Non-production
  60. 60. © 2017 Delphix. All Rights Reserved. Private and Confidential. Confidential data • Encryption is reversible data obfuscation, which is essential for production access and MUST be DONE. • Data masking is non-reversible. • It solves the issue at the data level. • Is authentication and authorization in non-production in compliance with security goals? • All organizations will soon need to review if critical data in non-production environments be accessible to developers, testers and users.
  61. 61. © 2017 Delphix. All Rights Reserved. Private and Confidential. Why Masking is Important- • Masking personally-identifiable, (PII, HIPPA, PCI, etc.) information renders it useless from a security standpoint • Resolves both the technical and personal responsibility issue. • The data can be masked before it moves to non- production, removing unnecessary risk. • As we discussed, on average, 80% of data is non- production.
  62. 62. © 2017 Delphix. All Rights Reserved. Private and Confidential. Confidential Data with Masking Exposure Production Non-production Encryption Mask Solution
  63. 63. © 2017 Delphix. All Rights Reserved. Private and Confidential. Target Complete Solutions to Ease Management and Vulnerability • Virtualize and tune on-prem • Mask data before it’s in the cloud • Smaller footprint before its in flight. • It’s transformed by securely masking on-premises... • Then... • securely virtualized data can be safely replicated to IaaS
  64. 64. © 2017 Delphix. All Rights Reserved. Private and Confidential. Summary • Know Your Cloud • Educate on more than one Database Platform • Use a cloud monitoring and heterogeneous tools to simplify management. • Confidential Data • Encrypt production data • Data masking is often integrated with products and protects PCI, PII, HIPPA and other confidential data.
  65. 65. © 2017 Delphix. All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Kellyn Pot’Vin-Gorman Technical Intelligence Manager kellyn@delphix.com http://dbakevlar.com

×