Kellyn Pot'Vin-Gorman presented on copy data management and virtualization for DBAs. She discussed how virtualization can be used to provision databases more quickly and easily for tasks like patching and testing without needing to copy large amounts of physical data. She also covered how command line interfaces can be used to automate some of these processes.
Accelerating Devops via Data Virtualization | DelphixDelphixCorp
“Accelerating DevOps Using Data Virtualization” at the Collaborate 2016 conference in Las Vegas. It discusses the inevitability of data virtualization and its many use cases.
Accelerating Devops via Data Virtualization | DelphixDelphixCorp
“Accelerating DevOps Using Data Virtualization” at the Collaborate 2016 conference in Las Vegas. It discusses the inevitability of data virtualization and its many use cases.
Big-Data-as-a-Service (BDaaS) in an enterprise environment requires meeting the often contradictory goals of (1) providing your data scientists, analysts, and data engineers with a self-service consumption model; (2) delivering agile and scalable on-demand infrastructure for the rapidly evolving ecosystem of big data frameworks and application software; while (3) ensuring enterprise-grade capabilities for isolation, security, monitoring, etc.
In this presentation at our BDaaS meetup in Santa Clara, Tom Phelan (chief architect and co-founder of BlueData) reviewed these goals and how to resolve the potential contradictions. He also discussed the infrastructure, application, user experience, security, and maintainability considerations required before selecting (or designing and building) a Big-Data-as-a-Service platform for an enterprise big data deployment.
More info on this BDaaS meetup can be found at: http://www.meetup.com/Big-Data-as-a-Service/events/233999817
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...Alex Gorbachev
Modern big data solutions often incorporate Hadoop as one of the components and require the integration of Hadoop with other components including Oracle Database. This presentation explains how Hadoop integrates with Oracle products focusing specifically on the Oracle Database products. Explore various methods and tools available to move data between Oracle Database and Hadoop, how to transparently access data in Hadoop from Oracle Database, and review how other products, such as Oracle Business Intelligence Enterprise Edition and Oracle Data Integrator integrate with Hadoop.
Maybe your business has outgrown its file server and you’re thinking of replacing it. Or perhaps your server is dated and not supporting your business like it should, so you’re considering moving to the cloud. It might be that you’re starting a new business and wondering if an in-house server is adequate or if you should adopt cloud technology from the start.
Regardless of why you’re debating an in-house server versus a cloud-based server, it’s a tough decision that will impact your business on a daily basis. We know there’s a lot to think about, and we’re here to help show why you should consolidate your file servers and move your data to the cloud.
In this webinar with Talon Storage Solutions, we covered:
-Challenges of using a physical file server
-Benefits of using a cloud file server
-Current State of the File Server market
-Reference Architecture examples for cloud file servers
-Demo: how to architect a cloud file server with highly-available storage
Learn more at https://www.softnas.com
DataOps in Financial Services: enable higher-quality test ing + lower levels ...Ugo Pollio
In this session, you will learn how banks and financial services all over the world are using DataOps tools to:
- Comply with GDPR with fully masked test data
- Achieve faster environment refreshes
- Shift Left with production-like test data
- Reduce infrastructure requirements
- Enabling continuous integration and continuous delivery
Selecting a SQL Server Cloud Platform - IaaS, Amazon RDS or Azure SQL DB?Christopher Foot
RDX takes a deep-dive look into the three cloud platforms we have available for SQL Server – SQL Server on IaaS, Amazon RDS for SQL Server and Azure SQL DB. We evaluate features, costs, migration utilities, toolsets, ongoing support requirements and the pros and cons of each architecture. Learn from experts who support dozens of systems on all 3 platforms!
Slides from QConSF Nov 19th, 2011 focusing this time on describing the globally distributed and scaled industrial strength Java Platform as a Service that Netflix has built and run on top of AWS and Cassandra. Parts of that platform are being released as open source - Curator, Priam and Astyanax.
Moving Windows Applications to the CloudRightScale
RightScale Webinar: March 10, 2011 – This webinar will give you insight into how Windows instances can be dynamically configured using RightScale and equip you to move your Windows applications to the cloud.
Migrating on premises workload to azure sql databasePARIKSHIT SAVJANI
Azure SQL Database is a fully managed cloud database service with built-in intelligence, elastic scale, performance, reliability, and data protection that enables enterprises and ISVs to reduce their total cost of ownership and operational cost and overheads. In this session, I will share real-world experience of successfully migrated existing SaaS application and on-premises workload for some our tier 1 customers and ISV partners to Azure SQL Database service. The session walks through planning, assessment, migration tools and best practices from the proven experiences and practices of migrating real world applications to Azure SQL Database service.
Big-Data-as-a-Service (BDaaS) in an enterprise environment requires meeting the often contradictory goals of (1) providing your data scientists, analysts, and data engineers with a self-service consumption model; (2) delivering agile and scalable on-demand infrastructure for the rapidly evolving ecosystem of big data frameworks and application software; while (3) ensuring enterprise-grade capabilities for isolation, security, monitoring, etc.
In this presentation at our BDaaS meetup in Santa Clara, Tom Phelan (chief architect and co-founder of BlueData) reviewed these goals and how to resolve the potential contradictions. He also discussed the infrastructure, application, user experience, security, and maintainability considerations required before selecting (or designing and building) a Big-Data-as-a-Service platform for an enterprise big data deployment.
More info on this BDaaS meetup can be found at: http://www.meetup.com/Big-Data-as-a-Service/events/233999817
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...Alex Gorbachev
Modern big data solutions often incorporate Hadoop as one of the components and require the integration of Hadoop with other components including Oracle Database. This presentation explains how Hadoop integrates with Oracle products focusing specifically on the Oracle Database products. Explore various methods and tools available to move data between Oracle Database and Hadoop, how to transparently access data in Hadoop from Oracle Database, and review how other products, such as Oracle Business Intelligence Enterprise Edition and Oracle Data Integrator integrate with Hadoop.
Maybe your business has outgrown its file server and you’re thinking of replacing it. Or perhaps your server is dated and not supporting your business like it should, so you’re considering moving to the cloud. It might be that you’re starting a new business and wondering if an in-house server is adequate or if you should adopt cloud technology from the start.
Regardless of why you’re debating an in-house server versus a cloud-based server, it’s a tough decision that will impact your business on a daily basis. We know there’s a lot to think about, and we’re here to help show why you should consolidate your file servers and move your data to the cloud.
In this webinar with Talon Storage Solutions, we covered:
-Challenges of using a physical file server
-Benefits of using a cloud file server
-Current State of the File Server market
-Reference Architecture examples for cloud file servers
-Demo: how to architect a cloud file server with highly-available storage
Learn more at https://www.softnas.com
DataOps in Financial Services: enable higher-quality test ing + lower levels ...Ugo Pollio
In this session, you will learn how banks and financial services all over the world are using DataOps tools to:
- Comply with GDPR with fully masked test data
- Achieve faster environment refreshes
- Shift Left with production-like test data
- Reduce infrastructure requirements
- Enabling continuous integration and continuous delivery
Selecting a SQL Server Cloud Platform - IaaS, Amazon RDS or Azure SQL DB?Christopher Foot
RDX takes a deep-dive look into the three cloud platforms we have available for SQL Server – SQL Server on IaaS, Amazon RDS for SQL Server and Azure SQL DB. We evaluate features, costs, migration utilities, toolsets, ongoing support requirements and the pros and cons of each architecture. Learn from experts who support dozens of systems on all 3 platforms!
Slides from QConSF Nov 19th, 2011 focusing this time on describing the globally distributed and scaled industrial strength Java Platform as a Service that Netflix has built and run on top of AWS and Cassandra. Parts of that platform are being released as open source - Curator, Priam and Astyanax.
Moving Windows Applications to the CloudRightScale
RightScale Webinar: March 10, 2011 – This webinar will give you insight into how Windows instances can be dynamically configured using RightScale and equip you to move your Windows applications to the cloud.
Migrating on premises workload to azure sql databasePARIKSHIT SAVJANI
Azure SQL Database is a fully managed cloud database service with built-in intelligence, elastic scale, performance, reliability, and data protection that enables enterprises and ISVs to reduce their total cost of ownership and operational cost and overheads. In this session, I will share real-world experience of successfully migrated existing SaaS application and on-premises workload for some our tier 1 customers and ISV partners to Azure SQL Database service. The session walks through planning, assessment, migration tools and best practices from the proven experiences and practices of migrating real world applications to Azure SQL Database service.
CCI2017 - Considerations for Migrating Databases to Azure - Gianluca Sartoriwalk2talk srl
In questa sessione analizzeremo le opportunità e le sfide che ci aspettano nella migrazione di database in Microsoft Azure.
Vedremo le possibili soluzioni proposte dalla piattaforma Azure e cercheremo di capire quali si adattano meglio ai differenti scenari di utilizzo, massimizzando i vantaggi e riducendo il più possibile i rischi.
Per richiedere accesso al canale contenente le registrazioni audio/video delle sessioni tecniche di Cloud Conference Italia 2017 compila il seguente form:
https://goo.gl/Fq6DQE
Microsoft Azure zmienia się. Jego częśc poświęcona bazie danych (Windows Azure SQL Database) zmienia się jeszcze szybciej. Podczas tej sesji chciałbym pokazac tym, którzy nie widzieli, oraz przypomniec tym, którzy już coś wiedzą - o co chodzi z WASD, jakie zmiany nastapiły i czego możemy po tej bazie oczekiwać. Dla odważnych będzie okazja podłączenia się do konta w chmurze i przetestowania ych rozwiązań samemu.
Blue Green Sitecore Deployments on AzureRob Habraken
The sildes of my presentation on the Sitecore User Group Jordan meetup on April 24th 2017 and the Sitecore User Group Belarus meetup on April 27th, presenting and demoing the blue green provisioning of Sitecore into Azure using Azure Web Apps. Note that these slides do not contain the demo itself. For the demo, view the recording of the presentation or read my blog post, both accessable via https://www.robhabraken.nl
Maintenance Plans for Beginners (but not only) | Each of experienced administrators used (to some extent) what is called Maintenance Plans - Plans of Conservation. During this session, I'd like to discuss what can be useful for us to provide functionality when we use them and what to look out for. Session at 200 times the forward-300, with the opening of the discussion.
VMware vCloud Automation Center, which has been renamed vRealize Automation as part of the vRealize Cloud Management Platform, automates the process of provisioning database virtual machines, and is designed by VMware to help IT without sacrificing control, security, or flexibility. Automating time-consuming processes has the potential to enable growth, improve service quality, and free IT resources for innovation and process improvement. As businesses continue to evolve and grow, DBAs and IT departments must be able to keep up with demand. Quick and easy access to self-service portals, a streamlined provisioning process that incorporates IT best practices and security policies, and fast delivery of VMs all attribute to avoiding delays and providing for increasing demands. DBAs and IT retain control of the content, upgrades, provisioning, and accessibility of the database VMs while still able to quickly provide virtualized environments to meet the needs of their business.
Practical SQL Azure: Moving into the cloudTimothy Corey
SQL Azure has been around for a few years now but you are still running all of your databases locally. You would like to use the cloud but you just aren't sure where to start. This presentation will get you started down the right path. We will start by going over the basics of SQL Azure, including what it is, how to set it up and what the benefits and drawbacks are of it. Next, we will look at how to move an existing database into SQL Azure. Finally, we will look at how to take advantage of the benefits of the cloud to make your SQL database safer and more redundant. By the end of this presentation, you should have a good understanding of where you could go with SQL Azure and how to get there.
Dans cette session nous vous présenterons les différentes manières d'utiliser SQL Server dans une infrastructure Cloud (Microsoft Azure). Seront présentés des scénarios hybrides, de migration, de backup, et d'hébergement de bases de données SQL Server en mode IaaS ou PaaS.
Spark is fast becoming a critical part of Customer Solutions on Azure. Databricks on Microsoft Azure provides a first-class experience for building and running Spark applications. The Microsoft Azure CAT team engaged with many early adopter customers helping them build their solutions on Azure Databricks.
In this session, we begin by reviewing typical workload patterns, integration with other Azure services like Azure Storage, Azure Data Lake, IoT / Event Hubs, SQL DW, PowerBI etc. Most importantly, we will share real-world tips and learnings that you can take and apply in your Data Engineering / Data Science workloads
Should I move my database to the cloud?James Serra
So you have been running on-prem SQL Server for a while now. Maybe you have taken the step to move it from bare metal to a VM, and have seen some nice benefits. Ready to see a TON more benefits? If you said “YES!”, then this is the session for you as I will go over the many benefits gained by moving your on-prem SQL Server to an Azure VM (IaaS). Then I will really blow your mind by showing you even more benefits by moving to Azure SQL Database (PaaS/DBaaS). And for those of you with a large data warehouse, I also got you covered with Azure SQL Data Warehouse. Along the way I will talk about the many hybrid approaches so you can take a gradual approve to moving to the cloud. If you are interested in cost savings, additional features, ease of use, quick scaling, improved reliability and ending the days of upgrading hardware, this is the session for you!
This are my keynote slides from SQL Saturday Oregon 2023 on AI and the Intersection of AI, Machine Learning and Economnic Challenges as a Technical Specialist
This is the second session of the learning pathway at PASS Summit 2019, which is still a stand alone session to teach you how to write proper Linux BASH scripts
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
2. #SQLSatATL
Kellyn Pot’Vin-Gorman
Technical Intelligence Manager for the Office of CTO,
Delphix
• Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase,
PostgreSQL…..)
• Oracle ACE Director, (Alumni), Oak Table
• APEX Women in Technology Award, CTA
• STEM education with Raspberry Pi and Python
• Liaison for Denver SQL Server User Group
• Rocky Mountain Oracle Training Days Conference Director
and Board Director
• Author, blogger, (http://dbakevlar.com)
3. #SQLSatATL
Copy Data Management: Virtualized sanity for the DBA
Realist.
Provisioning: Patching, refreshing and if you
ask me one more time!
Cloud: Cloudy with a chance of failures.
Security: Yo Developer- Is that the SA password
taped to your monitor??
4. #SQLSatATL
The Life of a DBA
Provision Databases
Refresh and provide data to reporting, testing and
development
Secure database environments
Optimize data access
Collaborate to solve business challenges
5. #SQLSatATL
What is Copy Data Management, (CDM)
The management of all non-production databases.
Broad Term- Physical and virtual clones
Managed or unmanaged
Command line or User Interface, (or both)
Administrative, Infrastructure, security
6. #SQLSatATL
Storage costs
Thin-provisioning storage avoidance
Data transfer costs
Far less data transferred during provisioning/refresh
operations
As opposed to the volume of data transferred using
traditional cloning techniques
Simplifies Provisioning vs. archaic processes to copy data
Why Use Virtualization for CDM?
7. #SQLSatATL 7
▶▶▶
Virtualize and Deployed▶ ▶ ▶
Copies: 90%+ Repeated Data
Storage Pool for Delphix
QA
DEV PATCH TEST
PRODUCTION
Database/App Tier
1 TB
1 TB
0.6 TB
Read From Production
Spin a VIRTUAL database up a patch test, without having to remove a current development or test one.
TEST
10. #SQLSatATL
Data Virtualization: From Prod to
Virtual
Validated Sync
Environment
Source Environment
SCSI/SSL
Any Storage
Create as many VDBs as needed!
13. #SQLSatATL
• Using any storage and only fraction of space
• Syncs with native or third-party SQL Server backups
• Can maintain two weeks of data changes
• Managed just like any SQL Server database
• Users can instantly provision a read/write virtual copy
of a database
• Can be used for replication, mirroring, and change data
capture (CDC).
This is Data Version Control
16. #SQLSatATL 16
Epiphany
e·piph·a·ny
əˈpifənē/
noun
a (1) : a usually sudden manifestation or perception of the essential nature or meaning of
something (2) : an intuitive grasp of reality through something (as an event) usually simple and
striking (3) : an illuminating discovery, realization, or disclosure
b : a revealing scene or moment
20. #SQLSatATL 20
Patching and Upgrading Databases
Each patch, would need to be applied to a
development database, requiring outage
to development teams and then tested
before applying to test, UAT and then
onto production.
This has to be performed to EACH
environment, every SQL Server, each
quarter.
Finance
HR
CRM
UAT Test DevProd
21. #SQLSatATL 21
Downtime for valuable resources.
DBAs working afterhours
Each database must have it done and…
The tedious task must be performed over and over
again.
Little opportunity for advanced learning.
Each database may experience different bugs.
Risks/Challenges Of This Approach
22. #SQLSatATL 22
Spin up a new
VIRTUAL DB and SQL
bin files and apply
patch to it.
Environment Virtualization, DB Style
CRM
Finance
HR
23. #SQLSatATL 23
No need to keep the
extra VDBs post
patch to prod.
After Testing, Apply to Production
CRM
Finance
HR
24. #SQLSatATL 24
The Compressed Copies
in the Delphix Engine
are Upgraded!
Environment Virtualization, DB Style
HR
Finance
CRM
27. #SQLSatATL 27
• I didn’t have to take away a valuable resource’s database environment to test
the patches.
• I didn’t have to apply the patches to subsequent environments, as they are
virtualized copies of the source, simply requiring a refresh from production,
post final patch.
• I save significant time that commonly has to be allocated to quarterly and
annual maintenance for patching.
• I apply the patch twice- once to test, once to production. I only need to
refresh my environments after I’m done.
• For releases, this can be “containerized”, simplifying release and if required,
rollback.
Patching and Upgrading with
Virtualization
29. #SQLSatATL 29
The CLI, (Command Line Interface) for CDM should be robust and able to incorporate into DevOps
What if I’m Retro- Command Line?
• The name of the VDB you want to create
• The group in which to create the VDB
• The Oracle database name
• The Oracle database unique name
• The Oracle database instance number
• The Oracle database instance name
• The source dSource or VDB from which you wish to provision
• The SCN or timestamp of the point you want to provision from, (commands can be run
to get the list of snapshots or timeflow ranges.)
31. #SQLSatATL 31
What a CLI Scripted Refresh Looks Like
> database
> select <VDB name>
> refresh
> set timeflowPointParameters.type= (one of TimeflowPointBookmark,
TimeflowPointBookmarkTag, TimeflowPointLocation,
TimeflowPointSemantic, TimeflowPointTimestamp as appropriate)
> set timeflowPointParameters.location= (the location, timestamp,
or bookmark you wish to refresh to)
> set timeflowPointParameters.timeflow= (the timeflow associated
with location)
> commit
This can all be called via a Powershell script…
33. #SQLSatATL 33
• Monstrous deployments, (20+) benefit from a scripted, CLI option.
• Single or several VDB deployment.
• When investigating deep level issues, using the CLI can make details
easier to search through.
• When looking for “just the facts”, then a graphical UI is beneficial for high
level error messages.
• Self-service has great benefits and make it simple for those without deep
level skills to take advantage of virtualized environments for development
and testing.
• The CLI helps with DevOps automation and orchestration.
When Do I USE the CLI or Graphical UI?
35. #SQLSatATL
Just copy data and applications into the cloud…
• Straightforward approach
• Inefficient, non-incremental for large environments
• Open-source “bcp” uses encryption, compresses,
and multi-threads
• Archaic processes recommended by vendors
Start with backups to IaaS storage, then populate re-hosted
applications by restoring from those backups
• Cloud backups are easy, known technology
How Are Companies Migrating to
the Cloud?
36. #SQLSatATL
Cost Estimates for Azure
https://azure.microsoft.com/en-
us/pricing/details/storage/blobs/
Storage Capacity LRS
First 1 TB / Month $0.024 per GB
Next 49 TB (1 to 50 TB) / Month $0.0236 per GB
Next 450 TB (50 to 500 TB) / Month $0.0232 per GB
Next 500 TB (500 to 1,000 TB) / Month $0.0228 per GB
ZRS
$0.03 per GB
$0.0295 per GB
$0.029 per GB
$0.0285 per GB
38. #SQLSatATL
Migration Complete…Not so Fast…
• What if you only want dev and test in the cloud.
• Data is migrated, but this doesn’t count for ongoing data loads, application
connectivity across the network.
• To refresh will take considerable time to perform with traditional tools or cloning
methods.
• Rarely a consideration for the difference in cost structure for processing large
amounts of data from on-premise to the cloud.
41. #SQLSatATL
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-cloud-migrate
• Database must be MSSQL 2005 or higher, (easy)
• Ensure that the database is compatible with Azure SQL DB, (correct any incompatible
functions, etc.)
• Must have identified all performance issues that will be impacted beforehand.
• Ensure there is as little physical distance between the cloud data center and bacpac files to
be used for migration.
• Disable management jobs that will hinder migration processing.
• Drop any objects or historical data that can impact migration time and can be performed
post migration.
44. #SQLSatATL
Much Improved Option-
1. Data virtualization
Easy, secure, revolutionary
Optimized for faster agile development and testing
Optimized for new costing structures in IaaS
With a software appliance, no concerns about hardware or
software outside of virtualization lock-in.
Migrating from on-prem to IaaS with
Virtualization
46. #SQLSatATL
Data virtualization: from on-premises
into the Cloud
Source DB server
SQL Server 2008-2016
2TB storage
Delphix Virtualization Engine
2 TB storage
Target DB server
SQL Server 2008-2016
No database storage
47. #SQLSatATL
Optimized for the cloud in the first place…not after!
Different cost structures
Much smaller storage footprint, much less data-transfer
How Does Data Virtualization Enhance
this?
48. #SQLSatATL
Traditional copy data management techniques
Developed without concerns about infrastructure chargeback
This corresponds to higher cost.
IaaS vendors monitor storage and data transfers
Help meet SLAs, garner profits
It’s not just the data that exists in the end, so transformations can
equal big money for cloud vendors.
Different cost structures
49. #SQLSatATL
Know Thy Enemy…
• Tune SQL and Apps to perform efficiently as possible- natural
life of database is growth, (in processes, resources, etc.) before
migrating.
• The less network latency, the better- network tracing to
eliminate database blame is important.
• Many of the same tools and data provides value- DMVs provide
data internally to SQL Server.
• Look at management tools such as Cloudmonix, (formerly
AzureWatch) AppDynamics, Dynatrace, Zabbix or Logic Monitor.
50. #SQLSatATL
For Non-Production Systems…
Change the way you’ve always performed tasks.
Performing common tasks the same way as previously might
end up costing more.
Secure Data
All IaaS alternatives promote encryption for data in-flight and
for data at-rest, but encryption may not be the right
answer…
Secondary Considerations
51. #SQLSatATL
All IaaS solutions provide encryption in-flight and encryption at-rest
But encryption doesn’t protect data as much as it needs to be .
Europe already requires data masking, not just data encryption for any
confindential data, (GDPR):
http://ec.europa.eu/justice/data-protection/article-
29/documentation/opinion-
recommendation/files/2014/wp216_en.pdf
Confidential data
52. #SQLSatATL
Encryption is reversible data obfuscation, which is very different from
masking data.
• Data masking is non-reversible.
It solves the issue at the data level.
Is authentication and authorization in non-production in compliance with
security goals?
All organizations will soon need to review if critical data in non-
production environments be accessible to developers, testers and
users.
Confidential data
53. #SQLSatATL
Masking personally-identifiable, (PII, HIPPA, PCI, etc.) information
renders it useless from a security standpoint
Resolves both the technical and personal responsibility issue.
The data can be masked before it moves to non-production,
removing unnecessary risk.
Why Masking is Part of the Answer
55. #SQLSatATL
Data virtualization: transformation by
masking
SQL Server
Validated Sync
Environment
Delphix Masking Engine
Delphix Virtualization Engine
2TB storage
SQL Server Target
No database storage
56. #SQLSatATL
Data virtualization: Masking and then to the
Cloud
Delphix Virtualization Engine
2 TB storage
SQL Server Target
Storage Only for
Masked Data
Delphix Virtualization Engine
2TB storage
Delphix Masking Engine
SQL Server
Validated Sync
Environment
SQL Server Target
Storage Only for
Masked Data
58. #SQLSatATL
On-Prem, Masked, then Replicated to IaaS
Delphix Virtualization Engine
4 TB storage
SQL Server
Validated Sync
Environment
SQL Server Target
Storage Only for Masked
Data
SQL Server Target
Storage Only for Masked
Data
Delphix Virtualization Engine
4 TB storage
SQL Server Target
Storage Only for Masked
Data
Delphix Masking Engine
59. #SQLSatATL
Virtualization Makes Copy Data Management
Simple
Security with masking and encryption is best
Cloud migrations are more successful when
virtualized and planned accordingly.
Patching and Upgrading can be done with less
effort and resources.
60. #SQLSatATL
Want to try it out, download the Delphix AWS Trial- Available for Oracle, MSSQL Soon!
https://www.delphix.com/products/free-trial-request
Twittter: @DBAKevlar
Linked in: http://linkedin.com/in/kellynpotvin
Blog: http://dbakevlar.com
61. #SQLSatATL 61
Delphix with SQL Server- the Basics
https://docs.delphix.com/docs/delphix-administration/sql-server-environments-and-data-
sources/managing-sql-server-environments/overview-of-setting-up-sql-server-
environments
Delphix Upgrade Workflow: https://community.delphix.com/delphix/topics/tip-of-the-day-
upgrading-a-sql-server-dsource
Upgrading the Dsource after an Upgrade: https://docs.delphix.com/docs/delphix-
administration/sql-server-environments-and-data-sources/virtualizing-databases-using-
delphix-with-sql-server/managing-sql-server-dsources/additional-dsource-
topics/upgrading-a-dsource-after-a-sql-server-upgrade
Delphix in the Cloud
https://www.delphix.com/solutions/cloud-migration-virtual-data
References and Tips
Talk about the future of the DBA with DevOps-
Learn other database platforms
Learn Shell, other than Powershell, learn Python and automation tools for DevOps
ETL, subsets of data, as well as physical and virtual clones, backup, replication.
Where is all that data going?
DBA 1.0/2.0? Does it translate?
Manage all those copies.
80-90% storage savings from traditional migration methods.
Data In flight can be significant cost for many cloud vendors
Network is the new bottleneck. You can avoid that with less copies- one golden copy- we call it the “validated sync environment”
This is the interface for Developers and testers- they can bookmark before important tasks or rewind to any point in the process. They can bookmark and branch for full development/testing needs.
How often does Microsoft send patches?
Do we start picking priorities about what we apply depending on environment access, resources and such?
DBA has to commandeer a database for patch testing.
This has to be performed for EACH environment, 100’s or 1000’s of databases!
Most are not synchronized with production, different outcomes when released to production.
Bugs occurring in one, not another!
Testing upgrades and patches can be greatly simplified using the portability and ease-of-use of Delphix Virtual Databases (VDBs). Here are two approaches that can be used, depending on the upgrade or patch.
Link the production database with the Delphix Server.
Provision a VDB at the existing patch level.
Patch the existing SQL Server bin files against the live VDB.
or
Create the new SQL Server bin file directory and switch the VDB.
Rollback VDB or Refresh from production.
Repeat 3 or 4 until confident.
Once the process has been tested and confirmed, it can be rolled out with confidence into production.
Still, very simple vs. what we write every day to support a simple create table or create database…
Do any of you see the problem with the high level project steps?
We commonly leave optimizing the environment until after we’ve migrated to the cloud.
Standard backup and recovery methods
Replication
Cloning, SSIS Packages to push data to Azure
Continual feed to keep up to date or refresh on regular basis, via archaic tools- bcp, log shipping or paid replication tools.
How many of you have moved dev and test to the cloud? How many moved cloud or moved it first??
If you moved it, would you consider keeping processing the same?
How can the cost structure impact you?
What all has to be moved? What issues are you going to run into?
Optimize first? Why?
And if you choose wrong or use more resources than expected, you can experience severe performance issues.
What resources are you really using? DBAs know, but do the developers and other stakeholders in the cloud migration project?
This is for Azure migrations- the requirements
Create this report- what can be migrated and what can’t?
90% of environments won’t fulfill the requirements and won’t migrate with the cloud migration asst.
Once final tests are done- you are testing.
Perform final migration, final sync to prod and downtime to switch from on-prem to cloud.
By going to a single source, loading to a single source and maintaining a single source, a smaller footprint is attained.
Cost savings in the way of less storage required results in even bigger savings.
Different cloud manufacturers have different pricing structures- verify what you are being charged for and make sure those costs aren’t in contrast with your environment.
Many avoid RDS on Amazon- we don’t support it. For our Oracle customers, too limited.
Before you start, tune SQL instaead of after.
Use network tools like Nagios network analyzer or. Solarwinds Network Performance Monitor, (NPM)
Your performance data can assist you in identifying huge IO, CPU and remote resource work that should be minimized beforehand.
Data in flight can cost you and data processing that was normal on-prem, may need to be redesigned post cloud migration.
Inspect pricing small print carefully and know you’re final decision on *what* choice in cloud and type of service will determine.
Encryption is important for production.
SQL 2016 dynamic data masking isn’t production ready- three steps and I had ‘un-masked
data!
Or does it shift the problem toward authentication and authorization?