This document discusses database deployment automation. It begins with introductions and an example of a problematic Friday deployment. It then reviews the concept of automation and different visions of it within an organization. Potential tools and frameworks for automation are discussed, along with common pitfalls. Basic deployment workflows using Oracle Cloud Control are demonstrated, including setting credentials, creating a proxy user, adding target properties, and using a job template. The document concludes by emphasizing that database deployment automation is possible but requires effort from multiple teams.
My experience with embedding PostgreSQLJignesh Shah
At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.
We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.
My experience with embedding PostgreSQLJignesh Shah
At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.
We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
In this session we will deep dive into the exciting features of Amazon RDS for PostgreSQL, including new versions of PostgreSQL releases, new extensions, larger instances. We will also show benchmarks of new RDS instance types, and their value proposition. We will also look at how high availability and read scaling works on RDS PostgreSQL. We will also explore lessons we have learned managing a large fleet of PostgreSQL instances, including important tunables.
OpenStack is rapidly gaining popularity with businesses as they realize the benefits of a private cloud architecture. This presentation was delivered by Dave Page, Chief Architect, Tools & Installers at EnterpriseDB & PostgreSQL Core Team member during PG Open 2014. He addressed some of the common components of OpenStack deployments, how they can affect Postgres servers, and how users might best utilize some of the features they offer when deploying Postgres, including:
• Different configurations for the Nova compute service
• Use of the Cinder block store
• Virtual networking options with Neutron
• WAL archiving with the Swift object store
Right-Sizing your SQL Server Virtual Machineheraflux
Virtualizing your top-tier production SQL Servers is not as easy as P2V’ing it. Sometimes allocating more resources to the VM is the wrong approach, and getting it wrong will silently hurt performance. What is the most effective method for determining the ‘right’ amount of resources to allocate? What happens if the workload changes a month from now?
The methods for understanding the performance of your mission-critical SQL Servers gathered over the past ten years of SQL Server virtualization will be addressed, and valuable processes for performance statistic collection and analysis will be displayed. Come learn how to properly ‘right-size’ the resources allocated to a VM, improve the performance of your SQL Servers, and keep it maximized well into the future.
This presentation answer a lot of your questions about PostgreSQL and the Red Hat Cluster Suite.
It reviews how you can create failover/standby capabilities with the following activities:
General PostgreSQL clustering options
Overview of Red Hat Cluster Service
Identification of candidate databases for clustering
Identification of hardware for clustering
Analysis of uptime requirements and data latency
Implementation of clustering
Testing of clustering
PostgreSQL installation tips for RHCS
Database Provisioning in EM12c: Provision me a Database Now!Maaz Anjum
My presentation for Georgia Oracle User Group on December 12, 2013. In it, I discuss the Database Provisioning feature in Enterprise Manager 12c with an example of how I architected a solution by leveraging it.
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
In this session we will deep dive into the exciting features of Amazon RDS for PostgreSQL, including new versions of PostgreSQL releases, new extensions, larger instances. We will also show benchmarks of new RDS instance types, and their value proposition. We will also look at how high availability and read scaling works on RDS PostgreSQL. We will also explore lessons we have learned managing a large fleet of PostgreSQL instances, including important tunables.
OpenStack is rapidly gaining popularity with businesses as they realize the benefits of a private cloud architecture. This presentation was delivered by Dave Page, Chief Architect, Tools & Installers at EnterpriseDB & PostgreSQL Core Team member during PG Open 2014. He addressed some of the common components of OpenStack deployments, how they can affect Postgres servers, and how users might best utilize some of the features they offer when deploying Postgres, including:
• Different configurations for the Nova compute service
• Use of the Cinder block store
• Virtual networking options with Neutron
• WAL archiving with the Swift object store
Right-Sizing your SQL Server Virtual Machineheraflux
Virtualizing your top-tier production SQL Servers is not as easy as P2V’ing it. Sometimes allocating more resources to the VM is the wrong approach, and getting it wrong will silently hurt performance. What is the most effective method for determining the ‘right’ amount of resources to allocate? What happens if the workload changes a month from now?
The methods for understanding the performance of your mission-critical SQL Servers gathered over the past ten years of SQL Server virtualization will be addressed, and valuable processes for performance statistic collection and analysis will be displayed. Come learn how to properly ‘right-size’ the resources allocated to a VM, improve the performance of your SQL Servers, and keep it maximized well into the future.
This presentation answer a lot of your questions about PostgreSQL and the Red Hat Cluster Suite.
It reviews how you can create failover/standby capabilities with the following activities:
General PostgreSQL clustering options
Overview of Red Hat Cluster Service
Identification of candidate databases for clustering
Identification of hardware for clustering
Analysis of uptime requirements and data latency
Implementation of clustering
Testing of clustering
PostgreSQL installation tips for RHCS
Database Provisioning in EM12c: Provision me a Database Now!Maaz Anjum
My presentation for Georgia Oracle User Group on December 12, 2013. In it, I discuss the Database Provisioning feature in Enterprise Manager 12c with an example of how I architected a solution by leveraging it.
What should I do now?! JCS for WebLogic AdminsSimon Haslam
The Java Cloud Service offers all the power of WebLogic but runs as a managed service in the Oracle Cloud. Does this mean your job as a WebLogic Administrator will disappear? No, far from it - you'll be able to put your skills to far more valuable use!
Instead of worrying about how much hardware you need, tracking down patches, or just keeping the systems running, with JCS, your focus will be different: liaising with your development teams to deploy new versions of applications safely into production, shaking out and fixing performance issues, scheduling patching and planning for future growth while maintaining consistent service levels - all tasks of high value to your users.
This is an administrator's view of JCS and how it can work for you.
WinOps Conf 2016 - Michael Greene - Release PipelinesWinOps Conf
There are benefits to be gained when patterns and practices from developer techniques are applied to operations. Notably, a fully automated solution where infrastructure is managed as code and all changes are automatically validated before reaching production. This is a process shift that is recognized among industry innovators. For organizations already leveraging these processes, it should be clear how to leverage Microsoft platforms. For organizations that are new to the topic, it should be clear how to bring this process to your environment and what it means to your organizational culture. This presentation explains the components of a Release Pipeline for configuration as code, the value to operations, and solutions that are used when designing a new Release Pipeline architecture.
Backroll: Production Grade KVM Backup Solution Integrated in CloudStackShapeBlue
Backroll is not only a production-grade KVM backup solution. It is also being integrated inside Apache Cloudstack using the Backup and restore framework. Pierre and Quentin will show how it works, the feature list, and how the integration has been made.
Quentin is in charge of DIMSI custom developments on Apache Cloudstack deployment : customer portal, backup solution. On a daily basis, he helps our customers and our developers to use and embrace Devops methodology, by building CI/CD pipelines (GitLab, Azure Devops), dockerizing apps and automate things as much as possible... When not DevOps'ing, Quentin loves to binge watch series and movies, play with his cat "Boogie" and is a crazy fan of street food.
Grégoire is a software architect who spends most of his time designing infrastructure applications and CRM systems, on-premise or multi-cloud based. He’s been using Apache Cloudstack for many years, and likes to keep knowledge and data outside black-boxes Father of 4 children, you can meet him in Southern Brittany, sailing Hobbie Cat or supporting Lorient football club at Moustoir stadium.
Pierre is in charge of Backroll integration inside Cloudstack. Pierre has a proven track record of successful c# and Java projects. When not playing with his keyboard, Pierre is surfing, WingFoiling or bodyboarding on Brittany coast.
-----------------------------------------
CloudStack Collaboration Conference 2022 took place on 14th-16th November in Sofia, Bulgaria and virtually. The day saw a hybrid get-together of the global CloudStack community hosting 370 attendees. The event hosted 43 sessions from leading CloudStack experts, users and skilful engineers from the open-source world, which included: technical talks, user stories, new features and integrations presentations and more.
Moving to the cloud isn’t easy, transforming your engineering team to adopt to the cloud and services lifestyle is therefore crucial. It all starts with creating a common understanding of the engineering and development principles which are important in the cloud, which are different then building regular applications. This session will take you on a road trip based on the presenters experience developing and more importantly operating Azure Active Directory, SQL Server Azure and most recently the Xbox Live Services to support Xbox One.
SQLSaturday is a training event for SQL Server professionals and those wanting to learn about SQL Server. This event will be held Jun 13 2015 at Hochschule Bonn-Rhein-Sieg, Grantham-Allee 20, St. Augustin, Rheinland, 53757, Germany. Admittance to this event is free, all costs are covered by donations and sponsorships. Please register soon as seating is limited, and let friends and colleagues know about the event.
###
Maintenance Plans for Beginners (but not only) | Each of experienced administrators used (to some extent) what is called Maintenance Plans - Plans of Conservation. During this session, I'd like to discuss what can be useful for us to provide functionality when we use them and what to look out for. Session at 200 times the forward-300, with the opening of the discussion.
Practical management of development & QA environments for SharePoint 2013SharePointRadi
Speakers: Ognyan Guglev & Radi Atanassov
In this session we will share how we maintain our environments for development, quality assurance and demonstration purposes. We've put in a lot of thought into optimising what we do and to deliver a highly-available, performing experience to our delivery teams.
For our work we have over 90 farms, so the challenges in maintaining them are not insignificant. Due to advances in the SharePoint platform we believe it is becoming increasingly difficult to maintain SharePoint environments for every project, client or product. We have a strong requirement to be flexible and efficient on hardware and at the same time be able to spawn development environments on demand. Automation here with SCVMM is key to a sustainable work front.
We will discuss our goals as a consultancy company, how we deal with licenses, whether we prefer centralised or decentralised team environments, how to automate VM's with Service Center Virtual Machine Manager, how to deal with Microsoft SQL and Active Directory, DNS and IP addresses, what we do to make the developer's time as productive as possible and a whole set of other tips and tricks we put in place. We will also share our Apps development and Office 365 development landscapes.
Overall, this session is infrastructure focused, but will be valuable and practical both for administrators and developers, it will cover experiences for both sides of the spectrum.
Database continuous integration, unit test and functional testHarry Zheng
Discuss continuous integration for database projects, including building project, deploying project to database, and executing unit tests and functional tests.
This presentation will also discuss database test standards, tips and tricks.
Migrating on premises workload to azure sql databasePARIKSHIT SAVJANI
Azure SQL Database is a fully managed cloud database service with built-in intelligence, elastic scale, performance, reliability, and data protection that enables enterprises and ISVs to reduce their total cost of ownership and operational cost and overheads. In this session, I will share real-world experience of successfully migrated existing SaaS application and on-premises workload for some our tier 1 customers and ISV partners to Azure SQL Database service. The session walks through planning, assessment, migration tools and best practices from the proven experiences and practices of migrating real world applications to Azure SQL Database service.
Everything gets faster and more agile in our Data Center.
Connectivity means everything.
Resource sharing is the new default.
Latency is the currency of the modern world to processes more data faster and smarter than ever.
Sometimes you are happy with you Desktop System, but you just need more RAM. Normally you would need to buy a Server Board ... or you try our Desktop Memory & PCIe Expansion solution and save a lot of money.
Converged Network Adapters have been around for a while. Now UniPlex takes things a step further giving you the very first Converged Fabric with Fabric Attached Memory, NVMe over Fabrics, SDN and PCIe Device Sharing capabilities.
Building an open memory-centric computing architecture using intel optaneUniFabric
OOW 2017 presentation showcasing Fabric Attached Memory with 2 node RAC system based on two standard x86 servers running 200 GB/s with a per licensed cpu core data rate of 25 GB/s.
A deep five into offloading techniques for Oracle database servers, that takes both hardware and software solution into consideration. The focus is clearly to boost the efficiency of your already paid licenses.
After a few months tuning various parts, we are on good way. Still there is work to be done on the throughput side at our UniPlex Units or as Jeremy Clarkson would elegantly put it: POWERRR !!!
2. Agenda
Introduction
A Friday afternoon story
The term automation reviewed
Automation Visions
Potential Tools and Frameworks
Building Basic Deployment Workflows
Conclusion
Q&A
3. Introduction
About Myself
◦ Married with children (one daughter, who I gave the advise to never work in IT)
◦ Linux since 1998
◦ Oracle since 2000
◦ OCM & OCP
◦ Chairman of NGENSTOR since 2014
◦ CEO of DATABASE-SERVERS since 2015
DATABASE-SERVERS.COM
The fastest way to run databases
4. Introduction
About DATABASE-SERVERS
◦ Founded in 2015
◦ Custom license-optimized whitebox servers
◦ Standard Operating Environment Toolkit for Oracle/Redhat/Suse Linux
◦ CLX Cloud Systems based on OVM/LXC Stack
◦ Performance kits for all brands (HP/Oracle/Dell/Lenovo)
◦ Water Cooling
◦ Overclocking
◦ HBA/SSD Upgrades
◦ Tuning/Redesign of Oracle Engineered Systems (ODA, Exadata)
◦ Storage Extension with NGENSTOR Arrays
◦ Performance Kits
◦ Merger with NGENSTOR
◦ More Appliances and more competition for Oracle Engineered Systems giving you more ROI
5. A Friday afternoon
story
Imagine
… it’s Friday 16:55 the developer finished the last lines of code and uploaded it to the versioning
system.
It’s just 5 minutes separating him from the weekend.
He sends out a mail to operations, who could send him to hell.
«Him again!!!!», yells Pete the DBA.
Before he can even think about looking at the code, the big boss calls him asking for a status.
Pete deploys the script, which throws an error.
Rollback is tricky and Pete’s weekend fucked up…
6. The term automation
reviewed
In our the database world we can separate into infrastructure and application tasks:
Infrastructure
◦ Server / VM deployment
◦ Database Creation
◦ Backup/Restore/Replication
◦ …
Application
◦ New Application Version
◦ New Interfaces
◦ New Attributes
◦ …
7. The term automation
reviewed
The two main areas we work on are:
Housekeeping / Recurring Tasks
◦ Gather Statics of your Databases (Schema)
◦ Cleanup of ADR
Change Management & Change Deployment
◦ Data Correction Patches
◦ New Application / Database / Host Setup
◦ OS / Database / Application Patching
8. The term automation
reviewed
Since the days of mainframe…
◦ You write a deployment or housekeeping script
◦ You eventually add this code to a versioning system and check it out
◦ You add it to a scheduler with conditions (e.g. JOB Control or cron)
◦ It works for your most important application, but only for that one
◦ You do not have a lifecycle
◦ You do not have configuration managment
◦ Your goal is to automate to deploy/run
9. The term automation
reviewed
In most companies Automation atm realized is:
Either
◦ Something they code theirselves
◦ To deploy stuff
◦ Using a scheduler
Or
◦ Something they bought a huge software suite for to kill the task and are still programming
◦ One department is leading
◦ Another Agent on the Machine besides the other 10 to 20
10. Automation
Visions
What everyone think it is
CIO (10’000m)
«Press button to deploy»
Architect (1000m)
«One solution fits all»
Developer (100m)
«Tool to deploy my scripts»
DBA (1m)
«Alot of work to clean up the mess and sleepless weekends"
11. Automation
Visions
What they think they actually need
CIO
Automation capability
Architect
Documented and standardized workflows
Developer
«A workflow to deploy my script»
DBA
Workflows, that can be executed by a Job Scheduler to:
Check scripts
Do Backup
Run Scripts
Post Check
Post Backup
12. Automation Visions
What Organization needs
Company
«Capability to map business processes into deployable workflows for the company’s IS applications»
IT Department
«Rebuild capability for their core applications such as their ERP System»
13. Potential Tools and
Frameworks
Tools
Job Schedulers (e.g. Control M)
Software Packaging (RPM, Redhat Satellite)
Provisioning (Puppet, HP Server Automation)
Versioning Systems (SourceSafe, SVN)
DevOps Tools (Delphix)
OS-Tools (System Center)
Host Tools
14. Potential Tools and
Frameworks
Key decisions for selecting tools
◦ Audience / Scope (entire IT, Development only)
◦ Reuse existing work and tool suites y/n?
◦ Do I want to have interfaces y/n?
◦ Do I want to map it to current organisation or do I want to make a transition?
◦ What are my KPI (no of deployments, failure rate)?
◦ Cost
◦ Time to market
◦ Additional added value
◦ Enforce Change Management
◦ Configuration Management
◦ Trackability / Reporting
15. Potential Tools and
Frameworks
Potential Pitfalls
◦ Automation/Provisioning Suites are often tuned for specific areas
◦ Provisioning, Development Tasks
◦ Often gaps, eg. bad Database Plugins => Coding
◦ Automation/Provisioning Suites have alot of features, but are stubborn
◦ You can spend a lot of time defining workflows
◦ Pure Schedulers have only few pre-defined Workflows
◦ Again: alot of coding
◦ Your solution does only work for one team (e.g. Middleware)
◦ Acceptance problem with other teams
◦ Requirement gap
◦ Code around the problems (customizing)
16. Potential Tools and
Frameworks
Solution Approaches
◦ Define what’s feasible and what not
◦ Database Versioning?
◦ Micro Changes
◦ Think in Services
◦ What does my team provide to other peers?
◦ Deployment
◦ Configuration Data
◦ How can I realize my services in my «environment»?
◦ How can I deploy the developer’s scripts using Cloud Control?
◦ How do I provide and lifecycle for my application scripts?
◦ Define Interfaces
◦ Development Suite to Oracle Cloud Control Job Submission
17. Potential Tools and
Frameworks
How we do it
◦ Use what’s available
◦ IBM UrbanCode Deploy for
◦ Script Versioning
◦ Middleware Workflows
◦ Cloud Control for
◦ Provisioning
◦ Store Target Properties (custom ones) for UrbanCode Interface
◦ Database Jobs
◦ SQL Script Execution
◦ Housekeeping
◦ Backup
18. Building Basic Deployment
Workflows
Non-Database People often understimate Database Deployments
◦ They think in files
◦ packages are most add/delete files/compile
◦ New installation means delete, config and deploy
◦ They have no user Data
◦ They do not see the database engine’s complexity is os-like
◦ An upgrade/change on the engine/appliacation will change the behaviour
◦ New data will be written in a different «format»
◦ Rollback is tricky, as the customer want’s to keep it’s data
◦ Backup is not enough, we need to migrate the data and it’s attributes
◦ Using database links, we can also touch data in a remote schema (for which we hopefully have a backup…)
19. Building Basic Deployment
Workflows
A minimal set of tasks for a database deployment
◦ Check, what databases are touched
◦ Backup
◦ Check Script Synthax
◦ Deploy
◦ Check Application
◦ Go/No Go
As simple as it sounds, mostly people overdo it right from the start
22. Building Basic Deployment
Workflows
Set up credentials (required for script) execution
◦ To run host commands via Cloud Control or EMCLI (emcli execute_hostcmd)
◦ emcli set_credential -target_type=host -target_name=$host -credential_set=HostCredsNormal -user=UCD -
column="username:oracle;password:$1"
◦ To run sql commands via Cloud Control or EMCLI (emcli execute_sql)
◦ emcli set_credential -target_type=host -target_name=$host -credential_set=DBHostCreds -user=UCD -
column="username:oracle;password:$1"
◦ emcli set_credential -target_type=oracle_database -target_name=$db -credential_set=DBCredsNormal -user=UCD -
columns="username:sys;password:${ORA_PW_PREFIX}c${SID};role:sysdba"
◦ emcli set_credential -target_type=oracle_database -target_name=$db -credential_set=DBCredsSYSDBA -user=UCD -
columns="sysDBAUserName:sys;sysDBAPassword:${ORA_PW_PREFIX}c${SID};sysDBARole:sysdba"
23. Building Basic Deployment
Workflows
Create Proxy User
◦ Proxy Users and Auditing Proxy Users (Doc ID 782078.1)
CREATE USER UCD IDENTIFIED BY *********;
GRANT CREATE SESSION TO UCD;
ALTER USER DBSNMP GRANT CONNECT through UCD;
AUDIT SESSION;
AUDIT SELECT TABLE, UPDATE TABLE, INSERT TABLE, DELETE TABLE BY UCD ON BEHALF OF DBSNMP;
AUDIT SELECT TABLE, UPDATE TABLE, INSERT TABLE, DELETE TABLE BY DBSNMP BY ACCESS;
CONNECT UCD[DBSNMP]/*********;
CREATE TABLE test (id NUMBER);
INSERT INTO test VALUES(1);
commit;
27. Building Basic Deployment
Workflows
Fancy: Generate Delphix CLI Template
◦ Create a Multi-Exec Job in the Cloud Control Library
◦ Source0: https://community.delphix.com/delphix/topics/tip_of_the_day_using_the_cli_to_provision_an_oracle_vdb
◦ Integrate VDB Creation into your testing
◦ We use Delphix to create an ad-hoc VDB or a set, if changes span multiple databases (db-links)
◦ Workflow
◦ Checkout Nightly Build from IBM UCD
◦ Create VDB Env
◦ Test Script in VDB
◦ Check Result
◦ Approve/Reject Deployment
28. Building Basic Deployment
Workflows
WIP: Liquidbase
◦ Not all database types are supported
◦ Used for quality assurance
◦ Deploy to Dev
◦ Deploy to Test
◦ Run Liquidbase diff for QAS
◦ Generate Changelog file, if needed to correct the delta ;-)
29. Conclusion
Database Deployment Automation
◦ Is doable
◦ Still requires efforts from multiple teams
◦ Release management
◦ Development
◦ Dba
◦ Comes with no OOTB-Solution for all you individual Application needs
◦ There will always be a slight amount of code required
◦ Can be simplified using Automation Tools
◦ Oracle Cloud Control provides a very good job schedulder, CMDB facilicy and overall good Warehouse
◦ Wheither you script yourself or buy the management pack, it’s there: USE IT ;-)
◦ A Synthax Checker would be very welcome in future releases or generally more Developer-Friendly features like a Code
Versioning Plugin or SQL Script Repository