Devops for
databases
Osama Mustafa
About me
oFounder/CEO for Gurus Solutions
oLeader & Director for JAOUG
oOracle ACE Director
o6x of OCI, 4x AWS and 5x Azure
oAutomation Lover
oStart as DBA and still working on different Of Databases.
o Author of two books and working on my 3rd one.
oPresenter & Contributor in Oracle Community
oPublished articles in different magazines
oBlogger on osamaoracle.com
My Books
Agenda
Questions & Answers.
Automation Examples.
How DevOps will make DBA life Easier ?
Understand the DevOps ? What , Why , When
?
DevOps is a software development
method that stresses communication,
collaboration, integration, automation and
measurement of cooperation between
software developers and other IT
professionals.
DevOps is sometimes treated
as a concept, a philosophy, a
job role or a set of tools.
DevOps is not all about
automation, but it enables fast
feedback loops and encourages
a culture of collaboration
Evolution of DevOps
Dev Vs Ops
Why DevOps ?
1. Speed deployment processes
2. Enable fast response to changing business requirements
3. Reduce the risk of production changes causing outages
4. Break down silos
Is DevOps Agile ?
• When a task becomes (extremely) repetitive.
• The Task will take so much manual work.
• The task output either will be too much or less.
• Some other reason why to use DevOps :-
• Eliminate human interaction.
• Errors, things not able to work.
• Differences in paths
• Differences in settings
Why DevOps With Oracle -
Problem
• Terraform
• Ansible
• Jenkins
• Puppet
• Chef
• Salt
Oracle Using DevOps – Tools
Examples
Tools How it work Focusing
Terraform Agentless Admin Focused , IAAC
Ansible agentless Admin Focused , Configuration
Management
Jenkins Agentless Dev and can be work for admin
also, CI/CD
Puppet agent based Dev Focused
Chef agent based Dev Focused
Salt Agent, Agentless Admin Focused
How Ansible works ?
• Agentless:
Ansible doesn’t require agents to be installed on your target servers.
Instead, it connects through the secured SSH protocol to execute its tasks.
• Facts:
Facts are a set of variables that Ansible will collect on target hosts at run
time. Those variables are then usable within your scripts. Facts are almost
all information you may need from a target host like the IPs, NICs, Devices,
etc. You can even add your fact like the list of Oracle instance running with
their Oracle Home for example.
Two levels:
• You can use the so-called ad-hoc command line tool. This tool will permit
you with one command line to execute actions, like creating OS users, on
multiple servers.
• Then you may want to script a set of operations. For that purpose, you’ll
use Playbook. A playbook is a file contention your sequence of operations
in a YAML format.
• RHEL/CentOS Linux
• $ sudo yum install ansible
• Debian/Ubuntu Linux
• $ sudo apt-get install software-properties-common
• $ sudo apt-add-repository ppa:ansible/ansible
• $ sudo apt-get update
• $ sudo apt-get install ansible
• Install Ansible using pip
• $ sudo pip install ansible
• Github
• $ git clone git://github.com/ansible/ansible.git
• $ cd ./ansible
• $ source ./hacking/env-setup
Note : you need to have passwordless SSH connectivity from the Ansible host to the Oracle
servers
Install Ansible
• Automate a single task on one or more managed nodes. Ad-hoc commands are quick and easy,
but they are not reusable. So why learn about ad-hoc commands first? Ad-hoc commands
demonstrate the simplicity and power of Ansible.
• Ad-hoc commands are great for tasks you repeat rarely. For example, if you want to power off
all the machines in your lab for Christmas vacation, you could execute a quick one-liner in
Ansible without writing a playbook. An ad-hoc command looks like this:
• $ ansible [pattern] -m [module] -a "[module options]“
• Ad-hoc tasks can be used to reboot servers, copy files, manage packages and users, and much
more. You can use any Ansible module in an ad-hoc task
•
Ansible ad-Hoc command line tool
Ansible
Modules
Ansible ad-Hoc
command line
tool Use Cases
Rebooting servers
•I want to reboot all the servers in group called “WebServer” , I have 10 webserver.
•ansible Webserver -a "/sbin/reboot“Rebooting
Managing file
•SCP to transfer many files to multiple machines in parallel.
•ansible atlanta -m copy -a "src=/etc/hosts dest=/tmp/hosts"Managing
Managing users and groups
•You can create, manage, and remove user accounts on your managed nodes with ad-hoc
tasks:
•ansible all -m user -a "name=foo password=<crypted password here>"
Managing
• Now that you can run ad-hoc commands, you may be interested in combining multiple
modules to perform more elaborate actions. Instead of putting all commands sequentially into
a bash script, Ansible provides the concept of playbook. A playbook is a YAML formatted file
in which you list the modules sequentially to be executed on a group of hosts. This is an
example:
Ansible PlayBook
Use Ansible to Install RDBMS 19c
Control Server
Ansible
Python
Managed Server
Server#1
Managed Server
Server#2
/etc/ansible/hosts
[Server#1]
192.168.1.1
[Server#2]
192.168.1.2
[dbservers]
192.168.1.1
192.168.1.2
oracle19c_rdbmsinstall.yml
- hosts: dbservers
user: root
roles: -
- dbsoftware19c_install
roles/dbsoftware19c_install/files/Software_installation.r
sp
oracle.install.responseFileVersion=/oracle/install/rspfmt_dbi
nstall_response_schema_v19.0.0
oracle.install.option=INSTALL_DB_SWONLY
UNIX_GROUP_NAME=oinstall
INVENTORY_LOCATION={{ oracle_inventory }}
ORACLE_HOME={{ oracle_home }}
ORACLE_BASE={{ oracle_base }}
oracle.install.db.InstallEdition=EE
oracle.install.db.OSDBA_GROUP=dba
oracle.install.db.OSOPER_GROUP=oper
oracle.install.db.OSBACKUPDBA_GROUP=oinstall
oracle.install.db.OSDGDBA_GROUP=oinstall
oracle.install.db.OSKMDBA_GROUP=oinstall
oracle.install.db.OSRACDBA_GROUP=dba
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false
DECLINE_SECURITY_UPDATES=true
oracle.installer.autoupdates.option=SKIP_UPDATES
• cd /etc/ansible
• ansible-playbook oracle19c_rdbmsinstall.yml
Error:
ansible
requires a
json
module,
none
found!
This means the python-simplejson rpm is not installed
In our case: to
add the install of
pythonsimplejson
to the playbook
•install python-simplejson rpm
Purpose: run
command without
the need for
python
Resolution: raw
module
Use Ansible to Upgrade
pre_upgrade.yml
Upgrade.yml
Final.ym
l
• Oracle Cloud Infrastructure Ansible Modules Collection Sample
• Compute
• Database
• Object Storage
• Launch a Compute Instance
• Set Up an Autonomous Database
• List Objects and Buckets
• Delete Objects
• OCI Ansible Modules
• Block Volume
• Compute
• Container Engine for Kubernetes
• Database
• File Storage
• IAM
• Load Balancing
Ansible Playbook for OCI
Launch
a Compute Instance
• generates a temporary host-specific
SSH key-pair
• specifies the public key from that
key-pair to connect to the instance
during instance launch and
• demonstrates how the newly
launched instance can be connected
to using SSH.
https://github.com/oracle/oci-ansible-
collection/tree/master/samples/compute/launch_compute_instance
• Set up an Autonomous Database with Autonomous Transaction
Processing.
• List all the Autonomous Transaction Processing instances available
in a compartment, filtered by display name.
• Get the "facts" for a specified database.
• Stop and start an Autonomous Database.
• Delete an Autonomous Database.
Set Up an Autonomous Database
https://github.com/oracle/oci-ansible-
collection/tree/master/samples/database/autonomous_database
Delete
Objects
https://github.com/oracle/oci-ansible-
collection/tree/master/samples/object_storage/delete_objects
Terraform
• Terraform is a tool for building, changing, and
versioning infrastructure safely and
efficiently. Terraform can manage existing and
popular service providers as well as custom in-
house solutions. Configuration files describe
to Terraform the components needed to run a
single application or your entire datacenter.
• Advantages of Terraform
1. Platform Agnostic
2. State Management
3. Operator Confidence
Install Terraform
• Terraform and integrated providers available for all major operating
environments:
• Linux (x86 & ARM)
• FreeBSD (x86 & ARM)
• OpenBSD (x86)
• Windows (x86)
• Mac OS X (x86)
• Solaris (x86)
• Installation
• Download the terraform binary for your OS
• Integrated providers automatically downloaded/installed when needed
• Non-integrated providers: manual download of the binary provider
needed
• Frequent updates advised
• Easy on Oracle Linux 7 (ol7_developer on public yum,
ol7_x86_64_developer on ULN)
• # yum install terraform (Installation)
• # yum upgrade terraform (Upgrade)
Terraform
Commands
• terraform init
• Before initial use or after upgrade
• terraform plan
• terraform apply
• terraform apply –-auto-approve
• terraform destroy
• terraform destroy -force
Terraform Variables
Variables must be defined in a .tf file
values can be assigned
during
definition with « default = »
Variables can be used anywhere in .tf files
Input variables from environment variables
(TF_VAR_name)
$ export TF_VAR_access_key="xxxxxxxx"
$ export TF_VAR_secret_key="xxxxxxxx"
Input variables from
command line
Input variables from one or more files
Default file name is terraform.tfvars
If not default name, just give the name(s) of
file(s)
Terraform Output
Outputs are optional but useful to display
relevant information
public IP randomly assigned (AWS, OCI, OCI-C)
Temporary password for Windows instances in OCI
Outputs are displayed at the end of « terraform
apply » or with « terraform output »
Terraform data-source
Useful to automatically get objects names or IDs
that are needed to create resources
Alternative is to hard-code names/IDs
Examples of data sources within OCI
Get the list of Availability Domains (name different in each
tenant)
Best Practice
• Putting all code in main.tf is a good idea when you are getting started or writing an example
code. In all other cases you will be better having several files split logically like this:
• main.tf - call modules, locals and data-sources to create all resources
• variables.tf - contains declarations of variables used in main.tf
• outputs.tf - contains outputs from the resources created in main.tf
• terraform.tfvars should not be used anywhere except composition.
Example – Single Web Server
main.tf terraform {
required_version = ">= 0.12"
}
# -------------------------------------------------------
# CONFIGURE OUR AWS CONNECTION
# -------------------------------------------------------
provider "aws" {
region = "us-east-2"
}
# ----------------------------
# DEPLOY A SINGLE EC2 INSTANCE
# ----------------------------
resource "aws_instance" "example" {
# Ubuntu Server 18.04 LTS (HVM), SSD Volume Type in
us-east-2
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
vpc_security_group_ids = [aws_security_group.instance.id]
user_data = <<-EOF
#!/bin/bash
echo "Hello, World" > index.html
nohup busybox httpd -f -p "${var.server_port}" &
EOF
tags = {
Name = "terraform-example"
}
}
# ------------------------------------------------------------
# CREATE THE SECURITY GROUP THAT'S APPLIED TO THE
EC2 INSTANCE
resource "aws_security_group" "instance" {
name = "terraform-example-instance"
# Inbound HTTP from anywhere
ingress {
from_port = var.server_port
to_port = var.server_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
variable "server_port" {
description = "The port the server will use for
HTTP requests"
type = number
default = 8080
}
outputs.tf
variables.tf
output "public_ip" {
value = aws_instance.example.public_ip
description = "The public IP of the web server"
}
Thank you!
Twitter
@OsamaOracle
My Blog
OsamaOracle.com
LinkedIn
https://jo.linkedin.com/in/osamamustafa

DevOps for database

  • 1.
  • 2.
    About me oFounder/CEO forGurus Solutions oLeader & Director for JAOUG oOracle ACE Director o6x of OCI, 4x AWS and 5x Azure oAutomation Lover oStart as DBA and still working on different Of Databases. o Author of two books and working on my 3rd one. oPresenter & Contributor in Oracle Community oPublished articles in different magazines oBlogger on osamaoracle.com
  • 3.
  • 4.
    Agenda Questions & Answers. AutomationExamples. How DevOps will make DBA life Easier ? Understand the DevOps ? What , Why , When ?
  • 5.
    DevOps is asoftware development method that stresses communication, collaboration, integration, automation and measurement of cooperation between software developers and other IT professionals. DevOps is sometimes treated as a concept, a philosophy, a job role or a set of tools. DevOps is not all about automation, but it enables fast feedback loops and encourages a culture of collaboration
  • 6.
  • 7.
  • 8.
    Why DevOps ? 1.Speed deployment processes 2. Enable fast response to changing business requirements 3. Reduce the risk of production changes causing outages 4. Break down silos
  • 9.
  • 10.
    • When atask becomes (extremely) repetitive. • The Task will take so much manual work. • The task output either will be too much or less. • Some other reason why to use DevOps :- • Eliminate human interaction. • Errors, things not able to work. • Differences in paths • Differences in settings Why DevOps With Oracle - Problem
  • 11.
    • Terraform • Ansible •Jenkins • Puppet • Chef • Salt Oracle Using DevOps – Tools Examples
  • 12.
    Tools How itwork Focusing Terraform Agentless Admin Focused , IAAC Ansible agentless Admin Focused , Configuration Management Jenkins Agentless Dev and can be work for admin also, CI/CD Puppet agent based Dev Focused Chef agent based Dev Focused Salt Agent, Agentless Admin Focused
  • 14.
  • 15.
    • Agentless: Ansible doesn’trequire agents to be installed on your target servers. Instead, it connects through the secured SSH protocol to execute its tasks. • Facts: Facts are a set of variables that Ansible will collect on target hosts at run time. Those variables are then usable within your scripts. Facts are almost all information you may need from a target host like the IPs, NICs, Devices, etc. You can even add your fact like the list of Oracle instance running with their Oracle Home for example. Two levels: • You can use the so-called ad-hoc command line tool. This tool will permit you with one command line to execute actions, like creating OS users, on multiple servers. • Then you may want to script a set of operations. For that purpose, you’ll use Playbook. A playbook is a file contention your sequence of operations in a YAML format.
  • 16.
    • RHEL/CentOS Linux •$ sudo yum install ansible • Debian/Ubuntu Linux • $ sudo apt-get install software-properties-common • $ sudo apt-add-repository ppa:ansible/ansible • $ sudo apt-get update • $ sudo apt-get install ansible • Install Ansible using pip • $ sudo pip install ansible • Github • $ git clone git://github.com/ansible/ansible.git • $ cd ./ansible • $ source ./hacking/env-setup Note : you need to have passwordless SSH connectivity from the Ansible host to the Oracle servers Install Ansible
  • 17.
    • Automate asingle task on one or more managed nodes. Ad-hoc commands are quick and easy, but they are not reusable. So why learn about ad-hoc commands first? Ad-hoc commands demonstrate the simplicity and power of Ansible. • Ad-hoc commands are great for tasks you repeat rarely. For example, if you want to power off all the machines in your lab for Christmas vacation, you could execute a quick one-liner in Ansible without writing a playbook. An ad-hoc command looks like this: • $ ansible [pattern] -m [module] -a "[module options]“ • Ad-hoc tasks can be used to reboot servers, copy files, manage packages and users, and much more. You can use any Ansible module in an ad-hoc task • Ansible ad-Hoc command line tool Ansible Modules
  • 18.
    Ansible ad-Hoc command line toolUse Cases Rebooting servers •I want to reboot all the servers in group called “WebServer” , I have 10 webserver. •ansible Webserver -a "/sbin/reboot“Rebooting Managing file •SCP to transfer many files to multiple machines in parallel. •ansible atlanta -m copy -a "src=/etc/hosts dest=/tmp/hosts"Managing Managing users and groups •You can create, manage, and remove user accounts on your managed nodes with ad-hoc tasks: •ansible all -m user -a "name=foo password=<crypted password here>" Managing
  • 19.
    • Now thatyou can run ad-hoc commands, you may be interested in combining multiple modules to perform more elaborate actions. Instead of putting all commands sequentially into a bash script, Ansible provides the concept of playbook. A playbook is a YAML formatted file in which you list the modules sequentially to be executed on a group of hosts. This is an example: Ansible PlayBook
  • 20.
    Use Ansible toInstall RDBMS 19c Control Server Ansible Python Managed Server Server#1 Managed Server Server#2 /etc/ansible/hosts [Server#1] 192.168.1.1 [Server#2] 192.168.1.2 [dbservers] 192.168.1.1 192.168.1.2 oracle19c_rdbmsinstall.yml - hosts: dbservers user: root roles: - - dbsoftware19c_install
  • 21.
    roles/dbsoftware19c_install/files/Software_installation.r sp oracle.install.responseFileVersion=/oracle/install/rspfmt_dbi nstall_response_schema_v19.0.0 oracle.install.option=INSTALL_DB_SWONLY UNIX_GROUP_NAME=oinstall INVENTORY_LOCATION={{ oracle_inventory }} ORACLE_HOME={{oracle_home }} ORACLE_BASE={{ oracle_base }} oracle.install.db.InstallEdition=EE oracle.install.db.OSDBA_GROUP=dba oracle.install.db.OSOPER_GROUP=oper oracle.install.db.OSBACKUPDBA_GROUP=oinstall oracle.install.db.OSDGDBA_GROUP=oinstall oracle.install.db.OSKMDBA_GROUP=oinstall oracle.install.db.OSRACDBA_GROUP=dba SECURITY_UPDATES_VIA_MYORACLESUPPORT=false DECLINE_SECURITY_UPDATES=true oracle.installer.autoupdates.option=SKIP_UPDATES
  • 22.
    • cd /etc/ansible •ansible-playbook oracle19c_rdbmsinstall.yml
  • 23.
    Error: ansible requires a json module, none found! This meansthe python-simplejson rpm is not installed In our case: to add the install of pythonsimplejson to the playbook •install python-simplejson rpm Purpose: run command without the need for python Resolution: raw module
  • 24.
    Use Ansible toUpgrade pre_upgrade.yml Upgrade.yml Final.ym l
  • 25.
    • Oracle CloudInfrastructure Ansible Modules Collection Sample • Compute • Database • Object Storage • Launch a Compute Instance • Set Up an Autonomous Database • List Objects and Buckets • Delete Objects • OCI Ansible Modules • Block Volume • Compute • Container Engine for Kubernetes • Database • File Storage • IAM • Load Balancing Ansible Playbook for OCI
  • 26.
    Launch a Compute Instance •generates a temporary host-specific SSH key-pair • specifies the public key from that key-pair to connect to the instance during instance launch and • demonstrates how the newly launched instance can be connected to using SSH. https://github.com/oracle/oci-ansible- collection/tree/master/samples/compute/launch_compute_instance
  • 27.
    • Set upan Autonomous Database with Autonomous Transaction Processing. • List all the Autonomous Transaction Processing instances available in a compartment, filtered by display name. • Get the "facts" for a specified database. • Stop and start an Autonomous Database. • Delete an Autonomous Database. Set Up an Autonomous Database https://github.com/oracle/oci-ansible- collection/tree/master/samples/database/autonomous_database
  • 28.
  • 29.
    Terraform • Terraform isa tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in- house solutions. Configuration files describe to Terraform the components needed to run a single application or your entire datacenter. • Advantages of Terraform 1. Platform Agnostic 2. State Management 3. Operator Confidence
  • 30.
    Install Terraform • Terraformand integrated providers available for all major operating environments: • Linux (x86 & ARM) • FreeBSD (x86 & ARM) • OpenBSD (x86) • Windows (x86) • Mac OS X (x86) • Solaris (x86) • Installation • Download the terraform binary for your OS • Integrated providers automatically downloaded/installed when needed • Non-integrated providers: manual download of the binary provider needed • Frequent updates advised • Easy on Oracle Linux 7 (ol7_developer on public yum, ol7_x86_64_developer on ULN) • # yum install terraform (Installation) • # yum upgrade terraform (Upgrade)
  • 31.
    Terraform Commands • terraform init •Before initial use or after upgrade • terraform plan • terraform apply • terraform apply –-auto-approve • terraform destroy • terraform destroy -force
  • 32.
    Terraform Variables Variables mustbe defined in a .tf file values can be assigned during definition with « default = » Variables can be used anywhere in .tf files Input variables from environment variables (TF_VAR_name) $ export TF_VAR_access_key="xxxxxxxx" $ export TF_VAR_secret_key="xxxxxxxx" Input variables from command line Input variables from one or more files Default file name is terraform.tfvars If not default name, just give the name(s) of file(s)
  • 33.
    Terraform Output Outputs areoptional but useful to display relevant information public IP randomly assigned (AWS, OCI, OCI-C) Temporary password for Windows instances in OCI Outputs are displayed at the end of « terraform apply » or with « terraform output »
  • 34.
    Terraform data-source Useful toautomatically get objects names or IDs that are needed to create resources Alternative is to hard-code names/IDs Examples of data sources within OCI Get the list of Availability Domains (name different in each tenant)
  • 35.
    Best Practice • Puttingall code in main.tf is a good idea when you are getting started or writing an example code. In all other cases you will be better having several files split logically like this: • main.tf - call modules, locals and data-sources to create all resources • variables.tf - contains declarations of variables used in main.tf • outputs.tf - contains outputs from the resources created in main.tf • terraform.tfvars should not be used anywhere except composition.
  • 36.
    Example – SingleWeb Server main.tf terraform { required_version = ">= 0.12" } # ------------------------------------------------------- # CONFIGURE OUR AWS CONNECTION # ------------------------------------------------------- provider "aws" { region = "us-east-2" } # ---------------------------- # DEPLOY A SINGLE EC2 INSTANCE # ---------------------------- resource "aws_instance" "example" { # Ubuntu Server 18.04 LTS (HVM), SSD Volume Type in us-east-2 ami = "ami-0c55b159cbfafe1f0" instance_type = "t2.micro" vpc_security_group_ids = [aws_security_group.instance.id] user_data = <<-EOF #!/bin/bash echo "Hello, World" > index.html nohup busybox httpd -f -p "${var.server_port}" & EOF tags = { Name = "terraform-example" } } # ------------------------------------------------------------ # CREATE THE SECURITY GROUP THAT'S APPLIED TO THE EC2 INSTANCE resource "aws_security_group" "instance" { name = "terraform-example-instance" # Inbound HTTP from anywhere ingress { from_port = var.server_port to_port = var.server_port protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } }
  • 37.
    variable "server_port" { description= "The port the server will use for HTTP requests" type = number default = 8080 } outputs.tf variables.tf output "public_ip" { value = aws_instance.example.public_ip description = "The public IP of the web server" }
  • 39.