SlideShare a Scribd company logo
Relax-and-Recover
Relax-and-Recover (ReaR)
Automated Testing
Gratien D'haeseGratien D'haese
IT3 ConsultantsIT3 Consultants
http://it3.be
http://relax­and­recover.org
IT3 Consultants Relax-and-Recover Automated Testing 2
Agenda
● Who am I?
● What is Relax-and-Recover?
● Relax-and-Recover Automated Testing
● DEMO
IT3 Consultants Relax-and-Recover Automated Testing 3
Who am I?
● Gratien D'haese
● IT3 Consultants (company)
● > 25 years Unix experience
● Unix/Linux Engineer (incl. Devops)
● ReaR Support Contracts
● Relax-and-Recover (ReaR)
● Major Open Source project
● https://github.com/gdha
IT3 Consultants Relax-and-Recover Automated Testing 4
Some Basics
● What is Disaster Recovery?
The process by which a business function is
restored to the normal, steady state after a
disaster
● What is Business Continuity?
The way that a business function will operate
after a disaster, until such time as the normal,
steady state is restored
IT3 Consultants Relax-and-Recover Automated Testing 5
Rehearse, maintain
and review
Business Continuity
Recovery
Recovery Plan
Preparedness
Business Impact
Analysis
Response
Incident Response
Prevention
Risk Management
ReaR
IT3 Consultants Relax-and-Recover Automated Testing 6
Linux Disaster Recovery
Like any other UNIX Operating System, Linux is
vulnerable for disaster to strike
Question: “What shall I do if a disaster strikes?”
Dependent on:
● Hardware failure (e.g. boot disk lost)
● Lost everything (fire, water, earthquake, theft)
● The answer: “Act immediately (with a disaster
recovery plan)”
IT3 Consultants Relax-and-Recover Automated Testing 7
Why are backups not enough?
Install
backup sw
Config
backup
sw
Full
data
restore
Linux OS install +
Patching
DR recovery
with ReaR
Restore
incremental
Replay
recipes
Replay
recipes
Disaster Recovery via re-install of Linux and restore data
Disaster Recovery with ReaR and restore data
Time
IT3 Consultants Relax-and-Recover Automated Testing 8
Relax-and-Recover (ReaR) as
DR solution
● ReaR is a tool that implements a DR work-flow
for Linux
● Basically meaning:
● Modular framework written in Bash
● Easy to extend to own needs
● Easy to deploy (set up and forget)
● Integration for various Linux technologies
● Integration with various back-up solutions
● Attempts to make system recovery as easy as
possible
IT3 Consultants Relax-and-Recover Automated Testing 9
Disaster Recovery – How It
Works
● Store the disk layout
● Partitioning, LVM and RAID configuration
● File systems, file system labels ...
● Boot loader (GRUB, GRUB2, LILO, UEFI)
● Store the files (tgz, rsync, through backup
software ...)
● Create bootable rescue media with system
configuration (and backup data)
● Can be done online (no business interruption)
IT3 Consultants Relax-and-Recover Automated Testing 10
Disaster Recovery – Rescue
Media
● Create “rescue linux” from running system
● Optimally compatible “tool box”
● Clone the system environment
● Linux kernel and modules
● Device driver configuration
● Network configuration
● Basic system software and tools
● Operate entirely in RAM (initrd)
IT3 Consultants Relax-and-Recover Automated Testing 11
DR Flow – BACKUP and
OUTPUT
BACKUP
Basic OS archive
(tar, rsync)
OUTPUT
Rescue boot image
BACKUP
Basic OS archive
(external backup sw)
internal
external
IT3 Consultants Relax-and-Recover Automated Testing 12
Usage of rear
● Shell scripts are stored under /usr/share/rear
● Scripts are executed via work-flows:
● mkrescue (only make rescue image)
● mkbackup (including make rescue image)
● mkbackuponly (excluding make rescue image)
● recover (the actual recovery part)
● Easy to incorporate new scripts, e.g. for
information gathering of Hard- and Software, or
other goodies
IT3 Consultants Relax-and-Recover Automated Testing 13
Getting started with ReaR
● Download it from
● Our web-site
– http://relax-and-recover.org/download/
● The rear-snapshot rpm's build from GitHub
– http://download.opensuse.org/repositories/Archiving:/Backup
:/Rear:/Snapshot/
● The official source
– https://github.com/rear/rear
● The official repo's (Fedora, RHEL and SLES)
– yum install rear
– zypper install rear
IT3 Consultants Relax-and-Recover Automated Testing 14
Testing ReaR
● ReaR is due to a wide range of options difficult
to test
● Different Linux flavors are using similar tools
with some minor differences :-(
● Too much to be able to test everything before a
new release, and sometimes it is even not
possible just because we do not have the
hardware
● Is and will stay a challenge for the future...
IT3 Consultants Relax-and-Recover Automated Testing 15
Automated ReaR Testing
● Finding a right balance between CI Testing and
Automated ReaR Testing
● Via OpenSuse Build Services we build daily a
fresh ReaR package (for free)
● We have some excellent ReaR developers who
do lots of coding (for free)
● Support via GitHub issues
● Free support
● Commercial support
IT3 Consultants Relax-and-Recover Automated Testing 16
Automated ReaR Testing
● For customers with a support contract we wrote
“Automated ReaR Testing”
● Currently only for CentOS 7 Linux distribution
● We start the process and it automatically does
● DR image creation
● Backup of system
● PXE boot the recover system with DR image made
● Restore backup
● Reboot the recover system
IT3 Consultants Relax-and-Recover Automated Testing 17
Test Configuration
Hypervisor (vagrant-host computer)
192.168.33.1
PXE
Boot
(libvirt)
PXE
Boot
(virtualbox)
Virtual
machines
DHCP network
IT3 Consultants Relax-and-Recover Automated Testing 18
Set up vagrant environment
● Host system must be GNU/Linux based
● A hypervisor like KVM or VirtualBox
● Install “vagrant” from your distro, or from
https://www.vagrantup.com/downloads.html
● KVM with libvirt needs the vagrant-libvirt plugin
vagrant plugin install vagrant-libvirt
● Install “git” software to clone the Vagrantfile and
scripts
IT3 Consultants Relax-and-Recover Automated Testing 19
Install the automated ReaR
Testing software
● Is Open Source and licensed under GPLv3
● New code is written only for customers with a
valid ReaR Support contracts
● git clone git@github.com:gdha/rear-automated-
testing.git
● Go into directory “rear-automated-testing”
● Type “./rear-automated-test.sh -h” to see info
● Uses “vagrant” to drive the creation of the VMs
● Account vagrant/vagrant (and root/vagrant)
IT3 Consultants Relax-and-Recover Automated Testing 20
Login via vagrant or ssh
● Login via vagrant:
● sudo vagrant ssh client
● sudo vagrant ssh server
● Another way to login is via ssh:
● ssh root@192.168.33.10 (client root pw is vagrant)
● ssh root@192.168.33.15 (server root pw is vagrant)
IT3 Consultants Relax-and-Recover Automated Testing 21
Login via vncviewer
● If you install “tigervnc” you can use vncviewer
● Use address 127.0.0.1 (localhost)
● Port 5991 for “client”
● Port 5992 for “server”
● Port 5993 for “recover”
IT3 Consultants Relax-and-Recover Automated Testing 22
Try it yourself?
● https://github.com/gdha/rear-automated-testing
 $ sudo ./rear­automated­test.sh ­p libvirt
+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+
|    Relax­and­Recover Automated Testing script    |
|             version 1.0                          |
+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+
Author: Gratien D'haese
Copyright: GPL v3
Current distro directory is centos7
Copy the Vagrantfile.libvirt to Vagrantfile
Bringing up the vagrant VMs client and server
Bringing machine 'client' up with 'libvirt' provider...
Bringing machine 'server' up with 'libvirt' provider...
==> server: Starting domain.
==> client: Starting domain.
IT3 Consultants Relax-and-Recover Automated Testing 23
Automated ReaR Testing
● Automated ReaR Testing speeds up
● Validation processes
● Bug hunting
● Testing with beakerlib (option -t)
● Trusworthy of “unstable” ReaR repository
● Acceptance of ReaR within corporations and Open
Source vendors
● Currently only available for CentOS 7 Linux
distribution (situation March 2017)
● We accept pull requestspull requests and sponsoringsponsoring
IT3 Consultants Relax-and-Recover Automated Testing 24
ReaR Automated Testing
IT3 Consultants Relax-and-Recover Automated Testing 25
Need more of this?
We can foresee in a customized workshop
on consultancy basis, or set-up in-house full
automated ReaR testing for customers with a
valid support contract
See http://www.it3.be/rear-support
Gratien D'haeseGratien D'haese
IT3 ConsultantsIT3 Consultants
IT3 Consultants Relax-and-Recover Automated Testing 26
Backup Slides in case live demonstration
Is not possible
IT3 Consultants Relax-and-Recover Automated Testing 27
The help and usage page
$ sudo ./rear-automated-test.sh -h
+----------------------------------------------------------------+
| Relax-and-Recover Automated Testing script |
| version 1.0 |
+----------------------------------------------------------------+
Author: Gratien D'haese
Copyright: GPL v3
Usage: rear-automated-test.sh [-d distro] [-b <boot method>] [-s <server IP>] [-p
provider] [-c rear-config-file.conf] [-t test] -vh
-d: The distribution to use for this automated test (default: centos7)
-b: The boot method to use by our automated test (default: PXE)
-s: The <boot server> IP address (default: 192.168.33.15)
-p: The vagrant <provider> to use (default: virtualbox)
-c: The ReaR config file we want to use with this test (default: PXE-
booting-example-with-URL-style.conf)
-t: The ReaR validation test directory (see tests directory; no default)
-h: This help message.
-v: Revision number of this script.
IT3 Consultants Relax-and-Recover Automated Testing 28
Starting the script
$ sudo ./rear-automated-test.sh
+-----------------------------------------------------+
| Relax-and-Recover Automated Testing script |
| version 1.0 |
+-----------------------------------------------------+
Author: Gratien D'haese
Copyright: GPL v3
Current distro directory is centos7
Copy the Vagrantfile.virtualbox to Vagrantfile
Bringing up the vagrant VMs client and server
Bringing machine 'client' up with 'virtualbox' provider...
Bringing machine 'server' up with 'virtualbox' provider...
==> client: Checking if box 'centos/7' is up to date...
==> client: A newer version of the box 'centos/7' is available! You currently
IT3 Consultants Relax-and-Recover Automated Testing 29
==> server: Setting hostname...
==> server: Configuring and enabling network interfaces...
==> server: Rsyncing folder: /projects/rear/rear-automated-testing/centos7/ => /vagrant
==> server: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> server: flag to force provisioning. Provisioners marked to run always will still run.
Sleep for 5 seconds [Control-C is now possible]
------------------------------------------------------------------------------
Current machine states:
client running (virtualbox)
server running (virtualbox)
recover poweroff (virtualbox)
This environment represents multiple VMs. The VMs are all listed
above with their current state. For more information about a specific
VM, run `vagrant status NAME`.
------------------------------------------------------------------------------
IT3 Consultants Relax-and-Recover Automated Testing 30
Check if 'eth1' is active on client [known issue
https://github.com/mitchellh/vagrant/issues/8166]
Connection to 127.0.0.1 closed.
Check if 'eth1' is active on server
Connection to 127.0.0.1 closed.
Doing ping tests to VMs client and server
client is up and running - ping test OK
server is up and running - ping test OK
IT3 Consultants Relax-and-Recover Automated Testing 31
Update rear on the VM client
Loaded plugins: fastestmirror
Determining fastest mirrors
* base: ftp.belnet.be
Resolving Dependencies
--> Running transaction check
---> Package rear.x86_64 0:2.00-44.git201702131649.el7 will be updated
---> Package rear.x86_64 0:2.00-74.git201703170925.el7 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================
Package Arch Version Repository Size
==========================================================
Updating:
rear x86_64 2.00-74.git201703170925.el7 Archiving_Backup_Rear_Snapshot 459 k
IT3 Consultants Relax-and-Recover Automated Testing 32
ReaR version that will be tested is:
Relax-and-Recover 2.00-git201703170925 / 2017-03-17
Content of /etc/rear/local.conf is:
OUTPUT=PXE
OUTPUT_PREFIX_PXE=$HOSTNAME
BACKUP=NETFS
BACKUP_URL=nfs://192.168.33.15/export/nfs/tftpboot
PXE_TFTP_URL=nfs://10.0.2.2/root/.config/VirtualBox/TFTP
PXE_CONFIG_URL=nfs://10.0.2.2/root/.config/VirtualBox/TFTP/pxelinux.cfg
PXE_RECOVER_MODE="unattended"
PXE_CREATE_LINKS="IP"
USE_STATIC_NETWORKING=y
KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"
IT3 Consultants Relax-and-Recover Automated Testing 33
Run rear -v mkbackup
Relax-and-Recover 2.00-git201703170925 / 2017-03-17
Using log file: /var/log/rear/rear-client.log
Using backup archive '/tmp/rear.mbxNVLH8555SO9t/outputfs/client/backup.tar.gz'
Creating disk layout
Creating root filesystem layout
Copying logfile /var/log/rear/rear-client.log into initramfs as '/tmp/rear-client-partial-
2017-03-19T09:43:41+0100.log'
Copied kernel+initrd 108M to nfs://10.0.2.2/root/.config/VirtualBox/TFTP
Created pxelinux config 'rear-client' and symlinks for IP adresses in
nfs://10.0.2.2/root/.config/VirtualBox/TFTP/pxelinux.cfg
Copying resulting files to nfs location
Saving /var/log/rear/rear-client.log as rear-client.log to nfs location
Creating tar archive '/tmp/rear.mbxNVLH8555SO9t/outputfs/client/backup.tar.gz'
Preparing archive operation
Archived 410 MiB in 64 seconds [avg 6567 KiB/sec]
The rear mkbackup was successful
IT3 Consultants Relax-and-Recover Automated Testing 34
Make client area readable for others on PXE boot server 10.0.2.2
Halting the client VM before doing the recovery
Recover VM will use the client IP address after it has been fully restored
==> client: Attempting graceful shutdown of VM...
Copied private key of client VB to recover VB config area
Starting the recover VM
Bringing machine 'recover' up with 'virtualbox' provider...
==> recover: Checking if box 'clink15/pxe' is up to date...
IT3 Consultants Relax-and-Recover Automated Testing 35
IT3 Consultants Relax-and-Recover Automated Testing 36
IT3 Consultants Relax-and-Recover Automated Testing 37
IT3 Consultants Relax-and-Recover Automated Testing 38
IT3 Consultants Relax-and-Recover Automated Testing 39
IT3 Consultants Relax-and-Recover Automated Testing 40
IT3 Consultants Relax-and-Recover Automated Testing 41
IT3 Consultants Relax-and-Recover Automated Testing 42
Beaker Tests
● sudo ./rear-automated-test.sh -t check-changed-
disk-layout
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Test option rear checklayout
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   PASS   ] :: Command 'rear ­v checklayout > output' (Expected 0, got 0)
:: [   PASS   ] :: Command 'grep 'Disk layout is identical.' output' (Expected 0, got 0)
:: [   PASS   ] :: There should be at least one swap disk (Assert: "3" should be >= "2")
:: [   PASS   ] :: Stop all swap disks (Expected 0, got 0)
:: [   PASS   ] :: Command 'rear ­v checklayout > output' (Expected 1, got 1)
:: [   PASS   ] :: Command 'grep 'Disk layout has changed.' output' (Expected 0, got 0)
:: [   PASS   ] :: Start all swap disks (Expected 0, got 0)
:: [   PASS   ] :: Command 'rear ­v checklayout > output' (Expected 0, got 0)
:: [   PASS   ] :: Command 'grep 'Disk layout is identical.' output' (Expected 0, got 0)
:: [   LOG    ] :: Duration: 2s
:: [   LOG    ] :: Assertions: 9 good, 0 bad
:: [   PASS   ] :: RESULT: Test option rear checklayout

More Related Content

What's hot

XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...
The Linux Foundation
 
Kvm and libvirt
Kvm and libvirtKvm and libvirt
Kvm and libvirt
plarsen67
 
Evolution of ota_update_in_the_io_t_world
Evolution of ota_update_in_the_io_t_worldEvolution of ota_update_in_the_io_t_world
Evolution of ota_update_in_the_io_t_world
Stefano Babic
 
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE SystemsXPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
The Linux Foundation
 
XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...
XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...
XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...
The Linux Foundation
 
Cobbler - Fast and reliable multi-OS provisioning
Cobbler - Fast and reliable multi-OS provisioningCobbler - Fast and reliable multi-OS provisioning
Cobbler - Fast and reliable multi-OS provisioning
RUDDER
 
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NECXPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
The Linux Foundation
 
AIXpert - AIX Security expert
AIXpert - AIX Security expertAIXpert - AIX Security expert
AIXpert - AIX Security expert
dlfrench
 
RunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edgeRunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edge
Stefano Stabellini
 
Linux sever building
Linux sever buildingLinux sever building
Linux sever building
Edmond Yu
 
RunX ELCE 2020
RunX ELCE 2020RunX ELCE 2020
RunX ELCE 2020
Stefano Stabellini
 
XPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, Huawei
XPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, HuaweiXPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, Huawei
XPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, Huawei
The Linux Foundation
 
S4 xen hypervisor_20080622
S4 xen hypervisor_20080622S4 xen hypervisor_20080622
S4 xen hypervisor_20080622
Todd Deshane
 
BSDCan2013
BSDCan2013BSDCan2013
BSDCan2013
krispcbsd
 
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISORLOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
Vanika Kapoor
 
Automação do físico ao NetSecDevOps
Automação do físico ao NetSecDevOpsAutomação do físico ao NetSecDevOps
Automação do físico ao NetSecDevOps
Raul Leite
 
BayLISA - FreeNAS 10 by Jordan Hubbard
BayLISA - FreeNAS 10 by Jordan HubbardBayLISA - FreeNAS 10 by Jordan Hubbard
BayLISA - FreeNAS 10 by Jordan Hubbard
iXsystems
 
XPDS16: CPUID handling for guests - Andrew Cooper, Citrix
XPDS16:  CPUID handling for guests - Andrew Cooper, CitrixXPDS16:  CPUID handling for guests - Andrew Cooper, Citrix
XPDS16: CPUID handling for guests - Andrew Cooper, Citrix
The Linux Foundation
 
XPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, Arm
XPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, ArmXPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, Arm
XPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, Arm
The Linux Foundation
 
PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)
PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)
PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)
iXsystems
 

What's hot (20)

XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...
 
Kvm and libvirt
Kvm and libvirtKvm and libvirt
Kvm and libvirt
 
Evolution of ota_update_in_the_io_t_world
Evolution of ota_update_in_the_io_t_worldEvolution of ota_update_in_the_io_t_world
Evolution of ota_update_in_the_io_t_world
 
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE SystemsXPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
 
XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...
XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...
XPDS16: Xen Orchestra: building a Cloud on top of Xen - Olivier Lambert & Jul...
 
Cobbler - Fast and reliable multi-OS provisioning
Cobbler - Fast and reliable multi-OS provisioningCobbler - Fast and reliable multi-OS provisioning
Cobbler - Fast and reliable multi-OS provisioning
 
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NECXPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
XPDS14 - Towards Massive Server Consolidation - Filipe Manco, NEC
 
AIXpert - AIX Security expert
AIXpert - AIX Security expertAIXpert - AIX Security expert
AIXpert - AIX Security expert
 
RunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edgeRunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edge
 
Linux sever building
Linux sever buildingLinux sever building
Linux sever building
 
RunX ELCE 2020
RunX ELCE 2020RunX ELCE 2020
RunX ELCE 2020
 
XPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, Huawei
XPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, HuaweiXPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, Huawei
XPDS16: Xen Scalability Analysis - Weidong Han, Zhichao Huang & Wei Yang, Huawei
 
S4 xen hypervisor_20080622
S4 xen hypervisor_20080622S4 xen hypervisor_20080622
S4 xen hypervisor_20080622
 
BSDCan2013
BSDCan2013BSDCan2013
BSDCan2013
 
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISORLOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
 
Automação do físico ao NetSecDevOps
Automação do físico ao NetSecDevOpsAutomação do físico ao NetSecDevOps
Automação do físico ao NetSecDevOps
 
BayLISA - FreeNAS 10 by Jordan Hubbard
BayLISA - FreeNAS 10 by Jordan HubbardBayLISA - FreeNAS 10 by Jordan Hubbard
BayLISA - FreeNAS 10 by Jordan Hubbard
 
XPDS16: CPUID handling for guests - Andrew Cooper, Citrix
XPDS16:  CPUID handling for guests - Andrew Cooper, CitrixXPDS16:  CPUID handling for guests - Andrew Cooper, Citrix
XPDS16: CPUID handling for guests - Andrew Cooper, Citrix
 
XPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, Arm
XPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, ArmXPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, Arm
XPDDS18: The Art of Virtualizing Cache Maintenance - Julien Grall, Arm
 
PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)
PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)
PCI Pass-through - FreeBSD VM on Hyper-V (MeetBSD California 2016)
 

Viewers also liked

Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...
Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...
Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...
Michael Larsen
 
Agile Testing
Agile TestingAgile Testing
Agile Testing
Naresh Jain
 
Big Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data QualityBig Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data Quality
RTTS
 
Agile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile TesterAgile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile Tester
Declan Whelan
 
Software Testing Basics
Software Testing BasicsSoftware Testing Basics
Software Testing Basics
Belal Raslan
 
Guerilla Usability Testing
Guerilla Usability TestingGuerilla Usability Testing
Guerilla Usability Testing
Andy Budd
 
Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...
Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...
Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...
Andrew Morgan
 
Introduction to Agile software testing
Introduction to Agile software testingIntroduction to Agile software testing
Introduction to Agile software testing
KMS Technology
 
Software testing ppt
Software testing pptSoftware testing ppt
Software testing ppt
Heritage Institute Of Tech,India
 
What Makes Great Infographics
What Makes Great InfographicsWhat Makes Great Infographics
What Makes Great Infographics
SlideShare
 
Masters of SlideShare
Masters of SlideShareMasters of SlideShare
Masters of SlideShare
Kapost
 
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to SlideshareSTOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
Empowered Presentations
 
You Suck At PowerPoint!
You Suck At PowerPoint!You Suck At PowerPoint!
You Suck At PowerPoint!
Jesse Desjardins - @jessedee
 
10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization
Oneupweb
 
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content MarketingHow To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
Content Marketing Institute
 
2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare
SlideShare
 
What to Upload to SlideShare
What to Upload to SlideShareWhat to Upload to SlideShare
What to Upload to SlideShare
SlideShare
 
How to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & TricksHow to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & Tricks
SlideShare
 
Getting Started With SlideShare
Getting Started With SlideShareGetting Started With SlideShare
Getting Started With SlideShare
SlideShare
 

Viewers also liked (19)

Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...
Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...
Get the Balance Right: Acceptance Test Driven Development, GUI Automation and...
 
Agile Testing
Agile TestingAgile Testing
Agile Testing
 
Big Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data QualityBig Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data Quality
 
Agile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile TesterAgile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile Tester
 
Software Testing Basics
Software Testing BasicsSoftware Testing Basics
Software Testing Basics
 
Guerilla Usability Testing
Guerilla Usability TestingGuerilla Usability Testing
Guerilla Usability Testing
 
Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...
Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...
Simulating APIs for Effective Testing: (Micro)Service Virtualisation for the ...
 
Introduction to Agile software testing
Introduction to Agile software testingIntroduction to Agile software testing
Introduction to Agile software testing
 
Software testing ppt
Software testing pptSoftware testing ppt
Software testing ppt
 
What Makes Great Infographics
What Makes Great InfographicsWhat Makes Great Infographics
What Makes Great Infographics
 
Masters of SlideShare
Masters of SlideShareMasters of SlideShare
Masters of SlideShare
 
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to SlideshareSTOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
 
You Suck At PowerPoint!
You Suck At PowerPoint!You Suck At PowerPoint!
You Suck At PowerPoint!
 
10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization
 
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content MarketingHow To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
 
2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare
 
What to Upload to SlideShare
What to Upload to SlideShareWhat to Upload to SlideShare
What to Upload to SlideShare
 
How to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & TricksHow to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & Tricks
 
Getting Started With SlideShare
Getting Started With SlideShareGetting Started With SlideShare
Getting Started With SlideShare
 

Similar to Relax-and-Recover Automated Testing

Rear automated testing with Bareos
Rear automated testing with BareosRear automated testing with Bareos
Rear automated testing with Bareos
Gratien D'haese
 
The Domino 10 RHEL 7 Primer
The Domino 10 RHEL 7 PrimerThe Domino 10 RHEL 7 Primer
The Domino 10 RHEL 7 Primer
Bill Malchisky Jr.
 
The 10 Commandments of Release Engineering
The 10 Commandments of Release EngineeringThe 10 Commandments of Release Engineering
The 10 Commandments of Release Engineering
Solano Labs
 
What’s New in UniVerse 11.2
What’s New in UniVerse 11.2What’s New in UniVerse 11.2
What’s New in UniVerse 11.2
Rocket Software
 
EuroPython 2013 - Python3 TurboGears Training
EuroPython 2013 - Python3 TurboGears TrainingEuroPython 2013 - Python3 TurboGears Training
EuroPython 2013 - Python3 TurboGears Training
Alessandro Molina
 
Prometheus and Docker (Docker Galway, November 2015)
Prometheus and Docker (Docker Galway, November 2015)Prometheus and Docker (Docker Galway, November 2015)
Prometheus and Docker (Docker Galway, November 2015)
Brian Brazil
 
Cinder Project On-Boarding - OpenInfra Summit Denver 2019
Cinder Project On-Boarding - OpenInfra Summit Denver 2019Cinder Project On-Boarding - OpenInfra Summit Denver 2019
Cinder Project On-Boarding - OpenInfra Summit Denver 2019
Jay Bryant
 
Backups
BackupsBackups
Backups
Payal Singh
 
2009-01-20 RHEL 5.3 for System z
2009-01-20 RHEL 5.3 for System z2009-01-20 RHEL 5.3 for System z
2009-01-20 RHEL 5.3 for System z
Shawn Wells
 
Archivematica Technical Training Diagnostics Guide (September 2018)
Archivematica Technical Training Diagnostics Guide (September 2018)Archivematica Technical Training Diagnostics Guide (September 2018)
Archivematica Technical Training Diagnostics Guide (September 2018)
Artefactual Systems - Archivematica
 
Containerize your Blackbox tests
Containerize your Blackbox testsContainerize your Blackbox tests
Containerize your Blackbox tests
Kevin Beeman
 
Maximizing Oracle RAC Uptime
Maximizing Oracle RAC UptimeMaximizing Oracle RAC Uptime
Maximizing Oracle RAC Uptime
Markus Michalewicz
 
A3Sec Advanced Deployment System
A3Sec Advanced Deployment SystemA3Sec Advanced Deployment System
A3Sec Advanced Deployment System
a3sec
 
Devops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShiftDevops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShift
Yaniv cohen
 
Design and Optimize your code for high-performance with Intel® Advisor and I...
Design and Optimize your code for high-performance with Intel®  Advisor and I...Design and Optimize your code for high-performance with Intel®  Advisor and I...
Design and Optimize your code for high-performance with Intel® Advisor and I...
Tyrone Systems
 
Embedded linux build systems
Embedded linux build systems  Embedded linux build systems
Embedded linux build systems
Mender.io
 
Powervc upgrade from_1.3.0.2_to_1.3.2.0
Powervc upgrade from_1.3.0.2_to_1.3.2.0Powervc upgrade from_1.3.0.2_to_1.3.2.0
Powervc upgrade from_1.3.0.2_to_1.3.2.0
Gobinath Panchavarnam
 
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios
 
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
OpenShift Origin
 
Introduction to Git for Network Engineers (Lab Guide)
Introduction to Git for Network Engineers (Lab Guide)Introduction to Git for Network Engineers (Lab Guide)
Introduction to Git for Network Engineers (Lab Guide)
Joel W. King
 

Similar to Relax-and-Recover Automated Testing (20)

Rear automated testing with Bareos
Rear automated testing with BareosRear automated testing with Bareos
Rear automated testing with Bareos
 
The Domino 10 RHEL 7 Primer
The Domino 10 RHEL 7 PrimerThe Domino 10 RHEL 7 Primer
The Domino 10 RHEL 7 Primer
 
The 10 Commandments of Release Engineering
The 10 Commandments of Release EngineeringThe 10 Commandments of Release Engineering
The 10 Commandments of Release Engineering
 
What’s New in UniVerse 11.2
What’s New in UniVerse 11.2What’s New in UniVerse 11.2
What’s New in UniVerse 11.2
 
EuroPython 2013 - Python3 TurboGears Training
EuroPython 2013 - Python3 TurboGears TrainingEuroPython 2013 - Python3 TurboGears Training
EuroPython 2013 - Python3 TurboGears Training
 
Prometheus and Docker (Docker Galway, November 2015)
Prometheus and Docker (Docker Galway, November 2015)Prometheus and Docker (Docker Galway, November 2015)
Prometheus and Docker (Docker Galway, November 2015)
 
Cinder Project On-Boarding - OpenInfra Summit Denver 2019
Cinder Project On-Boarding - OpenInfra Summit Denver 2019Cinder Project On-Boarding - OpenInfra Summit Denver 2019
Cinder Project On-Boarding - OpenInfra Summit Denver 2019
 
Backups
BackupsBackups
Backups
 
2009-01-20 RHEL 5.3 for System z
2009-01-20 RHEL 5.3 for System z2009-01-20 RHEL 5.3 for System z
2009-01-20 RHEL 5.3 for System z
 
Archivematica Technical Training Diagnostics Guide (September 2018)
Archivematica Technical Training Diagnostics Guide (September 2018)Archivematica Technical Training Diagnostics Guide (September 2018)
Archivematica Technical Training Diagnostics Guide (September 2018)
 
Containerize your Blackbox tests
Containerize your Blackbox testsContainerize your Blackbox tests
Containerize your Blackbox tests
 
Maximizing Oracle RAC Uptime
Maximizing Oracle RAC UptimeMaximizing Oracle RAC Uptime
Maximizing Oracle RAC Uptime
 
A3Sec Advanced Deployment System
A3Sec Advanced Deployment SystemA3Sec Advanced Deployment System
A3Sec Advanced Deployment System
 
Devops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShiftDevops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShift
 
Design and Optimize your code for high-performance with Intel® Advisor and I...
Design and Optimize your code for high-performance with Intel®  Advisor and I...Design and Optimize your code for high-performance with Intel®  Advisor and I...
Design and Optimize your code for high-performance with Intel® Advisor and I...
 
Embedded linux build systems
Embedded linux build systems  Embedded linux build systems
Embedded linux build systems
 
Powervc upgrade from_1.3.0.2_to_1.3.2.0
Powervc upgrade from_1.3.0.2_to_1.3.2.0Powervc upgrade from_1.3.0.2_to_1.3.2.0
Powervc upgrade from_1.3.0.2_to_1.3.2.0
 
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
 
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
Extending OpenShift Origin: Build Your Own Cartridge with Bill DeCoste of Red...
 
Introduction to Git for Network Engineers (Lab Guide)
Introduction to Git for Network Engineers (Lab Guide)Introduction to Git for Network Engineers (Lab Guide)
Introduction to Git for Network Engineers (Lab Guide)
 

More from Gratien D'haese

Open Technology Assemby Open Source Support Program
Open Technology Assemby Open Source Support ProgramOpen Technology Assemby Open Source Support Program
Open Technology Assemby Open Source Support Program
Gratien D'haese
 
What did you inspec?
What did you inspec?What did you inspec?
What did you inspec?
Gratien D'haese
 
Business Continuity Planning with Bareos and rear (Loadays 2015)
Business Continuity Planning with Bareos and rear (Loadays 2015)Business Continuity Planning with Bareos and rear (Loadays 2015)
Business Continuity Planning with Bareos and rear (Loadays 2015)
Gratien D'haese
 
Upgrade ux-fosdem-2015-gdhaese
Upgrade ux-fosdem-2015-gdhaeseUpgrade ux-fosdem-2015-gdhaese
Upgrade ux-fosdem-2015-gdhaese
Gratien D'haese
 
Cfg2html fosdem2014
Cfg2html fosdem2014Cfg2html fosdem2014
Cfg2html fosdem2014
Gratien D'haese
 
Adhocr T-dose 2012
Adhocr T-dose 2012Adhocr T-dose 2012
Adhocr T-dose 2012
Gratien D'haese
 
LinuxTag2012 Rear
LinuxTag2012 RearLinuxTag2012 Rear
LinuxTag2012 Rear
Gratien D'haese
 
Linux Disaster Recovery Best Practices with rear
Linux Disaster Recovery Best Practices with rearLinux Disaster Recovery Best Practices with rear
Linux Disaster Recovery Best Practices with rear
Gratien D'haese
 
Exploring I Pv6
Exploring I Pv6Exploring I Pv6
Exploring I Pv6
Gratien D'haese
 
Linux Disaster Recovery Solutions
Linux Disaster Recovery SolutionsLinux Disaster Recovery Solutions
Linux Disaster Recovery Solutions
Gratien D'haese
 

More from Gratien D'haese (10)

Open Technology Assemby Open Source Support Program
Open Technology Assemby Open Source Support ProgramOpen Technology Assemby Open Source Support Program
Open Technology Assemby Open Source Support Program
 
What did you inspec?
What did you inspec?What did you inspec?
What did you inspec?
 
Business Continuity Planning with Bareos and rear (Loadays 2015)
Business Continuity Planning with Bareos and rear (Loadays 2015)Business Continuity Planning with Bareos and rear (Loadays 2015)
Business Continuity Planning with Bareos and rear (Loadays 2015)
 
Upgrade ux-fosdem-2015-gdhaese
Upgrade ux-fosdem-2015-gdhaeseUpgrade ux-fosdem-2015-gdhaese
Upgrade ux-fosdem-2015-gdhaese
 
Cfg2html fosdem2014
Cfg2html fosdem2014Cfg2html fosdem2014
Cfg2html fosdem2014
 
Adhocr T-dose 2012
Adhocr T-dose 2012Adhocr T-dose 2012
Adhocr T-dose 2012
 
LinuxTag2012 Rear
LinuxTag2012 RearLinuxTag2012 Rear
LinuxTag2012 Rear
 
Linux Disaster Recovery Best Practices with rear
Linux Disaster Recovery Best Practices with rearLinux Disaster Recovery Best Practices with rear
Linux Disaster Recovery Best Practices with rear
 
Exploring I Pv6
Exploring I Pv6Exploring I Pv6
Exploring I Pv6
 
Linux Disaster Recovery Solutions
Linux Disaster Recovery SolutionsLinux Disaster Recovery Solutions
Linux Disaster Recovery Solutions
 

Recently uploaded

Generative AI and Large Language Models (LLMs)
Generative AI and Large Language Models (LLMs)Generative AI and Large Language Models (LLMs)
Generative AI and Large Language Models (LLMs)
rkpv2002
 
JORC_Review_presentation. 2024 código jorcpdf
JORC_Review_presentation. 2024 código jorcpdfJORC_Review_presentation. 2024 código jorcpdf
JORC_Review_presentation. 2024 código jorcpdf
WilliamsNuezEspetia
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
rawankhanlove256
 
Updated Limitations of Simplified Methods for Evaluating the Potential for Li...
Updated Limitations of Simplified Methods for Evaluating the Potential for Li...Updated Limitations of Simplified Methods for Evaluating the Potential for Li...
Updated Limitations of Simplified Methods for Evaluating the Potential for Li...
Robert Pyke
 
Chapter 1 Introduction to Software Engineering and Process Models.pdf
Chapter 1 Introduction to Software Engineering and Process Models.pdfChapter 1 Introduction to Software Engineering and Process Models.pdf
Chapter 1 Introduction to Software Engineering and Process Models.pdf
MeghaGupta952452
 
Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...
Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...
Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...
John Gallagher
 
Red Hat Enterprise Linux Administration 9.0 RH124 pdf
Red Hat Enterprise Linux Administration 9.0 RH124 pdfRed Hat Enterprise Linux Administration 9.0 RH124 pdf
Red Hat Enterprise Linux Administration 9.0 RH124 pdf
mdfkobir
 
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented ArchitectureANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
Divya Rajasekar
 
If we're running two pumps, why aren't we getting twice as much flow? v.17
If we're running two pumps, why aren't we getting twice as much flow? v.17If we're running two pumps, why aren't we getting twice as much flow? v.17
If we're running two pumps, why aren't we getting twice as much flow? v.17
Brian Gongol
 
Safety Operating Procedure for Testing Lifting Tackles
Safety Operating Procedure for Testing Lifting TacklesSafety Operating Procedure for Testing Lifting Tackles
Safety Operating Procedure for Testing Lifting Tackles
ssuserfcf701
 
DPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYS
DPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYSDPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYS
DPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYS
RyanMacayan
 
# Smart Parking Management System.pptx using IOT
# Smart Parking Management System.pptx using IOT# Smart Parking Management System.pptx using IOT
# Smart Parking Management System.pptx using IOT
Yesh20
 
Probability and Statistics by sheldon ross (8th edition).pdf
Probability and Statistics by sheldon ross (8th edition).pdfProbability and Statistics by sheldon ross (8th edition).pdf
Probability and Statistics by sheldon ross (8th edition).pdf
utkarshakusnake
 
"Operational and Technical Overview of Electric Locomotives at the Kanpur Ele...
"Operational and Technical Overview of Electric Locomotives at the Kanpur Ele..."Operational and Technical Overview of Electric Locomotives at the Kanpur Ele...
"Operational and Technical Overview of Electric Locomotives at the Kanpur Ele...
nanduchaihan9
 
RMC FPV.docx_fpv solar energy panels----
RMC FPV.docx_fpv solar energy panels----RMC FPV.docx_fpv solar energy panels----
RMC FPV.docx_fpv solar energy panels----
Khader Mallah
 
杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<
杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<
杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<
amzhoxvzidbke
 
李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<
李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<
李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<
amzhoxvzidbke
 
The world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptxThe world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptx
engrasjadshahzad
 
Sustainable construction is the use of renewable and recyclable materials in ...
Sustainable construction is the use of renewable and recyclable materials in ...Sustainable construction is the use of renewable and recyclable materials in ...
Sustainable construction is the use of renewable and recyclable materials in ...
RohitGhulanavar2
 
The Control of Relative Humidity & Moisture Content in The Air
The Control of Relative Humidity & Moisture Content in The AirThe Control of Relative Humidity & Moisture Content in The Air
The Control of Relative Humidity & Moisture Content in The Air
Ashraf Ismail
 

Recently uploaded (20)

Generative AI and Large Language Models (LLMs)
Generative AI and Large Language Models (LLMs)Generative AI and Large Language Models (LLMs)
Generative AI and Large Language Models (LLMs)
 
JORC_Review_presentation. 2024 código jorcpdf
JORC_Review_presentation. 2024 código jorcpdfJORC_Review_presentation. 2024 código jorcpdf
JORC_Review_presentation. 2024 código jorcpdf
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Updated Limitations of Simplified Methods for Evaluating the Potential for Li...
Updated Limitations of Simplified Methods for Evaluating the Potential for Li...Updated Limitations of Simplified Methods for Evaluating the Potential for Li...
Updated Limitations of Simplified Methods for Evaluating the Potential for Li...
 
Chapter 1 Introduction to Software Engineering and Process Models.pdf
Chapter 1 Introduction to Software Engineering and Process Models.pdfChapter 1 Introduction to Software Engineering and Process Models.pdf
Chapter 1 Introduction to Software Engineering and Process Models.pdf
 
Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...
Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...
Fix Production Bugs Quickly - The Power of Structured Logging in Ruby on Rail...
 
Red Hat Enterprise Linux Administration 9.0 RH124 pdf
Red Hat Enterprise Linux Administration 9.0 RH124 pdfRed Hat Enterprise Linux Administration 9.0 RH124 pdf
Red Hat Enterprise Linux Administration 9.0 RH124 pdf
 
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented ArchitectureANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
 
If we're running two pumps, why aren't we getting twice as much flow? v.17
If we're running two pumps, why aren't we getting twice as much flow? v.17If we're running two pumps, why aren't we getting twice as much flow? v.17
If we're running two pumps, why aren't we getting twice as much flow? v.17
 
Safety Operating Procedure for Testing Lifting Tackles
Safety Operating Procedure for Testing Lifting TacklesSafety Operating Procedure for Testing Lifting Tackles
Safety Operating Procedure for Testing Lifting Tackles
 
DPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYS
DPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYSDPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYS
DPWH - DEPARTMENT OF PUBLIC WORKS AND HIGHWAYS
 
# Smart Parking Management System.pptx using IOT
# Smart Parking Management System.pptx using IOT# Smart Parking Management System.pptx using IOT
# Smart Parking Management System.pptx using IOT
 
Probability and Statistics by sheldon ross (8th edition).pdf
Probability and Statistics by sheldon ross (8th edition).pdfProbability and Statistics by sheldon ross (8th edition).pdf
Probability and Statistics by sheldon ross (8th edition).pdf
 
"Operational and Technical Overview of Electric Locomotives at the Kanpur Ele...
"Operational and Technical Overview of Electric Locomotives at the Kanpur Ele..."Operational and Technical Overview of Electric Locomotives at the Kanpur Ele...
"Operational and Technical Overview of Electric Locomotives at the Kanpur Ele...
 
RMC FPV.docx_fpv solar energy panels----
RMC FPV.docx_fpv solar energy panels----RMC FPV.docx_fpv solar energy panels----
RMC FPV.docx_fpv solar energy panels----
 
杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<
杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<
杨洋李一桐做爱视频流出【网芷:ht28.co】国产国产午夜精华>>>[网趾:ht28.co】]<<<
 
李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<
李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<
李易峰祝绪丹做爱视频流出【网芷:ht28.co】可爱学生妹>>>[网趾:ht28.co】]<<<
 
The world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptxThe world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptx
 
Sustainable construction is the use of renewable and recyclable materials in ...
Sustainable construction is the use of renewable and recyclable materials in ...Sustainable construction is the use of renewable and recyclable materials in ...
Sustainable construction is the use of renewable and recyclable materials in ...
 
The Control of Relative Humidity & Moisture Content in The Air
The Control of Relative Humidity & Moisture Content in The AirThe Control of Relative Humidity & Moisture Content in The Air
The Control of Relative Humidity & Moisture Content in The Air
 

Relax-and-Recover Automated Testing

  • 1. Relax-and-Recover Relax-and-Recover (ReaR) Automated Testing Gratien D'haeseGratien D'haese IT3 ConsultantsIT3 Consultants http://it3.be http://relax­and­recover.org
  • 2. IT3 Consultants Relax-and-Recover Automated Testing 2 Agenda ● Who am I? ● What is Relax-and-Recover? ● Relax-and-Recover Automated Testing ● DEMO
  • 3. IT3 Consultants Relax-and-Recover Automated Testing 3 Who am I? ● Gratien D'haese ● IT3 Consultants (company) ● > 25 years Unix experience ● Unix/Linux Engineer (incl. Devops) ● ReaR Support Contracts ● Relax-and-Recover (ReaR) ● Major Open Source project ● https://github.com/gdha
  • 4. IT3 Consultants Relax-and-Recover Automated Testing 4 Some Basics ● What is Disaster Recovery? The process by which a business function is restored to the normal, steady state after a disaster ● What is Business Continuity? The way that a business function will operate after a disaster, until such time as the normal, steady state is restored
  • 5. IT3 Consultants Relax-and-Recover Automated Testing 5 Rehearse, maintain and review Business Continuity Recovery Recovery Plan Preparedness Business Impact Analysis Response Incident Response Prevention Risk Management ReaR
  • 6. IT3 Consultants Relax-and-Recover Automated Testing 6 Linux Disaster Recovery Like any other UNIX Operating System, Linux is vulnerable for disaster to strike Question: “What shall I do if a disaster strikes?” Dependent on: ● Hardware failure (e.g. boot disk lost) ● Lost everything (fire, water, earthquake, theft) ● The answer: “Act immediately (with a disaster recovery plan)”
  • 7. IT3 Consultants Relax-and-Recover Automated Testing 7 Why are backups not enough? Install backup sw Config backup sw Full data restore Linux OS install + Patching DR recovery with ReaR Restore incremental Replay recipes Replay recipes Disaster Recovery via re-install of Linux and restore data Disaster Recovery with ReaR and restore data Time
  • 8. IT3 Consultants Relax-and-Recover Automated Testing 8 Relax-and-Recover (ReaR) as DR solution ● ReaR is a tool that implements a DR work-flow for Linux ● Basically meaning: ● Modular framework written in Bash ● Easy to extend to own needs ● Easy to deploy (set up and forget) ● Integration for various Linux technologies ● Integration with various back-up solutions ● Attempts to make system recovery as easy as possible
  • 9. IT3 Consultants Relax-and-Recover Automated Testing 9 Disaster Recovery – How It Works ● Store the disk layout ● Partitioning, LVM and RAID configuration ● File systems, file system labels ... ● Boot loader (GRUB, GRUB2, LILO, UEFI) ● Store the files (tgz, rsync, through backup software ...) ● Create bootable rescue media with system configuration (and backup data) ● Can be done online (no business interruption)
  • 10. IT3 Consultants Relax-and-Recover Automated Testing 10 Disaster Recovery – Rescue Media ● Create “rescue linux” from running system ● Optimally compatible “tool box” ● Clone the system environment ● Linux kernel and modules ● Device driver configuration ● Network configuration ● Basic system software and tools ● Operate entirely in RAM (initrd)
  • 11. IT3 Consultants Relax-and-Recover Automated Testing 11 DR Flow – BACKUP and OUTPUT BACKUP Basic OS archive (tar, rsync) OUTPUT Rescue boot image BACKUP Basic OS archive (external backup sw) internal external
  • 12. IT3 Consultants Relax-and-Recover Automated Testing 12 Usage of rear ● Shell scripts are stored under /usr/share/rear ● Scripts are executed via work-flows: ● mkrescue (only make rescue image) ● mkbackup (including make rescue image) ● mkbackuponly (excluding make rescue image) ● recover (the actual recovery part) ● Easy to incorporate new scripts, e.g. for information gathering of Hard- and Software, or other goodies
  • 13. IT3 Consultants Relax-and-Recover Automated Testing 13 Getting started with ReaR ● Download it from ● Our web-site – http://relax-and-recover.org/download/ ● The rear-snapshot rpm's build from GitHub – http://download.opensuse.org/repositories/Archiving:/Backup :/Rear:/Snapshot/ ● The official source – https://github.com/rear/rear ● The official repo's (Fedora, RHEL and SLES) – yum install rear – zypper install rear
  • 14. IT3 Consultants Relax-and-Recover Automated Testing 14 Testing ReaR ● ReaR is due to a wide range of options difficult to test ● Different Linux flavors are using similar tools with some minor differences :-( ● Too much to be able to test everything before a new release, and sometimes it is even not possible just because we do not have the hardware ● Is and will stay a challenge for the future...
  • 15. IT3 Consultants Relax-and-Recover Automated Testing 15 Automated ReaR Testing ● Finding a right balance between CI Testing and Automated ReaR Testing ● Via OpenSuse Build Services we build daily a fresh ReaR package (for free) ● We have some excellent ReaR developers who do lots of coding (for free) ● Support via GitHub issues ● Free support ● Commercial support
  • 16. IT3 Consultants Relax-and-Recover Automated Testing 16 Automated ReaR Testing ● For customers with a support contract we wrote “Automated ReaR Testing” ● Currently only for CentOS 7 Linux distribution ● We start the process and it automatically does ● DR image creation ● Backup of system ● PXE boot the recover system with DR image made ● Restore backup ● Reboot the recover system
  • 17. IT3 Consultants Relax-and-Recover Automated Testing 17 Test Configuration Hypervisor (vagrant-host computer) 192.168.33.1 PXE Boot (libvirt) PXE Boot (virtualbox) Virtual machines DHCP network
  • 18. IT3 Consultants Relax-and-Recover Automated Testing 18 Set up vagrant environment ● Host system must be GNU/Linux based ● A hypervisor like KVM or VirtualBox ● Install “vagrant” from your distro, or from https://www.vagrantup.com/downloads.html ● KVM with libvirt needs the vagrant-libvirt plugin vagrant plugin install vagrant-libvirt ● Install “git” software to clone the Vagrantfile and scripts
  • 19. IT3 Consultants Relax-and-Recover Automated Testing 19 Install the automated ReaR Testing software ● Is Open Source and licensed under GPLv3 ● New code is written only for customers with a valid ReaR Support contracts ● git clone git@github.com:gdha/rear-automated- testing.git ● Go into directory “rear-automated-testing” ● Type “./rear-automated-test.sh -h” to see info ● Uses “vagrant” to drive the creation of the VMs ● Account vagrant/vagrant (and root/vagrant)
  • 20. IT3 Consultants Relax-and-Recover Automated Testing 20 Login via vagrant or ssh ● Login via vagrant: ● sudo vagrant ssh client ● sudo vagrant ssh server ● Another way to login is via ssh: ● ssh root@192.168.33.10 (client root pw is vagrant) ● ssh root@192.168.33.15 (server root pw is vagrant)
  • 21. IT3 Consultants Relax-and-Recover Automated Testing 21 Login via vncviewer ● If you install “tigervnc” you can use vncviewer ● Use address 127.0.0.1 (localhost) ● Port 5991 for “client” ● Port 5992 for “server” ● Port 5993 for “recover”
  • 22. IT3 Consultants Relax-and-Recover Automated Testing 22 Try it yourself? ● https://github.com/gdha/rear-automated-testing  $ sudo ./rear­automated­test.sh ­p libvirt +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ |    Relax­and­Recover Automated Testing script    | |             version 1.0                          | +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ Author: Gratien D'haese Copyright: GPL v3 Current distro directory is centos7 Copy the Vagrantfile.libvirt to Vagrantfile Bringing up the vagrant VMs client and server Bringing machine 'client' up with 'libvirt' provider... Bringing machine 'server' up with 'libvirt' provider... ==> server: Starting domain. ==> client: Starting domain.
  • 23. IT3 Consultants Relax-and-Recover Automated Testing 23 Automated ReaR Testing ● Automated ReaR Testing speeds up ● Validation processes ● Bug hunting ● Testing with beakerlib (option -t) ● Trusworthy of “unstable” ReaR repository ● Acceptance of ReaR within corporations and Open Source vendors ● Currently only available for CentOS 7 Linux distribution (situation March 2017) ● We accept pull requestspull requests and sponsoringsponsoring
  • 24. IT3 Consultants Relax-and-Recover Automated Testing 24 ReaR Automated Testing
  • 25. IT3 Consultants Relax-and-Recover Automated Testing 25 Need more of this? We can foresee in a customized workshop on consultancy basis, or set-up in-house full automated ReaR testing for customers with a valid support contract See http://www.it3.be/rear-support Gratien D'haeseGratien D'haese IT3 ConsultantsIT3 Consultants
  • 26. IT3 Consultants Relax-and-Recover Automated Testing 26 Backup Slides in case live demonstration Is not possible
  • 27. IT3 Consultants Relax-and-Recover Automated Testing 27 The help and usage page $ sudo ./rear-automated-test.sh -h +----------------------------------------------------------------+ | Relax-and-Recover Automated Testing script | | version 1.0 | +----------------------------------------------------------------+ Author: Gratien D'haese Copyright: GPL v3 Usage: rear-automated-test.sh [-d distro] [-b <boot method>] [-s <server IP>] [-p provider] [-c rear-config-file.conf] [-t test] -vh -d: The distribution to use for this automated test (default: centos7) -b: The boot method to use by our automated test (default: PXE) -s: The <boot server> IP address (default: 192.168.33.15) -p: The vagrant <provider> to use (default: virtualbox) -c: The ReaR config file we want to use with this test (default: PXE- booting-example-with-URL-style.conf) -t: The ReaR validation test directory (see tests directory; no default) -h: This help message. -v: Revision number of this script.
  • 28. IT3 Consultants Relax-and-Recover Automated Testing 28 Starting the script $ sudo ./rear-automated-test.sh +-----------------------------------------------------+ | Relax-and-Recover Automated Testing script | | version 1.0 | +-----------------------------------------------------+ Author: Gratien D'haese Copyright: GPL v3 Current distro directory is centos7 Copy the Vagrantfile.virtualbox to Vagrantfile Bringing up the vagrant VMs client and server Bringing machine 'client' up with 'virtualbox' provider... Bringing machine 'server' up with 'virtualbox' provider... ==> client: Checking if box 'centos/7' is up to date... ==> client: A newer version of the box 'centos/7' is available! You currently
  • 29. IT3 Consultants Relax-and-Recover Automated Testing 29 ==> server: Setting hostname... ==> server: Configuring and enabling network interfaces... ==> server: Rsyncing folder: /projects/rear/rear-automated-testing/centos7/ => /vagrant ==> server: Machine already provisioned. Run `vagrant provision` or use the `--provision` ==> server: flag to force provisioning. Provisioners marked to run always will still run. Sleep for 5 seconds [Control-C is now possible] ------------------------------------------------------------------------------ Current machine states: client running (virtualbox) server running (virtualbox) recover poweroff (virtualbox) This environment represents multiple VMs. The VMs are all listed above with their current state. For more information about a specific VM, run `vagrant status NAME`. ------------------------------------------------------------------------------
  • 30. IT3 Consultants Relax-and-Recover Automated Testing 30 Check if 'eth1' is active on client [known issue https://github.com/mitchellh/vagrant/issues/8166] Connection to 127.0.0.1 closed. Check if 'eth1' is active on server Connection to 127.0.0.1 closed. Doing ping tests to VMs client and server client is up and running - ping test OK server is up and running - ping test OK
  • 31. IT3 Consultants Relax-and-Recover Automated Testing 31 Update rear on the VM client Loaded plugins: fastestmirror Determining fastest mirrors * base: ftp.belnet.be Resolving Dependencies --> Running transaction check ---> Package rear.x86_64 0:2.00-44.git201702131649.el7 will be updated ---> Package rear.x86_64 0:2.00-74.git201703170925.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved ========================================================== Package Arch Version Repository Size ========================================================== Updating: rear x86_64 2.00-74.git201703170925.el7 Archiving_Backup_Rear_Snapshot 459 k
  • 32. IT3 Consultants Relax-and-Recover Automated Testing 32 ReaR version that will be tested is: Relax-and-Recover 2.00-git201703170925 / 2017-03-17 Content of /etc/rear/local.conf is: OUTPUT=PXE OUTPUT_PREFIX_PXE=$HOSTNAME BACKUP=NETFS BACKUP_URL=nfs://192.168.33.15/export/nfs/tftpboot PXE_TFTP_URL=nfs://10.0.2.2/root/.config/VirtualBox/TFTP PXE_CONFIG_URL=nfs://10.0.2.2/root/.config/VirtualBox/TFTP/pxelinux.cfg PXE_RECOVER_MODE="unattended" PXE_CREATE_LINKS="IP" USE_STATIC_NETWORKING=y KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"
  • 33. IT3 Consultants Relax-and-Recover Automated Testing 33 Run rear -v mkbackup Relax-and-Recover 2.00-git201703170925 / 2017-03-17 Using log file: /var/log/rear/rear-client.log Using backup archive '/tmp/rear.mbxNVLH8555SO9t/outputfs/client/backup.tar.gz' Creating disk layout Creating root filesystem layout Copying logfile /var/log/rear/rear-client.log into initramfs as '/tmp/rear-client-partial- 2017-03-19T09:43:41+0100.log' Copied kernel+initrd 108M to nfs://10.0.2.2/root/.config/VirtualBox/TFTP Created pxelinux config 'rear-client' and symlinks for IP adresses in nfs://10.0.2.2/root/.config/VirtualBox/TFTP/pxelinux.cfg Copying resulting files to nfs location Saving /var/log/rear/rear-client.log as rear-client.log to nfs location Creating tar archive '/tmp/rear.mbxNVLH8555SO9t/outputfs/client/backup.tar.gz' Preparing archive operation Archived 410 MiB in 64 seconds [avg 6567 KiB/sec] The rear mkbackup was successful
  • 34. IT3 Consultants Relax-and-Recover Automated Testing 34 Make client area readable for others on PXE boot server 10.0.2.2 Halting the client VM before doing the recovery Recover VM will use the client IP address after it has been fully restored ==> client: Attempting graceful shutdown of VM... Copied private key of client VB to recover VB config area Starting the recover VM Bringing machine 'recover' up with 'virtualbox' provider... ==> recover: Checking if box 'clink15/pxe' is up to date...
  • 35. IT3 Consultants Relax-and-Recover Automated Testing 35
  • 36. IT3 Consultants Relax-and-Recover Automated Testing 36
  • 37. IT3 Consultants Relax-and-Recover Automated Testing 37
  • 38. IT3 Consultants Relax-and-Recover Automated Testing 38
  • 39. IT3 Consultants Relax-and-Recover Automated Testing 39
  • 40. IT3 Consultants Relax-and-Recover Automated Testing 40
  • 41. IT3 Consultants Relax-and-Recover Automated Testing 41
  • 42. IT3 Consultants Relax-and-Recover Automated Testing 42 Beaker Tests ● sudo ./rear-automated-test.sh -t check-changed- disk-layout :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [   LOG    ] :: Test option rear checklayout :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [   PASS   ] :: Command 'rear ­v checklayout > output' (Expected 0, got 0) :: [   PASS   ] :: Command 'grep 'Disk layout is identical.' output' (Expected 0, got 0) :: [   PASS   ] :: There should be at least one swap disk (Assert: "3" should be >= "2") :: [   PASS   ] :: Stop all swap disks (Expected 0, got 0) :: [   PASS   ] :: Command 'rear ­v checklayout > output' (Expected 1, got 1) :: [   PASS   ] :: Command 'grep 'Disk layout has changed.' output' (Expected 0, got 0) :: [   PASS   ] :: Start all swap disks (Expected 0, got 0) :: [   PASS   ] :: Command 'rear ­v checklayout > output' (Expected 0, got 0) :: [   PASS   ] :: Command 'grep 'Disk layout is identical.' output' (Expected 0, got 0) :: [   LOG    ] :: Duration: 2s :: [   LOG    ] :: Assertions: 9 good, 0 bad :: [   PASS   ] :: RESULT: Test option rear checklayout