Gabriele Bartolini presented on using Barman for PostgreSQL disaster recovery. Barman is an open-source tool that automates backups and disaster recovery for PostgreSQL databases. It supports features like continuous archiving, retention policies, recovery to specific points in time, and recovery across multiple servers. Barman hides the complexity of PostgreSQL's point-in-time recovery functionality while keeping its strengths. It provides an easy-to-use interface and standard approach for PostgreSQL disaster recovery.
PostgreSQL Disaster Recovery with Barman (PGConf.EU 2013)Gabriele Bartolini
Are you tired of managing backup and recovery processes of your Postgres server using custom scripts? Are you worried about being called in the night or worse whilst on holiday to perform a recovery operation of your company's business critical PostgreSQL server? Barman, Backup and Recovery Manager, standardises backup and recovery operations, allowing database administrators and system administrators to easily integrate their PostgreSQL solutions in their disaster recovery plan. Barman is distributed as open source under GNU GPL 3 terms. Further information available at www.pgbarman.org.
Webinar: PostgreSQL continuous backup and PITR with BarmanGabriele Bartolini
How can you achieve an RTO of 5 minutes for the backups of your PostgreSQL databases? And what about RPO=0 for zero data loss backups?
This webinar gave an answer to those questions, by providing an overview of Disaster Recovery of PostgreSQL databases with Barman, covering its major features.
Barman, Backup and Recovery Manager for PostgreSQL, is an open source tool that was conceived by 2ndQuadrant about 10 years ago and released open source in 2012 under GNU GPL 3.
It is now one of the most popular backup and recovery tools in the PostgreSQL ecosystem.
Video available at https://resources.2ndquadrant.com/en/webinar-postgresql-continuous-backup-and-pitr-with-barman
A journey through the infinite combinations of processes and architectures that PostgreSQL and its open source ecosystem provide within disaster recovery and high availability to help you achieve peace of mind. Starting from zero.
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...Command Prompt., Inc
Alex Alexander & Linas Virbalas
Hot standby and streaming replication will move the needle forward for high availability and scaling for a wide number of applications. Tungsten already supports clustering using warm standby. In this talk we will describe how to build clusters using the new PostgreSQL features and give our report from the trenches.
This talk will cover how hot standby and streaming replication work from a user perspective, then dive into a description of how to use them, taking Tungsten as an example. We'll cover the following issues:
* Configuration of warm standby and streaming replication
* Provisioning new standby instances
* Strategies for balancing reads across primary and standby database
* Managing failover
* Troubleshooting and gotchas
Please join us for an enlightening discussion a set of PostgreSQL features that are interesting to a wide range of PostgreSQL users.
My experience with embedding PostgreSQLJignesh Shah
At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.
We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.
This document discusses tuning DB2 in a Solaris environment. It provides background on the presenters, Tom Bauch from IBM and Jignesh Shah from Sun Microsystems. The agenda covers general considerations, memory usage and bottlenecks, disk I/O considerations and bottlenecks, and tuning DB2 V8.1 specifically in Solaris 9. It discusses supported Solaris versions, kernel settings, required patches, installation methods, and the configuration wizard. Specific topics covered in more depth include the Data Partitioning Feature, DB2 Enterprise Server Edition, and analyzing and addressing potential memory bottlenecks.
Best Practices of HA and Replication of PostgreSQL in Virtualized EnvironmentsJignesh Shah
This document discusses best practices for high availability (HA) and replication of PostgreSQL databases in virtualized environments. It covers enterprise needs for HA, technologies like VMware HA and replication that can provide HA, and deployment blueprints for HA, read scaling, and disaster recovery within and across datacenters. The document also discusses PostgreSQL's different replication modes and how they can be used for HA, read scaling, and disaster recovery.
Gabriele Bartolini presented on using Barman for PostgreSQL disaster recovery. Barman is an open-source tool that automates backups and disaster recovery for PostgreSQL databases. It supports features like continuous archiving, retention policies, recovery to specific points in time, and recovery across multiple servers. Barman hides the complexity of PostgreSQL's point-in-time recovery functionality while keeping its strengths. It provides an easy-to-use interface and standard approach for PostgreSQL disaster recovery.
PostgreSQL Disaster Recovery with Barman (PGConf.EU 2013)Gabriele Bartolini
Are you tired of managing backup and recovery processes of your Postgres server using custom scripts? Are you worried about being called in the night or worse whilst on holiday to perform a recovery operation of your company's business critical PostgreSQL server? Barman, Backup and Recovery Manager, standardises backup and recovery operations, allowing database administrators and system administrators to easily integrate their PostgreSQL solutions in their disaster recovery plan. Barman is distributed as open source under GNU GPL 3 terms. Further information available at www.pgbarman.org.
Webinar: PostgreSQL continuous backup and PITR with BarmanGabriele Bartolini
How can you achieve an RTO of 5 minutes for the backups of your PostgreSQL databases? And what about RPO=0 for zero data loss backups?
This webinar gave an answer to those questions, by providing an overview of Disaster Recovery of PostgreSQL databases with Barman, covering its major features.
Barman, Backup and Recovery Manager for PostgreSQL, is an open source tool that was conceived by 2ndQuadrant about 10 years ago and released open source in 2012 under GNU GPL 3.
It is now one of the most popular backup and recovery tools in the PostgreSQL ecosystem.
Video available at https://resources.2ndquadrant.com/en/webinar-postgresql-continuous-backup-and-pitr-with-barman
A journey through the infinite combinations of processes and architectures that PostgreSQL and its open source ecosystem provide within disaster recovery and high availability to help you achieve peace of mind. Starting from zero.
Building tungsten-clusters-with-postgre sql-hot-standby-and-streaming-replica...Command Prompt., Inc
Alex Alexander & Linas Virbalas
Hot standby and streaming replication will move the needle forward for high availability and scaling for a wide number of applications. Tungsten already supports clustering using warm standby. In this talk we will describe how to build clusters using the new PostgreSQL features and give our report from the trenches.
This talk will cover how hot standby and streaming replication work from a user perspective, then dive into a description of how to use them, taking Tungsten as an example. We'll cover the following issues:
* Configuration of warm standby and streaming replication
* Provisioning new standby instances
* Strategies for balancing reads across primary and standby database
* Managing failover
* Troubleshooting and gotchas
Please join us for an enlightening discussion a set of PostgreSQL features that are interesting to a wide range of PostgreSQL users.
My experience with embedding PostgreSQLJignesh Shah
At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.
We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.
This document discusses tuning DB2 in a Solaris environment. It provides background on the presenters, Tom Bauch from IBM and Jignesh Shah from Sun Microsystems. The agenda covers general considerations, memory usage and bottlenecks, disk I/O considerations and bottlenecks, and tuning DB2 V8.1 specifically in Solaris 9. It discusses supported Solaris versions, kernel settings, required patches, installation methods, and the configuration wizard. Specific topics covered in more depth include the Data Partitioning Feature, DB2 Enterprise Server Edition, and analyzing and addressing potential memory bottlenecks.
Best Practices of HA and Replication of PostgreSQL in Virtualized EnvironmentsJignesh Shah
This document discusses best practices for high availability (HA) and replication of PostgreSQL databases in virtualized environments. It covers enterprise needs for HA, technologies like VMware HA and replication that can provide HA, and deployment blueprints for HA, read scaling, and disaster recovery within and across datacenters. The document also discusses PostgreSQL's different replication modes and how they can be used for HA, read scaling, and disaster recovery.
PostgreSQL High Availability in a Containerized WorldJignesh Shah
This document discusses high availability for PostgreSQL in a containerized environment. It outlines typical enterprise requirements for high availability including recovery time objectives and recovery point objectives. Shared storage-based high availability is described as well as the advantages and disadvantages of PostgreSQL replication. The use of Linux containers and orchestration tools like Kubernetes and Consul for managing containerized PostgreSQL clusters is also covered. The document advocates for using PostgreSQL replication along with services and self-healing tools to provide highly available and scalable PostgreSQL deployments in modern container environments.
Best Practices with PostgreSQL on SolarisJignesh Shah
This document provides best practices for deploying PostgreSQL on Solaris, including:
- Using Solaris 10 or latest Solaris Express for support and features
- Separating PostgreSQL data files onto different file systems tuned for each type of IO
- Tuning Solaris parameters like maxphys, klustsize, and UFS buffer cache size
- Configuring PostgreSQL parameters like fdatasync, commit_delay, wal_buffers
- Monitoring key metrics like memory, CPU, and IO usage at the Solaris and PostgreSQL level
This document provides an overview of Linux containers and Docker containers. It begins with definitions of containers and their advantages over virtual machines. It describes early implementations of containers like chroot, Jails, and Zones. It explains the underlying kernel technologies like cgroups and namespaces that enable Linux containers. It provides instructions for using LXC and systemd-nspawn to deploy basic containers. It then focuses on Docker containers, covering installation, images, volumes, and best practices for running applications like PostgreSQL in Docker containers.
This document discusses PostgreSQL streaming replication and switchover/switchback capabilities. It covers limitations in earlier PostgreSQL versions, timelines, new features in version 9.3 that enable switchover/switchback without needing fresh backups, and things to know like using clean shutdown and the recovery.conf file. A demo of these features is promised at the end.
OpenStack is rapidly gaining popularity with businesses as they realize the benefits of a private cloud architecture. This presentation was delivered by Dave Page, Chief Architect, Tools & Installers at EnterpriseDB & PostgreSQL Core Team member during PG Open 2014. He addressed some of the common components of OpenStack deployments, how they can affect Postgres servers, and how users might best utilize some of the features they offer when deploying Postgres, including:
• Different configurations for the Nova compute service
• Use of the Cinder block store
• Virtual networking options with Neutron
• WAL archiving with the Swift object store
Lightweight locks (LWLocks) in PostgreSQL provide mutually exclusive access to shared memory structures. They support both shared and exclusive locking modes. The LWLocks framework uses wait queues, semaphores, and spinlocks to efficiently manage acquiring and releasing locks. Dynamic monitoring of LWLock events is possible through special builds that incorporate statistics collection.
Introduction to PostgreSQL for System AdministratorsJignesh Shah
This document provides an introduction and overview of PostgreSQL for system administrators. It covers why to use open source databases like PostgreSQL, a quick start guide to installing and configuring PostgreSQL, PostgreSQL internals like file system layout and processes, and monitoring PostgreSQL performance using tools like iostat and top. The presentation is aimed at helping system administrators get started with managing PostgreSQL databases.
This presentation answer a lot of your questions about PostgreSQL and the Red Hat Cluster Suite.
It reviews how you can create failover/standby capabilities with the following activities:
General PostgreSQL clustering options
Overview of Red Hat Cluster Service
Identification of candidate databases for clustering
Identification of hardware for clustering
Analysis of uptime requirements and data latency
Implementation of clustering
Testing of clustering
PostgreSQL installation tips for RHCS
Various HA and DR setups for Postgres Plus Advanced Server -
Active – Passive OS HA Clustering
Log Shipping Replication (Hot Standby Mode)
Hot Streaming Replication (Hot Standby Mode)
EDB Postgres Plus Failover Manager
HA with read scaling (with pg-pool)
xDB Single Master Replication (SMR)
xDB Multi Master Replication (MMR)
Use Cases
Denish Patel deployed PostgreSQL on Amazon EC2 for a startup that had its entire IT architecture on the Amazon cloud. The initial deployment involved setting up master-slave configurations across two EC2 environments with issues like weekly instance failures and lack of monitoring. Patel then consolidated the environments, configured high availability using replication across availability zones and regions, implemented automation using Puppet, and added monitoring and backups to improve the stability and management of the PostgreSQL deployment.
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
In this session we will deep dive into the exciting features of Amazon RDS for PostgreSQL, including new versions of PostgreSQL releases, new extensions, larger instances. We will also show benchmarks of new RDS instance types, and their value proposition. We will also look at how high availability and read scaling works on RDS PostgreSQL. We will also explore lessons we have learned managing a large fleet of PostgreSQL instances, including important tunables.
This document discusses various goals, techniques, and solutions for replicating PostgreSQL databases. The goals covered are high availability, performance for reads and writes, supporting wide area networks, and handling offline peers. Techniques include master-slave and multi-master replication, proxies, and using standby systems. Specific solutions described are Slony-I, Slony-II, PGCluster, DBMirror, pgpool, WAL replication, Sequoia, DRBD, and shared storage. The document provides an overview of how each solution can help achieve different replication goals.
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataJignesh Shah
This document discusses PostgreSQL performance on multi-core systems with multi-terabyte data. It covers current market trends towards more cores and larger data sizes. Benchmark results show that PostgreSQL scales well on inserts up to a certain number of clients/cores but struggles with OLTP and TPC-E workloads due to lock contention. Issues are identified with sequential scans, index scans, and maintenance tasks like VACUUM as data sizes increase. The document proposes making PostgreSQL utilities and tools able to leverage multiple cores/processes to improve performance on modern hardware.
The document discusses PostgreSQL high availability and scaling options. It covers horizontal scaling using load balancing and data partitioning across multiple servers. It also covers high availability techniques like master-slave replication, warm standby servers with point-in-time recovery, and using a heartbeat to prevent multiple servers from becoming a master. The document recommends an initial architecture with two servers using warm standby and point-in-time recovery with a heartbeat for high availability. It suggests scaling the application servers horizontally later on if more capacity is needed.
This document summarizes a presentation about PostgreSQL replication. It discusses different replication terms like master/slave and primary/secondary. It also covers replication mechanisms like statement-based and binary replication. The document outlines how to configure and administer replication through files like postgresql.conf and recovery.conf. It discusses managing replication including failover, failback, remastering and replication lag. It also covers synchronous replication and cascading replication setups.
Highly efficient backups with percona xtrabackupNilnandan Joshi
Percona XtraBackup is an open source, free MySQL hot backup software that performs non-blocking backups for InnoDB and XtraDB databases. In this talk we'll describe below things.
- How it works with MySQL/Percona Server and what are the features provided
- Difference between Xtrabackup and Innobackupex
- How to take full/increment/partial backup and restore
- How to use features like streaming, compression, remote and compact backups
- How to troubleshoot the issue with xtrabackup
This document provides an overview of key differences between SQL Server and PostgreSQL databases. It covers topics such as extensions, cost, case sensitivity, operating systems, processor configuration, write-ahead logging (WAL), checkpoints, disabling writes, page corruptions, MVCC, vacuum, database snapshots, system databases, tables, indexes, statistics, triggers, functions, security, backups, replication, imports/exports, maintenance, and monitoring. The document aims to help SQL Server DBAs understand how to administer and work with PostgreSQL databases.
Cobbler - Fast and reliable multi-OS provisioningRUDDER
In a lot of companies, machine deployment is a delicate subject: every administrator has his own recipe, using CD-ROMs, static binary images deployed via the network, peer delegation ...
However, one solution makes the consensus when it comes to automated mass deployments ( except in the Cloud ): PXE boot. The main cons are that the deployment and the management of such a service is a pain, and every OS has its own installation automation system.
This is where Cobbler saves the day: it enables a painless and reliably to create a PXE service, usable on either virtual or physical machines, while beeing the most agnostic possible towards the target OSes and its preconfiguration system (preseed, kickstart, sysprep, ...) while offering the possibility to handle lots of configuration parameters in a modular fashion (network, partitionning, user accounts, configuration management agent...)
This conference aims to introduce the audience to the general concepts of Cobbler, and some scenarios where it would be a useful solution.
Out of the box replication in postgres 9.4Denish Patel
This document provides an overview of setting up out of the box replication in PostgreSQL 9.4 without third party tools. It discusses write-ahead logs (WAL), replication slots, pg_basebackup, and pg_receivexlog. The document then demonstrates setting up replication on VMs with pg_basebackup to initialize a standby server, configuration of primary and standby servers, and monitoring of replication.
O documento fornece instruções e diretrizes sobre técnicas de serviço de bar, incluindo regras de higiene, organização do espaço, equipamentos necessários e procedimentos padrão. É destacada a importância de manter os mais altos padrões de limpeza, organização e qualidade dos produtos para garantir a satisfação dos clientes e a reputação do estabelecimento.
O documento fornece dicas importantes para estudantes sobre como aprender de forma efetiva, incluindo ler o conteúdo com atenção, explorar ilustrações e aplicar o aprendizado na prática. Também apresenta detalhes sobre um curso de barman, incluindo a carga horária da primeira parte e os tópicos que serão abordados.
PostgreSQL High Availability in a Containerized WorldJignesh Shah
This document discusses high availability for PostgreSQL in a containerized environment. It outlines typical enterprise requirements for high availability including recovery time objectives and recovery point objectives. Shared storage-based high availability is described as well as the advantages and disadvantages of PostgreSQL replication. The use of Linux containers and orchestration tools like Kubernetes and Consul for managing containerized PostgreSQL clusters is also covered. The document advocates for using PostgreSQL replication along with services and self-healing tools to provide highly available and scalable PostgreSQL deployments in modern container environments.
Best Practices with PostgreSQL on SolarisJignesh Shah
This document provides best practices for deploying PostgreSQL on Solaris, including:
- Using Solaris 10 or latest Solaris Express for support and features
- Separating PostgreSQL data files onto different file systems tuned for each type of IO
- Tuning Solaris parameters like maxphys, klustsize, and UFS buffer cache size
- Configuring PostgreSQL parameters like fdatasync, commit_delay, wal_buffers
- Monitoring key metrics like memory, CPU, and IO usage at the Solaris and PostgreSQL level
This document provides an overview of Linux containers and Docker containers. It begins with definitions of containers and their advantages over virtual machines. It describes early implementations of containers like chroot, Jails, and Zones. It explains the underlying kernel technologies like cgroups and namespaces that enable Linux containers. It provides instructions for using LXC and systemd-nspawn to deploy basic containers. It then focuses on Docker containers, covering installation, images, volumes, and best practices for running applications like PostgreSQL in Docker containers.
This document discusses PostgreSQL streaming replication and switchover/switchback capabilities. It covers limitations in earlier PostgreSQL versions, timelines, new features in version 9.3 that enable switchover/switchback without needing fresh backups, and things to know like using clean shutdown and the recovery.conf file. A demo of these features is promised at the end.
OpenStack is rapidly gaining popularity with businesses as they realize the benefits of a private cloud architecture. This presentation was delivered by Dave Page, Chief Architect, Tools & Installers at EnterpriseDB & PostgreSQL Core Team member during PG Open 2014. He addressed some of the common components of OpenStack deployments, how they can affect Postgres servers, and how users might best utilize some of the features they offer when deploying Postgres, including:
• Different configurations for the Nova compute service
• Use of the Cinder block store
• Virtual networking options with Neutron
• WAL archiving with the Swift object store
Lightweight locks (LWLocks) in PostgreSQL provide mutually exclusive access to shared memory structures. They support both shared and exclusive locking modes. The LWLocks framework uses wait queues, semaphores, and spinlocks to efficiently manage acquiring and releasing locks. Dynamic monitoring of LWLock events is possible through special builds that incorporate statistics collection.
Introduction to PostgreSQL for System AdministratorsJignesh Shah
This document provides an introduction and overview of PostgreSQL for system administrators. It covers why to use open source databases like PostgreSQL, a quick start guide to installing and configuring PostgreSQL, PostgreSQL internals like file system layout and processes, and monitoring PostgreSQL performance using tools like iostat and top. The presentation is aimed at helping system administrators get started with managing PostgreSQL databases.
This presentation answer a lot of your questions about PostgreSQL and the Red Hat Cluster Suite.
It reviews how you can create failover/standby capabilities with the following activities:
General PostgreSQL clustering options
Overview of Red Hat Cluster Service
Identification of candidate databases for clustering
Identification of hardware for clustering
Analysis of uptime requirements and data latency
Implementation of clustering
Testing of clustering
PostgreSQL installation tips for RHCS
Various HA and DR setups for Postgres Plus Advanced Server -
Active – Passive OS HA Clustering
Log Shipping Replication (Hot Standby Mode)
Hot Streaming Replication (Hot Standby Mode)
EDB Postgres Plus Failover Manager
HA with read scaling (with pg-pool)
xDB Single Master Replication (SMR)
xDB Multi Master Replication (MMR)
Use Cases
Denish Patel deployed PostgreSQL on Amazon EC2 for a startup that had its entire IT architecture on the Amazon cloud. The initial deployment involved setting up master-slave configurations across two EC2 environments with issues like weekly instance failures and lack of monitoring. Patel then consolidated the environments, configured high availability using replication across availability zones and regions, implemented automation using Puppet, and added monitoring and backups to improve the stability and management of the PostgreSQL deployment.
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
In this session we will deep dive into the exciting features of Amazon RDS for PostgreSQL, including new versions of PostgreSQL releases, new extensions, larger instances. We will also show benchmarks of new RDS instance types, and their value proposition. We will also look at how high availability and read scaling works on RDS PostgreSQL. We will also explore lessons we have learned managing a large fleet of PostgreSQL instances, including important tunables.
This document discusses various goals, techniques, and solutions for replicating PostgreSQL databases. The goals covered are high availability, performance for reads and writes, supporting wide area networks, and handling offline peers. Techniques include master-slave and multi-master replication, proxies, and using standby systems. Specific solutions described are Slony-I, Slony-II, PGCluster, DBMirror, pgpool, WAL replication, Sequoia, DRBD, and shared storage. The document provides an overview of how each solution can help achieve different replication goals.
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataJignesh Shah
This document discusses PostgreSQL performance on multi-core systems with multi-terabyte data. It covers current market trends towards more cores and larger data sizes. Benchmark results show that PostgreSQL scales well on inserts up to a certain number of clients/cores but struggles with OLTP and TPC-E workloads due to lock contention. Issues are identified with sequential scans, index scans, and maintenance tasks like VACUUM as data sizes increase. The document proposes making PostgreSQL utilities and tools able to leverage multiple cores/processes to improve performance on modern hardware.
The document discusses PostgreSQL high availability and scaling options. It covers horizontal scaling using load balancing and data partitioning across multiple servers. It also covers high availability techniques like master-slave replication, warm standby servers with point-in-time recovery, and using a heartbeat to prevent multiple servers from becoming a master. The document recommends an initial architecture with two servers using warm standby and point-in-time recovery with a heartbeat for high availability. It suggests scaling the application servers horizontally later on if more capacity is needed.
This document summarizes a presentation about PostgreSQL replication. It discusses different replication terms like master/slave and primary/secondary. It also covers replication mechanisms like statement-based and binary replication. The document outlines how to configure and administer replication through files like postgresql.conf and recovery.conf. It discusses managing replication including failover, failback, remastering and replication lag. It also covers synchronous replication and cascading replication setups.
Highly efficient backups with percona xtrabackupNilnandan Joshi
Percona XtraBackup is an open source, free MySQL hot backup software that performs non-blocking backups for InnoDB and XtraDB databases. In this talk we'll describe below things.
- How it works with MySQL/Percona Server and what are the features provided
- Difference between Xtrabackup and Innobackupex
- How to take full/increment/partial backup and restore
- How to use features like streaming, compression, remote and compact backups
- How to troubleshoot the issue with xtrabackup
This document provides an overview of key differences between SQL Server and PostgreSQL databases. It covers topics such as extensions, cost, case sensitivity, operating systems, processor configuration, write-ahead logging (WAL), checkpoints, disabling writes, page corruptions, MVCC, vacuum, database snapshots, system databases, tables, indexes, statistics, triggers, functions, security, backups, replication, imports/exports, maintenance, and monitoring. The document aims to help SQL Server DBAs understand how to administer and work with PostgreSQL databases.
Cobbler - Fast and reliable multi-OS provisioningRUDDER
In a lot of companies, machine deployment is a delicate subject: every administrator has his own recipe, using CD-ROMs, static binary images deployed via the network, peer delegation ...
However, one solution makes the consensus when it comes to automated mass deployments ( except in the Cloud ): PXE boot. The main cons are that the deployment and the management of such a service is a pain, and every OS has its own installation automation system.
This is where Cobbler saves the day: it enables a painless and reliably to create a PXE service, usable on either virtual or physical machines, while beeing the most agnostic possible towards the target OSes and its preconfiguration system (preseed, kickstart, sysprep, ...) while offering the possibility to handle lots of configuration parameters in a modular fashion (network, partitionning, user accounts, configuration management agent...)
This conference aims to introduce the audience to the general concepts of Cobbler, and some scenarios where it would be a useful solution.
Out of the box replication in postgres 9.4Denish Patel
This document provides an overview of setting up out of the box replication in PostgreSQL 9.4 without third party tools. It discusses write-ahead logs (WAL), replication slots, pg_basebackup, and pg_receivexlog. The document then demonstrates setting up replication on VMs with pg_basebackup to initialize a standby server, configuration of primary and standby servers, and monitoring of replication.
O documento fornece instruções e diretrizes sobre técnicas de serviço de bar, incluindo regras de higiene, organização do espaço, equipamentos necessários e procedimentos padrão. É destacada a importância de manter os mais altos padrões de limpeza, organização e qualidade dos produtos para garantir a satisfação dos clientes e a reputação do estabelecimento.
O documento fornece dicas importantes para estudantes sobre como aprender de forma efetiva, incluindo ler o conteúdo com atenção, explorar ilustrações e aplicar o aprendizado na prática. Também apresenta detalhes sobre um curso de barman, incluindo a carga horária da primeira parte e os tópicos que serão abordados.
Cardapio iba (international bartender association)Mateus Rocha
Este documento fornece informações sobre vários coquetéis reconhecidos pela Associação Internacional de Bartenders (IBA), incluindo sua categoria, classificação, copo ideal e, em alguns casos, histórico. Os coquetéis descritos incluem o Americano, Brandy Alexander, B52, Bacardi Cocktail e Caipirinha.
O documento descreve a origem e evolução histórica dos bares, desde os primeiros estabelecimentos na França no século XVIII até se tornarem populares nos Estados Unidos. Detalha também os diferentes tipos de bares, equipamento, layout típico e tarefas diárias de um barman.
O documento descreve:
1) O mercado-alvo da empresa, composto por homens e mulheres de classe média entre 25 e 68 anos;
2) Os principais atributos desejados em fornecedores, como pontualidade, qualidade e preços competitivos;
3) Os principais concorrentes da empresa no ramo de buffets para eventos.
O documento classifica drinks de acordo com sua modalidade de preparação (coquetéis mexidos, batidos ou montados), categoria (curtos ou longos), classificação nutricional ou estimulante (apetite, digestivos, etc) e fornece exemplos como Manhatan, Cosmopolitan e Caipirinha.
Este documento fornece receitas de coquetéis que utilizam a cachaça Sagatiba. Inclui várias variações de caipirinhas, além de aperitivos, martinis, drinks longos e bebidas quentes e geladas. As receitas variam de clássicas a criações mais elaboradas com diversos ingredientes e frutas.
This document provides an overview of replicating a PostgreSQL database. It discusses setting up a primary server for reads and writes and standby servers that are kept in sync with the primary to serve as backups. The primary server writes data to its write-ahead log (WAL) files, which are streamed in real-time to the standby servers via WAL shipping. This allows the standby servers to keep an identical copy of the database. The document also covers configuration of both the primary and standby servers for replication as well as tools for testing the replication setup.
Out of the Box Replication in Postgres 9.4(PgConfUS)Denish Patel
This document contains notes from a presentation on PostgreSQL replication. It discusses write-ahead logs (WAL), replication history in PostgreSQL from versions 7.0 to 9.4, how to set up basic replication, tools for backups and monitoring replication, and demonstrates setting up replication without third party tools using pg_basebackup, replication slots, and pg_receivexlog. Contact information is provided for the presenter and information on their employer, Medallia, is included at the end.
Out of the box replication in postgres 9.4(pg confus)Denish Patel
This document contains notes from a presentation on PostgreSQL replication. It discusses write-ahead logs (WAL), replication history in PostgreSQL from versions 7.0 to 9.4, how to set up basic replication, tools for backups and monitoring replication, and demonstrates setting up replication without third party tools using pg_basebackup, replication slots, and pg_receivexlog. It also includes contact information for the presenter and an invitation to join the PostgreSQL Slack channel.
The Magic of Hot Streaming Replication, Bruce MomjianFuenteovejuna
This document discusses PostgreSQL 9.0's new capabilities for maintaining a current standby server and issuing read-only queries on the standby server. It covers how to configure continuous archiving from the primary server to the standby, and how to configure the standby for streaming replication so it can accept queries. The summary demonstrates streaming replication by creating and inserting a row on the primary which then appears on the standby.
PostgreSQL Hangout Replication Features v9.4Ashnikbiz
See the new enhancements in v9.4 which takes away the pain of guessing right wal_keep_segment
See the new time lagging replication capability in v9.4
Short intro to logical replication introduced in v9.4
This document discusses database backup and recovery concepts including:
1. Mean time between failures and mean time to recover should have high and low values respectively. Database failures can occur due to statements, processes, instances, and user errors.
2. Solutions for failures include adding filespaces, using PMON to recover from backups, restarting instances, and SQL commands like ALTER SYSTEM SWITCH LOGFILE.
3. Views like V$INSTANCE, V$FAST_START_SERVERS, V$BACKUP are queried to obtain database information for managing backups and different types of recovery including time-based, cancel-based, and change-based.
Freezer is an OpenStack backup and restore service that allows users to automate backup processes. It includes components like an API, scheduler, and agent. The scheduler retrieves backup jobs from the API and executes them via the agent. Freezer supports different backup types including file system, Cinder volume, and MySQL backups. It can store backups in OpenStack Swift, locally, or remotely using SSH. Backups can be restored locally or by recreating volumes/instances in Cinder/Nova.
This talk cover various advanced topics in the area of backups:
- incremental backups;
- archive management;
- backup validation;
- retention policies;
etc.
Based on these features, we'll compare various backup/recovery solutions for PostgreSQL.
This information will help you to choose the most appropriate tool for your system.
The document agenda includes topics on PostgreSQL WAL concepts, streaming replication, point-in-time recovery (PITR), connection pooling with PgBouncer, backups and restores, and PostgreSQL internal tables. Configuration details are provided for setting up streaming replication between a master and slave node, and using PgBouncer for connection pooling. Backup and restore commands using pg_dump and pg_basebackup are also listed.
The document outlines tips for troubleshooting CloudStack. It discusses ACS error codes, debugging in development, SSVM troubleshooting, and ACS ports. It also covers installation, configuration, log analysis, global parameters, the cloud database, reusing hypervisors, and best practices for CloudStack administrators. References and Q&A are included at the end.
Out of the Box Replication in Postgres 9.4(PgCon)Denish Patel
This document provides an overview of setting up out of the box replication in PostgreSQL 9.4 without third party tools. It discusses write-ahead logs (WAL), replication slots, pg_basebackup, pg_receivexlog and monitoring replication. The presenter then demonstrates setting up replication on VMs with pg_basebackup to initialize a standby, configuration of primary and standby servers, and monitoring replication status.
Out of the Box Replication in Postgres 9.4(PgCon)Denish Patel
The document provides an overview of out-of-the-box replication in PostgreSQL 9.4. It discusses PostgreSQL write-ahead logging (WAL), setting up basic streaming replication between a primary and standby server, taking base backups with pg_basebackup, and using replication slots and pg_receivexlog to archive WAL files without external tools. The presentation includes steps to set up a demo of this replication method on a virtual machine.
This document discusses PostgreSQL backups and disaster recovery. It covers the need for different types of backups like logical and physical backups. It discusses how to store backups and automate the backup process. The document also covers how to validate backups are working properly and tools that can be used. It emphasizes that both logical and physical backups are important to have for different recovery scenarios. Automation is recommended to manage the complex backup processes.
EWD 3 Training Course Part 34: QEWD Resilient ModeRob Tweed
This is part 34 of the EWD 3 Training Course. This presentation explains how you can add resilience to the message queue used by QEWD by making it save all incoming requests to your embedded Global Storage database. In Resilient Mode, QEWD also saves a copy of all responses sent to clients, so the stored activity information can be used as an audit trail.
PGDay.Amsterdam 2018 - Stefan Fercot - Save your data with pgBackRestPGDay.Amsterdam
Ever heard of Point-in-time recovery? pgBackRest is an awsome tool to handle backups, restores and even helps you build streaming replication ! This talk will introduce the tool, its basic features and how to use it.
This technical presentation by EDB Dave Thomas, Systems Engineer provides an overview of:
1) BGWriter/Writer Process
2) Wall Writer Process
3) Stats Collector Process
4) Autovacuum Launch Process
5) Syslogger Process/Logger process
6) Archiver Process
7) WAL Send/Receive Processes
Joshua D. Drake
Are you tired of not having a real solution for PITR? Enter PITRTools, a single and secure solution for using Point In Time Recovery for PostgreSQL.
LVOUG meetup #4 - Case Study 10g to 11gMaris Elsins
My presentation on a case study of 10g to 11g upgrade at LVOUG meetup #4 in 2012. Includes preserving execution plans by exporting them from 10g and importing as SQL Plan Baselines in 11gR2
This document discusses high availability, disaster recovery, and backup considerations for Microsoft Hyper-V virtual machines. It covers Hyper-V architecture, anatomy of a virtual machine, challenges with backing up virtual machines including transactional consistency, and different approaches to backups including file-level and image-level. It also discusses high availability options for Hyper-V like live migration and replication, and disaster recovery strategies ranging from days to immediate recovery depending on budget and needs.
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...PsychoTech Services
A proprietary approach developed by bringing together the best of learning theories from Psychology, design principles from the world of visualization, and pedagogical methods from over a decade of training experience, that enables you to: Learn better, faster!
06-20-2024-AI Camp Meetup-Unstructured Data and Vector DatabasesTimothy Spann
Tech Talk: Unstructured Data and Vector Databases
Speaker: Tim Spann (Zilliz)
Abstract: In this session, I will discuss the unstructured data and the world of vector databases, we will see how they different from traditional databases. In which cases you need one and in which you probably don’t. I will also go over Similarity Search, where do you get vectors from and an example of a Vector Database Architecture. Wrapping up with an overview of Milvus.
Introduction
Unstructured data, vector databases, traditional databases, similarity search
Vectors
Where, What, How, Why Vectors? We’ll cover a Vector Database Architecture
Introducing Milvus
What drives Milvus' Emergence as the most widely adopted vector database
Hi Unstructured Data Friends!
I hope this video had all the unstructured data processing, AI and Vector Database demo you needed for now. If not, there’s a ton more linked below.
My source code is available here
https://github.com/tspannhw/
Let me know in the comments if you liked what you saw, how I can improve and what should I show next? Thanks, hope to see you soon at a Meetup in Princeton, Philadelphia, New York City or here in the Youtube Matrix.
Get Milvused!
https://milvus.io/
Read my Newsletter every week!
https://github.com/tspannhw/FLiPStackWeekly/blob/main/141-10June2024.md
For more cool Unstructured Data, AI and Vector Database videos check out the Milvus vector database videos here
https://www.youtube.com/@MilvusVectorDatabase/videos
Unstructured Data Meetups -
https://www.meetup.com/unstructured-data-meetup-new-york/
https://lu.ma/calendar/manage/cal-VNT79trvj0jS8S7
https://www.meetup.com/pro/unstructureddata/
https://zilliz.com/community/unstructured-data-meetup
https://zilliz.com/event
Twitter/X: https://x.com/milvusio https://x.com/paasdev
LinkedIn: https://www.linkedin.com/company/zilliz/ https://www.linkedin.com/in/timothyspann/
GitHub: https://github.com/milvus-io/milvus https://github.com/tspannhw
Invitation to join Discord: https://discord.com/invite/FjCMmaJng6
Blogs: https://milvusio.medium.com/ https://www.opensourcevectordb.cloud/ https://medium.com/@tspann
https://www.meetup.com/unstructured-data-meetup-new-york/events/301383476/?slug=unstructured-data-meetup-new-york&eventId=301383476
https://www.aicamp.ai/event/eventdetails/W2024062014
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Marlon Dumas
This webinar discusses the limitations of traditional approaches for business process simulation based on had-crafted model with restrictive assumptions. It shows how process mining techniques can be assembled together to discover high-fidelity digital twins of end-to-end processes from event data.
Do People Really Know Their Fertility Intentions? Correspondence between Sel...Xiao Xu
Fertility intention data from surveys often serve as a crucial component in modeling fertility behaviors. Yet, the persistent gap between stated intentions and actual fertility decisions, coupled with the prevalence of uncertain responses, has cast doubt on the overall utility of intentions and sparked controversies about their nature. In this study, we use survey data from a representative sample of Dutch women. With the help of open-ended questions (OEQs) on fertility and Natural Language Processing (NLP) methods, we are able to conduct an in-depth analysis of fertility narratives. Specifically, we annotate the (expert) perceived fertility intentions of respondents and compare them to their self-reported intentions from the survey. Through this analysis, we aim to reveal the disparities between self-reported intentions and the narratives. Furthermore, by applying neural topic modeling methods, we could uncover which topics and characteristics are more prevalent among respondents who exhibit a significant discrepancy between their stated intentions and their probable future behavior, as reflected in their narratives.
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)Rebecca Bilbro
To honor ten years of PyData London, join Dr. Rebecca Bilbro as she takes us back in time to reflect on a little over ten years working as a data scientist. One of the many renegade PhDs who joined the fledgling field of data science of the 2010's, Rebecca will share lessons learned the hard way, often from watching data science projects go sideways and learning to fix broken things. Through the lens of these canon events, she'll identify some of the anti-patterns and red flags she's learned to steer around.
2. Database backups
A. SQL Dump
– .sql or custom format for pg_dump/pg_restore
– high CPU and disk demands
B. File system level backup
– offline database server or LVM / file system snapshot
C. Continuous Archiving and Point-in-Time Recovery
3. Point-in-Time Recovery
– Utilizes write ahead log (WAL)
– Pros
– ability to restore to a specific timestamp or transaction ID
– minimum CPU overhead
– Cons
– disk space requirements
– possibly long restore time
– several caveats
4. Common use cases
– Unintentional errors recovery
– Disaster recovery
– Sandbox server (BI, staging)
http://www.slideshare.net/openobject/odoo-disaster-recovery-with-barman
5. – Incremental backups (hard links)
– WAL compression
– Easy installation and config
– pip install barman
postgresql.conf:
wal_level = 'archive' # For PostgreSQL >= 9.0
archive_mode = on
archive_command = 'rsync -a %p barman@backup:WALS_DIRECTORY/%f'
6. Periodical and differential
backups
– Full backup (weekly / monthly)
– reduces time needed for recovery
# m h dom mon dow command
0 8 * * 1 barman backup SERVER_ID
– Differential backup (WAL files)
– continuos archiving (rsync / ssh)
8. Disaster recovery
– Recover to specified time
$ barman recover main 20150715T230232 /path/to/recover/directory --target-time
"2015-07-16 9:00:00 CEST”
– Run standalone Postgres server
$ pg_ctl -D /path/to/recover/directory start
2015-09-15 23:17:57 CEST LOG: database system was interrupted; last known up at 2015-09-14 08:05:06 CEST
2015-09-15 23:20:18 CEST LOG: redo starts at 52/5333C080
2015-09-15 23:20:19 CEST LOG: consistent recovery state reached at 52/543A28E8
2015-09-15 23:20:19 CEST LOG: consistent recovery state reached at 52/543A28E8
2015-09-15 23:21:11 CEST LOG: redo done at 53/1FFFE2E0
2015-09-15 23:21:11 CEST LOG: last completed transaction was at log time 2015-09-15 23:18:02.661924+02
2015-09-15 23:22:30 CEST LOG: MultiXact member wraparound protections are now enabled
2015-09-15 23:22:30 CEST LOG: database system is ready to accept connections
2015-09-15 23:22:30 CEST LOG: autovacuum launcher started