PostgreSQL 10 is not even one year old and PostgreSQL 11 is just around the corner. This is a (very personal) selection of improvements and new features with some demos. There are improvements to partitioning, parallel query and much more.
The document discusses the top 10 mistakes Oracle DBAs make when migrating to PostgreSQL. These include: configuring PostgreSQL settings based on Oracle without understanding PostgreSQL's different architecture, preserving uppercase metadata which is lowercase in PostgreSQL, unnecessary use of tablespaces, not using PostgreSQL's native features like PL languages and data types, treating exceptions the same between the databases, and not properly handling null values. It emphasizes that PostgreSQL and Oracle have meaningful differences and to avoid porting Oracle concepts directly without understanding PostgreSQL.
Using MySQL without Maatkit is like taking a photo without removing the camera's lens cap. Professional MySQL experts use this toolkit to help keep complex MySQL installations running smoothly and efficiently. This session will show you practical ways to use Maatkit every day.
This document summarizes PL/Java, which allows writing server-side functions in Java for PostgreSQL. It discusses how to define and deploy Java functions, configure PL/Java, handle parameters and return types, use JDBC from functions, and write triggers in Java. While compatible with Oracle's SQL/JRT standard, PL/Java has some limitations around memory usage and performance. It works best on Linux and is a stable option for adding Java code to PostgreSQL databases.
Top 10 Mistakes When Migrating From Oracle to PostgreSQLJim Mlodgenski
As more and more people are moving to PostgreSQL from Oracle, a pattern of mistakes is emerging. They can be caused by the tools being used or just not understanding how PostgreSQL is different than Oracle. In this talk we will discuss the top mistakes people generally make when moving to PostgreSQL from Oracle and what the correct course of action.
PostgreSQL Procedural Languages: Tips, Tricks and GotchasJim Mlodgenski
One of the most powerful features of PostgreSQL is its diversity of procedural languages, but with that diversity comes a lot of options.
Did you ever wonder:
- What all of those options are on the CREATE FUNCTION statement?
- How do they affect my application?
- Does my choice of procedural language affect the performance of my statements?
- Should I create a single trigger with IF statements or several simple triggers?
- How do I debug my code?
- Can I tell which line in my function is taking all of the time?
Crafting Beautiful CLI Applications in RubyNikhil Mungel
The document describes a presentation on crafting beautiful command line applications using Ruby. The presentation discusses why the command line is useful, what makes a good CLI app, and some key principles for CLI apps like least astonishment, reversibility, config files, graceful failure, and honoring piping. It also covers input/output in CLI apps and libraries like OptionParser, Mixlib::CLI, and Thor that can help build CLI apps in Ruby.
Les slides du Tools in Action que j'ai donné à Devoxx France 2019.
Le JDK est open source et son développement aussi. L'ensemble des nouveautés vit sur des branches Mercurial qui peuvent être téléchargées et compilées à la demande. Cela permet de jouer avec des fonctionnalités encore en chantier, de regarder comment les nouvelles syntaxes fonctionneront, bref, de jouer avec un JDK du futur, qui n'existera peut-être jamais. L'objet de ce tools in action est simple : montrer comment tout ceci fonctionne, de l'installation d'une machine Ubuntu à l'exécution d'un JDK recompilé maison.
Many IT operations teams are used to managing infrastructure manually or with simple one-off scripts. This manual work and lack of verifiable behavior results in many issues and in uncertainty. In software development, Test Driven Development (TDD) is well recognized for improving design, increasing code quality, and allowing refactoring and better knowledge sharing.
Similar benefits can be gained in infrastructure projects when infrastructure is treated as code, driving that code development with tests. Configuration management tools such as Chef and Puppet allow infrastructure to be easily described as code and provide a complete support to introduce and run tests. This can allow development and operations teams to collaborate and confidently deliver working infrastructure code.
The document discusses the top 10 mistakes Oracle DBAs make when migrating to PostgreSQL. These include: configuring PostgreSQL settings based on Oracle without understanding PostgreSQL's different architecture, preserving uppercase metadata which is lowercase in PostgreSQL, unnecessary use of tablespaces, not using PostgreSQL's native features like PL languages and data types, treating exceptions the same between the databases, and not properly handling null values. It emphasizes that PostgreSQL and Oracle have meaningful differences and to avoid porting Oracle concepts directly without understanding PostgreSQL.
Using MySQL without Maatkit is like taking a photo without removing the camera's lens cap. Professional MySQL experts use this toolkit to help keep complex MySQL installations running smoothly and efficiently. This session will show you practical ways to use Maatkit every day.
This document summarizes PL/Java, which allows writing server-side functions in Java for PostgreSQL. It discusses how to define and deploy Java functions, configure PL/Java, handle parameters and return types, use JDBC from functions, and write triggers in Java. While compatible with Oracle's SQL/JRT standard, PL/Java has some limitations around memory usage and performance. It works best on Linux and is a stable option for adding Java code to PostgreSQL databases.
Top 10 Mistakes When Migrating From Oracle to PostgreSQLJim Mlodgenski
As more and more people are moving to PostgreSQL from Oracle, a pattern of mistakes is emerging. They can be caused by the tools being used or just not understanding how PostgreSQL is different than Oracle. In this talk we will discuss the top mistakes people generally make when moving to PostgreSQL from Oracle and what the correct course of action.
PostgreSQL Procedural Languages: Tips, Tricks and GotchasJim Mlodgenski
One of the most powerful features of PostgreSQL is its diversity of procedural languages, but with that diversity comes a lot of options.
Did you ever wonder:
- What all of those options are on the CREATE FUNCTION statement?
- How do they affect my application?
- Does my choice of procedural language affect the performance of my statements?
- Should I create a single trigger with IF statements or several simple triggers?
- How do I debug my code?
- Can I tell which line in my function is taking all of the time?
Crafting Beautiful CLI Applications in RubyNikhil Mungel
The document describes a presentation on crafting beautiful command line applications using Ruby. The presentation discusses why the command line is useful, what makes a good CLI app, and some key principles for CLI apps like least astonishment, reversibility, config files, graceful failure, and honoring piping. It also covers input/output in CLI apps and libraries like OptionParser, Mixlib::CLI, and Thor that can help build CLI apps in Ruby.
Les slides du Tools in Action que j'ai donné à Devoxx France 2019.
Le JDK est open source et son développement aussi. L'ensemble des nouveautés vit sur des branches Mercurial qui peuvent être téléchargées et compilées à la demande. Cela permet de jouer avec des fonctionnalités encore en chantier, de regarder comment les nouvelles syntaxes fonctionneront, bref, de jouer avec un JDK du futur, qui n'existera peut-être jamais. L'objet de ce tools in action est simple : montrer comment tout ceci fonctionne, de l'installation d'une machine Ubuntu à l'exécution d'un JDK recompilé maison.
Many IT operations teams are used to managing infrastructure manually or with simple one-off scripts. This manual work and lack of verifiable behavior results in many issues and in uncertainty. In software development, Test Driven Development (TDD) is well recognized for improving design, increasing code quality, and allowing refactoring and better knowledge sharing.
Similar benefits can be gained in infrastructure projects when infrastructure is treated as code, driving that code development with tests. Configuration management tools such as Chef and Puppet allow infrastructure to be easily described as code and provide a complete support to introduce and run tests. This can allow development and operations teams to collaborate and confidently deliver working infrastructure code.
The why and how of moving to php 5.4/5.5Wim Godden
With PHP 5.5 out and many production environments still running 5.2 (or older), it's time to paint a clear picture on why everyone should move to 5.4 and 5.5 and how to get code ready for the latest version of PHP. In this talk, we'll look at some handy tools and techniques to ease the migration.
The document discusses strategic autovacuum configuration and monitoring in PostgreSQL. It begins by explaining the ACID properties and how MVCC and transactions work. It then discusses how to monitor workloads for heavily updated tables, adjust per-table autovacuum thresholds to prioritize those tables, monitor autovacuum behavior over time using logs and queries, and tune the autovacuum throttle settings based on that monitoring to optimize autovacuum performance. The key steps are to start with defaults, monitor workload changes, adjust settings for busy tables, continue monitoring, and refine settings as needed.
The document summarizes 11 new features in Oracle Database 11g Release 2. It discusses improvements to parallelism, analytics, external tables, recursive queries, and flashback features. Key points include automated parallel DML, improved analytic functions like LISTAGG, using external tables with preprocessors on directories, recursive queries with common table expressions, and enhanced time travel capabilities.
Taking Jenkins Pipeline to the Extremeyinonavraham
Slide deck from Jenkins User Conference Tel Aviv 2018.
Talking about suggested (best?) practices, tips and tricks, using Jenkins pipeline scripts with shared libraries, managing shared libraries, using docker compose, and more.
Introducing Command Line Applications with RubyNikhil Mungel
This document discusses building command line applications in Ruby. It covers using OptionParser and Mixlib::CLI for parsing arguments, Thor for building commands, testing CLI apps, interacting with the system via libraries like Mixlib::ShellOut, and logging to standard output and error. The target audience is developers looking to get started with Ruby command line programs.
Test: DML with NOLOGGING
NOLOGGING: Oracle will generate a minimal number of redo log entries in order to protect
the data dictionary, and the operation will probably run faster. Logging can be disabled at
the table level or the tablespace level.
Diapositivas de la charla dada por la gente de uno21.com.ar (@luke_ar y @matitanio) en la UP, el día 21/08/2012. Próximamente en otras universidades :)
♥ Play Framework is an open-source web application framework for Java and Scala that follows the model-view-controller (MVC) architectural pattern. It supports dependency injection, routing, and asynchronous programming. Some key features include routing, controllers, database access using Slick, evolutions for database schema changes, and support for functional programming concepts like Option, Either, and Future.
Custom deployments with sbt-native-packagerGaryCoady
sbt-native-packager offers a comprehensive approach to packaging artifacts with SBT. The user describes a generic layout, which can then be extended for different types of software and deployments. For example, it is flexible enough to describe both a Zip-based archive format, and an RPM package with appropriate Systemd configuration for a service.
This talk will cover the essentials needed to understand the design of sbt-native-packager, and how to extend its structure to create custom layouts and deployments.
The document discusses using Xdebug to debug PHP code. It begins with an overview of Xdebug, explaining that it is a PHP extension that allows for step-by-step debugging and profiling of PHP applications. It then covers how to set up Xdebug, including installing the extension, configuring an IDE like PhpStorm to work with it, setting breakpoints in code, and activating the debugger. Finally, it discusses how to use Xdebug, such as setting and working with breakpoints, stepping through code, and using watches and the console window. The overall document serves as an introduction to debugging PHP applications with Xdebug.
Basic - Oracle Edition Based Redefinition PresentationN/A
The document discusses Oracle Edition Based Redefinition, which allows upgrading an application while it is in use. Key points include: Edition Based Redefinition introduces editions, editioning views, and crossedition triggers to enable in-place application upgrades. Editions allow different versions of objects to coexist. Editioning views provide a transition between table structures. Crossedition triggers propagate transactions between editions to support concurrent usage of pre- and post-upgrade applications.
This document discusses using sampling to diagnose buffer busy wait issues in an Oracle database. It provides an example of using the v$session_wait view to identify the specific buffer busy wait type, file, and block number involved. This allows finding the impacted object and SQL statement. The example identifies an insert statement on a table with a single freelist as the cause. It recommends adding more freelists to improve concurrency for inserts on that table.
Slides of the talk we gave with David Delabassee at Oracle Code One 2018.
The FnProject was released about a year ago, proposing new ways to design asynchronous systems in a very flexible way. This tutorial shows you how to set up a Fn Server on a local development machine running Windows with a first simple function. Then we cover the design of multi-functions systems, connected together in an asynchronous way, leveraging the Fn Flow API. This API is first presented, including how it compares to the the Java 8 CompletableFuture API. We then set up a real example made of several functions, producing multiple results and see how we can leverage their asynchronous nature to make a non-blocking system, handling errors in a simple and intuitive way. Most of the examples are shown both on slides and live coding parts.
Automatic PHP 7 Compatibility Checking Using php7cc (and PHPCompatibility)Mark Niebergall
The document discusses automatic compatibility checking for PHP 7 using the php7cc and PHPCompatibility tools. It provides an overview of why upgrading to PHP 7 is important for performance, security and new features. It then demonstrates how to use the php7cc and PHPCompatibility tools to identify compatibility issues, including various configuration options. Common issues that can be missed by static analysis like test coverage are also discussed. An analysis of a PHP codebase found errors and warnings using both tools.
The document describes steps to identify and repair a block corruption in an Oracle database:
1. Use RMAN's Data Recovery Advisor to list, analyze, and repair the corruption. It identified a corrupted block in the USERS tablespace datafile and recommended restoring it from backup with block media recovery.
2. Verify the corruption using DBVERIFY and validate the tablespace with RMAN backup. Both tools confirmed the single corrupted block.
3. Restore the corrupted block using RMAN block recovery to fix the issue, and revalidate that the tablespace is no longer corrupted.
Nagios Conference 2011 - Nathan Vonnahme - Integrating Nagios With Test Drive...Nagios
Nathan Vonnahme's presentation on integrating Nagios with test driven development. The presentation was given during the Nagios World Conference North America held Sept 27-29th, 2011 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
This document discusses the Play framework and how it starts a Play application from the command line using the ProdServerStart object.
1. The ProdServerStart object's main method creates a RealServerProcess and calls the start method, passing the process.
2. The start method reads server configuration settings, creates a PID file, starts the Play application, and starts the HTTP server provider.
3. It loads the Play application using an ApplicationLoader, which can use Guice for dependency injection. The loaded application is then started by Play.
This document provides instructions on how to use HANGANALYZE and interpret HANGANALYZE trace files to diagnose hangs or lock waits in Oracle databases. It describes how to run HANGANALYZE on single node and RAC configurations, alternative diagnostic tools like HANGFG, and how to interpret the output files including identifying blocking sessions and chains.
Large scale machine learning projects with r suiteWit Jakuczun
Agenda for the workshop I conducted at ML@Enterprise conference that took place on 14th of December 2017 in Warsaw.
Machine Learning is not only about algorithms. Machine learning is about value and this can be achieved only after proper deployment of Machine Learning solutions. I will present best practices regarding managing R based ML projects. I will use our open-source tool R Suite (http://rsuite.io/). During the workshop I will talk about:
– project structure
– development cycle
– deployment
– test
PostgreSQL is an open-source relational database management system that runs on Linux, Unix, Windows and Mac OS. It supports SQL queries, transactions, foreign keys, triggers and views. To install PostgreSQL, download the installer package for your platform and run through the installation process, which sets up the database files and creates a default database and user account. The psql command-line tool can then be used to interact with and administer the PostgreSQL database using SQL commands.
The GPars (Groovy Parallel Systems) project provides multiple abstractions for concurrent, parallel programming in Groovy and Java. Rather than dealing directly with threads, synchronization, and locks, or even the java.util.concurrent classes added in Java 5, the project allows you to think in terms of actors, data flows, or composable asynchronous functions (to name a few).
In this talk, I covered the basics of GPars, including what it's like to learn to use it. Although I've done a fair amount of concurrent programming, I've just started using GPars. As such, this talk should be suitable for Groovy beginners.
The why and how of moving to php 5.4/5.5Wim Godden
With PHP 5.5 out and many production environments still running 5.2 (or older), it's time to paint a clear picture on why everyone should move to 5.4 and 5.5 and how to get code ready for the latest version of PHP. In this talk, we'll look at some handy tools and techniques to ease the migration.
The document discusses strategic autovacuum configuration and monitoring in PostgreSQL. It begins by explaining the ACID properties and how MVCC and transactions work. It then discusses how to monitor workloads for heavily updated tables, adjust per-table autovacuum thresholds to prioritize those tables, monitor autovacuum behavior over time using logs and queries, and tune the autovacuum throttle settings based on that monitoring to optimize autovacuum performance. The key steps are to start with defaults, monitor workload changes, adjust settings for busy tables, continue monitoring, and refine settings as needed.
The document summarizes 11 new features in Oracle Database 11g Release 2. It discusses improvements to parallelism, analytics, external tables, recursive queries, and flashback features. Key points include automated parallel DML, improved analytic functions like LISTAGG, using external tables with preprocessors on directories, recursive queries with common table expressions, and enhanced time travel capabilities.
Taking Jenkins Pipeline to the Extremeyinonavraham
Slide deck from Jenkins User Conference Tel Aviv 2018.
Talking about suggested (best?) practices, tips and tricks, using Jenkins pipeline scripts with shared libraries, managing shared libraries, using docker compose, and more.
Introducing Command Line Applications with RubyNikhil Mungel
This document discusses building command line applications in Ruby. It covers using OptionParser and Mixlib::CLI for parsing arguments, Thor for building commands, testing CLI apps, interacting with the system via libraries like Mixlib::ShellOut, and logging to standard output and error. The target audience is developers looking to get started with Ruby command line programs.
Test: DML with NOLOGGING
NOLOGGING: Oracle will generate a minimal number of redo log entries in order to protect
the data dictionary, and the operation will probably run faster. Logging can be disabled at
the table level or the tablespace level.
Diapositivas de la charla dada por la gente de uno21.com.ar (@luke_ar y @matitanio) en la UP, el día 21/08/2012. Próximamente en otras universidades :)
♥ Play Framework is an open-source web application framework for Java and Scala that follows the model-view-controller (MVC) architectural pattern. It supports dependency injection, routing, and asynchronous programming. Some key features include routing, controllers, database access using Slick, evolutions for database schema changes, and support for functional programming concepts like Option, Either, and Future.
Custom deployments with sbt-native-packagerGaryCoady
sbt-native-packager offers a comprehensive approach to packaging artifacts with SBT. The user describes a generic layout, which can then be extended for different types of software and deployments. For example, it is flexible enough to describe both a Zip-based archive format, and an RPM package with appropriate Systemd configuration for a service.
This talk will cover the essentials needed to understand the design of sbt-native-packager, and how to extend its structure to create custom layouts and deployments.
The document discusses using Xdebug to debug PHP code. It begins with an overview of Xdebug, explaining that it is a PHP extension that allows for step-by-step debugging and profiling of PHP applications. It then covers how to set up Xdebug, including installing the extension, configuring an IDE like PhpStorm to work with it, setting breakpoints in code, and activating the debugger. Finally, it discusses how to use Xdebug, such as setting and working with breakpoints, stepping through code, and using watches and the console window. The overall document serves as an introduction to debugging PHP applications with Xdebug.
Basic - Oracle Edition Based Redefinition PresentationN/A
The document discusses Oracle Edition Based Redefinition, which allows upgrading an application while it is in use. Key points include: Edition Based Redefinition introduces editions, editioning views, and crossedition triggers to enable in-place application upgrades. Editions allow different versions of objects to coexist. Editioning views provide a transition between table structures. Crossedition triggers propagate transactions between editions to support concurrent usage of pre- and post-upgrade applications.
This document discusses using sampling to diagnose buffer busy wait issues in an Oracle database. It provides an example of using the v$session_wait view to identify the specific buffer busy wait type, file, and block number involved. This allows finding the impacted object and SQL statement. The example identifies an insert statement on a table with a single freelist as the cause. It recommends adding more freelists to improve concurrency for inserts on that table.
Slides of the talk we gave with David Delabassee at Oracle Code One 2018.
The FnProject was released about a year ago, proposing new ways to design asynchronous systems in a very flexible way. This tutorial shows you how to set up a Fn Server on a local development machine running Windows with a first simple function. Then we cover the design of multi-functions systems, connected together in an asynchronous way, leveraging the Fn Flow API. This API is first presented, including how it compares to the the Java 8 CompletableFuture API. We then set up a real example made of several functions, producing multiple results and see how we can leverage their asynchronous nature to make a non-blocking system, handling errors in a simple and intuitive way. Most of the examples are shown both on slides and live coding parts.
Automatic PHP 7 Compatibility Checking Using php7cc (and PHPCompatibility)Mark Niebergall
The document discusses automatic compatibility checking for PHP 7 using the php7cc and PHPCompatibility tools. It provides an overview of why upgrading to PHP 7 is important for performance, security and new features. It then demonstrates how to use the php7cc and PHPCompatibility tools to identify compatibility issues, including various configuration options. Common issues that can be missed by static analysis like test coverage are also discussed. An analysis of a PHP codebase found errors and warnings using both tools.
The document describes steps to identify and repair a block corruption in an Oracle database:
1. Use RMAN's Data Recovery Advisor to list, analyze, and repair the corruption. It identified a corrupted block in the USERS tablespace datafile and recommended restoring it from backup with block media recovery.
2. Verify the corruption using DBVERIFY and validate the tablespace with RMAN backup. Both tools confirmed the single corrupted block.
3. Restore the corrupted block using RMAN block recovery to fix the issue, and revalidate that the tablespace is no longer corrupted.
Nagios Conference 2011 - Nathan Vonnahme - Integrating Nagios With Test Drive...Nagios
Nathan Vonnahme's presentation on integrating Nagios with test driven development. The presentation was given during the Nagios World Conference North America held Sept 27-29th, 2011 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
This document discusses the Play framework and how it starts a Play application from the command line using the ProdServerStart object.
1. The ProdServerStart object's main method creates a RealServerProcess and calls the start method, passing the process.
2. The start method reads server configuration settings, creates a PID file, starts the Play application, and starts the HTTP server provider.
3. It loads the Play application using an ApplicationLoader, which can use Guice for dependency injection. The loaded application is then started by Play.
This document provides instructions on how to use HANGANALYZE and interpret HANGANALYZE trace files to diagnose hangs or lock waits in Oracle databases. It describes how to run HANGANALYZE on single node and RAC configurations, alternative diagnostic tools like HANGFG, and how to interpret the output files including identifying blocking sessions and chains.
Large scale machine learning projects with r suiteWit Jakuczun
Agenda for the workshop I conducted at ML@Enterprise conference that took place on 14th of December 2017 in Warsaw.
Machine Learning is not only about algorithms. Machine learning is about value and this can be achieved only after proper deployment of Machine Learning solutions. I will present best practices regarding managing R based ML projects. I will use our open-source tool R Suite (http://rsuite.io/). During the workshop I will talk about:
– project structure
– development cycle
– deployment
– test
PostgreSQL is an open-source relational database management system that runs on Linux, Unix, Windows and Mac OS. It supports SQL queries, transactions, foreign keys, triggers and views. To install PostgreSQL, download the installer package for your platform and run through the installation process, which sets up the database files and creates a default database and user account. The psql command-line tool can then be used to interact with and administer the PostgreSQL database using SQL commands.
The GPars (Groovy Parallel Systems) project provides multiple abstractions for concurrent, parallel programming in Groovy and Java. Rather than dealing directly with threads, synchronization, and locks, or even the java.util.concurrent classes added in Java 5, the project allows you to think in terms of actors, data flows, or composable asynchronous functions (to name a few).
In this talk, I covered the basics of GPars, including what it's like to learn to use it. Although I've done a fair amount of concurrent programming, I've just started using GPars. As such, this talk should be suitable for Groovy beginners.
Стажировка 2016-07-27 02 Денис Нелюбин. PostgreSQL и jsonbSmartTools
The document discusses using PostgreSQL and JSONB data. It covers installing PostgreSQL, connecting to a database, configuring network access and authentication, creating a database and user, inserting and querying JSONB data using operators like ->> and ->, updating and deleting rows, and creating a functional index to query on fields within the JSONB data.
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuRedis Labs
Postgres and Redis Sitting in a Tree | In today’s world of polyglot persistence, it’s likely that companies will be using multiple data stores for storing and working with data based on the use case. Typically a company will
start with a relational database like Postgres and then add Redis for more high velocity use-cases. What if you could tie the two systems together to enable so much more?
MySQL 5.7 Tutorial Dutch PHP Conference 2015Dave Stokes
MySQL 5.7 is the latest version of the MySQL database. It includes new features such as support for JSON as a native data type with functions for manipulating JSON documents. Security has also been improved with secure defaults, password rotation/expiration controls, and SSL encryption enabled by default for the C client library. The release candidate for 5.7 was released in April 2015 and includes patches, contributions, and enhancements over previous versions.
MySQL 5.7. Tutorial - Dutch PHP Conference 2015Dave Stokes
MySQL 5.7 is the latest version of the MySQL database. It includes new features such as support for JSON as a native data type with functions for manipulating JSON documents. Security has also been improved with secure defaults, password rotation/expiration controls, and SSL encryption enabled by default for the C client library. The release candidate for 5.7 was released in April 2015 and includes patches, contributions, and enhancements over previous versions.
Mark Wong
pg_proctab is a collection of PostgreSQL stored functions that provide access to the operating system process table using SQL. We'll show you which functions are available and where they collect the data, and give examples of their use to collect processor and I/O statistics on SQL queries.
pg_proctab: Accessing System Stats in PostgreSQLMark Wong
pg_proctab is a collection of PostgreSQL stored functions that provide access to the operating system process table using SQL. We'll show you which functions are available and where they collect the data, and give examples of their use to collect processor and I/O statistics on SQL queries.
This document provides an overview of PostgreSQL topics including:
- Installation and configuration best practices such as using package management and configuring logging
- Routine maintenance activities like vacuuming and backups
- Upgrades and the differences between major, minor, and bugfix versions
- Advanced SQL topics like window functions, common table expressions, and querying slow queries
Reproducible Computational Research in RSamuel Bosch
A short presentation with pointers on getting started with reproducible computational research in R. Some of the topics include git, R package development, document generation with R markdown, saving plots, saving tables and using packrat.
This document provides instructions for setting up DBACockpit in SAP Solution Manager to monitor remote databases. It describes downloading the necessary database libraries, installing the Oracle client, configuring tnsnames.ora and sqlnet.ora, setting environment variables, testing the connection, and importing the databases into DBACockpit from the system landscape directory.
Large scale machine learning projects with R SuiteWLOG Solutions
Presentation from a workshop delivered by company's CEO Wit Jakuczun during ML@Enterprise conference that took place on 14th of December 2017 in Warsaw. Check https://github.com/WLOGSolutions/MLForum2017) for R code generated during this workshop.
Machine Learning is not only about algorithms. Machine learning is about value and this can be achieved only after proper deployment of Machine Learning solutions. I will present best practices regarding managing R based ML projects. I will use our open-source tool R Suite (http://rsuite.io/). During the workshop I will talk about:
– project structure
– development cycle
– deployment
– test
Javascript unit testing, yes we can e bigAndy Peterson
This document discusses test-driven development for JavaScript using ScrewUnit and BlueRidge. It introduces the Carbon Five consulting firm and covers why JavaScript unit testing is important. It then demonstrates how to write behavioral tests using ScrewUnit's BDD style and shows a live example testing a wizard component. Some challenges of JavaScript testing like DOM cleanup and structure are addressed. The document emphasizes that JavaScript testing is possible and can be integrated into the development process.
This document provides an overview and introduction to PostgreSQL for new users. It covers getting started with PostgreSQL, including installing it, configuring authentication and logging, upgrading to new versions, routine maintenance tasks, hardware recommendations, availability and scalability options, and query tuning and optimization. The document is presented as a slide deck with different sections labeled by letters (e.g. K-0, S-0, U-0).
The document discusses deploying Firebase Cloud Functions using Firebase CLI commands. It also discusses types for defining Cloud Function interfaces and callable functions. There are examples of managing Algolia indexes using Cloud Functions and security rules for Firestore.
The Ring programming language version 1.7 book - Part 31 of 196Mahmoud Samir Fayed
The document provides documentation on ODBC functions in Ring for connecting to databases and executing SQL queries. It describes functions for initializing an ODBC connection, getting lists of drivers and data sources, connecting and disconnecting from databases, executing queries, fetching and retrieving result sets, and performing transactions with commit and rollback. It also shows examples of using these functions to connect to an Access database, retrieve table and column metadata, insert records, and save/retrieve an image from a database table.
The document discusses the different states that a package's contents can be stored in, including as a source, bundle, binary, or installed in an R library or online repository. It also lists several functions that can be used to move a package between these states, such as install.packages(), devtools::install(), and library(). The bottom portion provides a cheat sheet on common parts of an R package like the DESCRIPTION file, namespaces, documentation, data, testing, and more.
Similar to PGDay.Amsterdam 2018 - Daniel Westermann - What we already know about PostgreSQL 11 (20)
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
20 Comprehensive Checklist of Designing and Developing a WebsitePixlogix Infotech
Dive into the world of Website Designing and Developing with Pixlogix! Looking to create a stunning online presence? Look no further! Our comprehensive checklist covers everything you need to know to craft a website that stands out. From user-friendly design to seamless functionality, we've got you covered. Don't miss out on this invaluable resource! Check out our checklist now at Pixlogix and start your journey towards a captivating online presence today.
2. About me
12.07.2018 Page 2What we (already) know about PostgreSQL 11
Daniel Westermann
Delivery Manager
Open Infrastructure Technology Leader
+41 79 927 24 46
daniel.westermann[at]dbi-services.com
3. Who we are
12.07.2018 Page 3What we (already) know about PostgreSQL 11
Experts At Your Service
> Over 60 specialists in IT infrastructure
> Certified, experienced, passionate
Based In Switzerland
> 100% self-financed Swiss company
> Over CHF10 mio. Turnover
Leading In Infrastructure Services
> More than 150 customers in CH, D & F
> Over 50 SLAs dbi FlexService contracted
4. Agenda
1.How you can test development features
2.What is (probably) coming with PostgreSQL 11?
3.Conclusion
12.07.2018 Page 4What we (already) know about PostgreSQL 11
5. 12.07.2018 Page 5What we (already) know about PostgreSQL 11
1
2
3
How you can test development features
6. 12.07.2018What we (already) know about PostgreSQL 11 Page 6
How you can test development features
Nightly development snapshots
When you want to test something in the development tree
> https://www.postgresql.org/download/
> https://www.postgresql.org/download/snapshots/
7. 12.07.2018What we (already) know about PostgreSQL 11 Page 7
How you can test development features
Nightly development snapshots
https://www.postgresql.org/ftp/snapshot/dev
8. 12.07.2018What we (already) know about PostgreSQL 11 Page 8
How you can test development features
Nightly development snapshots
How can you run that?
$ wget https://ftp.postgresql.org/pub/snapshot/dev/postgresql-snapshot.tar.bz2
$ tar -axf postgresql-snapshot.tar.bz2
$ cd postgresql-*
$ ./configure --prefix=/tmp/postgresql-test
$ make -j 4 all
$ make install
$ cd contrib
$ make -j 4 install
$ /tmp/postgresql-test/bin/initdb -D /tmp/testpg
$ /tmp/postgresql-test/bin/pg_ctl -D /tmp/testpg -l /dev/null start
$ /tmp/postgresql-test/bin/psql -c "select version()" postgres
version
----------------------------------------------------------------------------------------
PostgreSQL 11beta1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 , 64-bit
(1 row)
9. 12.07.2018What we (already) know about PostgreSQL 11 Page 9
How you can test development features
Working with git
The PostgreSQL source code is managed in a git repository
> https://git.postgresql.org/gitweb/
> To clone the git repository
$ git clone git://git.postgresql.org/git/postgresql.git
$ cd postgresql
$ ./configure --prefix=/tmp/postgresql-test
$ make -j 4 all
$ make install
$ cd contrib
$ make -j 4 install
$ /tmp/postgresql-test/bin/initdb -D /tmp/testpg
$ /tmp/postgresql-test/bin/pg_ctl -D /tmp/testpg -l /dev/null start
$ /tmp/postgresql-test/bin/psql -c "select version()" postgres
version
----------------------------------------------------------------------------------------
PostgreSQL 11beta1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 , 64-bit
(1 row)
10. 12.07.2018What we (already) know about PostgreSQL 11 Page 10
How you can test development features
What to test
And now? What do you want to test?
$ /tmp/postgresql-test/bin/psql -X postgres
psql (11beta1)
Type "help" for help.
postgres=# select '???????';
?column?
----------
???????
(1 row)
11. 12.07.2018What we (already) know about PostgreSQL 11 Page 11
How you can test development features
The development documentation
Either have a look at the development documentation
> https://www.postgresql.org/docs/devel/static/index.html
12. 12.07.2018What we (already) know about PostgreSQL 11 Page 12
How you can test development features
Git web interface
Or check the latest commits in git
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=summary
13. 12.07.2018What we (already) know about PostgreSQL 11 Page 13
How you can test development features
Git web interface
Select one which seems interesting
Read the commit message and test
14. 12.07.2018What we (already) know about PostgreSQL 11 Page 14
How you can test development features
Testing features not yet committed
But what when you want to test something that is not yet committed?
> https://commitfest.postgresql.org/
15. 12.07.2018What we (already) know about PostgreSQL 11 Page 15
How you can test development features
Testing features not yet committed
Select one which seems interesting
Read, apply patch and test
16. 12.07.2018What we (already) know about PostgreSQL 11 Page 16
How you can test development features
Testing features not yet committed
How to apply a patch?
> https://wiki.postgresql.org/wiki/Working_with_Git
$ git checkout -b feature-to-review
$ patch -p1 < feature.patch
17. 12.07.2018What we (already) know about PostgreSQL 11 Page 17
How you can test development features
Testing features not yet committed
Next year, probably in June, probably here ...
It is up to someone of you to present the PostgreSQL 12 new
features!
No, no excuses, now you know how to do it!
18. 12.07.2018 Page 18What we (already) know about PostgreSQL 11
1
2
3
What is (probably) coming with
PostgreSQL11
19. 12.07.2018What we (already) know about PostgreSQL 11 Page 19
Demo time
See it, feel it, understand it
We will skip this chapter and do a demo instead
> What is in the demo is also on the slides
20. 12.07.2018What we (already) know about PostgreSQL 11 Page 20
PostgreSQL 11 new features
psql
Did you ever face this?
What most newcomers miss
$ psql -X postgres
psql (10.3)
Type "help" for help.
postgres=# addad
postgres-# adass
postgres-# exit
postgres-# quit
postgres-# let me out
$ psql -X postgres
psql (10.3)
Type "help" for help.
21. 12.07.2018What we (already) know about PostgreSQL 11 Page 21
PostgreSQL 11 new features
psql
What most newcomers miss
$ psql -X postgres
psql (10.3)
Type "help" for help.
postgres=# help
You are using psql, the command-line interface to PostgreSQL.
Type: copyright for distribution terms
h for help with SQL commands
? for help with psql commands
g or terminate with semicolon to execute query
q to quit
22. 12.07.2018What we (already) know about PostgreSQL 11 Page 22
PostgreSQL 11 new features
psql
My favorite new feature for those not willing to read
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# exit
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# quit
$
23. 12.07.2018What we (already) know about PostgreSQL 11 Page 23
PostgreSQL 11 new features
psql
My favorite new feature for those not willing to read
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# assa
postgres-# asas
postgres-# quit
Use q to quit.
postgres-#
24. 12.07.2018What we (already) know about PostgreSQL 11 Page 24
PostgreSQL 11 new features
Partitioning improvements
What happens when you try to update the partition key in PostgreSQL 10?
$ psql -X postgres
psql (10.0)
Type "help" for help.
postgres=# select version();
postgres=# create table part ( a int, list varchar(5) ) partition by list (list);
postgres=# create table part_1 partition of part for values in ('beer');
postgres=# create table part_2 partition of part for values in ('wine');
postgres=# d+ part
postgres=# insert into part (a,list) select i,'beer' from generate_series(1,10) i;
postgres=# insert into part (a,list) select i,'wine' from generate_series(11,20) i;
postgres=# update part set list = 'beer' where a = 15;
ERROR: new row for relation "part_2" violates partition constraint
DETAIL: Failing row contains (15, beer).
25. 12.07.2018What we (already) know about PostgreSQL 11 Page 25
PostgreSQL 11 new features
Partitioning improvements
Updating the partition key now works in PostgreSQL 11
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# select version();
postgres=# create table part ( a int, list varchar(5) ) partition by list (list);
postgres=# create table part_1 partition of part for values in ('beer');
postgres=# create table part_2 partition of part for values in ('wine');
postgres=# d+ part
postgres=# insert into part (a,list) select i,'beer' from generate_series(1,10) i;
postgres=# insert into part (a,list) select i,'wine' from generate_series(11,20) i;
postgres=# update part set list = 'beer' where a = 15;
UPDATE 1
26. 12.07.2018What we (already) know about PostgreSQL 11 Page 26
PostgreSQL 11 new features
Partitioning improvements
What happens when you try to create an index on a partitioned table in PostgreSQL 10?
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# select version();
postgres=# create table part ( a int, list varchar(5) ) partition by list (list);
postgres=# create table part_1 partition of part for values in ('beer');
postgres=# create table part_2 partition of part for values in ('wine');
postgres=# d+ part
postgres=# insert into part (a,list) select i,'beer' from generate_series(1,10) i;
postgres=# insert into part (a,list) select i,'wine' from generate_series(11,20) i;
postgres=# create index i_test on part (a);
ERROR: cannot create index on partitioned table "part"
27. 12.07.2018What we (already) know about PostgreSQL 11 Page 27
PostgreSQL 11 new features
Partitioning improvements
Creating an index on a partitioned table works PostgreSQL 11
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# select version();
postgres=# create table part ( a int, list varchar(5) ) partition by list (list);
postgres=# create table part_1 partition of part for values in ('beer');
postgres=# create table part_2 partition of part for values in ('wine');
postgres=# d+ part
postgres=# insert into part (a,list) select i,'beer' from generate_series(1,10) i;
postgres=# insert into part (a,list) select i,'wine' from generate_series(11,20) i;
postgres=# create index i_test on part (a);
CREATE INDEX
28. 12.07.2018What we (already) know about PostgreSQL 11 Page 28
PostgreSQL 11 new features
Partitioning improvements
Insert on conflict...do, against a partitioned tabled in PostgreSQL 10
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# select version();
postgres=# create table part ( a int, list varchar(5) ) partition by list (list);
postgres=# create table part_1 partition of part for values in ('beer');
postgres=# create table part_2 partition of part for values in ('wine');
postgres=# d+ part
postgres=# insert into part (a,list) select i,'beer' from generate_series(1,10) i;
postgres=# insert into part (a,list) select i,'wine' from generate_series(11,20) i;
postgres=# insert into part(a,list) values (1,'beer')
on conflict (a,list) do nothing;
ERROR: ON CONFLICT clause is not supported with partitioned tables
29. 12.07.2018What we (already) know about PostgreSQL 11 Page 29
PostgreSQL 11 new features
Partitioning improvements
Insert on conflict...do, in PostgreSQL 11
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# select version();
postgres=# create table part ( a int, list varchar(5) ) partition by list (list);
postgres=# create table part_1 partition of part for values in ('beer');
postgres=# create table part_2 partition of part for values in ('wine');
postgres=# d+ part
postgres=# insert into part (a,list) select i,'beer' from generate_series(1,10) i;
postgres=# insert into part (a,list) select i,'wine' from generate_series(11,20) i;
postgres=# insert into part(a,list) values (1,'beer')
on conflict (a,list) do nothing;
INSERT 0 0
30. 12.07.2018What we (already) know about PostgreSQL 11 Page 30
PostgreSQL 11 new features
Partitioning improvements
What is partition pruning?
Avoiding to scan partitions which are known to not contain any mathing tuples
> In PostgreSQL 10 this can be only done at planning time
> PostgreSQL 11 will be able to do partition pruning at execution time as well
31. 12.07.2018What we (already) know about PostgreSQL 11 Page 31
PostgreSQL 11 new features
Partitioning improvements
PostgreSQL 10 can already do that at planning time
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# select version();
postgres=# create table part ( a int, list varchar(5) ) partition by list (list);
postgres=# create table part_1 partition of part for values in ('beer');
postgres=# create table part_2 partition of part for values in ('wine');
postgres=# d+ part
postgres=# insert into part (a,list) select i,'beer' from generate_series(1,10) i;
postgres=# insert into part (a,list) select i,'wine' from generate_series(11,20) i;
postgres=# explain select * from part where list = 'beer';
QUERY PLAN
--------------------------------------------------------------
Append (cost=0.00..28.12 rows=7 width=28)
-> Seq Scan on part_1 (cost=0.00..28.12 rows=7 width=28)
Filter: ((list)::text = 'beer'::text)
32. 12.07.2018What we (already) know about PostgreSQL 11 Page 32
PostgreSQL 11 new features
Partitioning improvements
PostgreSQL 10 can not do this (subquery)
postgres=# explain select * from part where list = ( select 'beer' );
QUERY PLAN
--------------------------------------------------------------
Append (cost=0.01..56.26 rows=14 width=28)
InitPlan 1 (returns $0)
-> Result (cost=0.00..0.01 rows=1 width=32)
-> Seq Scan on part_1 (cost=0.00..28.12 rows=7 width=28)
Filter: ((list)::text = $0)
-> Seq Scan on part_2 (cost=0.00..28.12 rows=7 width=28)
Filter: ((list)::text = $0)
33. 12.07.2018What we (already) know about PostgreSQL 11 Page 33
PostgreSQL 11 new features
Partitioning improvements
Doing the same in PostgreSQL 11
postgres=# explain select * from part where list = ( select 'beer' );
QUERY PLAN
--------------------------------------------------------------
Append (cost=0.01..56.26 rows=14 width=28)
InitPlan 1 (returns $0)
-> Result (cost=0.00..0.01 rows=1 width=32)
-> Seq Scan on part_1 (cost=0.00..28.12 rows=7 width=28)
Filter: ((list)::text = $0)
-> Seq Scan on part_2 (cost=0.00..28.12 rows=7 width=28)
Filter: ((list)::text = $0)
Didn't I tell you that works in PostgreSQL 11
34. 12.07.2018What we (already) know about PostgreSQL 11 Page 34
PostgreSQL 11 new features
Partitioning improvements
Pruning at execution times means the query must be executed
postgres=# explain (analyze) select * from part where list = ( select 'beer' );
QUERY PLAN
------------------------------------------------------------------------------------------
Append (cost=0.01..56.33 rows=14 width=28) (actual time=0.010..0.010 rows=0 loops=1)
InitPlan 1 (returns $0)
-> Result ( rows=1 width=32) (actual time=0.001..0.001 rows=1 loops=1)
-> Seq Scan on part_1 (rows=7 width=28) (actual time=0.003..0.003 rows=0 loops=1)
Filter: ((list)::text = $0)
-> Seq Scan on part_2 (cost=0.00..28.12 rows=7 width=28) (never executed)
Filter: ((list)::text = $0)
postgres=# show enable_partition_pruning;
enable_partition_pruning
--------------------------
on
35. 12.07.2018What we (already) know about PostgreSQL 11 Page 35
PostgreSQL 11 new features
Partitioning improvements
PostgreSQL 11 will come with hash partitioning
> Tuples will be evenly distributed by the remainder of the modules
postgres=# create table part2 ( a int, list varchar(5) ) partition by hash (a);
postgres=# create table part2_1
partition of part2 FOR VALUES WITH (MODULUS 3, REMAINDER 0);
postgres=# create table part2_2
partition of part2 FOR VALUES WITH (MODULUS 3, REMAINDER 1);
postgres=# create table part2_3
partition of part2 FOR VALUES WITH (MODULUS 3, REMAINDER 2);
postgres=# insert into part2 (a,list) values (1,'beer');
postgres=# insert into part2 (a,list) values (2,'whine');
postgres=# insert into part2 (a,list) values (3,'schnaps');
36. 12.07.2018What we (already) know about PostgreSQL 11 Page 36
PostgreSQL 11 new features
Partitioning improvements
PostgreSQL 11 will come with hash partitioning, evenly distributed data
postgres=# select * from only part2_1;
a | list
---+-------
2 | whine
(1 row)
postgres=# select * from only part2_2;
a | list
---+---------
3 | schnaps
(1 row)
postgres=# select * from only part2_3;
a | list
---+------
1 | beer
(1 row)
37. 12.07.2018What we (already) know about PostgreSQL 11 Page 37
PostgreSQL 11 new features
Indexes
PostgreSQL 11 will have support for indexes with include columns
postgres=# create table tt (a int, b text, c date );
postgres=# insert into tt (a,b,c)
postgres=# select aa.*, md5(aa::text), now()
postgres=# from generate_series(1,1000000) aa;
postgres=# create unique index i1 on tt(a) include(b);
postgres=# d+ tt
Table "public.tt"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------+---------+-----------+----------+---------+----------+--------------+------------
a | integer | | | | plain | |
b | text | | | | extended | |
c | date | | | | plain | |
Indexes:
"i1" UNIQUE, btree (a) INCLUDE (b)
38. 12.07.2018What we (already) know about PostgreSQL 11 Page 38
PostgreSQL 11 new features
New hashing functions
New SHA hashing functions show up in PostgreSQL 11
$ psql -X postgres
psql (11devel)
Type "help" for help.
postgres=# df sha*
List of functions
Schema | Name | Result data type | Argument data types | Type
------------+--------+------------------+---------------------+------
pg_catalog | sha224 | bytea | bytea | func
pg_catalog | sha256 | bytea | bytea | func
pg_catalog | sha384 | bytea | bytea | func
pg_catalog | sha512 | bytea | bytea | func
(4 rows)
39. 12.07.2018What we (already) know about PostgreSQL 11 Page 39
PostgreSQL 11 new features
Procedures with transaction control
In PostgreSQL 10 you can create functions which return void/nothing
... but you can not do this:
postgres=# CREATE FUNCTION dummy_func (id int) RETURNS VOID AS $$
DECLARE
BEGIN
RAISE NOTICE 'id is %', id;
END;
$$ LANGUAGE plpgsql;
postgres=# CREATE PROCEDURE dummy_proc (id int) AS $$
DECLARE
BEGIN
raise notice 'id is %', id;
END;
$$ LANGUAGE plpgsql;
40. 12.07.2018What we (already) know about PostgreSQL 11 Page 40
PostgreSQL 11 new features
Procedures with transaction control
PostgreSQL 11 comes with support for procedures
postgres=# CREATE PROCEDURE dummy_proc (id int) AS $$
DECLARE
BEGIN
raise notice 'id is %', id;
END;
$$ LANGUAGE plpgsql;
postgres=# call dummy_proc(1);
NOTICE: id is 1
CALL
41. 12.07.2018What we (already) know about PostgreSQL 11 Page 41
PostgreSQL 11 new features
Procedures with transaction control
PostgreSQL 11 comes with support for procedures & transaction control
postgres=# create table t1 ( a int primary key );
postgres=# CREATE OR REPLACE PROCEDURE dummy_proc2 (id int) AS $$
DECLARE
l_id int := id;
BEGIN
raise notice 'id is %', l_id;
insert into t1 (a) values (l_id);
commit;
l_id := l_id + 1;
raise notice 'id is %', l_id;
insert into t1 (a) values (l_id);
commit;
raise notice 'id is %', l_id;
insert into t1 (a) values (l_id);
END;
$$ LANGUAGE plpgsql;
42. 12.07.2018What we (already) know about PostgreSQL 11 Page 42
PostgreSQL 11 new features
Procedures with transaction control
PostgreSQL 11 comes with support for procedures & transaction control
postgres=# call dummy_proc2(1);
NOTICE: id is 1
NOTICE: id is 2
NOTICE: id is 2
ERROR: duplicate key value violates unique constraint "t1_pkey"
DETAIL: Key (a)=(2) already exists.
CONTEXT: SQL statement "insert into t1 (a) values (l_id)"
PL/pgSQL function dummy_proc2(integer) line 13 at SQL statement
postgres=# select * from t1;
a
---
1
2
(2 rows)
43. 12.07.2018What we (already) know about PostgreSQL 11 Page 43
PostgreSQL 11 new features
Faster add column with a non null default value
Up to PostgreSQL 10, when you did this:
> The entire table gets re-written
postgres=# create table ttt (a int, b text, c date );
postgres=# insert into ttt (a,b,c)
select aa.*, md5(aa::text), now()
from generate_series(1,1000000) aa;
postgres=# timing
Timing is on.
postgres=# alter table ttt add column d text default 'a' ;
ALTER TABLE
Time: 1307.764 ms (00:01.308)
44. 12.07.2018What we (already) know about PostgreSQL 11 Page 44
PostgreSQL 11 new features
Faster add column with a non null default value
In PostgreSQL 11 adding a column with a default value is instant
> But how can that work?
postgres=# create table ttt (a int, b text, c date );
postgres=# insert into ttt (a,b,c)
select aa.*, md5(aa::text), now()
from generate_series(1,1000000) aa;
postgres=# timing
Timing is on.
postgres=# alter table ttt add column d text default 'a' ;
ALTER TABLE
Time: 20.949 ms
45. 12.07.2018What we (already) know about PostgreSQL 11 Page 45
PostgreSQL 11 new features
Faster add column with a non null default value
In PostgreSQL 11 pg_attribute got two new columns
postgres=# d pg_attribute
Table "pg_catalog.pg_attribute"
Column | Type | Collation | Nullable | Default
---------------+-----------+-----------+----------+---------
attrelid | oid | | not null |
attname | name | | not null |
atttypid | oid | | not null |
attstattarget | integer | | not null |
...
atthasmissing | boolean | | not null |
...
attinhcount | integer | | not null |
attcollation | oid | | not null |
attacl | aclitem[] | | |
attoptions | text[] | | |
attfdwoptions | text[] | | |
attmissingval | anyarray | | |
46. 12.07.2018What we (already) know about PostgreSQL 11 Page 46
PostgreSQL 11 new features
Faster add column with a non null default value
Once a column is added with a default value those columns get set
> Every row that is updated and older than the change of the table will get the new value set
> Every row that is selected and older than the change of the table will get the new value set
> New rows get it set anyway
> Once the table is re-written the flags are cleared
postgres=# select attmissingval
, atthasmissing
from pg_attribute
where attrelid = 'ttt'::regclass
and attname = 'd';
attmissingval | atthasmissing
---------------+---------------
{a} | t
(1 row)
47. 12.07.2018What we (already) know about PostgreSQL 11 Page 47
PostgreSQL 11 new features
JIT compilation
PostgreSQL will come with JIT compilation
> JIT = Just-in-time
> Turning interpreted programs into machine code at runtime
Example
> Instead of having "something" than can evaluate WHERE a = 3
> Create a function that natively can be executed on the CPU to just do that
PostgreSQL needs to be build with "--with-lvm"
48. 12.07.2018What we (already) know about PostgreSQL 11 Page 48
PostgreSQL 11 new features
JIT compilation
There are a few parameters which control JIT
postgres=# select name
, setting
from pg_settings
where name like 'jit%';
name | setting
-------------------------+---------
jit | on
jit_above_cost | 100000
jit_debugging_support | off
jit_dump_bitcode | off
jit_expressions | on
jit_inline_above_cost | 500000
jit_optimize_above_cost | 500000
jit_profiling_support | off
jit_provider | llvmjit
jit_tuple_deforming | on
(10 rows)
49. 12.07.2018What we (already) know about PostgreSQL 11 Page 49
PostgreSQL 11 new features
JIT compilation
A (fake) JIT example
postgres=# set jit_above_cost=5;
postgres=# set jit_optimize_above_cost=5;
postgres=# set jit_inline_above_cost=5;
postgres=# explain select sum(a) from ttt;
QUERY PLAN
---------------------------------------------------------------------------------------
Finalize Aggregate (cost=15554.55..15554.56 rows=1 width=8)
-> Gather (cost=15554.33..15554.54 rows=2 width=8)
Workers Planned: 2
-> Partial Aggregate (cost=14554.33..14554.34 rows=1 width=8)
-> Parallel Seq Scan on ttt (cost=0.00..13512.67 rows=416667 width=4)
JIT:
Functions: 8
Inlining: true
Optimization: true
(9 rows)
50. 12.07.2018What we (already) know about PostgreSQL 11 Page 50
PostgreSQL 11 new features
psql
New psql variables for ERROR, SQLSTATE and ROWCOUNT
postgres=# select count(*) from ttt;
count
---------
1000000
postgres=# echo :ROW_COUNT
1
postgres=# echo :SQLSTATE
00000
postgres=# select count(*.) from ttt;
ERROR: syntax error at or near "."
LINE 1: select count(*.) from ttt;
^
postgres=# echo :ERROR
True
postgres=# echo :SQLSTATE
postgres=# ! grep 42601 $PGHOME/share/errcodes.txt
postgres=# ! vi $PGHOME/share/errcodes.txt
51. 12.07.2018What we (already) know about PostgreSQL 11 Page 51
PostgreSQL 11 new features
Parallel (concurrent) btree index build
PostgreSQL 11 supports parallel (concurrent) btree index builds
> The parameters which control that
postgres=# show max_parallel_workers;
max_parallel_workers
----------------------
8
postgres=# show max_parallel_maintenance_workers;
max_parallel_maintenance_workers
----------------------------------
2
postgres=# show maintenance_work_mem;
maintenance_work_mem
----------------------
64MB
52. 12.07.2018What we (already) know about PostgreSQL 11 Page 52
PostgreSQL 11 new features
Parallel (concurrent) btree index build
You will not see it
> Was it parallel or serial?
> Of course you can disable the parallel operations and compare time
postgres=# create index ip on iptest (a,b,c);
CREATE INDEX
postgres=#
53. 12.07.2018What we (already) know about PostgreSQL 11 Page 53
PostgreSQL 11 new features
Parallel (concurrent) btree index build
Or you open a second session before the index build
Create the index
Check the first session
postgres=# select query,state,backend_type from pg_stat_activity;
postgres=# watch
postgres=# create index ip on iptest (a,b,c);
CREATE INDEX
postgres=# select query,state,backend_type from pg_stat_activity;
query | state | backend_type
--------------------------------------------------------+--------+-----------------------
…
create index ip on iptest (a,b,c); | active | client backend
select query,state,backend_type from pg_stat_activity; | active | client backend
create index ip on iptest (a,b,c); | active | parallel worker
…
54. 12.07.2018What we (already) know about PostgreSQL 11 Page 54
PostgreSQL 11 new features
Parallel hash joins
With PostgreSQL 10 you can get a plan like this:
postgres=# explain select max(t1.a) from iptest t1, iptest t2
where t1.a = t2.a;
QUERY PLAN
------------------------------------------------------------------------------------------
Finalize Aggregate (cost=1036388347.67..1036388347.68 rows=1 width=32)
-> Gather (cost=1036388347.45..1036388347.66 rows=2 width=32)
Workers Planned: 2
-> Partial Aggregate (cost=1036387347.45..1036387347.46 rows=1 width=32)
-> Merge Join (cost=1429874.39..888539787.42 rows=59139024013 width=32)
Merge Cond: (t1.a = t2.a)
-> Sort (cost=444407.57..449957.49 rows=2219967 width=32)
Sort Key: t1.a
-> Parallel Seq Scan on iptest t1
-> Materialize (cost=985466.82..1012106.42 rows=5327920 width=32)
-> Sort (cost=985466.82..998786.62 rows=5327920 width=32)
Sort Key: t2.a
-> Seq Scan on iptest t2
55. 12.07.2018What we (already) know about PostgreSQL 11 Page 55
PostgreSQL 11 new features
Parallel hash joins
PostgreSQL 11 comes with parallel hash joins
postgres=# explain select max(t1.a) from iptest t1, iptest t2
where t1.a = t2.a;
QUERY PLAN
-----------------------------------------------------------------------------------------
Finalize Aggregate (cost=322058.98..322058.99 rows=1 width=32)
-> Gather (cost=322058.77..322058.98 rows=2 width=32)
Workers Planned: 2
-> Partial Aggregate (cost=321058.77..321058.78 rows=1 width=32)
-> Parallel Hash Join (cost=145120.45..315850.38 rows=2083353 width=33)
Hash Cond: (t1.a = t2.a)
-> Parallel Seq Scan on iptest t1
-> Parallel Hash (cost=102801.53..102801.53 rows=2083353 width=33)
-> Parallel Seq Scan on iptest t2
56. 12.07.2018What we (already) know about PostgreSQL 11 Page 56
PostgreSQL 11 new features
WALSIZE with initdb
In PostgreSQL 10 you can specify the WALSEGSIZE only at compile time
> Once it was compiled and installed this is fixed
> Changes to the WAL segment size require new binaries to be compiled
postgres@pgbox:# ./configure --prefix=${PGHOME}
--with-wal-segsize=${WALSEGSIZE}
57. 12.07.2018What we (already) know about PostgreSQL 11 Page 57
PostgreSQL 11 new features
WALSIZE with initdb
In PostgreSQL 11 this becomes configurable with initdb
> The same binaries can be used for clusters with different WAL segment size configurations
postgres@pgbox:# initdb --help
initdb initializes a PostgreSQL database cluster.
Usage:
initdb [OPTION]... [DATADIR]
Options:
…
-W, --pwprompt prompt for a password for the new superuser
-X, --waldir=WALDIR location for the write-ahead log directory
--wal-segsize=SIZE size of WAL segments, in megabytes
58. 12.07.2018What we (already) know about PostgreSQL 11 Page 58
PostgreSQL 11 new features
Multiple table arguments for vacuum and analyze
In PostgreSQL 10 you can only vacuum/analyze one table at a time
postgres=# vacuum part;
VACUUM
postgres=# vacuum ttt;
VACUUM
postgres=# vacuum ttt,part;
ERROR: syntax error at or near ","
LINE 1: vacuum ttt,part;
59. 12.07.2018What we (already) know about PostgreSQL 11 Page 59
PostgreSQL 11 new features
Multiple table arguments for vacuum and analyze
PostgreSQL 11 allows to do that on multiple tables
postgres=# vacuum part;
VACUUM
postgres=# vacuum ttt;
VACUUM
postgres=# vacuum ttt,part;
VACUUM
postgres=# analyze ttt,part;
ANALYZE
60. 12.07.2018What we (already) know about PostgreSQL 11 Page 60
PostgreSQL 11 new features
Specify a separate PAGER for psql
A new environment variable PSQL_PAGER controls the PAGER psql will use
> PAGER is still used when PSQL_PAGER is not set
postgres@pgbox:# echo $PAGER
sed "s/^(([0-9]+ [rows]+))/1/;s/^(-[ RECORD [0-9]+ ][-
+]+)/1/;s/|/|/g;s/^([-+]+)/1/" 2>/dev/null | less
postgres@pgbox:# psql -c "select * from pg_stat_attribute" postgres
postgres@pgbox:# export PSQL_PAGER=cat
postgres@pgbox:# psql -c "select * from pg_stat_attribute" postgres
61. 12.07.2018What we (already) know about PostgreSQL 11 Page 61
PostgreSQL 11 new features
Restoring shared buffer content automatically after restart
The pg_prewarm extension in PostgreSQL 11 can restore the shared buffer content
automatically
postgres=# create extension pg_prewarm;
postgres=# create extension pg_buffercache;
postgres=# alter system set shared_preload_libraries='pg_prewarm';
postgres=# ! pg_ctl -D $PGDATA restart -m fast
postgres=# ! ps -ef | grep warm
postgres=# select * from ttt limit 30;
postgres=# SELECT pg_relation_filepath('ttt');
postgres=# select count(*) from pg_buffercache where relfilenode = [FILE_NODE];
postgres=# ! pg_ctl -D $PGDATA restart -m fast
postgres=# select count(*) from pg_buffercache where relfilenode = [FILE_NODE];
postgres=# select 'database,tablespace,table,fork,block';
postgres=# ! cat $PGDATA/autoprewarm.blocks | more
62. 12.07.2018What we (already) know about PostgreSQL 11 Page 62
PostgreSQL 11 new features
Allow locking of views by locking an underlying table
Before PostgreSQL 11 locking an underlying table does not lock the view on top of it
postgres=# create table tt1 ( a int );
CREATE TABLE
postgres=# create view vv1 as select * from tt1;
CREATE VIEW
postgres=# begin;
BEGIN
postgres=# lock table tt1;
LOCK TABLE
postgres=# select count(*) from vv1;
count
-------
0
1
2
63. 12.07.2018What we (already) know about PostgreSQL 11 Page 63
PostgreSQL 11 new features
Allow locking of views by locking an underlying table
PostgreSQL 11 by default locks the view
postgres=# create table tt1 ( a int );
CREATE TABLE
postgres=# create view vv1 as select * from tt1;
CREATE VIEW
postgres=# begin;
BEGIN
postgres=# lock table tt1;
LOCK TABLE
postgres=# select count(*) from vv1;
-- BLOCKS
1
2
65. 12.07.2018What we (already) know about PostgreSQL 11 Page 65
Conclusion
As always, many, many improvements in each release
Full list of changes here:
> https://www.postgresql.org/docs/11/static/release-11.html#id-1.11.6.5.5
You must read the release notes, always!!!
66. Any questions?
Please do ask!
We would love to boost
your IT-Infrastructure
How about you?
Zürich
Basel
Delémont
Nyon
12.07.2018 Page 66What we (already) know about PostgreSQL 11