The document discusses new features and improvements in PostgreSQL 9.0, including performance enhancements like improved VACUUM FULL and index usage, and new administration features like granting permissions on all tables in a schema at once and improved initialization process using pg_ctl initdb. It also covers areas like development, procedures, and replication.
PostgreSQL has kept up the momentum around JSON with version 9.4 featuring JSONB as demand for working with unstructured data continues to grow. In this talk delivered during Postgres Open 2014, Vibhor Kumar, principal systems engineer at EnterpriseDB, offered some scenarios for working with JSON in PostgreSQL and demonstrated performance metrics. This session also gave some instruction on how to use different operations and explored comparisons to BSON.
This ppt was used by Devrim at pgDay Asia 2017. He talked about some important facts about WAL - Transaction Logs or xlogs in PostgreSQL. Some of these can really come handy on a bad day
PostgreSQL has kept up the momentum around JSON with version 9.4 featuring JSONB as demand for working with unstructured data continues to grow. In this talk delivered during Postgres Open 2014, Vibhor Kumar, principal systems engineer at EnterpriseDB, offered some scenarios for working with JSON in PostgreSQL and demonstrated performance metrics. This session also gave some instruction on how to use different operations and explored comparisons to BSON.
This ppt was used by Devrim at pgDay Asia 2017. He talked about some important facts about WAL - Transaction Logs or xlogs in PostgreSQL. Some of these can really come handy on a bad day
Slides from my Feature Bits presentation at the 2010 Lean Software and Systems conference in Atlanta. See http://atlanta2010.leanssc.org/home/erik-sowa/ and http://www.leanssc.org/files/201004/videos/20100421_Sowa_EnabilingFlowWithinAndAcrossTeams/20100421_Sowa_EnabilingFlowWithinAndAcrossTeams.html
Presentation for the Workshop at the TYPO3 Developer Days 2013 (July 4th) in Hamburg:
- What happened since TYPO3 4.5?
- Roadmap for TYPO3 CMS 6.2 LTS
- Current Status of 6.2 Development
The Art of Database Experiments – PostgresConf Silicon Valley 2018 / San JoseNikolay Samokhvalov
Future database administration will be highly automated. Until then, we still live in a world where extensive manual interactions are required from a skilled DBA. This will change soon as more "autonomous databases" reach maturity and enter the production environment.
Postgres-specific monitoring tools and systems continue to improve, detecting and analyzing performance issues and bottlenecks in production databases. However, while these tools can detect current issues, they require highly-experienced DBAs to analyze and recommend mitigations.
In this session, the speaker will present the initial results of the POSTGRES.AI project – Nancy CLI, a unified way to manage automated database experiments. Nancy CLI is an automated database management framework based on well-known open-source projects and incorporating major open-source tools and Postgres modules: pgBadger, pg_stat_kcache, auto_explain, pgreplay, and others.
Originally developed with the goal to simulate various SQL query use cases in various environments and collect data to train ML models, Nancy CLI turned out to be very a universal framework that can play a crucial role in CI/CD pipelines in any company.
Using Nancy CLI, casual DBAs and any engineers can easily conduct automated experiments today, either on AWS EC2 Spot instances or on any other servers. All you need is to tell Nancy which database to use, specify workload (synthetic or "real", generated based on the Postgres logs), and what you want to test – say, check how a new index will affect all most expensive query groups from pg_stat_statements, or compare various values of "default_statistics_target". All the collected information with a very high level of confidence will give you understanding, how various queries and overall Postgres performance will be affected when you apply this change to production.
Graduating To Go - A Jumpstart into the Go Programming LanguageKaylyn Gibilterra
This workshop jumps through a lot of what is covered in the Go Tour. The exercises are new and match more along with the class content, and some pieces (like testing and APIs) are not covered in the Go Tour.
Expanded slide set for my talk on dealing with integer overflow and generalized data type conversion techniques. Versions of this talk were given at PGConf NYC, CitusCon, and SCALE 2022
The original Explaining Explain talk, focused on how the Postgres query optimizer works, and how to use the Explain command to better tune queries. This was delivered at OSCon ~2005, though the fundamentals still mostly apply today.
Slides from my Feature Bits presentation at the 2010 Lean Software and Systems conference in Atlanta. See http://atlanta2010.leanssc.org/home/erik-sowa/ and http://www.leanssc.org/files/201004/videos/20100421_Sowa_EnabilingFlowWithinAndAcrossTeams/20100421_Sowa_EnabilingFlowWithinAndAcrossTeams.html
Presentation for the Workshop at the TYPO3 Developer Days 2013 (July 4th) in Hamburg:
- What happened since TYPO3 4.5?
- Roadmap for TYPO3 CMS 6.2 LTS
- Current Status of 6.2 Development
The Art of Database Experiments – PostgresConf Silicon Valley 2018 / San JoseNikolay Samokhvalov
Future database administration will be highly automated. Until then, we still live in a world where extensive manual interactions are required from a skilled DBA. This will change soon as more "autonomous databases" reach maturity and enter the production environment.
Postgres-specific monitoring tools and systems continue to improve, detecting and analyzing performance issues and bottlenecks in production databases. However, while these tools can detect current issues, they require highly-experienced DBAs to analyze and recommend mitigations.
In this session, the speaker will present the initial results of the POSTGRES.AI project – Nancy CLI, a unified way to manage automated database experiments. Nancy CLI is an automated database management framework based on well-known open-source projects and incorporating major open-source tools and Postgres modules: pgBadger, pg_stat_kcache, auto_explain, pgreplay, and others.
Originally developed with the goal to simulate various SQL query use cases in various environments and collect data to train ML models, Nancy CLI turned out to be very a universal framework that can play a crucial role in CI/CD pipelines in any company.
Using Nancy CLI, casual DBAs and any engineers can easily conduct automated experiments today, either on AWS EC2 Spot instances or on any other servers. All you need is to tell Nancy which database to use, specify workload (synthetic or "real", generated based on the Postgres logs), and what you want to test – say, check how a new index will affect all most expensive query groups from pg_stat_statements, or compare various values of "default_statistics_target". All the collected information with a very high level of confidence will give you understanding, how various queries and overall Postgres performance will be affected when you apply this change to production.
Graduating To Go - A Jumpstart into the Go Programming LanguageKaylyn Gibilterra
This workshop jumps through a lot of what is covered in the Go Tour. The exercises are new and match more along with the class content, and some pieces (like testing and APIs) are not covered in the Go Tour.
Expanded slide set for my talk on dealing with integer overflow and generalized data type conversion techniques. Versions of this talk were given at PGConf NYC, CitusCon, and SCALE 2022
The original Explaining Explain talk, focused on how the Postgres query optimizer works, and how to use the Explain command to better tune queries. This was delivered at OSCon ~2005, though the fundamentals still mostly apply today.
It is hard to believe, but plpgsql used to be a thing. Now lost in all the hype of REST APIs and JSON wizardry, the idea of doing server-side database functions gives most people the shivers. But as it turns out, doing things server-side can be pretty useful. So useful that Postgres 11 recently upped the plpgsql game by introducing support for true stored procedures. What does that mean for you? It's time to take another look at plpgsql and what new options are available inside everyone's favorite database.
This talk aims to cover
A brief overview of postgres functions
An equally brief look at plpgsql
At least one slide on DO scripts
A slightly more extensive look at the new stored procedure functionality
A primer for advocating on using server side logic
Always with the trade-offs
Ok, plpgsql probably isn't going to take over the world, but its a handy toolset and one too many DBA's and Developers simply overlook in favor of more cumbersome solutions buried in their app code. We need to at least give it a fighting chance.
Managing Chaos In Production: Testing vs MonitoringRobert Treat
While no one disputes the good in finding and fixing issues before deploying to production, relying on traditional testing methods in the age of data-intensive, internet scale software has proven to be incomplete. The ability to identify and fix production issues quickly is crucial and requires insight into usage patterns and trends across the entire application architecture. This talk touches on deficiencies of common testing methods, provide real-world examples of discovering odd edge cases with both testing and monitoring, and offers recommendations on metric instrumentation to help companies identify and act on business-affecting problems.
Managing Databases In A DevOps Environment 2016Robert Treat
Given at #pgdayphilly2016, this talk covers how configuration management, monitoring, and rapid deployments are impacting how we think about database management.
Alert overload is bad for people and bad for business. In this presentation we discuss ways to get your business on board with actionable alerts, and how to prune poor alerts from your system.
Delivered at Velocity Europe in Barcelona, this talk introduces "ops" people to the idea of user centered design, touching on several techniques long used in the design world, and talks about how those ideas might be applied to software and processes that we use every day.
Pretty much every company that has computers on the internet has someone who gets called when those computers go down. While this practice isn’t surprising, what is surprising is that we spend very little time as an industry discussing the right way to design and implement alerts. Not from a technical sense; what we need to discuss are how to make alerts something that are actually of value for the business, and worth the disruption they cause in peoples lives. That may sound a bit dramatic, but “pager fatigue” is a real risk to business, and “phantom pages” are a sign that things have gotten out of hand. We have terms for the bad things, it’s time to start talking about the good things. Topics we’ll cover include:
* The difference between metrics, alerts, alarms, and other particulars.
* How do you determine who should be called when a problem arises.
* Simple and effective techniques for your team to responding to alerts & alarms.
* How to attack your monitoring setup to eliminate alerts without adding risk.
* Defining what “production ready” ready software is in a way that the business people will agree to.
At OmniTI, we’re often forced to walk into the middle of an existing infrastructure that is already set on fire. The only thing worse than having no alerts in that situation is having hundreds of alerts screaming at you constantly. Over the years we’ve had to come up with a way to help keep our operations team sane while also providing business value, and most importantly giving comfort to the folks that have brought us in. The methods that we’ve developed can be used by any operations team to help bring sanity back to their world, and end the cycle of “pager fatigue”.
Past, Present, and Pachyderm - All Things Open - 2013Robert Treat
Slides based on my talk at the All Things Open conference, held in Raleigh, North Carolina. This talk covers some basic history on Postgres, new features in the 9.3 release, and some thoughts on what might be in the future for Postgres.
This talk covers a long running upgrade project of a multi-terabyte database from Postgres 8.3 to Postgres 9.1, by way of pg_migrator. We discuss both technical and non-technical reasons why the project took several years to complete.
Managing Databases In A DevOps EnvironmentRobert Treat
There’s a lot of talk in the devops world about bringing developer concepts to system administration, and discussion the other way about bringing the awareness of operations to developers, but a lot of the conversation leaves out what is often the most critical part of your technology stack: the database. Perhaps that’s because DBA’s have always had to keep one foot in development and one in production, before there was a devops. Or maybe DBA’s just suck at playing well with others. Bottom line; it doesn’t matter. If you are going to store data, you need a plan that both developers and operations people can understand and embrace.
At OmniTI we’ve worked with many of the leaders in the devops movement and we’ve found there are commonalties across these organizations. It’s not so much about the tools, but about the techniques they use to help people break down barriers between different roles and establish a common ownership of technology within their organizations.
Monitoring and visibility, managing schema changes and production pushes, optimization, configuration and backups; there are aspects to data storage that bring about unique challenges. You won’t need to adopt all of these techniques to be successful, but it’s time you had a frank conversation about what it takes to make your database truly “webscale”.
Slides from PGOpen 2011, But this talk was also delivered at Velocity 2011 as well.
This talk gives an overview of the Postgres community, the Postgres database, and several add-on tools for Postgres. It is intended to provide a guide for someone wanting to do full time Postgres administration.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
16. Postgres 9.0 / Stats
• New feature patches: 204
Wednesday, June 2, 2010
17. Postgres 9.0 / Stats
• New feature patches: 204
• doesn’t include direct commits
Wednesday, June 2, 2010
18. Postgres 9.0 / Stats
• New feature patches: 204
• doesn’t include direct commits
• Submitters: 84
Wednesday, June 2, 2010
19. Postgres 9.0 / Stats
• New feature patches: 204
• doesn’t include direct commits
• Submitters: 84
• doesn’t include committers
Wednesday, June 2, 2010
20. Postgres 9.0 / Stats
• New feature patches: 204
• doesn’t include direct commits
• Submitters: 84
• doesn’t include committers
• 1860 files changed
Wednesday, June 2, 2010
21. Postgres 9.0 / Stats
• New feature patches: 204
• doesn’t include direct commits
• Submitters: 84
• doesn’t include committers
• 1860 files changed
• 150951 insertions
Wednesday, June 2, 2010
22. Postgres 9.0 / Stats
• New feature patches: 204
• doesn’t include direct commits
• Submitters: 84
• doesn’t include committers
• 1860 files changed
• 150951 insertions
• 82558 deletions
Wednesday, June 2, 2010
23. Postgres 9.0
• Performance
• Administration
• Development
• Procedures
• Replication
Wednesday, June 2, 2010
24. Postgres 9.0 / Perf / vacuum full
Wednesday, June 2, 2010
25. Postgres 9.0 / Perf / vacuum full
• VACUUM FULL now works like CLUSTER
• The Old
• move rows around, heavy scans, bloat indexes
• syntax available with VACUUM FULL INPLACE
• still used for system catalogs
Wednesday, June 2, 2010
26. Postgres 9.0 / Perf / vacuum full
• VACUUM FULL now works like CLUSTER
• The Old
• move rows around, heavy scans, bloat indexes
• syntax available with VACUUM FULL INPLACE
• still used for system catalogs
• NEW
• rewrite table and indexes
• ~ 2% less efficient for tables, 90% more efficient for indexes
• 1/3 the amount of time
• continue to avoid it :-)
Wednesday, June 2, 2010
27. Postgres 9.0 / Perf / vacuum full
Wednesday, June 2, 2010
28. Postgres 9.0 / Perf / explain buffers
explain can now show buffers information
pagila=# explain (analyze, buffers) select * from actor;
QUERY PLAN
----------------------------------------------------------------------------------------------------
Seq Scan on actor (cost=0.00..4.00 rows=200 width=25) (actual time=0.034..0.089 rows=200 loops=1)
Buffers: shared read=2
Total runtime: 0.149 ms
(3 rows)
pagila=# explain (analyze, buffers) select count(*) from actor;
QUERY PLAN
---------------------------------------------------------------------------------------------------------
Aggregate (cost=4.50..4.51 rows=1 width=0) (actual time=0.109..0.109 rows=1 loops=1)
Buffers: shared hit=2
-> Seq Scan on actor (cost=0.00..4.00 rows=200 width=0) (actual time=0.010..0.048 rows=200 loops=1)
Buffers: shared hit=2
Total runtime: 0.173 ms
(5 rows)
Wednesday, June 2, 2010
29. Postgres 9.0 / Perf / index not null
support index use for IS NOT NULL
PostgreSQL 8.4
pagila=# explain select * from address where address2 is not null;
QUERY PLAN
---------------------------------------------------------
Seq Scan on address (cost=0.00..20.03 rows=1 width=70)
Filter: (address2 IS NOT NULL)
(2 rows)
PostgreSQL 9.0 Beta 1
pagila=# explain select * from address where address2 is not null;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using address_address2_idx on address (cost=0.00..8.27 rows=1 width=70)
Index Cond: (address2 IS NOT NULL)
(2 rows)
Wednesday, June 2, 2010
30. Postgres 9.0 / Perf / join removal
remove joins from execution plan where not needed
PostgreSQL 9.0 Alpha 4
pagila=# explain analyze select actor.last_name from actor left join film_actor using (actor_id) where
actor.last_update > current_date;
QUERY PLAN
------------------------------------------------------------------------------------------------------
Nested Loop Left Join (cost=4.46..38.48 rows=27 width=35) (actual time=0.258..0.258 rows=0 loops=1)
-> Seq Scan on actor (cost=0.00..5.50 rows=1 width=25) (actual time=0.257..0.257 rows=0 loops=1)
Filter: (last_update > ('now'::text)::date)
-> Bitmap Heap Scan on film_actor (cost=4.46..32.64 rows=27 width=12) (never executed)
Recheck Cond: (actor.actor_id = film_actor.actor_id)
-> Bitmap Index Scan on film_actor_pkey (cost=0.00..4.45 rows=27 width=0) (never executed)
Index Cond: (actor.actor_id = film_actor.actor_id)
Total runtime: 0.340 ms
(8 rows)
this is not join removal!
Wednesday, June 2, 2010
31. Postgres 9.0 / Perf / join removal
PostgreSQL 8.4
pagila=# explain analyze select city from city left join country using (country_id) where city.last_update
> current_date;
QUERY PLAN
--------------------------------------------------------------------------------------------------------
Nested Loop Left Join (cost=0.00..17.95 rows=1 width=9) (actual time=28.022..28.022 rows=0 loops=1)
Join Filter: (city.country_id = country.country_id)
-> Seq Scan on city (cost=0.00..14.50 rows=1 width=11) (actual time=28.021..28.021 rows=0 loops=1)
Filter: (last_update > ('now'::text)::date)
-> Seq Scan on country (cost=0.00..2.09 rows=109 width=4) (never executed)
Total runtime: 30.567 ms
(6 rows)
PostgreSQL 9.0 Beta 1
pagila=# explain analyze select city from city left join country using (country_id) where city.last_update
> current_date;
QUERY PLAN
------------------------------------------------------------------------------------------------
Seq Scan on city (cost=0.00..14.50 rows=1 width=11) (actual time=0.748..0.748 rows=0 loops=1)
Filter: (last_update > ('now'::text)::date)
Total runtime: 0.777 ms
(3 rows)
Wednesday, June 2, 2010
38. Postgres 9.0
• Performance
• Administration
• Development
• Procedures
• Replication
Wednesday, June 2, 2010
39. Postgres 9.0 / Admin / pg_ctl initdb
initdb is dead (well, to me)
sa-x:postgres rob$ pgsql90/bin/pg_ctl -D data/ initdb -o "--locale=en_US.UTF-8"
The files belonging to this database system will be owned by user "rob".
This user must also own the server process.
The database cluster will be initialized with locale en_US.UTF-8.
The default database encoding has accordingly been set to UTF8.
The default text search configuration will be set to "english".
...
Wednesday, June 2, 2010
40. Postgres 9.0 / Admin / psql
greatest feature in postgres 9?
sa-x:postgres rob$ pgsql90b1/bin/psql -d pagila
FATAL: unrecognized configuration parameter "application_name"
psql (9.0beta1, server 8.3.5)
WARNING: psql version 9.0, server version 8.3.
Some psql features might not work.
Type "help" for help.
pagila=# d actor
Table "public.actor"
Column | Type | Modifiers
-------------+-----------------------------+----------------------------------------------------------
actor_id | integer | not null default nextval('actor_actor_id_seq'::regclass)
first_name | character varying(45) | not null
last_name | character varying(45) | not null
last_update | timestamp without time zone | not null default now()
Indexes:
"actor_pkey" PRIMARY KEY, btree (actor_id)
"idx_actor_last_name" btree (last_name)
Referenced by:
TABLE "film_actor" CONSTRAINT "film_actor_actor_id_fkey" FOREIGN KEY (actor_id) REFERENCES
actor(actor_id) ON UPDATE CASCADE ON DELETE RESTRICT
Triggers:
last_updated BEFORE UPDATE ON actor FOR EACH ROW EXECUTE PROCEDURE last_updated()
Wednesday, June 2, 2010
41. Postgres 9.0 / Admin / grant all
GRANT/REVOKE ON ALL object IN SCHEMA
pagila=# create role dylan;
CREATE ROLE
pagila=# grant select on all tables in schema public to dylan;
GRANT
pagila=# select oid::regclass, relacl from pg_class where relkind='r' and relnamespace='2200';
oid | relacl
------------------+----------------------------------------------
film_actor | {postgres=arwdDxt/postgres,dylan=r/postgres}
payment_p2007_02 | {postgres=arwdDxt/postgres,dylan=r/postgres}
payment_p2007_03 | {postgres=arwdDxt/postgres,dylan=r/postgres}
city | {postgres=arwdDxt/postgres,dylan=r/postgres}
actor | {postgres=arwdDxt/postgres,dylan=r/postgres}
category | {postgres=arwdDxt/postgres,dylan=r/postgres}
film | {postgres=arwdDxt/postgres,dylan=r/postgres}
address | {postgres=arwdDxt/postgres,dylan=r/postgres}
store | {postgres=arwdDxt/postgres,dylan=r/postgres}
staff | {postgres=arwdDxt/postgres,dylan=r/postgres}
payment_p2007_04 | {postgres=arwdDxt/postgres,dylan=r/postgres}
payment_p2007_05 | {postgres=arwdDxt/postgres,dylan=r/postgres}
payment_p2007_06 | {postgres=arwdDxt/postgres,dylan=r/postgres}
rental | {postgres=arwdDxt/postgres,dylan=r/postgres}
payment_p2007_01 | {postgres=arwdDxt/postgres,dylan=r/postgres}
country | {postgres=arwdDxt/postgres,dylan=r/postgres}
payment | {postgres=arwdDxt/postgres,dylan=r/postgres}
film_category | {postgres=arwdDxt/postgres,dylan=r/postgres}
language | {postgres=arwdDxt/postgres,dylan=r/postgres}
customer | {postgres=arwdDxt/postgres,dylan=r/postgres}
inventory | {postgres=arwdDxt/postgres,dylan=r/postgres}
Wednesday, June 2, 2010
42. Postgres 9.0 / Admin / default privs
ALTER DEFAULT PRIVILEGES
pagila=# alter default privileges grant select on tables to dylan;
ALTER DEFAULT PRIVILEGES
pagila=# create table payment_p2007_07 () inherits (payment);
CREATE TABLE
pagila=# z payment_p2007_07
Access privileges
Schema | Name | Type | Access privileges | Column access
privileges
--------+------------------+-------+---------------------------
+--------------------------
public | payment_p2007_07 | table | postgres=arwdDxt/postgres+|
| | | dylan=r/postgres |
(1 row)
Wednesday, June 2, 2010
43. Postgres 9.0 / Admin / d for children
pagila=# d payment
Table "public.payment"
Column | Type | Modifiers
--------------+-----------------------------+-------------------------------------------------------------
payment_id | integer | not null default nextval('payment_payment_id_seq'::regclass)
customer_id | smallint | not null
staff_id | smallint | not null
rental_id | integer | not null
amount | numeric(5,2) | not null
payment_date | timestamp without time zone | not null
Indexes:
"payment_pkey" PRIMARY KEY, btree (payment_id)
"idx_fk_customer_id" btree (customer_id)
"idx_fk_staff_id" btree (staff_id)
"payment_rental_id_idx" btree (rental_id)
Foreign-key constraints:
"payment_customer_id_fkey" FOREIGN KEY (customer_id) REFERENCES customer(customer_id) ON UPDATE CASCADE ON DELETE RESTRICT
"payment_rental_id_fkey" FOREIGN KEY (rental_id) REFERENCES rental(rental_id) ON UPDATE CASCADE ON DELETE SET NULL
"payment_staff_id_fkey" FOREIGN KEY (staff_id) REFERENCES staff(staff_id) ON UPDATE CASCADE ON DELETE RESTRICT
Rules:
<snip>
Number of child tables: 7 (Use d+ to list them.)
Wednesday, June 2, 2010
44. Postgres 9.0 / Admin / d for children
pagila=# d+ payment
Table "public.payment"
Column | Type | Modifiers | Storage |
Description
--------------+-----------------------------+-------------------------------------------------------------
payment_id | integer | not null default nextval('payment_payment_id_seq'::regclass) | plain |
customer_id | smallint | not null | plain |
staff_id | smallint | not null | plain |
rental_id | integer | not null | plain |
amount | numeric(5,2) | not null | main |
payment_date | timestamp without time zone | not null | plain |
Indexes:
"payment_pkey" PRIMARY KEY, btree (payment_id)
"idx_fk_customer_id" btree (customer_id)
"idx_fk_staff_id" btree (staff_id)
"payment_rental_id_idx" btree (rental_id)
Foreign-key constraints:
"payment_customer_id_fkey" FOREIGN KEY (customer_id) REFERENCES customer(customer_id) ON UPDATE CASCADE ON DELETE RESTRICT
"payment_rental_id_fkey" FOREIGN KEY (rental_id) REFERENCES rental(rental_id) ON UPDATE CASCADE ON DELETE SET NULL
"payment_staff_id_fkey" FOREIGN KEY (staff_id) REFERENCES staff(staff_id) ON UPDATE CASCADE ON DELETE RESTRICT
Rules:
<snip>
Child tables: payment_p2007_01,
payment_p2007_02,
payment_p2007_03,
payment_p2007_04,
payment_p2007_05,
payment_p2007_06,
payment_p2007_07
Has OIDs: no
Wednesday, June 2, 2010
46. Postgres 9.0 / Admin / Wait, there’s more!
• show value when unique
• reindexing shared system
constraints are violated
catalogs is now crash safe
• radius authentication
• add ability to reset statistics
on individual tables
• add SQLSTATE as option to
log_line_prefix
• support samehost/samenet
options in pg_hba.conf
• allow collection of statistics
on sequences
• added string_agg function
(group_concat)
• GUC can now be configured
based on user/database
combination • improved tabular display for
psql rows with null data
• log changes to postgresql.conf
• add index methods to di
during reload
Wednesday, June 2, 2010
48. Postgres 9.0
• Performance
• Administration
• Development
• Procedures
• Replication
Wednesday, June 2, 2010
49. Postgres 9.0 / Dev / conditional triggers
CREATE TRIGGER notify_waiting_list
AFTER UPDATE ON rental FOR EACH ROW
WHEN NEW.return_date <> OLD.return_date
THEN EXECUTE procedure send_notices();
• improves control for execution of triggers
• any boolean expression
• shows up in d output
• some after triggers will gain significant speed up
Wednesday, June 2, 2010
50. Postgres 9.0 / Dev / conditional triggers
CREATE TRIGGER film_fulltext_trigger
BEFORE INSERT OR UPDATE of title, description
ON FILM FOR EACH ROW EXECUTE PROCEDURE
tsvector_update_trigger('fulltext', 'pg_catalog.english', 'title', 'description')
• improves control for execution of triggers
• shows up in d output
• simplify code
• trigger only fires based on column declaration in update clause
Wednesday, June 2, 2010
51. Postgres 9.0 / Dev / exclusion constraints
This is not the same thing as constraint exclusion!
“Yes, we suck at naming things”
- Magnus Hagander
Wednesday, June 2, 2010
52. Postgres 9.0 / Dev / exclusion constraints
Wednesday, June 2, 2010
53. Postgres 9.0 / Dev / exclusion constraints
• Generic constraint infrastructure
• UNIQUE constraints are like =
• any indexable commutative operator
• BETWEEN-rows constraint
• each row can conflict with any other row in
the table
Wednesday, June 2, 2010
54. Postgres 9.0 / Dev / exclusion constraints
Wednesday, June 2, 2010
55. Postgres 9.0 / Dev / exclusion constraints
Wednesday, June 2, 2010
56. Postgres 9.0 / Dev / exclusion constraints
• Examples:
• non-overlapping time intervals
• calendars, scheduling, reservations
• non-overlapping geometric shapes
• zoning
Wednesday, June 2, 2010
57. Postgres 9.0 / Dev / exclusion constraints
Wednesday, June 2, 2010
58. Postgres 9.0 / Dev / exclusion constraints
Jeff Davis, “Not Just Unique”, Thu @ 1:30PM
Wednesday, June 2, 2010
59. Postgres 9.0 / Dev / exclusion constraints
• Examples:
• non-overlapping time intervals
• calendars, scheduling, reservations
• non-overlapping geometric shapes
• zoning
Jeff Davis, “Not Just Unique”, Thu @ 1:30PM
Wednesday, June 2, 2010
60. Postgres 9.0 / Dev / exclusion constraints
Jeff Davis, “Not Just Unique”, Thu @ 1:30PM
Wednesday, June 2, 2010
61. Postgres 9.0 / Dev / window functions
pagila=# pagila=# select date_trunc('week',payment_date), sum(avg(sum(amount))
pagila-# over (order by date_trunc('week', payment_date) rows between 1 preceding and 1 following)
pagila-# from payment_p2007_04 group by date_trunc('week',payment_date) order by 1;
date_trunc | sum | avg
---------------------+---------+-----------------------
2007-04-02 00:00:00 | 6562.62 | 7237.2300000000000000
2007-04-09 00:00:00 | 7911.84 | 7611.8566666666666667
2007-04-23 00:00:00 | 8361.11 | 7332.2800000000000000
2007-04-30 00:00:00 | 5723.89 | 7042.5000000000000000
(4 rows)
• enhancements to the frame clause
• now able to compute moving averages (easily)
• several options, check the docs!
Wednesday, June 2, 2010
62. Postgres 9.0 / Dev / Wait, there’s more!
Wednesday, June 2, 2010
63. Postgres 9.0 / Dev / Wait, there’s more!
• deferrable unique constraints
• new bytea hex format output
• hstore... more efficient, kv • drop if exists columns
storage
• drop if exists constraints
• show value when unique
• ordered aggregates
constraints are violated
• added string_agg function • rewrite listen/notify
(payloads)
(group_concat)
• support SQL Standard
LIMIT/OFFSET
Wednesday, June 2, 2010
64. Postgres 9.0 / Dev / Wait, there’s more!
Wednesday, June 2, 2010
65. Postgres 9.0
• Performance
• Administration
• Development
• Procedures
• Replication
Wednesday, June 2, 2010
66. Postgres 9.0 / Proc / by default
plpgsql is now created by default
Wednesday, June 2, 2010
67. Postgres 9.0 / Proc / just DO it!
pagila=# DO $$
pagila$# DECLARE
pagila$# v_part record;
pagila$# v_sql text;
pagila$# BEGIN
pagila$# set search_path = 'public';
pagila$# for v_part in select tablename from pg_tables where tablename ~ 'payment' loop
pagila$# v_sql := 'create index '||v_part.tablename||'_rental_id_idx ON '||v_part.tablename||'(rental_id)';
pagila$# raise notice '%',v_sql;
pagila$# execute v_sql;
pagila$# end loop;
pagila$# END
pagila$# $$ language plpgsql;
NOTICE: create index payment_p2007_02_rental_id_idx ON payment_p2007_02(rental_id)
NOTICE: create index payment_p2007_03_rental_id_idx ON payment_p2007_03(rental_id)
NOTICE: create index payment_p2007_07_rental_id_idx ON payment_p2007_07(rental_id)
NOTICE: create index payment_p2007_04_rental_id_idx ON payment_p2007_04(rental_id)
NOTICE: create index payment_p2007_05_rental_id_idx ON payment_p2007_05(rental_id)
NOTICE: create index payment_p2007_06_rental_id_idx ON payment_p2007_06(rental_id)
NOTICE: create index payment_p2007_01_rental_id_idx ON payment_p2007_01(rental_id)
NOTICE: create index payment_rental_id_idx ON payment(rental_id)
DO
pagila=#
Wednesday, June 2, 2010
68. Postgres 9.0 / Proc / just DO it!
pagila=# di payment_p2007_0*rental*
List of relations
Schema | Name | Type | Owner | Table
--------+--------------------------------+-------+----------+------------------
public | payment_p2007_01_rental_id_idx | index | postgres | payment_p2007_01
public | payment_p2007_02_rental_id_idx | index | postgres | payment_p2007_02
public | payment_p2007_03_rental_id_idx | index | postgres | payment_p2007_03
public | payment_p2007_04_rental_id_idx | index | postgres | payment_p2007_04
public | payment_p2007_05_rental_id_idx | index | postgres | payment_p2007_05
public | payment_p2007_06_rental_id_idx | index | postgres | payment_p2007_06
public | payment_p2007_07_rental_id_idx | index | postgres | payment_p2007_07
(7 rows)
Wednesday, June 2, 2010
70. Postgres 9.0 / Proc / Wait, there’s more!
• can now use expressions in
OPEN cursor FOR EXECUTE
• improve variable recognition • use strict now works in plperl
within plpgsql
• change Perl warnings to
• allow assignment of values to elog(WARNING)
IN parameters
• Add Python 3 support
• Allow MOVE FORWARD n/
ALL, MOVE BACKWARD n/ • support for C++ stored procs
ALL for plpgsql cursors
• named parameters
• add utility functions for
plperl (quote_literal and
friends)
Wednesday, June 2, 2010
72. Postgres 9.0
• Performance
• Administration
• Development
• Procedures
• Replication
Wednesday, June 2, 2010
73. Postgres 9.0 / repl / hot standby
• allows for (read only) queries against a database during crash recovery
• aka pitr replication / xlog shipping
• GUC Settings
• primary
• wal_level = ‘hot standby’
• vacuum_defer_cleanup_age = 0
• secondary
• hot_standby = on
• max_standby_delay = 30s
Wednesday, June 2, 2010
74. Postgres 9.0 / repl / streaming replication
• Process to connect to the primary from a secondary server and
“stream” changes to slaves
• Config primary for archive logging, then
• max_wal_senders
• Config secondary for streaming replication
• in recovery.conf
• standby_mode = on
• primary_conninfo = ...
• in postgresql.conf
• wal_level = archive
• hot_standby = on
Wednesday, June 2, 2010