This document summarizes the key features and changes in PostgreSQL 9.0 beta release. It highlights major new features like replication, permissions, and anonymous code blocks. It also briefly outlines many other enhancements, including performance improvements, monitoring tools, JSON/XML output for EXPLAIN, and mobile app contest. The presentation aims to excite developers about trying the new beta version.
Psycopg2 - Connect to PostgreSQL using Python ScriptSurvey Department
It's the presentation slides I prepared for my college workshop. This demonstrates how you can talk with PostgreSql db using python scripting.For queries, mail at dipeshsuwal@gmail.com
Psycopg2 - Connect to PostgreSQL using Python ScriptSurvey Department
It's the presentation slides I prepared for my college workshop. This demonstrates how you can talk with PostgreSql db using python scripting.For queries, mail at dipeshsuwal@gmail.com
SCALE 15x Minimizing PostgreSQL Major Version Upgrade DowntimeJeff Frost
Let's face it, major version upgrades can be a pain. Most people are familiar with the dump/restore method, but if your database happens to be larger than a few GB, the downtime required for dump/restore is likely going to exceed your maximum maintenance window.
We'll briefly explore upgrade options including dump/restore, pg_upgrade and logical replication tools like Slony and why I think Slony is currently the best option. Then we'll run through a tutorial on how to use slony for a major version upgrade with minimal downtime.
You can find the scripts used in the demos here:
https://github.com/jfrost/scale-15x-talk
Python Ireland Nov 2010 Talk: Unit TestingPython Ireland
Unit testing for those seeking instant gratification - Maciej Bliziński
Abstract: Unit testing has long term benefits. However, depending on how you use it, it can have short term benefits too. This is an introductory talk, aimed at both beginner and experienced Python programmers who would like to get started testing their code.
Dr. Hsieh is teaching how to use the state-of-the-art libraries, Spark by Apache, to conduct data analysis on hadoop platform in ISSNIP 2015, Singapore. He started with teaching the basic operations like “map, reduce, flatten, and more,” followed by explaining the extension of Spark, including MLib, GraphX, and SparkSQL.
QA Fest 2019. Saar Rachamim. Developing Tools, While TestingQAFest
Our daily work is comprised of testing a product and improve its quality. However, here and there, we can come to a state where we find a need to build a tool, that can make our work easierbetter.
I will share from my experience when I found myself in a situation where building a tool was needed.
We will start with a web application that allows you to know when a food delivery you ordered arrives to the office, and then we will focus on a tool that test the performance of an app from the UI side. We will do a live demo for both of them.
An introduction to Apache Cassandra, covering the clustering model and the data model.
Presented by Tyler Hobbs at the October 2011 Austin NoSQL meetup.
This talk was part tongue in cheek, part serious, but entirely fun and given twice as a lightning talk - once at Europython & once at the ACCU python uk 05. It presents a generic python like language parser which does actually work. Think of it as an alternative to brackets in Lisp!
SCALE 15x Minimizing PostgreSQL Major Version Upgrade DowntimeJeff Frost
Let's face it, major version upgrades can be a pain. Most people are familiar with the dump/restore method, but if your database happens to be larger than a few GB, the downtime required for dump/restore is likely going to exceed your maximum maintenance window.
We'll briefly explore upgrade options including dump/restore, pg_upgrade and logical replication tools like Slony and why I think Slony is currently the best option. Then we'll run through a tutorial on how to use slony for a major version upgrade with minimal downtime.
You can find the scripts used in the demos here:
https://github.com/jfrost/scale-15x-talk
Python Ireland Nov 2010 Talk: Unit TestingPython Ireland
Unit testing for those seeking instant gratification - Maciej Bliziński
Abstract: Unit testing has long term benefits. However, depending on how you use it, it can have short term benefits too. This is an introductory talk, aimed at both beginner and experienced Python programmers who would like to get started testing their code.
Dr. Hsieh is teaching how to use the state-of-the-art libraries, Spark by Apache, to conduct data analysis on hadoop platform in ISSNIP 2015, Singapore. He started with teaching the basic operations like “map, reduce, flatten, and more,” followed by explaining the extension of Spark, including MLib, GraphX, and SparkSQL.
QA Fest 2019. Saar Rachamim. Developing Tools, While TestingQAFest
Our daily work is comprised of testing a product and improve its quality. However, here and there, we can come to a state where we find a need to build a tool, that can make our work easierbetter.
I will share from my experience when I found myself in a situation where building a tool was needed.
We will start with a web application that allows you to know when a food delivery you ordered arrives to the office, and then we will focus on a tool that test the performance of an app from the UI side. We will do a live demo for both of them.
An introduction to Apache Cassandra, covering the clustering model and the data model.
Presented by Tyler Hobbs at the October 2011 Austin NoSQL meetup.
This talk was part tongue in cheek, part serious, but entirely fun and given twice as a lightning talk - once at Europython & once at the ACCU python uk 05. It presents a generic python like language parser which does actually work. Think of it as an alternative to brackets in Lisp!
Como analisar planos de execução e estatísticas no PostgreSQL.
- Rastreamento de consultas lentas
- Uso do EXPLAIN
- Métodos de acesso
- Junções
- Parâmetros relevantes para o otimizador
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak PROIDEA
Speaker: Andrzej Dyjak
Language: English
In recent years security industry started to grow fond of Apple’s iOS and OS X platforms. This talk will cover one of XNU's flagship debugging utilities: DTrace, a dynamic tracing framework for troubleshooting kernel and application problems on production systems in real time. It will be shown how it can be used in order to ease various tasks within the realm of dynamic binary analysis and beyond.
CONFidence: http://confidence.org.pl/
Emerging Languages: A Tour of the HorizonAlex Payne
A tour of a number of new programming languages, organized by the job they're best suited for. Presented at Philadelphia Emerging Technology for the Enterprise 2012.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond PHP - It's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"DataStax Academy
The ColumnFamily data model and wide-row support provides the ability to store and access data efficiently in a de-normalized state. Recent enhancements for CQL's spare tables and built-in indexing provide the capability to store data in a manner similar to that of relational databases. For many use cases hybrid approaches are needed, because complete de-normalization is appropriate for some access patterns whereas more structured data is appropriate for others. At times a single logical event becomes multiple insertions across multiple column families. Likewise a user request might require a several reads across different column families. This talk describes some of these scenarios and demonstrates how advanced operations such multiple step procedures, filtering, intersection, and paging can be implemented client side or server side with the help of the IntraVert plugin.
BigQuery JavaScript User-Defined Functions by THOMAS PARK and FELIPE HOFFA at...Big Data Spain
Performing ETL on big data can be slow, expensive and painful - but it doesn't have to be! In this session, we'll take an in-depth look at several real-world examples of computations that don't fit well with the SQL language model and how to solve them with user-defined functions in Google BigQuery.
Session presented at Big Data Spain 2014 Conference
18th Nov 2014
Kinépolis Madrid
http://www.bigdataspain.org
Event promoted by: http://www.paradigmatecnologico.com
Abstract: http://www.bigdataspain.org/2014/conference/hands-on-with-bigquery-javascript-user-defined-functions
Ever hire someone and have it “just not work out”? How long did that take to figure out? Weeks? Days? Before your morning coffee?
Hiring, firing and retention are some of the most difficult tasks that a business faces. And we all make mistakes. The tech industry is currently in the middle of a competitive hiring bubble and it’s really hard to find good people. It’s even harder to retain them. So how do you find good people, and keep them?
We’ll show what mistakes we’ve made in our combined 30 years in open source and tech.
Postgres has long been a database that easily scales vertically. It has also supported durable and reliable binary replication.
In the last two years, work on HA, clustering and extensions to our MVCC system has produced clustering features that help Postgres efficiently scale across many smaller systems.
This talk will start with a demo of the latest Postgres replication features. Then, we'll walk through open source and commercially-supported stacks that supporting the most demanding applications backed by Postgres.
In version 9.1, Postgres added streaming and synchronous replication that can be configured in about 5 steps. In 9.2 (to be released in 2012), we're adding cascaded replication and optimizations for common developer requests.
These features set the stage for a fast, efficient and distributed SQL cluster -- one that has configurable durability, is easy to administer and supported by "cloud" and virtualization vendors.
Have a project that just needs some cash to get off the ground? Need someone to fund beer and food for an event? Have a great idea and want to get paid for implementing it? Come find out how we did it.
Asking for money doesn’t have to be so hard. Whether you’re working on a non-profit, or a small side project. Or you’re bootstrapping a sweet new developer event. Or you want to find angel investing or venture capital… The same skills apply.
Come learn all about how to ask for money from Selena Deckelmann, J Chris Anderson and Teyo Tyree. Selena co-founded Open Source Bridge, raises money for the open source project PostgreSQL and has found funding for many small non- and for-profit projects. J Chris and Teyo both founded VC-backed startups – Couchbase and Puppet Labs – based on open source products.
They’ll lead you through their process, their successes and failures.
You’ll leave with proven strategies for developing relationships, asking the right questions and providing the right information to people who want to give you their money.
Size can creep up on you. Some day you may wake up to a multi-terabyte Postgres system handling over 3000 tps staring you down. Learn the best ways to manage these systems as they grow, and find out what new features in 9.0 have made life easier for administrators and application developers working with big data.
This talk will lead you through solutions to problems Postgres faces when it gets big: backups, transaction wraparound, bloat, huge catalogs and upgrades. You need to monitor the right things, find the gems in DBA-friendly database functions and catalog tables, and know the right places to look to spot problems early. We’ll also go over monitoring best practices and open source tools to get the job done.
Working with multiple versions of Postgres back to version 8.2 will be included, and as well as tips on making the most out of new features in 9.0. War stories will be taken from real-world work with Emma, an email marketing company with a few large databases.
Open Source software development starts with the code. But if you want a sustainable project, you need people too. This is a meditation on the idea of making friendship the core of your community's values.
Bucardo is a replication system for PostgreSQL. It supports both master-slave (to multiple slaves) and master-master replication, and does not require any modifications to PostgreSQL to run. Maintained by Greg Sabino Mullane, its management application (bucardo_ctl) has recently been significantly improved, making initial setup of replication very fast and easy.
Open Source Bridge 2009 Conference opening day slides. Includes some statistics about attendees, why they're here, goals for the conference, and a pointer about session/wiki tie-in.
4. PostgreSQL
- Current version 8.4.4
- 9.0 is in Beta
- Release in August?
- Get it here:
http://www.postgresql.org/ftp/source/9.0beta3/
http://www.postgresql.org/developer/beta
5. Contents
Full list of features:
http://developer.postgresql.org/pgdocs/postgres/
release-9-0.html
6. To boldly go...
___________________ _-_
__(==========/_=_/ ____.---’---‘---.____
_ ----._________.----/
/ / ‘-_-’
__,--‘.‘-’..’-_
/____ ||
‘--.____,-’
Based on overwhelming user demand, this release of PostgreSQL
adds features that have been requested for years, like easy-to-use
replication, a mass permission facility, and anonymous code blocks.
7. Seriously. It’s a big release
- New feature patches: 204
- Submitters: 84
- 2189 files changed, 214227 insertions(+), 125369 deletions(-)
8. Add the ability to make mass permission changes per
schema using the new GRANT/REVOKE IN SCHEMA
clause (Petr Jelinek)
postgres# GRANT SELECT,INSERT,UPDATE ON
ALL TABLES IN SCHEMA conference TO selena;
9. Add ALTER DEFAULT PRIVILEGES command to control
privileges of newly-created objects (Petr Jelinek)
postgres# ALTER DEFAULT PRIVILEGES
IN SCHEMA conference
GRANT INSERT ON TABLES TO selena;
Use psql’s ddp command to get info about
existing assignments of default privileges.
10. Install server-side language PL/pgSQL by default (Bruce
Momjian)
Yes.
Also recall: CREATE [ OR REPLACE ] [ PROCEDURAL ]
LANGUAGE name
11. Implement anonymous functions using the DO statement,
a.k.a anonymous blocks (Petr Jelinek, Joshua Tolley,
Hannu Valtonen)
DO $$
BEGIN
FOR x IN 1..10 LOOP
RAISE NOTICE ’Now at %’, x;
END LOOP;
END;$$;
12. More DO...
DO $$
for i in range(10):
plpy.notice(i)
$$ language ’plpythonu’;
13. MOAR DO...
DO $$
HAI
I HAS A VAR ITZ 0
IM IN YR LOOP
VAR R SUM OF VAR AN 1
VISIBLE VAR
BOTH SAEM VAR AN 10, O RLY?
YA RLY, GTFO
OIC
IM OUTTA YR LOOP
KTHXBYE
$$ language ’pllolcode’;
14. Allow function calls to supply parameter names and match
them to named parameters in the function definition
(Pavel Stehule)
(Demo.)
15. Add support for to char() scientific notation output
(’EEEE’) (Pavel Stehule, Brendan Jurd)
postgres=# create table exponents_demo
(example numeric);
CREATE TABLE
postgres=# INSERT INTO exponents_demo
VALUES
(’10000000’),
(’2.54’),
(’3.1415’),
(’666’),
(’6.02e+23’),
(’5.29e-11’),
(’8675309’)
;
16. Con’t
postgres=# SELECT * from exponents_demo ;
example
--------------------------
10000000
2.54
3.1415
666
602000000000000000000000
0.0000000000529
8675309
(7 rows)
18. RADIUS Authentication (Magnus Hagander)
#host database user CIDR-address auth-method [auth-opt
host all all 0.0.0.0/0 radius
radiusserver=1.2.3.4 radiussecret=canttouchthis
radiusport=1234 radiusidentifier=gabscooldatabase
19. Add samehost and samenet designations to pg hba.conf
(Stef Walter)
# IPv4 local connections:
host all all 127.0.0.1/32 md5
host all all samehost md5
host all all samenet md5
20. Log changed parameter values when postgresql.conf is
reloaded (Peter Eisentraut)
LOG: parameter "log_rotation_age" changed to "7d"
LOG: parameter "port" cannot be changed without restarting
LOG: parameter "log_rotation_age" changed to "1d"
LOG: parameter "log_min_messages" changed to "notice"
LOG: parameter "autovacuum" changed to "off"
<2010-04-22 14:00:50 PDT >LOG: parameter "log_line_prefix
<2010-04-22 14:00:50 PDT >LOG: parameter "log_statement"
21. Add an SQL state option (%e) to log line prefix
(Guillaume Smet)
e.g.:
log_line_prefix = ’<%t %d %u %e>’
gives you something like:
<2010-04-22 21:02:02 PDT postgres postgres 00000>
LOG: statement: ALTER TABLE drop_column_demo DROP COLUMN
<2010-04-22 21:02:54 PDT postgres postgres 00000>
LOG: statement: GRANT SELECT on drop_column_demo to groth
<2010-04-22 21:02:54 PDT postgres postgres 42704>
ERROR: role "groth" does not exist
22. Add new EXPLAIN (BUFFERS) to report query buffer
activity (Itagaki Takahiro)
# EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM
pg_attribute;
QUERY PLA
-----------------------------------------------------------
Seq Scan on pg_attribute (cost=0.00..52.35 rows=1935 widt
(actual time=0.010..0.948 rows=1935 loops=1)
Buffers: shared hit=16 read=17
Total runtime: 1.841 ms
(3 rows)
selena@postgres:5432=# EXPLAIN (ANALYZE, BUFFERS) SELECT *
-----------------------------------------------------------
Seq Scan on pg_attribute (cost=0.00..52.35 rows=1935 widt
(actual time=0.010..0.589 rows=1935 loops=1)
Buffers: shared hit=33
Total runtime: 0.992 ms
23. Allow EXPLAIN output in XML, JSON, and YAML
formats (Robert Haas, Greg Sabino Mullane)
EXPLAIN
SELECT * FROM user_history ORDER BY "timestamp";
QUERY PLAN
-----------------------------------------------------------
Sort (cost=18494.32..18852.33 rows=143204 width=34)
Sort Key: "timestamp"
-> Seq Scan on user_history (cost=0.00..2559.04 rows=1
(3 rows)
24. Con’t
EXPLAIN (FORMAT XML)
SELECT * FROM user_history ORDER BY "timestamp";
QUERY PLAN
-----------------------------------------------------------
<explain xmlns="http://www.postgresql.org/2009/explain"> +
<Query> +
<Plan> +
<Node-Type>Sort</Node-Type> +
<Startup-Cost>18494.32</Startup-Cost> +
<Total-Cost>18852.33</Total-Cost> +
<Plan-Rows>143204</Plan-Rows> +
<Plan-Width>34</Plan-Width> +
<Sort-Key> +
<Item>"timestamp"</Item> +
</Sort-Key> +
...
28. Add WHEN clause to CREATE TRIGGER to allow control
over whether a trigger is fired (Takahiro Itagaki)
CREATE TRIGGER name { BEFORE | AFTER } { event [ OR ... ] }
ON table [ FOR [ EACH ] { ROW | STATEMENT } ]
[ WHEN ( condition ) ]
EXECUTE PROCEDURE function_name ( arguments )
29. Allow NOTIFY to pass an optional string to listeners
(Joachim Wieland)
lfnw-9.0=# LISTEN l9_0;
LISTEN
lfnw-9.0=# NOTIFY l9_0, ’need coffee...’;
NOTIFY
Asynchronous notification "l9_0"
with payload "need coffee..."
received from server process with PID 10480.
Also, now in-memory (faster!)
30. Have columns defined with storage type MAIN remain on
the main heap page unless it cannot fit (Kevin Grittner)
8k page size
Before: a tuple might get TOAST’d prematurely
LIVE DEMO!
31. Add pg stat reset single table counters() and
pg stat reset single function counters() to allow the
resetting of statistics counters for individual tables and
indexes (Magnus Hagander)
(Discuss.)
32. Add pg last xlog receive location() and
pg last xlog replay location(), which can be used to
monitor standby server WAL activity (Simon, Fujii Masao,
Heikki)
(Discuss.)
34. And now for something completely different...
Pg Android App Contest!
http://wiki.postgresql.org/wiki/AndroidAppContest
35. Odds & Ends
VACUUM FULL is dead! Long live VACUUM FULL (like
CLUSTER).
Add deferrable unique constraints (Dean Rasheed)
vacuumdb –analyze-only (Bruce Momjian)
GRANT/REVOKE IN SCHEMA (Petr Jelinek)
pg table size and pg indexes size (Bernd Helmle)
Remove the use of flat files for system table bootstrapping
(Tom, Alvaro)
DROP COLUMN IF EXISTS (Andreas Freund)
64-bit Windows support (Tsutomu Yamada, Magnus
Hagander)
Improvements to PL/Perl (Tim Bunce)
/contrib/passwordcheck (Laurenz Albe)
36. Final Notes
These slides:
http://www.slideshare.net/selenamarie
Try it!
http://www.postgresql.org/ftp/source/9.0beta3/
http://www.postgresql.org/developer/beta
http://wiki.postgresql.org/wiki/HowToBetaTest
37. Acknowledgements
Gabrielle Roth
Mark Wong
Magnus Hagander
Hayley Jane Wakenshaw
__ __
/ ~~~/ . o O ( Thank you! )
,----( oo )
/ __ __/
/| ( |(
^ /___ / |
|__| |__|-"
38. License
This work is licensed under a Creative Commons Attribution 3.0
Unported License. To view a copy of this license, (a) visit
http://creativecommons.org/licenses/by/3.0/us/; or, (b)
send a letter to Creative Commons, 171 2nd Street, Suite 300, San
Francisco, California, 94105, USA.