SlideShare a Scribd company logo
1 of 80
Download to read offline
Programming with Python
    and PostgreSQL
            Peter Eisentraut
          peter@eisentraut.org

            F-Secure Corporation



    PostgreSQL Conference East 2011



                                      CC-BY
Partitioning



   • Part I: Client programming (60 min)
   • Part II: PL/Python (30 min)
Why Python?
Why Python?
 Pros:
   • widely used
   • easy
   • strong typing
   • scripting, interactive use
   • good PostgreSQL support
   • client and server (PL) interfaces
   • open source, community-based
Why Python?
 Pros:
   • widely used
   • easy
   • strong typing
   • scripting, interactive use
   • good PostgreSQL support
   • client and server (PL) interfaces
   • open source, community-based
 Pros:
   • no static syntax checks, must rely on test coverage
   • Python community has varying interest in RDBMS
Part I

Client Programming
Example
 import psycopg2

 dbconn = psycopg2.connect('dbname=dellstore2')
 cursor = dbconn.cursor()
 cursor.execute("""
 SELECT firstname, lastname
 FROM customers
 ORDER BY 1, 2
 LIMIT 10
 """)
 for row in cursor.fetchall():
     print "Name: %s %s" % (row[0], row[1])
 cursor.close()
 db.close()
Drivers

  Name             License   Platforms     Py Versions
  Psycopg          LGPL      Unix, Win     2.4–3.2
  PyGreSQL         BSD       Unix, Win     2.3–2.6
  ocpgdb           BSD       Unix          2.3–2.6
  py-postgresql    BSD       pure Python   3.0+
  bpgsql (alpha)   LGPL      pure Python   2.3–2.6
  pg8000           BSD       pure Python   2.5–3.0+
Drivers

   Name             License   Platforms     Py Versions
   Psycopg          LGPL      Unix, Win     2.4–3.2
   PyGreSQL         BSD       Unix, Win     2.3–2.6
   ocpgdb           BSD       Unix          2.3–2.6
   py-postgresql    BSD       pure Python   3.0+
   bpgsql (alpha)   LGPL      pure Python   2.3–2.6
   pg8000           BSD       pure Python   2.5–3.0+
  More details
    • http://wiki.postgresql.org/wiki/Python
    • http://wiki.python.org/moin/PostgreSQL
DB-API 2.0


   • the standard Python database API
   • all mentioned drivers support it
   • defined in PEP 249
   • discussions: db-sig@python.org
   • very elementary (from a PostgreSQL perspective)
   • outdated relative to Python language development
   • lots of extensions and incompatibilities possible
Higher-Level Interfaces



   • Zope
   • SQLAlchemy
   • Django
Psycopg Facts

   • Main authors: Federico Di Gregorio, Daniele Varrazzo
   • License: LGPLv3+
   • Web site: http://initd.org/psycopg/
        • Documentation: http://initd.org/psycopg/docs/
        • Git, Gitweb
   • Mailing list: psycopg@postgresql.org
   • Twitter: @psycopg
   • Latest version: 2.4 (February 27, 2011)
Using the Driver



  import psycopg2

  dbconn = psycopg2.connect(...)
  ...
Driver Independence?



 import psycopg2

 dbconn = psycopg2.connect(...)   # hardcodes driver name
Driver Independence?



 import psycopg2 as dbdriver

 dbconn = dbdriver.connect(...)
Driver Independence?


 dbtype = 'psycopg2'   # e.g. from config file
 dbdriver = __import__(dbtype,
                       globals(), locals(),
                       [], -1)

 dbconn = dbdriver.connect(...)
Connecting
 # libpq-like connection string
 dbconn = psycopg2.connect('dbname=dellstore2
     host=localhost port=5432')

 # same
 dbconn = psycopg2.connect(dsn='dbname=dellstore2
     host=localhost port=5432')

 # keyword arguments
 # (not all possible libpq options supported)
 dbconn = psycopg2.connect(database='dellstore2',
                           host='localhost',
                           port='5432')

 DB-API 2.0 says: arguments database dependent
“Cursors”



  cursor = dbconn.cursor()

    • not a real database cursor, only an API abstraction
    • think “statement handle”
Server-Side Cursors



  cursor = dbconn.cursor(name='mycursor')

    • a real database cursor
    • use for large result sets
Executing
 # queries
 cursor.execute("""
 SELECT firstname, lastname
 FROM customers
 ORDER BY 1, 2
 LIMIT 10
 """)

 # updates
 cursor.execute("UPDATE customers SET password = NULL")
 print "%d rows updated" % cursor.rowcount

 # or anything else
 cursor.execute("ANALYZE customers")
Fetching Query Results

  cursor.execute("SELECT firstname, lastname FROM ...")
  cursor.fetchall()

  [('AABBKO',   'DUTOFRPLOK'),
   ('AABTSI',   'ZFCKMPRVVJ'),
   ('AACOHS',   'EECCQPVTIW'),
   ('AACVVO',   'CLSXSGZYKS'),
   ('AADVMN',   'MEMQEWYFYE'),
   ('AADXQD',   'GLEKVVLZFV'),
   ('AAEBUG',   'YUOIINRJGE')]
Fetching Query Results



  cursor.execute("SELECT firstname, lastname FROM ...")
  for row in cursor.fetchall():
      print "Name: %s %s" % (row[0], row[1])
Fetching Query Results



  cursor.execute("SELECT firstname, lastname FROM ...")
  for row in cursor.fetchall():
      print "Name: %s %s" % (row[0], row[1])

  Note: field access only by number
Fetching Query Results



  cursor.execute("SELECT firstname, lastname FROM ...")
  row = cursor.fetchone()
  if row is not None:
      print "Name: %s %s" % (row[0], row[1])
Fetching Query Results



  cursor.execute("SELECT firstname, lastname FROM ...")
  for row in cursor:
      print "Name: %s %s" % (row[0], row[1])
Fetching Query Results in Batches


  cursor = dbconn.cursor(name='mycursor')
  cursor.arraysize = 500   # default: 1
  cursor.execute("SELECT firstname, lastname FROM ...")
  while True:
      batch = cursor.fetchmany()
      break if not batch
      for row in batch:
          print "Name: %s %s" % (row[0], row[1])
Fetching Query Results in Batches



  cursor = dbconn.cursor(name='mycursor')
  cursor.execute("SELECT firstname, lastname FROM ...")
  cursor.itersize = 2000   # default
  for row in cursor:
      print "Name: %s %s" % (row[0], row[1])
Getting Query Metadata

 cursor.execute("SELECT DISTINCT state, zip FROM
     customers")
 print cursor.description[0].name
 print cursor.description[0].type_code
 print cursor.description[1].name
 print cursor.description[1].type_code

 state
 1043    # == psycopg2.STRING
 zip
 23      # == psycopg2.NUMBER
Passing Parameters



 cursor.execute("""
 UPDATE customers
     SET password = %s
     WHERE customerid = %s
 """, ["sekret", 37])
Passing Parameters


 Not to be confused with (totally evil):
 cursor.execute("""
 UPDATE customers
     SET password = '%s'
     WHERE customerid = %d
 """ % ["sekret", 37])
Passing Parameters

 cursor.execute("INSERT INTO foo VALUES (%s)",
                "bar")    # WRONG

 cursor.execute("INSERT INTO foo VALUES (%s)",
                ("bar")) # WRONG

 cursor.execute("INSERT INTO foo VALUES (%s)",
                ("bar",)) # correct

 cursor.execute("INSERT INTO foo VALUES (%s)",
                ["bar"]) # correct

 (from Psycopg documentation)
Passing Parameters



 cursor.execute("""
 UPDATE customers
     SET password = %(pw)s
     WHERE customerid = %(id)s
 """, {'id': 37, 'pw': "sekret"})
Passing Many Parameter Sets


 cursor.executemany("""
 UPDATE customers
     SET password = %s
     WHERE customerid = %s
 """, [["ahTh4oip", 100],
       ["Rexahho7", 101],
       ["Ee1aetui", 102]])
Calling Procedures



  cursor.callproc('pg_start_backup', 'label')
Data Types


 from decimal import Decimal
 from psycopg2 import Date

 cursor.execute("""
 INSERT INTO orders (orderdate, customerid,
                     netamount, tax, totalamount)
 VALUES (%s, %s, %s, %s, %s)""",
 [Date(2011, 03, 23), 12345,
  Decimal("899.95"), 8.875, Decimal("979.82")])
Mogrify
  from decimal import Decimal
  from psycopg2 import Date

  cursor.mogrify("""
  INSERT INTO orders (orderdate, customerid,
                      netamount, tax, totalamount)
  VALUES (%s, %s, %s, %s, %s)""",
  [Date(2011, 03, 23), 12345,
   Decimal("899.95"), 8.875, Decimal("979.82")])

  Result:
  "nINSERT INTO orders (orderdate, customerid,n
      netamount, tax, totalamount)nVALUES
      ('2011-03-23'::date, 12345, 899.95, 8.875, 979.82)"
Data Types


 cursor.execute("""
 SELECT * FROM orders WHERE customerid = 12345
 """)

 Result:
 (12002, datetime.date(2011, 3, 23), 12345,
     Decimal('899.95'), Decimal('8.88'),
     Decimal('979.82'))
Nulls

  Input:
  cursor.mogrify("SELECT %s", [None])

  'SELECT NULL'

  Output:
  cursor.execute("SELECT NULL")
  cursor.fetchone()

  (None,)
Booleans



 cursor.mogrify("SELECT %s, %s", [True, False])

 'SELECT true, false'
Binary Data
  Standard way:
  from psycopg2 import Binary
  cursor.mogrify("SELECT %s", [Binary("foo")])

  "SELECT E'x666f6f'::bytea"
Binary Data
  Standard way:
  from psycopg2 import Binary
  cursor.mogrify("SELECT %s", [Binary("foo")])

  "SELECT E'x666f6f'::bytea"

  Other ways:
  cursor.mogrify("SELECT %s", [buffer("foo")])

  "SELECT E'x666f6f'::bytea"

  cursor.mogrify("SELECT %s",
                 [bytearray.fromhex(u"deadbeef")])

  "SELECT E'xdeadbeef'::bytea"

  There are more. Check the documentation. Check the versions.
Date/Time

 Standard ways:
 from psycopg2 import Date, Time, Timestamp

 cursor.mogrify("SELECT %s, %s, %s",
                [Date(2011, 3, 23),
                 Time(9, 0, 0),
                 Timestamp(2011, 3, 23, 9, 0, 0)])

 "SELECT '2011-03-23'::date, '09:00:00'::time,
     '2011-03-23T09:00:00'::timestamp"
Date/Time
 Other ways:
 import datetime

 cursor.mogrify("SELECT %s, %s, %s, %s",
                [datetime.date(2011, 3, 23),
                 datetime.time(9, 0, 0),
                 datetime.datetime(2011, 3, 23, 9, 0),
                 datetime.timedelta(minutes=90)])

 "SELECT '2011-03-23'::date, '09:00:00'::time,
     '2011-03-23T09:00:00'::timestamp, '0 days
     5400.000000 seconds'::interval"

 mx.DateTime   also supported
Arrays


  foo = [1, 2, 3]
  bar = [datetime.time(9, 0), datetime.time(10, 30)]

  cursor.mogrify("SELECT %s, %s",
                 [foo, bar])

  "SELECT ARRAY[1, 2, 3], ARRAY['09:00:00'::time,
      '10:30:00'::time]"
Tuples


 foo = (1, 2, 3)

 cursor.mogrify("SELECT * FROM customers WHERE
     customerid IN %s",
                [foo])

 'SELECT * FROM customers WHERE customerid IN (1, 2, 3)'
Hstore

 import psycopg2.extras

 psycopg2.extras.register_hstore(cursor)

 x = {'a': 'foo', 'b': 'bar'}

 cursor.mogrify("SELECT %s",
                [x])

 "SELECT hstore(ARRAY[E'a', E'b'], ARRAY[E'foo',
     E'bar'])"
Unicode Support


 Cause all result strings to be returned as Unicode strings:
 psycopg2.extensions.register_type(psycopg2.extensions.
     UNICODE)
 psycopg2.extensions.register_type(psycopg2.extensions.
     UNICODEARRAY)
Transaction Control


  Transaction blocks are used by default. Must use
  dbconn.commit()

  or
  dbconn.rollback()
Transaction Control: Autocommit


  import psycopg2.extensions

  dbconn.set_isolation_level(psycopg2.extensions.
      ISOLATION_LEVEL_AUTOCOMMIT)

  cursor = dbconn.cursor()
  cursor.execute("VACUUM")
Transaction Control: Isolation Mode


  import psycopg2.extensions

  dbconn.set_isolation_level(psycopg2.extensions.
      ISOLATION_LEVEL_SERIALIZABLE) # or other level

  cursor = dbconn.cursor()
  cursor.execute(...)
  ...
  dbconn.commit()
Exception Handling

  StandardError
  |__ Warning
  |__ Error
      |__ InterfaceError
      |__ DatabaseError
          |__ DataError
          |__ OperationalError
          |   |__ psycopg2.extensions.QueryCanceledError
          |   |__ psycopg2.extensions.TransactionRollbackError
          |__ IntegrityError
          |__ InternalError
          |__ ProgrammingError
          |__ NotSupportedError
Error Messages



 try:
     cursor.execute("boom")
 except Exception, e:
     print e.pgerror
Error Codes

 import psycopg2.errorcodes

 while True:
     try:
         cursor.execute("UPDATE something ...")
         cursor.execute("UPDATE otherthing ...")
         break
     except Exception, e:
         if e.pgcode == 
                 psycopg2.errorcodes.SERIALIZATION_FAILURE:
             continue
         else:
             raise
Connection and Cursor Factories

  Want: accessing result columns by name
  Recall:
  dbconn = psycopg2.connect(dsn='...')
  cursor = dbconn.cursor()
  cursor.execute("""
  SELECT firstname, lastname
  FROM customers
  ORDER BY 1, 2
  LIMIT 10
  """)
  for row in cursor.fetchall():
      print "Name: %s %s" % (row[0], row[1])   # stupid :(
Connection and Cursor Factories
  Solution 1: Using DictConnection:
  import psycopg2.extras

  dbconn = psycopg2.connect(dsn='...',
      connection_factory=psycopg2.extras.DictConnection)
  cursor = dbconn.cursor()
  cursor.execute("""
  SELECT firstname, lastname
  FROM customers
  ORDER BY 1, 2
  LIMIT 10
  """)
  for row in cursor.fetchall():
      print "Name: %s %s" % (row['firstname'], # or row[0]
                             row['lastname']) # or row[1]
Connection and Cursor Factories
  Solution 2: Using RealDictConnection:
  import psycopg2.extras

  dbconn = psycopg2.connect(dsn='...',
      connection_factory=psycopg2.extras.RealDictConnection)
  cursor = dbconn.cursor()
  cursor.execute("""
  SELECT firstname, lastname
  FROM customers
  ORDER BY 1, 2
  LIMIT 10
  """)
  for row in cursor.fetchall():
      print "Name: %s %s" % (row['firstname'],
                             row['lastname'])
Connection and Cursor Factories
  Solution 3: Using NamedTupleConnection:
  import psycopg2.extras

  dbconn = psycopg2.connect(dsn='...',
      connection_factory=psycopg2.extras.NamedTupleConnection)
  cursor = dbconn.cursor()
  cursor.execute("""
  SELECT firstname, lastname
  FROM customers
  ORDER BY 1, 2
  LIMIT 10
  """)
  for row in cursor.fetchall():
      print "Name: %s %s" % (row.firstname,    # or row[0]
                             row.lastname)     # or row[1]
Connection and Cursor Factories
  Alternative: Using
  DictCursor/RealDictCursor/NamedTupleCursor:

  import psycopg2.extras

  dbconn = psycopg2.connect(dsn='...')
  cursor = dbconn.cursor(cursor_factory=psycopg2.extras.
      DictCursor/RealDictCursor/NameTupleCursor)
  cursor.execute("""
  SELECT firstname, lastname
  FROM customers
  ORDER BY 1, 2
  LIMIT 10
  """)
  for row in cursor.fetchall():
      print "Name: %s %s" % (row['firstname'],
                             row['lastname'])
      # (resp. row.firstname, row.lastname)
Supporting New Data Types



 Only a finite list of types is supported by default: Date, Binary,
 etc.
   • map new PostgreSQL data types into Python
   • map new Python data types into PostgreSQL
Mapping New PostgreSQL Types Into
Python
 import psycopg2
 import psycopg2.extensions

 def cast_oidvector(value, _cursor):
     """Convert oidvector to Python array"""
     if value is None:
         return None
     return map(int, value.split(' '))


 OIDVECTOR = psycopg2.extensions.new_type((30,),
     'OIDVECTOR', cast_oidvector)
 psycopg2.extensions.register_type(OIDVECTOR)
Mapping New Python Types into
PostgreSQL
 from psycopg2.extensions import adapt,
     register_adapter, AsIs

 class Point(object):
     def __init__(self, x, y):
         self.x = x
         self.y = y

 def adapt_point(point):
     return AsIs("'(%s, %s)'" % (adapt(point.x),
         adapt(point.y)))

 register_adapter(Point, adapt_point)

 cur.execute("INSERT INTO atable (apoint) VALUES (%s)",
             (Point(1.23, 4.56),))

 (from Psycopg documentation)
Connection Pooling With Psycopg
 from psycopg2.pool import SimpleConnectionPool

 pool = SimpleConnectionPool(1, 20, dsn='...')
 dbconn = pool.getconn()
 ...
 pool.putconn(dbconn)
 pool.closeall()
Connection Pooling With Psycopg
 for non-threaded applications:
 from psycopg2.pool import SimpleConnectionPool

 pool = SimpleConnectionPool(1, 20, dsn='...')
 dbconn = pool.getconn()
 ...
 pool.putconn(dbconn)
 pool.closeall()

 for non-threaded applications:
 from psycopg2.pool import ThreadedConnectionPool

 pool = ThreadedConnectionPool(1, 20, dsn='...')
 dbconn = pool.getconn()
 cursor = dbconn.cursor()
 ...
 pool.putconn(dbconn)
 pool.closeall()
Connection Pooling With DBUtils


  import psycopg2
  from DBUtils.PersistentDB import PersistentDB

  dbconn = PersistentDB(psycopg2, dsn='...')
  cursor = dbconn.cursor()
  ...

  see http://pypi.python.org/pypi/DBUtils/
The Other Stuff

   • thread safety: can share connections, but not cursors
   • COPY support: cursor.copy_from(), cursor.copy_to()
   • large object support: connection.lobject()
   • 2PC: connection.xid(), connection.tpc_begin(), . . .
   • query cancel: dbconn.cancel()
   • notices: dbconn.notices
   • notifications: dbconn.notifies
   • asynchronous communication
   • coroutine support
   • logging cursor
Part II

PL/Python
Setup


   • included with PostgreSQL
        • configure --with-python
        • apt-get/yum install postgresql-plpython
   • CREATE LANGUAGE plpythonu;
   • Python 3: CREATE LANGUAGE plpython3u;
   • “untrusted”, superuser only
Basic Examples
 CREATE FUNCTION add(a int, b int) RETURNS int
 LANGUAGE plpythonu
 AS $$
 return a + b
 $$;

 CREATE FUNCTION longest(a text, b text) RETURNS text
 LANGUAGE plpythonu
 AS $$
 if len(a) > len(b):
     return a
 elif len(b) > len(a):
     return b
 else:
     return None
 $$;
Using Modules


 CREATE FUNCTION json_to_array(j text) RETURNS text[]
 LANGUAGE plpythonu
 AS $$
 import json

 return json.loads(j)
 $$;
Database Calls


 CREATE FUNCTION clear_passwords() RETURNS int
 LANGUAGE plpythonu
 AS $$
 rv = plpy.execute("UPDATE customers SET password =
     NULL")
 return rv.nrows
 $$;
Database Calls With Parameters


 CREATE FUNCTION set_password(username text, password
     text) RETURNS boolean
 LANGUAGE plpythonu
 AS $$
 plan = plpy.prepare("UPDATE customers SET password = $1
     WHERE username= $2", ['text', 'text'])
 rv = plpy.execute(plan, [username, password])
 return rv.nrows == 1
 $$;
Avoiding Prepared Statements

 CREATE FUNCTION set_password(username text, password
     text) RETURNS boolean
 LANGUAGE plpythonu
 AS $$
 rv = plpy.execute("UPDATE customers SET password = %s
     WHERE username= %s" %
     (plpy.quote_nullable(username),
     plpy.quote_literal(password)))
 return rv.nrows == 1
 $$;

 (available in 9.1-to-be)
Caching Plans

 CREATE FUNCTION set_password2(username text, password
     text) RETURNS boolean
 LANGUAGE plpythonu
 AS $$
 if 'myplan' in SD:
     plan = SD['myplan']
 else:
     plan = plpy.prepare("UPDATE customers SET password
         = $1 WHERE username= $2", ['text', 'text'])
     SD['myplan'] = plan
 rv = plpy.execute(plan, [username, password])
 return rv.nrows == 1
 $$;
Processing Query Results

 CREATE FUNCTION get_customer_name(username text)
     RETURNS boolean
 LANGUAGE plpythonu
 AS $$
 plan = plpy.prepare("SELECT firstname || ' ' ||
     lastname AS ""name"" FROM customers WHERE username =
     $1", ['text'])
 rv = plpy.execute(plan, [username], 1)
 return rv[0]['name']
 $$;
Compare: PL/Python vs. DB-API

 PL/Python:
 plan = plpy.prepare("SELECT ...")
 for row in plpy.execute(plan, ...):
     plpy.info(row["fieldname"])

 DB-API:
 dbconn = psycopg2.connect(...)
 cursor = dbconn.cursor()
 cursor.execute("SELECT ...")
 for row in cursor.fetchall() do:
     print row[0]
Set-Returning and Table Functions


  CREATE FUNCTION get_customers(id int) RETURNS SETOF
      customers
  LANGUAGE plpythonu
  AS $$
  plan = plpy.prepare("SELECT * FROM customers WHERE
      customerid = $1", ['int'])
  rv = plpy.execute(plan, [id])
  return rv
  $$;
Triggers

  CREATE FUNCTION delete_notifier() RETURNS trigger
  LANGUAGE plpythonu
  AS $$
  if TD['event'] == 'DELETE':
      plpy.notice("one row deleted from table %s" %
          TD['table_name'])
  $$;

  CREATE TRIGGER customers_delete_notifier AFTER DELETE
      ON customers FOR EACH ROW EXECUTE PROCEDURE
      delete_notifier();
Exceptions


 CREATE FUNCTION test() RETURNS text
 LANGUAGE plpythonu
 AS $$
 try:
     rv = plpy.execute("SELECT ...")
 except plpy.SPIError, e:
     plpy.notice("something went wrong")

 The transaction is still aborted in < 9.1.
New in PostgreSQL 9.1

   • SPI calls wrapped in subtransactions
   • custom SPI exceptions: subclass per SQLSTATE,
    .sqlstate    attribute
   • plpy.subtransaction() context manager
   • support for OUT parameters
   • quoting functions
   • validator
   • lots of internal improvements
The End

More Related Content

What's hot

Upgrading to MongoDB 4.0 from older versions
Upgrading to MongoDB 4.0 from older versionsUpgrading to MongoDB 4.0 from older versions
Upgrading to MongoDB 4.0 from older versionsAntonios Giannopoulos
 
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...Mydbops
 
GIT presentation
GIT presentationGIT presentation
GIT presentationNaim Latifi
 
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UIData Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UIAltinity Ltd
 
PGConf.ASIA 2017 Logical Replication Internals (English)
PGConf.ASIA 2017 Logical Replication Internals (English)PGConf.ASIA 2017 Logical Replication Internals (English)
PGConf.ASIA 2017 Logical Replication Internals (English)Noriyoshi Shinoda
 
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike SteenbergenMeet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergendistributed matters
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaAutovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaPostgreSQL-Consulting
 
MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0Mayank Prasad
 
Intro to git and git hub
Intro to git and git hubIntro to git and git hub
Intro to git and git hubVenkat Malladi
 
What is new in PostgreSQL 14?
What is new in PostgreSQL 14?What is new in PostgreSQL 14?
What is new in PostgreSQL 14?Mydbops
 
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...Altinity Ltd
 
All about Context API
All about Context APIAll about Context API
All about Context APISouvik Basu
 
Advanced Git Presentation By Swawibe
Advanced Git Presentation By SwawibeAdvanced Git Presentation By Swawibe
Advanced Git Presentation By SwawibeMd Swawibe Ul Alam
 
MongoDB - Aggregation Pipeline
MongoDB - Aggregation PipelineMongoDB - Aggregation Pipeline
MongoDB - Aggregation PipelineJason Terpko
 
Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners HubSpot
 
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performancePresto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performanceDataWorks Summit
 

What's hot (20)

PostgreSQL : Introduction
PostgreSQL : IntroductionPostgreSQL : Introduction
PostgreSQL : Introduction
 
Upgrading to MongoDB 4.0 from older versions
Upgrading to MongoDB 4.0 from older versionsUpgrading to MongoDB 4.0 from older versions
Upgrading to MongoDB 4.0 from older versions
 
Git & GitHub for Beginners
Git & GitHub for BeginnersGit & GitHub for Beginners
Git & GitHub for Beginners
 
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
 
GIT presentation
GIT presentationGIT presentation
GIT presentation
 
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UIData Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
 
PGConf.ASIA 2017 Logical Replication Internals (English)
PGConf.ASIA 2017 Logical Replication Internals (English)PGConf.ASIA 2017 Logical Replication Internals (English)
PGConf.ASIA 2017 Logical Replication Internals (English)
 
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike SteenbergenMeet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaAutovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
 
MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0
 
Intro to git and git hub
Intro to git and git hubIntro to git and git hub
Intro to git and git hub
 
What is new in PostgreSQL 14?
What is new in PostgreSQL 14?What is new in PostgreSQL 14?
What is new in PostgreSQL 14?
 
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
 
All about Context API
All about Context APIAll about Context API
All about Context API
 
GitHub Presentation
GitHub PresentationGitHub Presentation
GitHub Presentation
 
PHP and Mysql
PHP and MysqlPHP and Mysql
PHP and Mysql
 
Advanced Git Presentation By Swawibe
Advanced Git Presentation By SwawibeAdvanced Git Presentation By Swawibe
Advanced Git Presentation By Swawibe
 
MongoDB - Aggregation Pipeline
MongoDB - Aggregation PipelineMongoDB - Aggregation Pipeline
MongoDB - Aggregation Pipeline
 
Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners
 
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performancePresto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performance
 

Viewers also liked

Psycopg2 - Connect to PostgreSQL using Python Script
Psycopg2 - Connect to PostgreSQL using Python ScriptPsycopg2 - Connect to PostgreSQL using Python Script
Psycopg2 - Connect to PostgreSQL using Python ScriptSurvey Department
 
"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014Henning Jacobs
 
Socket Programming In Python
Socket Programming In PythonSocket Programming In Python
Socket Programming In Pythondidip
 
Programming with Python - Basic
Programming with Python - BasicProgramming with Python - Basic
Programming with Python - BasicMosky Liu
 
Functional programming in Python
Functional programming in PythonFunctional programming in Python
Functional programming in PythonColin Su
 
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQLTen Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQLanandology
 
Golang iran - tutorial go programming language - Preliminary
Golang iran - tutorial  go programming language - PreliminaryGolang iran - tutorial  go programming language - Preliminary
Golang iran - tutorial go programming language - Preliminarygo-lang
 
Golang #5: To Go or not to Go
Golang #5: To Go or not to GoGolang #5: To Go or not to Go
Golang #5: To Go or not to GoOliver N
 
Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1AkramWaseem
 
Rethink db with Python
Rethink db with PythonRethink db with Python
Rethink db with PythonPrabhu Raghav
 
Succumbing to the Python in Financial Markets
Succumbing to the Python in Financial MarketsSuccumbing to the Python in Financial Markets
Succumbing to the Python in Financial Marketsdcerezo
 
Massively Parallel Processing with Procedural Python (PyData London 2014)
Massively Parallel Processing with Procedural Python (PyData London 2014)Massively Parallel Processing with Procedural Python (PyData London 2014)
Massively Parallel Processing with Procedural Python (PyData London 2014)Ian Huston
 
Matando o Java e Mostrando o Python
Matando o Java e Mostrando o PythonMatando o Java e Mostrando o Python
Matando o Java e Mostrando o PythonOsvaldo Santana Neto
 
Relational Database Access with Python
Relational Database Access with PythonRelational Database Access with Python
Relational Database Access with PythonMark Rees
 
Postgresql + Python = Power!
Postgresql + Python = Power!Postgresql + Python = Power!
Postgresql + Python = Power!Juliano Atanazio
 
MySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQLMySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQLTed Leung
 
Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).Roberto Polli
 
Oracle Deep Internal 1 (ver.2)
Oracle Deep Internal 1 (ver.2)Oracle Deep Internal 1 (ver.2)
Oracle Deep Internal 1 (ver.2)EXEM
 

Viewers also liked (20)

Psycopg2 - Connect to PostgreSQL using Python Script
Psycopg2 - Connect to PostgreSQL using Python ScriptPsycopg2 - Connect to PostgreSQL using Python Script
Psycopg2 - Connect to PostgreSQL using Python Script
 
"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014
 
Socket Programming In Python
Socket Programming In PythonSocket Programming In Python
Socket Programming In Python
 
Programming with Python - Basic
Programming with Python - BasicProgramming with Python - Basic
Programming with Python - Basic
 
Functional programming in Python
Functional programming in PythonFunctional programming in Python
Functional programming in Python
 
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQLTen Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
 
Pl/Python
Pl/PythonPl/Python
Pl/Python
 
Golang iran - tutorial go programming language - Preliminary
Golang iran - tutorial  go programming language - PreliminaryGolang iran - tutorial  go programming language - Preliminary
Golang iran - tutorial go programming language - Preliminary
 
Golang #5: To Go or not to Go
Golang #5: To Go or not to GoGolang #5: To Go or not to Go
Golang #5: To Go or not to Go
 
Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1
 
Rethink db with Python
Rethink db with PythonRethink db with Python
Rethink db with Python
 
Succumbing to the Python in Financial Markets
Succumbing to the Python in Financial MarketsSuccumbing to the Python in Financial Markets
Succumbing to the Python in Financial Markets
 
Massively Parallel Processing with Procedural Python (PyData London 2014)
Massively Parallel Processing with Procedural Python (PyData London 2014)Massively Parallel Processing with Procedural Python (PyData London 2014)
Massively Parallel Processing with Procedural Python (PyData London 2014)
 
Matando o Java e Mostrando o Python
Matando o Java e Mostrando o PythonMatando o Java e Mostrando o Python
Matando o Java e Mostrando o Python
 
Relational Database Access with Python
Relational Database Access with PythonRelational Database Access with Python
Relational Database Access with Python
 
Postgresql + Python = Power!
Postgresql + Python = Power!Postgresql + Python = Power!
Postgresql + Python = Power!
 
MySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQLMySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQL
 
Get to know PostgreSQL!
Get to know PostgreSQL!Get to know PostgreSQL!
Get to know PostgreSQL!
 
Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).
 
Oracle Deep Internal 1 (ver.2)
Oracle Deep Internal 1 (ver.2)Oracle Deep Internal 1 (ver.2)
Oracle Deep Internal 1 (ver.2)
 

Similar to Programming with Python and PostgreSQL

.gradle 파일 정독해보기
.gradle 파일 정독해보기.gradle 파일 정독해보기
.gradle 파일 정독해보기경주 전
 
The Ring programming language version 1.2 book - Part 32 of 84
The Ring programming language version 1.2 book - Part 32 of 84The Ring programming language version 1.2 book - Part 32 of 84
The Ring programming language version 1.2 book - Part 32 of 84Mahmoud Samir Fayed
 
The Ring programming language version 1.6 book - Part 46 of 189
The Ring programming language version 1.6 book - Part 46 of 189The Ring programming language version 1.6 book - Part 46 of 189
The Ring programming language version 1.6 book - Part 46 of 189Mahmoud Samir Fayed
 
Emerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the HorizonEmerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the HorizonAlex Payne
 
Relational Database Access with Python ‘sans’ ORM
Relational Database Access with Python ‘sans’ ORM  Relational Database Access with Python ‘sans’ ORM
Relational Database Access with Python ‘sans’ ORM Mark Rees
 
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak   CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak PROIDEA
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web DevelopmentCheng-Yi Yu
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with ClojureDmitry Buzdin
 
The Ring programming language version 1.7 book - Part 48 of 196
The Ring programming language version 1.7 book - Part 48 of 196The Ring programming language version 1.7 book - Part 48 of 196
The Ring programming language version 1.7 book - Part 48 of 196Mahmoud Samir Fayed
 
The Ring programming language version 1.5.2 book - Part 43 of 181
The Ring programming language version 1.5.2 book - Part 43 of 181The Ring programming language version 1.5.2 book - Part 43 of 181
The Ring programming language version 1.5.2 book - Part 43 of 181Mahmoud Samir Fayed
 
OrientDB - The 2nd generation of (multi-model) NoSQL
OrientDB - The 2nd generation of  (multi-model) NoSQLOrientDB - The 2nd generation of  (multi-model) NoSQL
OrientDB - The 2nd generation of (multi-model) NoSQLRoberto Franchini
 
Graph Algorithms: Analytics for Understanding Data Relationships
Graph Algorithms: Analytics for Understanding Data RelationshipsGraph Algorithms: Analytics for Understanding Data Relationships
Graph Algorithms: Analytics for Understanding Data RelationshipsNeo4j
 
Norikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In RubyNorikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In RubySATOSHI TAGOMORI
 
The Ring programming language version 1.5 book - Part 8 of 31
The Ring programming language version 1.5 book - Part 8 of 31The Ring programming language version 1.5 book - Part 8 of 31
The Ring programming language version 1.5 book - Part 8 of 31Mahmoud Samir Fayed
 
2 BytesC++ course_2014_c3_ function basics&parameters and overloading
2 BytesC++ course_2014_c3_ function basics&parameters and overloading2 BytesC++ course_2014_c3_ function basics&parameters and overloading
2 BytesC++ course_2014_c3_ function basics&parameters and overloadingkinan keshkeh
 
Benchy, python framework for performance benchmarking of Python Scripts
Benchy, python framework for performance benchmarking  of Python ScriptsBenchy, python framework for performance benchmarking  of Python Scripts
Benchy, python framework for performance benchmarking of Python ScriptsMarcel Caraciolo
 
The Ring programming language version 1.4.1 book - Part 13 of 31
The Ring programming language version 1.4.1 book - Part 13 of 31The Ring programming language version 1.4.1 book - Part 13 of 31
The Ring programming language version 1.4.1 book - Part 13 of 31Mahmoud Samir Fayed
 
Managing Large-scale Networks with Trigger
Managing Large-scale Networks with TriggerManaging Large-scale Networks with Trigger
Managing Large-scale Networks with Triggerjathanism
 
The Ring programming language version 1.9 book - Part 53 of 210
The Ring programming language version 1.9 book - Part 53 of 210The Ring programming language version 1.9 book - Part 53 of 210
The Ring programming language version 1.9 book - Part 53 of 210Mahmoud Samir Fayed
 

Similar to Programming with Python and PostgreSQL (20)

.gradle 파일 정독해보기
.gradle 파일 정독해보기.gradle 파일 정독해보기
.gradle 파일 정독해보기
 
Pdxpugday2010 pg90
Pdxpugday2010 pg90Pdxpugday2010 pg90
Pdxpugday2010 pg90
 
The Ring programming language version 1.2 book - Part 32 of 84
The Ring programming language version 1.2 book - Part 32 of 84The Ring programming language version 1.2 book - Part 32 of 84
The Ring programming language version 1.2 book - Part 32 of 84
 
The Ring programming language version 1.6 book - Part 46 of 189
The Ring programming language version 1.6 book - Part 46 of 189The Ring programming language version 1.6 book - Part 46 of 189
The Ring programming language version 1.6 book - Part 46 of 189
 
Emerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the HorizonEmerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the Horizon
 
Relational Database Access with Python ‘sans’ ORM
Relational Database Access with Python ‘sans’ ORM  Relational Database Access with Python ‘sans’ ORM
Relational Database Access with Python ‘sans’ ORM
 
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak   CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web Development
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with Clojure
 
The Ring programming language version 1.7 book - Part 48 of 196
The Ring programming language version 1.7 book - Part 48 of 196The Ring programming language version 1.7 book - Part 48 of 196
The Ring programming language version 1.7 book - Part 48 of 196
 
The Ring programming language version 1.5.2 book - Part 43 of 181
The Ring programming language version 1.5.2 book - Part 43 of 181The Ring programming language version 1.5.2 book - Part 43 of 181
The Ring programming language version 1.5.2 book - Part 43 of 181
 
OrientDB - The 2nd generation of (multi-model) NoSQL
OrientDB - The 2nd generation of  (multi-model) NoSQLOrientDB - The 2nd generation of  (multi-model) NoSQL
OrientDB - The 2nd generation of (multi-model) NoSQL
 
Graph Algorithms: Analytics for Understanding Data Relationships
Graph Algorithms: Analytics for Understanding Data RelationshipsGraph Algorithms: Analytics for Understanding Data Relationships
Graph Algorithms: Analytics for Understanding Data Relationships
 
Norikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In RubyNorikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In Ruby
 
The Ring programming language version 1.5 book - Part 8 of 31
The Ring programming language version 1.5 book - Part 8 of 31The Ring programming language version 1.5 book - Part 8 of 31
The Ring programming language version 1.5 book - Part 8 of 31
 
2 BytesC++ course_2014_c3_ function basics&parameters and overloading
2 BytesC++ course_2014_c3_ function basics&parameters and overloading2 BytesC++ course_2014_c3_ function basics&parameters and overloading
2 BytesC++ course_2014_c3_ function basics&parameters and overloading
 
Benchy, python framework for performance benchmarking of Python Scripts
Benchy, python framework for performance benchmarking  of Python ScriptsBenchy, python framework for performance benchmarking  of Python Scripts
Benchy, python framework for performance benchmarking of Python Scripts
 
The Ring programming language version 1.4.1 book - Part 13 of 31
The Ring programming language version 1.4.1 book - Part 13 of 31The Ring programming language version 1.4.1 book - Part 13 of 31
The Ring programming language version 1.4.1 book - Part 13 of 31
 
Managing Large-scale Networks with Trigger
Managing Large-scale Networks with TriggerManaging Large-scale Networks with Trigger
Managing Large-scale Networks with Trigger
 
The Ring programming language version 1.9 book - Part 53 of 210
The Ring programming language version 1.9 book - Part 53 of 210The Ring programming language version 1.9 book - Part 53 of 210
The Ring programming language version 1.9 book - Part 53 of 210
 

More from Peter Eisentraut

Getting Started with PL/Proxy
Getting Started with PL/ProxyGetting Started with PL/Proxy
Getting Started with PL/ProxyPeter Eisentraut
 
Linux distribution for the cloud
Linux distribution for the cloudLinux distribution for the cloud
Linux distribution for the cloudPeter Eisentraut
 
Most Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesMost Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesPeter Eisentraut
 
Porting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPorting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPeter Eisentraut
 
Porting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPorting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPeter Eisentraut
 
XML Support: Specifications and Development
XML Support: Specifications and DevelopmentXML Support: Specifications and Development
XML Support: Specifications and DevelopmentPeter Eisentraut
 
PostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePeter Eisentraut
 
The Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsThe Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsPeter Eisentraut
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsPeter Eisentraut
 
Replication Solutions for PostgreSQL
Replication Solutions for PostgreSQLReplication Solutions for PostgreSQL
Replication Solutions for PostgreSQLPeter Eisentraut
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsPeter Eisentraut
 
Docbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLDocbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLPeter Eisentraut
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Peter Eisentraut
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...Peter Eisentraut
 
The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)Peter Eisentraut
 

More from Peter Eisentraut (20)

Getting Started with PL/Proxy
Getting Started with PL/ProxyGetting Started with PL/Proxy
Getting Started with PL/Proxy
 
Linux distribution for the cloud
Linux distribution for the cloudLinux distribution for the cloud
Linux distribution for the cloud
 
Most Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesMost Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL Features
 
Porting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPorting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQL
 
Porting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPorting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQL
 
PostgreSQL and XML
PostgreSQL and XMLPostgreSQL and XML
PostgreSQL and XML
 
XML Support: Specifications and Development
XML Support: Specifications and DevelopmentXML Support: Specifications and Development
XML Support: Specifications and Development
 
PostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie Datenbankalternative
 
The Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsThe Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future Developments
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
PostgreSQL and PL/Java
PostgreSQL and PL/JavaPostgreSQL and PL/Java
PostgreSQL and PL/Java
 
Replication Solutions for PostgreSQL
Replication Solutions for PostgreSQLReplication Solutions for PostgreSQL
Replication Solutions for PostgreSQL
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
Docbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLDocbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XML
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
 
Spaß mit PostgreSQL
Spaß mit PostgreSQLSpaß mit PostgreSQL
Spaß mit PostgreSQL
 
The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)
 

Recently uploaded

Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...ScyllaDB
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Skynet Technologies
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxFIDO Alliance
 
Navigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi DaparthiNavigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi DaparthiRaviKumarDaparthi
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard37
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data SciencePaolo Missier
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxMarkSteadman7
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctBrainSell Technologies
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfSrushith Repakula
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsLeah Henrickson
 
Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfOverkill Security
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)Samir Dash
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch TuesdayIvanti
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 

Recently uploaded (20)

Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Navigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi DaparthiNavigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi Daparthi
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdf
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 

Programming with Python and PostgreSQL

  • 1. Programming with Python and PostgreSQL Peter Eisentraut peter@eisentraut.org F-Secure Corporation PostgreSQL Conference East 2011 CC-BY
  • 2. Partitioning • Part I: Client programming (60 min) • Part II: PL/Python (30 min)
  • 4. Why Python? Pros: • widely used • easy • strong typing • scripting, interactive use • good PostgreSQL support • client and server (PL) interfaces • open source, community-based
  • 5. Why Python? Pros: • widely used • easy • strong typing • scripting, interactive use • good PostgreSQL support • client and server (PL) interfaces • open source, community-based Pros: • no static syntax checks, must rely on test coverage • Python community has varying interest in RDBMS
  • 7. Example import psycopg2 dbconn = psycopg2.connect('dbname=dellstore2') cursor = dbconn.cursor() cursor.execute(""" SELECT firstname, lastname FROM customers ORDER BY 1, 2 LIMIT 10 """) for row in cursor.fetchall(): print "Name: %s %s" % (row[0], row[1]) cursor.close() db.close()
  • 8. Drivers Name License Platforms Py Versions Psycopg LGPL Unix, Win 2.4–3.2 PyGreSQL BSD Unix, Win 2.3–2.6 ocpgdb BSD Unix 2.3–2.6 py-postgresql BSD pure Python 3.0+ bpgsql (alpha) LGPL pure Python 2.3–2.6 pg8000 BSD pure Python 2.5–3.0+
  • 9. Drivers Name License Platforms Py Versions Psycopg LGPL Unix, Win 2.4–3.2 PyGreSQL BSD Unix, Win 2.3–2.6 ocpgdb BSD Unix 2.3–2.6 py-postgresql BSD pure Python 3.0+ bpgsql (alpha) LGPL pure Python 2.3–2.6 pg8000 BSD pure Python 2.5–3.0+ More details • http://wiki.postgresql.org/wiki/Python • http://wiki.python.org/moin/PostgreSQL
  • 10. DB-API 2.0 • the standard Python database API • all mentioned drivers support it • defined in PEP 249 • discussions: db-sig@python.org • very elementary (from a PostgreSQL perspective) • outdated relative to Python language development • lots of extensions and incompatibilities possible
  • 11. Higher-Level Interfaces • Zope • SQLAlchemy • Django
  • 12. Psycopg Facts • Main authors: Federico Di Gregorio, Daniele Varrazzo • License: LGPLv3+ • Web site: http://initd.org/psycopg/ • Documentation: http://initd.org/psycopg/docs/ • Git, Gitweb • Mailing list: psycopg@postgresql.org • Twitter: @psycopg • Latest version: 2.4 (February 27, 2011)
  • 13. Using the Driver import psycopg2 dbconn = psycopg2.connect(...) ...
  • 14. Driver Independence? import psycopg2 dbconn = psycopg2.connect(...) # hardcodes driver name
  • 15. Driver Independence? import psycopg2 as dbdriver dbconn = dbdriver.connect(...)
  • 16. Driver Independence? dbtype = 'psycopg2' # e.g. from config file dbdriver = __import__(dbtype, globals(), locals(), [], -1) dbconn = dbdriver.connect(...)
  • 17. Connecting # libpq-like connection string dbconn = psycopg2.connect('dbname=dellstore2 host=localhost port=5432') # same dbconn = psycopg2.connect(dsn='dbname=dellstore2 host=localhost port=5432') # keyword arguments # (not all possible libpq options supported) dbconn = psycopg2.connect(database='dellstore2', host='localhost', port='5432') DB-API 2.0 says: arguments database dependent
  • 18. “Cursors” cursor = dbconn.cursor() • not a real database cursor, only an API abstraction • think “statement handle”
  • 19. Server-Side Cursors cursor = dbconn.cursor(name='mycursor') • a real database cursor • use for large result sets
  • 20. Executing # queries cursor.execute(""" SELECT firstname, lastname FROM customers ORDER BY 1, 2 LIMIT 10 """) # updates cursor.execute("UPDATE customers SET password = NULL") print "%d rows updated" % cursor.rowcount # or anything else cursor.execute("ANALYZE customers")
  • 21. Fetching Query Results cursor.execute("SELECT firstname, lastname FROM ...") cursor.fetchall() [('AABBKO', 'DUTOFRPLOK'), ('AABTSI', 'ZFCKMPRVVJ'), ('AACOHS', 'EECCQPVTIW'), ('AACVVO', 'CLSXSGZYKS'), ('AADVMN', 'MEMQEWYFYE'), ('AADXQD', 'GLEKVVLZFV'), ('AAEBUG', 'YUOIINRJGE')]
  • 22. Fetching Query Results cursor.execute("SELECT firstname, lastname FROM ...") for row in cursor.fetchall(): print "Name: %s %s" % (row[0], row[1])
  • 23. Fetching Query Results cursor.execute("SELECT firstname, lastname FROM ...") for row in cursor.fetchall(): print "Name: %s %s" % (row[0], row[1]) Note: field access only by number
  • 24. Fetching Query Results cursor.execute("SELECT firstname, lastname FROM ...") row = cursor.fetchone() if row is not None: print "Name: %s %s" % (row[0], row[1])
  • 25. Fetching Query Results cursor.execute("SELECT firstname, lastname FROM ...") for row in cursor: print "Name: %s %s" % (row[0], row[1])
  • 26. Fetching Query Results in Batches cursor = dbconn.cursor(name='mycursor') cursor.arraysize = 500 # default: 1 cursor.execute("SELECT firstname, lastname FROM ...") while True: batch = cursor.fetchmany() break if not batch for row in batch: print "Name: %s %s" % (row[0], row[1])
  • 27. Fetching Query Results in Batches cursor = dbconn.cursor(name='mycursor') cursor.execute("SELECT firstname, lastname FROM ...") cursor.itersize = 2000 # default for row in cursor: print "Name: %s %s" % (row[0], row[1])
  • 28. Getting Query Metadata cursor.execute("SELECT DISTINCT state, zip FROM customers") print cursor.description[0].name print cursor.description[0].type_code print cursor.description[1].name print cursor.description[1].type_code state 1043 # == psycopg2.STRING zip 23 # == psycopg2.NUMBER
  • 29. Passing Parameters cursor.execute(""" UPDATE customers SET password = %s WHERE customerid = %s """, ["sekret", 37])
  • 30. Passing Parameters Not to be confused with (totally evil): cursor.execute(""" UPDATE customers SET password = '%s' WHERE customerid = %d """ % ["sekret", 37])
  • 31. Passing Parameters cursor.execute("INSERT INTO foo VALUES (%s)", "bar") # WRONG cursor.execute("INSERT INTO foo VALUES (%s)", ("bar")) # WRONG cursor.execute("INSERT INTO foo VALUES (%s)", ("bar",)) # correct cursor.execute("INSERT INTO foo VALUES (%s)", ["bar"]) # correct (from Psycopg documentation)
  • 32. Passing Parameters cursor.execute(""" UPDATE customers SET password = %(pw)s WHERE customerid = %(id)s """, {'id': 37, 'pw': "sekret"})
  • 33. Passing Many Parameter Sets cursor.executemany(""" UPDATE customers SET password = %s WHERE customerid = %s """, [["ahTh4oip", 100], ["Rexahho7", 101], ["Ee1aetui", 102]])
  • 34. Calling Procedures cursor.callproc('pg_start_backup', 'label')
  • 35. Data Types from decimal import Decimal from psycopg2 import Date cursor.execute(""" INSERT INTO orders (orderdate, customerid, netamount, tax, totalamount) VALUES (%s, %s, %s, %s, %s)""", [Date(2011, 03, 23), 12345, Decimal("899.95"), 8.875, Decimal("979.82")])
  • 36. Mogrify from decimal import Decimal from psycopg2 import Date cursor.mogrify(""" INSERT INTO orders (orderdate, customerid, netamount, tax, totalamount) VALUES (%s, %s, %s, %s, %s)""", [Date(2011, 03, 23), 12345, Decimal("899.95"), 8.875, Decimal("979.82")]) Result: "nINSERT INTO orders (orderdate, customerid,n netamount, tax, totalamount)nVALUES ('2011-03-23'::date, 12345, 899.95, 8.875, 979.82)"
  • 37. Data Types cursor.execute(""" SELECT * FROM orders WHERE customerid = 12345 """) Result: (12002, datetime.date(2011, 3, 23), 12345, Decimal('899.95'), Decimal('8.88'), Decimal('979.82'))
  • 38. Nulls Input: cursor.mogrify("SELECT %s", [None]) 'SELECT NULL' Output: cursor.execute("SELECT NULL") cursor.fetchone() (None,)
  • 39. Booleans cursor.mogrify("SELECT %s, %s", [True, False]) 'SELECT true, false'
  • 40. Binary Data Standard way: from psycopg2 import Binary cursor.mogrify("SELECT %s", [Binary("foo")]) "SELECT E'x666f6f'::bytea"
  • 41. Binary Data Standard way: from psycopg2 import Binary cursor.mogrify("SELECT %s", [Binary("foo")]) "SELECT E'x666f6f'::bytea" Other ways: cursor.mogrify("SELECT %s", [buffer("foo")]) "SELECT E'x666f6f'::bytea" cursor.mogrify("SELECT %s", [bytearray.fromhex(u"deadbeef")]) "SELECT E'xdeadbeef'::bytea" There are more. Check the documentation. Check the versions.
  • 42. Date/Time Standard ways: from psycopg2 import Date, Time, Timestamp cursor.mogrify("SELECT %s, %s, %s", [Date(2011, 3, 23), Time(9, 0, 0), Timestamp(2011, 3, 23, 9, 0, 0)]) "SELECT '2011-03-23'::date, '09:00:00'::time, '2011-03-23T09:00:00'::timestamp"
  • 43. Date/Time Other ways: import datetime cursor.mogrify("SELECT %s, %s, %s, %s", [datetime.date(2011, 3, 23), datetime.time(9, 0, 0), datetime.datetime(2011, 3, 23, 9, 0), datetime.timedelta(minutes=90)]) "SELECT '2011-03-23'::date, '09:00:00'::time, '2011-03-23T09:00:00'::timestamp, '0 days 5400.000000 seconds'::interval" mx.DateTime also supported
  • 44. Arrays foo = [1, 2, 3] bar = [datetime.time(9, 0), datetime.time(10, 30)] cursor.mogrify("SELECT %s, %s", [foo, bar]) "SELECT ARRAY[1, 2, 3], ARRAY['09:00:00'::time, '10:30:00'::time]"
  • 45. Tuples foo = (1, 2, 3) cursor.mogrify("SELECT * FROM customers WHERE customerid IN %s", [foo]) 'SELECT * FROM customers WHERE customerid IN (1, 2, 3)'
  • 46. Hstore import psycopg2.extras psycopg2.extras.register_hstore(cursor) x = {'a': 'foo', 'b': 'bar'} cursor.mogrify("SELECT %s", [x]) "SELECT hstore(ARRAY[E'a', E'b'], ARRAY[E'foo', E'bar'])"
  • 47. Unicode Support Cause all result strings to be returned as Unicode strings: psycopg2.extensions.register_type(psycopg2.extensions. UNICODE) psycopg2.extensions.register_type(psycopg2.extensions. UNICODEARRAY)
  • 48. Transaction Control Transaction blocks are used by default. Must use dbconn.commit() or dbconn.rollback()
  • 49. Transaction Control: Autocommit import psycopg2.extensions dbconn.set_isolation_level(psycopg2.extensions. ISOLATION_LEVEL_AUTOCOMMIT) cursor = dbconn.cursor() cursor.execute("VACUUM")
  • 50. Transaction Control: Isolation Mode import psycopg2.extensions dbconn.set_isolation_level(psycopg2.extensions. ISOLATION_LEVEL_SERIALIZABLE) # or other level cursor = dbconn.cursor() cursor.execute(...) ... dbconn.commit()
  • 51. Exception Handling StandardError |__ Warning |__ Error |__ InterfaceError |__ DatabaseError |__ DataError |__ OperationalError | |__ psycopg2.extensions.QueryCanceledError | |__ psycopg2.extensions.TransactionRollbackError |__ IntegrityError |__ InternalError |__ ProgrammingError |__ NotSupportedError
  • 52. Error Messages try: cursor.execute("boom") except Exception, e: print e.pgerror
  • 53. Error Codes import psycopg2.errorcodes while True: try: cursor.execute("UPDATE something ...") cursor.execute("UPDATE otherthing ...") break except Exception, e: if e.pgcode == psycopg2.errorcodes.SERIALIZATION_FAILURE: continue else: raise
  • 54. Connection and Cursor Factories Want: accessing result columns by name Recall: dbconn = psycopg2.connect(dsn='...') cursor = dbconn.cursor() cursor.execute(""" SELECT firstname, lastname FROM customers ORDER BY 1, 2 LIMIT 10 """) for row in cursor.fetchall(): print "Name: %s %s" % (row[0], row[1]) # stupid :(
  • 55. Connection and Cursor Factories Solution 1: Using DictConnection: import psycopg2.extras dbconn = psycopg2.connect(dsn='...', connection_factory=psycopg2.extras.DictConnection) cursor = dbconn.cursor() cursor.execute(""" SELECT firstname, lastname FROM customers ORDER BY 1, 2 LIMIT 10 """) for row in cursor.fetchall(): print "Name: %s %s" % (row['firstname'], # or row[0] row['lastname']) # or row[1]
  • 56. Connection and Cursor Factories Solution 2: Using RealDictConnection: import psycopg2.extras dbconn = psycopg2.connect(dsn='...', connection_factory=psycopg2.extras.RealDictConnection) cursor = dbconn.cursor() cursor.execute(""" SELECT firstname, lastname FROM customers ORDER BY 1, 2 LIMIT 10 """) for row in cursor.fetchall(): print "Name: %s %s" % (row['firstname'], row['lastname'])
  • 57. Connection and Cursor Factories Solution 3: Using NamedTupleConnection: import psycopg2.extras dbconn = psycopg2.connect(dsn='...', connection_factory=psycopg2.extras.NamedTupleConnection) cursor = dbconn.cursor() cursor.execute(""" SELECT firstname, lastname FROM customers ORDER BY 1, 2 LIMIT 10 """) for row in cursor.fetchall(): print "Name: %s %s" % (row.firstname, # or row[0] row.lastname) # or row[1]
  • 58. Connection and Cursor Factories Alternative: Using DictCursor/RealDictCursor/NamedTupleCursor: import psycopg2.extras dbconn = psycopg2.connect(dsn='...') cursor = dbconn.cursor(cursor_factory=psycopg2.extras. DictCursor/RealDictCursor/NameTupleCursor) cursor.execute(""" SELECT firstname, lastname FROM customers ORDER BY 1, 2 LIMIT 10 """) for row in cursor.fetchall(): print "Name: %s %s" % (row['firstname'], row['lastname']) # (resp. row.firstname, row.lastname)
  • 59. Supporting New Data Types Only a finite list of types is supported by default: Date, Binary, etc. • map new PostgreSQL data types into Python • map new Python data types into PostgreSQL
  • 60. Mapping New PostgreSQL Types Into Python import psycopg2 import psycopg2.extensions def cast_oidvector(value, _cursor): """Convert oidvector to Python array""" if value is None: return None return map(int, value.split(' ')) OIDVECTOR = psycopg2.extensions.new_type((30,), 'OIDVECTOR', cast_oidvector) psycopg2.extensions.register_type(OIDVECTOR)
  • 61. Mapping New Python Types into PostgreSQL from psycopg2.extensions import adapt, register_adapter, AsIs class Point(object): def __init__(self, x, y): self.x = x self.y = y def adapt_point(point): return AsIs("'(%s, %s)'" % (adapt(point.x), adapt(point.y))) register_adapter(Point, adapt_point) cur.execute("INSERT INTO atable (apoint) VALUES (%s)", (Point(1.23, 4.56),)) (from Psycopg documentation)
  • 62. Connection Pooling With Psycopg from psycopg2.pool import SimpleConnectionPool pool = SimpleConnectionPool(1, 20, dsn='...') dbconn = pool.getconn() ... pool.putconn(dbconn) pool.closeall()
  • 63. Connection Pooling With Psycopg for non-threaded applications: from psycopg2.pool import SimpleConnectionPool pool = SimpleConnectionPool(1, 20, dsn='...') dbconn = pool.getconn() ... pool.putconn(dbconn) pool.closeall() for non-threaded applications: from psycopg2.pool import ThreadedConnectionPool pool = ThreadedConnectionPool(1, 20, dsn='...') dbconn = pool.getconn() cursor = dbconn.cursor() ... pool.putconn(dbconn) pool.closeall()
  • 64. Connection Pooling With DBUtils import psycopg2 from DBUtils.PersistentDB import PersistentDB dbconn = PersistentDB(psycopg2, dsn='...') cursor = dbconn.cursor() ... see http://pypi.python.org/pypi/DBUtils/
  • 65. The Other Stuff • thread safety: can share connections, but not cursors • COPY support: cursor.copy_from(), cursor.copy_to() • large object support: connection.lobject() • 2PC: connection.xid(), connection.tpc_begin(), . . . • query cancel: dbconn.cancel() • notices: dbconn.notices • notifications: dbconn.notifies • asynchronous communication • coroutine support • logging cursor
  • 67. Setup • included with PostgreSQL • configure --with-python • apt-get/yum install postgresql-plpython • CREATE LANGUAGE plpythonu; • Python 3: CREATE LANGUAGE plpython3u; • “untrusted”, superuser only
  • 68. Basic Examples CREATE FUNCTION add(a int, b int) RETURNS int LANGUAGE plpythonu AS $$ return a + b $$; CREATE FUNCTION longest(a text, b text) RETURNS text LANGUAGE plpythonu AS $$ if len(a) > len(b): return a elif len(b) > len(a): return b else: return None $$;
  • 69. Using Modules CREATE FUNCTION json_to_array(j text) RETURNS text[] LANGUAGE plpythonu AS $$ import json return json.loads(j) $$;
  • 70. Database Calls CREATE FUNCTION clear_passwords() RETURNS int LANGUAGE plpythonu AS $$ rv = plpy.execute("UPDATE customers SET password = NULL") return rv.nrows $$;
  • 71. Database Calls With Parameters CREATE FUNCTION set_password(username text, password text) RETURNS boolean LANGUAGE plpythonu AS $$ plan = plpy.prepare("UPDATE customers SET password = $1 WHERE username= $2", ['text', 'text']) rv = plpy.execute(plan, [username, password]) return rv.nrows == 1 $$;
  • 72. Avoiding Prepared Statements CREATE FUNCTION set_password(username text, password text) RETURNS boolean LANGUAGE plpythonu AS $$ rv = plpy.execute("UPDATE customers SET password = %s WHERE username= %s" % (plpy.quote_nullable(username), plpy.quote_literal(password))) return rv.nrows == 1 $$; (available in 9.1-to-be)
  • 73. Caching Plans CREATE FUNCTION set_password2(username text, password text) RETURNS boolean LANGUAGE plpythonu AS $$ if 'myplan' in SD: plan = SD['myplan'] else: plan = plpy.prepare("UPDATE customers SET password = $1 WHERE username= $2", ['text', 'text']) SD['myplan'] = plan rv = plpy.execute(plan, [username, password]) return rv.nrows == 1 $$;
  • 74. Processing Query Results CREATE FUNCTION get_customer_name(username text) RETURNS boolean LANGUAGE plpythonu AS $$ plan = plpy.prepare("SELECT firstname || ' ' || lastname AS ""name"" FROM customers WHERE username = $1", ['text']) rv = plpy.execute(plan, [username], 1) return rv[0]['name'] $$;
  • 75. Compare: PL/Python vs. DB-API PL/Python: plan = plpy.prepare("SELECT ...") for row in plpy.execute(plan, ...): plpy.info(row["fieldname"]) DB-API: dbconn = psycopg2.connect(...) cursor = dbconn.cursor() cursor.execute("SELECT ...") for row in cursor.fetchall() do: print row[0]
  • 76. Set-Returning and Table Functions CREATE FUNCTION get_customers(id int) RETURNS SETOF customers LANGUAGE plpythonu AS $$ plan = plpy.prepare("SELECT * FROM customers WHERE customerid = $1", ['int']) rv = plpy.execute(plan, [id]) return rv $$;
  • 77. Triggers CREATE FUNCTION delete_notifier() RETURNS trigger LANGUAGE plpythonu AS $$ if TD['event'] == 'DELETE': plpy.notice("one row deleted from table %s" % TD['table_name']) $$; CREATE TRIGGER customers_delete_notifier AFTER DELETE ON customers FOR EACH ROW EXECUTE PROCEDURE delete_notifier();
  • 78. Exceptions CREATE FUNCTION test() RETURNS text LANGUAGE plpythonu AS $$ try: rv = plpy.execute("SELECT ...") except plpy.SPIError, e: plpy.notice("something went wrong") The transaction is still aborted in < 9.1.
  • 79. New in PostgreSQL 9.1 • SPI calls wrapped in subtransactions • custom SPI exceptions: subclass per SQLSTATE, .sqlstate attribute • plpy.subtransaction() context manager • support for OUT parameters • quoting functions • validator • lots of internal improvements