Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Trees and More
            With PostgreSQL
   Common Table Expressions
   Percona Conference 2009


Copyright © 2009
David...
Some Approaches

• External Processing
• Functions and Stored Procedures
• Materialized Path Enumeration
• Nested Set
External Processing

•   Pros:

    •   Lots of Language Choices

•   Cons:

    •   Scaling

    •   No Relational Operat...
Functions and Stored
            Procedures
•   Pros:

    •   Easy to visualize

    •   Easy to write

•   Cons:

    • ...
Materialized Path Enumeration

  •   Pros:

      •   Easy to visualize

  •   Cons:

      •   DDL for each hierarchy

  ...
Nested Sets
•   Pros:

    •   Easy to manipulate on SELECT

    •   Sounds cool

    •   Endorsed by a famous bald guy

•...
Thanks to:
•   The ISO SQL Standards Committee

•   Yoshiyuki Asaba

•   Ishii Tatsuo

•   Jeff Davis

•   Gregory Stark

...
SQL Standard


• Common Table Expressions
 (CTE)
Recursion
Recursion in General

•
Initial Condition
•
Recursion step
•
Termination
condition
E1 List Table
CREATE TABLE employee(
    id INTEGER NOT NULL,
    boss_id INTEGER,
    UNIQUE(id, boss_id)/*, etc., etc. *...
Tree Query Initiation
WITH RECURSIVE t(node, path) AS (
   SELECT id, ARRAY[id] FROM employee WHERE boss_id IS NULL
   /* ...
Tree Query Recursion
WITH RECURSIVE t(node, path) AS (
   SELECT id, ARRAY[id] FROM employee WHERE boss_id IS NULL
UNION A...
Tree Query
                 Termination
WITH RECURSIVE t(node, path) AS (
    SELECT id, ARRAY[id] FROM employee WHERE bos...
Tree Query Display
WITH RECURSIVE t(node, path) AS (
   SELECT id, ARRAY[id] FROM employee WHERE boss_id IS NULL
UNION ALL...
Tree Query Initiation
           Branch
        ----------
         1
         +-2
         +---5
         +-----11
      ...
Travelling Salesman Problem
Given a number of cities and the costs of travelling
from any city to any other city, what is ...
TSP Schema
CREATE TABLE pairs (
    from_city TEXT NOT NULL,
    to_city TEXT NOT NULL,
    distance INTEGER NOT NULL,
   ...
TSP Data
INSERT INTO pairs
VALUES
    ('Bari','Bologna',672),
    ('Bari','Bolzano',939),
    ('Bari','Firenze',723),
    ...
TSP Program:
     Symmetric Setup

 WITH RECURSIVE both_ways(
      from_city,
      to_city,
      distance
 )           ...
TSP Program:
     Symmetric Setup

 WITH RECURSIVE both_ways(
     from_city,
     to_city,
     distance
 )
 AS (/* Dista...
TSP Program:
    Symmetric Setup

 WITH RECURSIVE both_ways(
      from_city,
      to_city,
      distance
 )
 AS (
     ...
TSP Program:
      Path Initialization Step
paths (
     from_city,
     to_city,
     distance,
     path
)
AS (
     SEL...
TSP Program:
             Path Recursion Step

    SELECT
         b2.from_city,
         b2.to_city,
         p.distance ...
TSP Program:
          Timely Termination Step

    SELECT
         b2.from_city,
         b2.to_city,
         p.distance...
TSP Program:
               Filter and Display

SELECT
     path || to_city AS quot;pathquot;,
     distance
FROM
     pat...
TSP Program:
                 Filter and Display



davidfetter@tsp=# i travelling_salesman.sql
               path       ...
Hausdorf-Besicovich-
          WTF Dimension
WITH RECURSIVE
Z(Ix, Iy, Cx, Cy, X, Y, I)
AS (
     SELECT Ix, Iy, X::float, ...
Filter
Zt (Ix, Iy, I) AS (
    SELECT Ix, Iy, MAX(I) AS I
    FROM Z
    GROUP BY Iy, Ix
    ORDER BY Iy, Ix
)
Display

SELECT array_to_string(
    array_agg(
         SUBSTRING(
             ' .,,,-----++++%%%%@@@@#### ',
          ...
Return Map
Questions?
Comments?
Straitjackets?
Thanks!
Copyright © 2009
David Fetter david.fetter@pgexperts.com
All Rights Reserved
Trees And  More With  Postgre S Q L
Upcoming SlideShare
Loading in …5
×

Trees And More With Postgre S Q L

4,428 views

Published on

Published in: Economy & Finance, Business
  • Be the first to comment

Trees And More With Postgre S Q L

  1. 1. Trees and More With PostgreSQL Common Table Expressions Percona Conference 2009 Copyright © 2009 David Fetter david.fetter@pgexperts.com All Rights Reserved
  2. 2. Some Approaches • External Processing • Functions and Stored Procedures • Materialized Path Enumeration • Nested Set
  3. 3. External Processing • Pros: • Lots of Language Choices • Cons: • Scaling • No Relational Operators
  4. 4. Functions and Stored Procedures • Pros: • Easy to visualize • Easy to write • Cons: • Scaling • Hard to do relational operators
  5. 5. Materialized Path Enumeration • Pros: • Easy to visualize • Cons: • DDL for each hierarchy • Full-table changes on write • Hard to do relational operators
  6. 6. Nested Sets • Pros: • Easy to manipulate on SELECT • Sounds cool • Endorsed by a famous bald guy • Cons: • DDL for each hierarchy • Full-table changes on write • Hard to do relational operators
  7. 7. Thanks to: • The ISO SQL Standards Committee • Yoshiyuki Asaba • Ishii Tatsuo • Jeff Davis • Gregory Stark • Tom Lane • etc., etc., etc.
  8. 8. SQL Standard • Common Table Expressions (CTE)
  9. 9. Recursion
  10. 10. Recursion in General • Initial Condition • Recursion step • Termination condition
  11. 11. E1 List Table CREATE TABLE employee( id INTEGER NOT NULL, boss_id INTEGER, UNIQUE(id, boss_id)/*, etc., etc. */ ); INSERT INTO employee(id, boss_id) VALUES(1,NULL), /* El capo di tutti capi */ (2,1),(3,1),(4,1), (5,2),(6,2),(7,2),(8,3),(9,3),(10,4), (11,5),(12,5),(13,6),(14,7),(15,8), (1,9);
  12. 12. Tree Query Initiation WITH RECURSIVE t(node, path) AS ( SELECT id, ARRAY[id] FROM employee WHERE boss_id IS NULL /* Initiation Step */ UNION ALL SELECT e1.id, t.path || ARRAY[e1.id] FROM employee e1 JOIN t ON (e1.boss_id = t.node) WHERE id NOT IN (t.path) ) SELECT CASE WHEN array_upper(path,1)>1 THEN '+-' ELSE '' END || REPEAT('--', array_upper(path,1)-2) || node AS quot;Branchquot; FROM t ORDER BY path;
  13. 13. Tree Query Recursion WITH RECURSIVE t(node, path) AS ( SELECT id, ARRAY[id] FROM employee WHERE boss_id IS NULL UNION ALL SELECT e1.id, t.path || ARRAY[e1.id] /* Recursion */ FROM employee e1 JOIN t ON (e1.boss_id = t.node) WHERE id NOT IN (t.path) ) SELECT CASE WHEN array_upper(path,1)>1 THEN '+-' ELSE '' END || REPEAT('--', array_upper(path,1)-2) || node AS quot;Branchquot; FROM t ORDER BY path;
  14. 14. Tree Query Termination WITH RECURSIVE t(node, path) AS ( SELECT id, ARRAY[id] FROM employee WHERE boss_id IS NULL UNION ALL SELECT e1.id, t.path || ARRAY[e1.id] FROM employee e1 JOIN t ON (e1.boss_id = t.node) WHERE id NOT IN (t.path) /* Termination Condition */ ) SELECT CASE WHEN array_upper(path,1)>1 THEN '+-' ELSE '' END || REPEAT('--', array_upper(path,1)-2) || node AS quot;Branchquot; FROM t ORDER BY path;
  15. 15. Tree Query Display WITH RECURSIVE t(node, path) AS ( SELECT id, ARRAY[id] FROM employee WHERE boss_id IS NULL UNION ALL SELECT e1.id, t.path || ARRAY[e1.id] FROM employee e1 JOIN t ON (e1.boss_id = t.node) WHERE id NOT IN (t.path) ) SELECT CASE WHEN array_upper(path,1)>1 THEN '+-' ELSE '' END || REPEAT('--', array_upper(path,1)-2) || node AS quot;Branchquot; /* Display */ FROM t ORDER BY path;
  16. 16. Tree Query Initiation Branch ---------- 1 +-2 +---5 +-----11 +-----12 +---6 +-----13 +---7 +-----14 +-3 +---8 +-----15 +---9 +-4 +---10 +-9 (16 rows)
  17. 17. Travelling Salesman Problem Given a number of cities and the costs of travelling from any city to any other city, what is the least- cost round-trip route that visits each city exactly once and then returns to the starting city?
  18. 18. TSP Schema CREATE TABLE pairs ( from_city TEXT NOT NULL, to_city TEXT NOT NULL, distance INTEGER NOT NULL, PRIMARY KEY(from_city, to_city), CHECK (from_city < to_city) );
  19. 19. TSP Data INSERT INTO pairs VALUES ('Bari','Bologna',672), ('Bari','Bolzano',939), ('Bari','Firenze',723), ('Bari','Genova',944), ('Bari','Milan',881), ('Bari','Napoli',257), ('Bari','Palermo',708), ('Bari','Reggio Calabria',464), ....
  20. 20. TSP Program: Symmetric Setup WITH RECURSIVE both_ways( from_city, to_city, distance ) /* Working Table */ AS ( SELECT from_city, to_city, distance FROM pairs UNION ALL SELECT to_city AS quot;from_cityquot;, from_city AS quot;to_cityquot;, distance FROM pairs ),
  21. 21. TSP Program: Symmetric Setup WITH RECURSIVE both_ways( from_city, to_city, distance ) AS (/* Distances One Way */ SELECT from_city, to_city, distance FROM pairs UNION ALL SELECT to_city AS quot;from_cityquot;, from_city AS quot;to_cityquot;, distance FROM pairs ),
  22. 22. TSP Program: Symmetric Setup WITH RECURSIVE both_ways( from_city, to_city, distance ) AS ( SELECT from_city, to_city, distance FROM pairs UNION ALL /* Distances Other Way */ SELECT to_city AS quot;from_cityquot;, from_city AS quot;to_cityquot;, distance FROM pairs ),
  23. 23. TSP Program: Path Initialization Step paths ( from_city, to_city, distance, path ) AS ( SELECT from_city, to_city, distance, ARRAY[from_city] AS quot;pathquot; FROM both_ways b1 WHERE b1.from_city = 'Roma' UNION ALL
  24. 24. TSP Program: Path Recursion Step SELECT b2.from_city, b2.to_city, p.distance + b2.distance, p.path || b2.from_city FROM both_ways b2 JOIN paths p ON ( p.to_city = b2.from_city AND b2.from_city <> ALL (p.path[ 2:array_upper(p.path,1) ]) /* Prevent re-tracing */ AND array_upper(p.path,1) < 6 ) )
  25. 25. TSP Program: Timely Termination Step SELECT b2.from_city, b2.to_city, p.distance + b2.distance, p.path || b2.from_city FROM both_ways b2 JOIN paths p ON ( p.to_city = b2.from_city AND b2.from_city <> ALL (p.path[ 2:array_upper(p.path,1) ]) /* Prevent re-tracing */ AND array_upper(p.path,1) < 6 /* Timely Termination */ ) )
  26. 26. TSP Program: Filter and Display SELECT path || to_city AS quot;pathquot;, distance FROM paths WHERE to_city = 'Roma' AND ARRAY['Milan','Firenze','Napoli'] <@ path ORDER BY distance, path LIMIT 1;
  27. 27. TSP Program: Filter and Display davidfetter@tsp=# i travelling_salesman.sql path | distance ----------------------------------+---------- {Roma,Firenze,Milan,Napoli,Roma} | 1553 (1 row) Time: 11679.503 ms
  28. 28. Hausdorf-Besicovich- WTF Dimension WITH RECURSIVE Z(Ix, Iy, Cx, Cy, X, Y, I) AS ( SELECT Ix, Iy, X::float, Y::float, X::float, Y::float, 0 FROM (SELECT -2.2 + 0.031 * i, i FROM generate_series(0,101) AS i) AS xgen(x,ix) CROSS JOIN (SELECT -1.5 + 0.031 * i, i FROM generate_series(0,101) AS i) AS ygen(y,iy) UNION ALL SELECT Ix, Iy, Cx, Cy, X * X - Y * Y + Cx AS X, Y * X * 2 + Cy, I + 1 FROM Z WHERE X * X + Y * Y < 16::float AND I < 100 ),
  29. 29. Filter Zt (Ix, Iy, I) AS ( SELECT Ix, Iy, MAX(I) AS I FROM Z GROUP BY Iy, Ix ORDER BY Iy, Ix )
  30. 30. Display SELECT array_to_string( array_agg( SUBSTRING( ' .,,,-----++++%%%%@@@@#### ', GREATEST(I,1) ),'' ) FROM Zt GROUP BY Iy ORDER BY Iy;
  31. 31. Return Map
  32. 32. Questions? Comments? Straitjackets?
  33. 33. Thanks! Copyright © 2009 David Fetter david.fetter@pgexperts.com All Rights Reserved

×