Fun with SQL

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN Fun With SQL INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Joshua Tolley Operations Subqueries End Point Corporation Set Operations Common Operations Advanced Operations Common Table October 5, 2009 Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN ”The degree of normality in a database is inversely Self Joins proportional to that of its DBA.” - Anon, twitter Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Why Not Do Stuff in SQL Joshua Tolley End Point Databases are harder to replicate, if you really need to Corporation scale out Why and Why Not Often, one complex SQL query is more efficient than Joins several simple ones CROSS JOIN Sometimes, indeed, it’s useful to reduce the load on the INNER JOIN OUTER JOIN database by moving logic into the application. Be NATURAL JOIN careful doing this Self Joins Other Useful c.f. Premature Optimization Operations Subqueries More complex queries are harder to write and debug Set Operations Common Operations True. But so is more complex programming. Advanced Operations More complex queries are harder for the next guy to Common Table Expressions maintain Window Functions Also, good DBAs are often more expensive than good Real, Live Queries Something Simple programmers Something Fun These are both true. But complex programming is also Key Points hard for the next guy to maintain Of all the reasons not to write fluent SQL, this is probably the most widely applicable

Fun With SQL Why do stuﬀ in SQL? Joshua Tolley End Point Corporation Why and Why Not The database is more eﬃcient than your application for Joins processing big chunks of data CROSS JOIN INNER JOIN ...especially if your code is in an interpreted language OUTER JOIN NATURAL JOIN The database is better tested than your application Self Joins Applications trying to do what SQL should be doing Other Useful Operations often get big and complex quickly Subqueries Set Operations ...and also buggy quickly Common Operations That’s what the database is there for Advanced Operations SQL is designed to express relations and conditions on Common Table Expressions Window Functions them. Your application’s language isn’t. Real, Live Queries A better understanding of SQL allows you to write Something Simple Something Fun queries that perform better Key Points

Fun With SQL Why do stuﬀ in SQL? Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins In short, the database exists to manage data, and your Other Useful Operations application exists to handle business logic. Write software Subqueries accordingly. Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins So let’s get started... Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Tables we’ll use Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a; # SELECT * FROM b; INNER JOIN OUTER JOIN NATURAL JOIN id | value id | value Self Joins ----+------- ----+------- Other Useful Operations 1 | a1 5 | b5 Subqueries Set Operations 2 | a2 4 | b4 Common Operations 3 | a3 3 | b3 Advanced Operations 4 | a4 6 | b6 Common Table Expressions (4 rows) (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL JOINs Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN If you want data from multiple tables, you probably Self Joins Other Useful want a join Operations ...but see also Subqueries, later on Subqueries Set Operations Common Operations There are several diﬀerent kinds of joins Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL JOINs Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN <table1> [alias1] OUTER JOIN [ [ [NATURAL] [ [FULL | RIGHT | LEFT] [OUTER] | NATURAL JOIN Self Joins INNER] ] | CROSS ] JOIN Other Useful Operations <table2> [alias2] Subqueries Set Operations [USING (...) | Common Operations ON (<value1> <op> <value2> Advanced Operations [,<value3> <op> <value4>...] ) ] Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL CROSS JOIN Joshua Tolley End Point Corporation Why and Why Not Joins SELECT <... >FROM table1 JOIN table2 CROSS JOIN INNER JOIN With no explicit join type and no join qualiﬁers (an ON OUTER JOIN NATURAL JOIN clause, WHERE clause involving both relations, etc.) Self Joins this is a CROSS JOIN Other Useful Operations Equivalent to Subqueries Set Operations Common Operations SELECT <... >FROM table1, table2 Advanced SELECT <... >FROM table1 CROSS JOIN table2 Operations Common Table ”Cartesian product” of the two relations Expressions Window Functions Combines every row of table1 with every row of table2 Real, Live Queries Makes LOTS of rows, and can thus be very slow Something Simple Something Fun Key Points

Fun With SQL CROSS JOIN Joshua Tolley End Point Corporation # SELECT * FROM a, b; Why and Why Not id | value | id | value Joins CROSS JOIN ----+-------+----+------- INNER JOIN OUTER JOIN 1 | a1 | 5 | b5 NATURAL JOIN Self Joins 1 | a1 | 4 | b4 Other Useful <snip> Operations Subqueries 3 | a3 | 3 | b3 Set Operations Common Operations 3 | a3 | 6 | b6 Advanced Operations 4 | a4 | 5 | b5 Common Table Expressions 4 | a4 | 4 | b4 Window Functions 4 | a4 | 3 | b3 Real, Live Queries Something Simple 4 | a4 | 6 | b6 Something Fun (16 rows) Key Points

Fun With SQL INNER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN SELECT <...>FROM table1 INNER JOIN table2 ON Self Joins Other Useful (table1.ﬁeld = table2.ﬁeld ...) Operations Subqueries Only returns rows satisfying the ON condition Set Operations Common Operations Equivalent to a CROSS JOIN with a WHERE clause Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL INNER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN # SELECT * FROM a INNER JOIN b USING (id); NATURAL JOIN Self Joins id | value | value Other Useful ----+-------+------- Operations Subqueries 3 | a3 | b3 Set Operations Common Operations 4 | a4 | b4 Advanced (2 rows) Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL OUTER JOIN Joshua Tolley End Point Corporation Why and Why Not Joins Return all rows from one or both relations CROSS JOIN INNER JOIN OUTER JOIN LEFT: Return all rows from the relation on the left NATURAL JOIN Self Joins RIGHT: Return all rows from the relation on the right Other Useful Operations FULL: Return all rows from both relations Subqueries Set Operations Returns nulls for values from one relation when it Common Operations Advanced contains to match with the other relation Operations Common Table The OUTER keyword is redundant Expressions Window Functions Requires ON or USING clause Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL RIGHT JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN # SELECT * FROM a RIGHT JOIN b USING (id); INNER JOIN OUTER JOIN id | value | value NATURAL JOIN Self Joins ----+-------+------- Other Useful 3 | a3 | b3 Operations Subqueries 4 | a4 | b4 Set Operations Common Operations 5 | | b5 Advanced Operations 6 | | b6 Common Table Expressions (4 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Applications Joshua Tolley End Point Corporation Why and Why Not Find rows with no match in table b: Joins CROSS JOIN INNER JOIN OUTER JOIN # SELECT * FROM a LEFT JOIN b USING (id) NATURAL JOIN Self Joins WHERE b.value IS NULL; Other Useful id | value | value Operations Subqueries ----+-------+------- Set Operations Common Operations 1 | a1 | Advanced Operations 2 | a2 | Common Table Expressions (2 rows) Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL NATURAL JOIN Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful NATURAL is syntactic sugar to match all columns with Operations the same name Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL NATURAL JOIN Joshua Tolley End Point Corporation # SELECT * FROM a NATURAL FULL JOIN b; id | value Why and Why Not Joins ----+------- CROSS JOIN 1 | a1 INNER JOIN OUTER JOIN 2 | a2 NATURAL JOIN Self Joins 3 | a3 Other Useful Operations 3 | b3 Subqueries Set Operations 4 | a4 Common Operations 4 | b4 Advanced Operations 5 | b5 Common Table Expressions 6 | b6 Window Functions (8 rows) Real, Live Queries Something Simple Something Fun Key Points This looked for matches in both the id and value columns, so no rows matched. It returned all rows of both relations because it’s a FULL JOIN.

Fun With SQL Self Joins Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins ”Self joins” are particularly counterintuitive Other Useful Operations Joins one table to itself Subqueries Set Operations It helps to give the table two diﬀerent aliases Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Self Joins Joshua Tolley Find all employees’ names, and each employee’s manager End Point Corporation SELECT Why and Why Not e.first || ’ ’ || e.last, Joins (SELECT CROSS JOIN INNER JOIN m.first || ’ ’ || m.last OUTER JOIN NATURAL JOIN FROM employee m Self Joins Other Useful WHERE m.id = e.manager); Operations Subqueries Set Operations Common Operations ... will generally be much faster rewritten as ... Advanced Operations Common Table SELECT Expressions Window Functions e.first || ’ ’ || e.last, Real, Live Queries m.first || ’ ’ || m.last Something Simple Something Fun FROM Key Points employee e JOIN employee m ON (e.manager = m.id)

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins More useful operations... Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Subqueries Joshua Tolley End Point Corporation Why and Why Not Joins Embeds one query within another CROSS JOIN INNER JOIN OUTER JOIN Examples (some bad, some good) NATURAL JOIN Self Joins SELECT id FROM table WHERE ﬁeld = (SELECT Other Useful MAX(ﬁeld) FROM table) Operations SELECT id, (SELECT COUNT(*) FROM table2 Subqueries Set Operations WHERE id = table1.id) FROM table1 Common Operations SELECT a, b FROM (SELECT a, COUNT(*) AS c Advanced Operations FROM table1) t1 JOIN (SELECT b, COUNT(*) AS c Common Table Expressions FROM table2) t2 on (t1.c = t2.c) Window Functions You can join subqueries just like you’d join tables Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Set Operations Joshua Tolley End Point Corporation INTERSECT Why and Why Not Returns the intersection of two sets Joins Doesn’t exist in MySQL CROSS JOIN INNER JOIN SELECT (SELECT a, b FROM table1) INTERSECT OUTER JOIN NATURAL JOIN (SELECT c, d FROM table2) Self Joins UNION Other Useful Operations Appends one set of rows to another set with matching Subqueries Set Operations column types Common Operations SELECT a FROM table1 UNION SELECT b FROM Advanced Operations table2 Common Table Expressions EXCEPT Window Functions Real, Live Queries Returns rows in one SELECT that aren’t in another Something Simple SELECT Something Fun SELECT a FROM table1 EXCEPT SELECT b FROM Key Points table2

Fun With SQL Common Operations Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN COALESCE(a, b) INNER JOIN OUTER JOIN If a is null, return b, else return a NATURAL JOIN Self Joins SELECT COALESCE(ﬁrst, ’<NULL>’) FROM table Other Useful Oracle calls this NVL() Operations Subqueries CASE...WHEN Set Operations Common Operations Conditional operation Advanced SELECT CASE WHEN langused IN (’Lisp’, ’OCaml’, Operations Common Table ’Haskell’) THEN ’Functional’ ELSE ’Imperative’ AS Expressions Window Functions langtype FROM software Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Series Generation Joshua Tolley End Point Corporation generate series() in PostgreSQL; might be something else in other databases Why and Why Not Joins Returns a series of numbers CROSS JOIN INNER JOIN Can be used like a for loop (example given later) OUTER JOIN NATURAL JOIN Self Joins # SELECT * FROM generate_series(1, 5); Other Useful Operations generate_series Subqueries Set Operations ----------------- Common Operations Advanced 1 Operations 2 Common Table Expressions Window Functions 3 Real, Live Queries 4 Something Simple Something Fun 5 Key Points (5 rows)

Fun With SQL Common Table Expressions Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN Abbreviated CTEs INNER JOIN OUTER JOIN Fairly advanced; not available in all databases NATURAL JOIN Self Joins Not in PostgreSQL before v. 8.4, or any version of Other Useful Operations MySQL Subqueries Set Operations It’s just like defining a one-time view for your query Common Operations One major benefit: CTEs allow recursion Advanced Operations Recursing with CTEs is much more efficent than Common Table Expressions processing recursive data in your application Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL A Simple CTE Example Joshua Tolley End Point Corporation Why and Why Not # SELECT * FROM GENERATE_SERIES(1,3) Joins CROSS JOIN CROSS JOIN INNER JOIN (SELECT * FROM GENERATE_SERIES(8,9)) AS f; OUTER JOIN NATURAL JOIN generate_series | generate_series Self Joins -----------------+----------------- Other Useful Operations 1 | 8 Subqueries Set Operations 1 | 9 Common Operations 2 | 8 Advanced Operations 2 | 9 Common Table Expressions 3 | 8 Window Functions Real, Live Queries 3 | 9 Something Simple Something Fun (6 rows) Key Points

Fun With SQL A Simple CTE Example Joshua Tolley End Point Corporation # WITH t AS ( Why and Why Not SELECT * FROM GENERATE_SERIES(8,9) Joins ) CROSS JOIN INNER JOIN SELECT * FROM GENERATE_SERIES(1,3) OUTER JOIN NATURAL JOIN CROSS JOIN t; Self Joins Other Useful generate_series | generate_series Operations -----------------+----------------- Subqueries Set Operations 1 | 8 Common Operations Advanced 1 | 9 Operations Common Table 2 | 8 Expressions Window Functions 2 | 9 Real, Live Queries 3 | 8 Something Simple Something Fun 3 | 9 Key Points (6 rows)

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN That last example was a bit cheesy, but the technique can Self Joins Other Useful be useful for complex queries in several parts Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Recursion Joshua Tolley Recursive CTE to retrieve management hierarchy: End Point Corporation # WITH RECURSIVE t (id, managernames) AS ( Why and Why Not SELECT e.id, first || ’ ’ || last Joins AS managernames CROSS JOIN INNER JOIN FROM employee e WHERE manager IS NULL OUTER JOIN NATURAL JOIN UNION ALL Self Joins Other Useful SELECT e.id, Operations Subqueries first || ’ ’ || last || ’, ’ || managernames Set Operations Common Operations AS managernames Advanced FROM employee e Operations Common Table JOIN t ON (e.manager = t.id) Expressions Window Functions WHERE manager IS NOT NULL Real, Live Queries ) Something Simple Something Fun SELECT e.id, first || ’ ’ || last AS name, Key Points managernames FROM employee e JOIN t ON (e.id = t.id);

Fun With SQL Fractals in SQL Joshua Tolley End Point Corporation WITH RECURSIVE x(i) AS (VALUES(0) UNION ALL SELECT i + 1 FROM x WHERE i < 101), Z(Ix, Iy, Cx, Cy, X, Y, I) Why and Why Not AS ( SELECT Ix, Iy, X::float, Y::float, X::float, Y::float, 0 Joins FROM (SELECT -2.2 + 0.031 * i, i FROM x) AS xgen(x,ix) CROSS JOIN CROSS JOIN INNER JOIN (SELECT -1.5 + 0.031 * i, i FROM x) AS ygen(y,iy) OUTER JOIN UNION ALL NATURAL JOIN Self Joins SELECT Ix, Iy, Cx, Cy, X * X - Y * Y + Cx AS X, Other Useful Y * X * 2 + Cy, I + 1 Operations FROM Z Subqueries WHERE X * X + Y * Y < 16.0 AND I < 27), Set Operations Zt (Ix, Iy, I) AS ( Common Operations SELECT Ix, Iy, MAX(I) AS I FROM Z GROUP BY Iy, Ix Advanced ORDER BY Iy, Ix Operations ) Common Table SELECT array_to_string( Expressions array_agg( Window Functions SUBSTRING(’ .,,,-----++++%%%%@@@@#### ’, Real, Live Queries GREATEST(I,1), 1) Something Simple ),’’ Something Fun ) FROM Zt GROUP BY Iy ORDER BY Iy; Key Points (yes, this query is SQL-spec compliant)

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Window Functions Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN Like CTEs, these are quite advanced NATURAL JOIN Self Joins Also unavailable in MySQL, and PostgreSQL before 8.4 Other Useful Operations Allow ranking, moving averages Subqueries Set Operations Like a set-returning aggregate function. Window Common Operations Advanced functions return results for each row based on a Operations ”window” of related rows Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Window Functions Example Joshua Tolley End Point Corporation Why and Why Not Joins Rank employees in each department by salary CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN SELECT first, last, salary, department, Self Joins RANK() OVER ( Other Useful Operations PARTITION BY department Subqueries Set Operations ORDER BY salary DESC Common Operations ) Advanced Operations FROM employee Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Real, live queries Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Something Simple Joshua Tolley End Point Corporation Why and Why Not The slow version: Joins CROSS JOIN SELECT DISTINCT(sync) FROM bucardo.bucardo_rate INNER JOIN OUTER JOIN ORDER BY 1 NATURAL JOIN Self Joins Other Useful Operations The fast version: Subqueries Set Operations Common Operations SELECT name FROM sync WHERE EXISTS ( Advanced Operations SELECT 1 FROM bucardo_rate Common Table Expressions WHERE sync = name LIMIT 1) Window Functions Real, Live Queries ORDER BY 1 Something Simple Something Fun Key Points

Fun With SQL Something Simple Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN The bucardo rate table is huge, with few distinct values NATURAL JOIN Self Joins finding ”DISTINCT sync” requires a long table scan Other Useful Operations The sync table contains a list of all possible values in Subqueries Set Operations the bucardo rate.sync column Common Operations Advanced So instead of a big table scan, we scan the small table, Operations and filter out values can’t find in bucardo rate Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Something Fun Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Joshua Tolley SELECT End Point id, idname, Corporation COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime, COALESCE(SUM(total), 0) AS count FROM ( Why and Why Not SELECT slavecommit, EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins total CROSS JOIN FROM bucardo.bucardo_rate INNER JOIN WHERE sync = ’RO_everything’ AND OUTER JOIN mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN ) i Self Joins RIGHT JOIN ( Other Useful SELECT id, idname, Operations TO_TIMESTAMP(start - start::INTEGER % 3600) AS start, Subqueries TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop Set Operations FROM ( Common Operations SELECT id, TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced ’Dy Mon DD HH:MI AM’) AS idname, Operations EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions FROM ( Window Functions SELECT GENERATE_SERIES(1, 15) AS id Real, Live Queries ) f ) g Something Simple Something Fun ) h ON (slavecommit BETWEEN start AND stop) GROUP BY id, idname Key Points ORDER BY id DESC;

Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN The table contains replication data NATURAL JOIN Self Joins Time of commit on master Other Useful Time of commit on slave Operations Subqueries Number of rows replicated Set Operations Common Operations The user wants a graph of replication speed over time, Advanced given a user-determined range of time Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Something Fun Joshua Tolley End Point We want to average replication times over a series of Corporation buckets. The ﬁrst part of our query creates those buckets, Why and Why Not based on generate series(). Here we create buckets for Joins 15 hours CROSS JOIN INNER JOIN OUTER JOIN SELECT NATURAL JOIN Self Joins id, Other Useful Operations TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Subqueries ’Dy Mon DD HH:MI AM’) AS idname, Set Operations Common Operations EXTRACT(EPOCH FROM NOW() - id * Advanced Operations INTERVAL ’1 HOUR’) AS start, Common Table Expressions EXTRACT(EPOCH FROM NOW() - (id - 1) * Window Functions INTERVAL ’1 HOUR’) AS stop Real, Live Queries Something Simple FROM ( Something Fun SELECT GENERATE_SERIES(1, 15) AS id Key Points ) f

Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Make the buckets end on nice time boundaries: Joins CROSS JOIN INNER JOIN SELECT id, idname, OUTER JOIN NATURAL JOIN TO_TIMESTAMP(start - start::INTEGER % 3600) Self Joins Other Useful AS start, Operations TO_TIMESTAMP(stop - stop::INTEGER % 3600) Subqueries Set Operations AS stop Common Operations Advanced FROM ( Operations -- The bucket query, shown earlier, goes here Common Table Expressions Window Functions ) g Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN That gives us this: NATURAL JOIN Self Joins id | idname | start | stop ----+---------------------+-------------------------------+------------------------------- Other Useful 1 | Sat Mar 14 10:23 PM | 2009-03-14 21:59:59.956568-06 | 2009-03-14 22:59:59.956568-06 Operations 2 | Sat Mar 14 09:23 PM | 2009-03-14 20:59:59.956568-06 | 2009-03-14 21:59:59.956568-06 Subqueries 3 | Sat Mar 14 08:23 PM | 2009-03-14 19:59:59.956568-06 | 2009-03-14 20:59:59.956568-06 Set Operations 4 | Sat Mar 14 07:23 PM | 2009-03-14 18:59:59.956568-06 | 2009-03-14 19:59:59.956568-06 Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Something Fun Joshua Tolley End Point Corporation In an diﬀerent subquery, select everything from the table of Why and Why Not the right time period and right sync. Call this the ”stats” Joins query: CROSS JOIN INNER JOIN OUTER JOIN SELECT NATURAL JOIN Self Joins slavecommit, Other Useful EXTRACT(EPOCH FROM slavecommit - mastercommit) Operations Subqueries AS synctime, Set Operations Common Operations total Advanced FROM bucardo.bucardo_rate Operations Common Table WHERE Expressions Window Functions sync = ’RO_everything’ AND Real, Live Queries Something Simple mastercommit > (NOW() - (15 + 1) * Something Fun INTERVAL ’1 HOUR’) Key Points

Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not ...which gives us this: Joins CROSS JOIN INNER JOIN slavecommit | synctime | total OUTER JOIN -------------------------------+------------------+-------- NATURAL JOIN Self Joins 2009-03-14 07:32:00.103759-06 | 5.65614098310471 | 1 Other Useful 2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 3 Operations 2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 5 Subqueries Set Operations 2009-03-14 07:32:08.700184-06 | 7.71899098157883 | 1 Common Operations 2009-03-14 07:32:08.700184-06 | 8.22490698099136 | 1 Advanced 2009-03-14 07:32:12.675518-06 | 7.85176599025726 | 6 Operations Common Table 2009-03-14 07:32:12.675518-06 | 7.15798497200012 | 6 Expressions Window Functions ... Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Something Fun Joshua Tolley End Point Corporation Now, join the two queries: Why and Why Not SELECT Joins CROSS JOIN id, idname, INNER JOIN OUTER JOIN COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) NATURAL JOIN Self Joins AS avgtime, Other Useful COALESCE(SUM(total), 0) AS count Operations Subqueries FROM ( Set Operations Common Operations <STATS QUERY> Advanced ) RIGHT JOIN ( Operations Common Table <CALENDAR QUERY> Expressions Window Functions ) ON (slavecommit BETWEEN start AND stop) Real, Live Queries Something Simple GROUP BY id, idname Something Fun ORDER BY id DESC; Key Points

Fun With SQL Something Fun Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Other Useful Operations That query again: Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Joshua Tolley SELECT End Point id, idname, Corporation COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime, COALESCE(SUM(total), 0) AS count FROM ( Why and Why Not SELECT slavecommit, EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins total CROSS JOIN FROM bucardo.bucardo_rate INNER JOIN WHERE sync = ’RO_everything’ AND OUTER JOIN mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN ) i Self Joins RIGHT JOIN ( Other Useful SELECT id, idname, Operations TO_TIMESTAMP(start - start::INTEGER % 3600) AS start, Subqueries TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop Set Operations FROM ( Common Operations SELECT id, TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced ’Dy Mon DD HH:MI AM’) AS idname, Operations EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions FROM ( Window Functions SELECT GENERATE_SERIES(1, 15) AS id Real, Live Queries ) f ) g Something Simple Something Fun ) h ON (slavecommit BETWEEN start AND stop) GROUP BY id, idname Key Points ORDER BY id DESC;

Fun With SQL Key Points Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Understand join types, and use them Self Joins Other Useful Know what functions and set operations your database Operations Subqueries provides Set Operations Common Operations Build large queries piece by piece Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun With SQL Joshua Tolley End Point Corporation Why and Why Not Joins CROSS JOIN INNER JOIN OUTER JOIN NATURAL JOIN Self Joins Questions? Other Useful Operations Subqueries Set Operations Common Operations Advanced Operations Common Table Expressions Window Functions Real, Live Queries Something Simple Something Fun Key Points

Fun with SQL

by eggyknap on Oct 10, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 7

Statistics

Fun with SQL — Presentation Transcript