SlideShare a Scribd company logo
Models for Hierarchical Data
         with SQL and PHP
             Bill Karwin, Percona Inc.
Me
• Software developer
• C, Java, Perl, PHP, Ruby
• SQL maven
• MySQL Consultant at Percona
• Author of SQL Antipatterns:
   Avoiding the Pitfalls of
   Database Programming




                                www.percona.com
Problem
• Store & query hierarchical data
  - Categories/subcategories
  - Bill of materials
  - Threaded discussions




                                    www.percona.com
Example: Bug Report
       Comments
                               (1) Fran:
                               What’s the cause
                               of this bug?



             (2) Ollie:                      (4) Kukla:
             I think it’s a null             We need to check
             pointer.                        valid input.



(3) Fran:                                                 (6) Fran:
                                    (5) Ollie:
No, I checked for                                         Yes, please add a
                                    Yes, that’s a bug.
that.                                                     check.




                                                          (7) Kukla:
                                                          That fixed it.



                                                                    www.percona.com
Solutions

•Adjacency list
•Path enumeration
•Nested sets
•Closure table


                      www.percona.com
Adjacency List




                 www.percona.com
Adjacency List
• Naive solution nearly everyone uses
• Each entry knows its immediate parent

     comment_id parent_id author   comment
     1           NULL     Fran     What’s the cause of this bug?
     2           1        Ollie    I think it’s a null pointer.
     3           2        Fran     No, I checked for that.
     4           1        Kukla    We need to check valid input.
     5           4        Ollie    Yes, that’s a bug.
     6           4        Fran     Yes, please add a check
     7           6        Kukla    That fixed it.

                                                            www.percona.com
Insert a New Node
INSERT INTO Comments (parent_id, author, comment)
  VALUES (5, ‘Fran’, ‘I agree!’);
                                                 (1) Fran:
                                                 What’s the cause of
                                                 this bug?




                           (2) Ollie:                            (4) Kukla:
                           I think it’s a null                   We need to check
                           pointer.                              valid input.




            (3) Fran:                                                          (6) Fran:
                                                       (5) Ollie:
            No, I checked for                                                  Yes, please add a
                                                       Yes, that’s a bug.
            that.                                                              check.




                                                                               (7) Kukla:
                                                                               That fixed it.




                                                                                                   www.percona.com
Insert a New Node
INSERT INTO Comments (parent_id, author, comment)
  VALUES (5, ‘Fran’, ‘I agree!’);
                                                 (1) Fran:
                                                 What’s the cause of
                                                 this bug?




                           (2) Ollie:                              (4) Kukla:
                           I think it’s a null                     We need to check
                           pointer.                                valid input.




            (3) Fran:                                                            (6) Fran:
                                                       (5) Ollie:
            No, I checked for                                                    Yes, please add a
                                                       Yes, that’s a bug.
            that.                                                                check.




                                                       (8) Fran:                 (7) Kukla:
                                                       I agree!                  That fixed it.




                                                                                                     www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:                                                          (6) Fran:
                                                  (5) Ollie:
       No, I checked for                                                  Yes, please add a
                                                  Yes, that’s a bug.
       that.                                                              check.




                                                                          (7) Kukla:
                                                                          That fixed it.




                                                                                              www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:
                                                  (5) Ollie:
       No, I checked for
                                                  Yes, that’s a bug.
       that.




                                                                               www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:
                                                  (5) Ollie:
       No, I checked for
                                                  Yes, that’s a bug.
       that.




                                                                               www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                              (1) Fran:
                                              What’s the cause of
                                              this bug?




                        (2) Ollie:                            (4) Kukla:
                        I think it’s a null                   We need to check
                        pointer.                              valid input.




       (3) Fran:
                                                    (5) Ollie:
       No, I checked for
                                                    Yes, that’s a bug.
       that.




       (6) Fran:
       Yes, please add a
       check.




       (7) Kukla:
       That fixed it.                                                            www.percona.com
Query Immediate Child/Parent
• Query a node’s children:
       SELECT * FROM Comments c1
        LEFT JOIN Comments c2
         ON (c2.parent_id = c1.comment_id);


• Query a node’s parent:
      SELECT * FROM Comments c1
       JOIN Comments c2
        ON (c1.parent_id = c2.comment_id);


                                        www.percona.com
Can’t Handle Deep Trees
SELECT * FROM Comments c1
  LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id)
  LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id)
  LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id)
  LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id)
  LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id)
  LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id)
  LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id)
  LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id)
  LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id)
  ...




                                              www.percona.com
Can’t Handle Deep Trees
SELECT * FROM Comments c1
  LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id)
  LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id)
  LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id)
  LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id)
  LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id)
  LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id)
  LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id)
  LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id)
  LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id)
  ...
                it still doesn’t support
                unlimited depth!



                                              www.percona.com
SQL-99 recursive syntax
WITH [RECURSIVE] CommentTree
       (comment_id, bug_id, parent_id, author, comment, depth)
  AS (
       SELECT *, 0 AS depth FROM Comments
       WHERE parent_id IS NULL
    UNION ALL
       SELECT c.*, ct.depth+1 AS depth FROM CommentTree ct
       JOIN Comments c ON (ct.comment_id = c.parent_id)
  )
  SELECT * FROM CommentTree WHERE bug_id = 1234;




   ✓
       PostgreSQL, Oracle 11g,
       IBM DB2, Microsoft SQL
       Server, Apache Derby            ✗   MySQL, SQLite, Informix,
                                           Firebird,etc.



                                                        www.percona.com
Path Enumeration




                   www.percona.com
Path Enumeration
• Store chain of ancestors in each node


     comment_id path        author   comment
     1           1/         Fran     What’s the cause of this bug?
     2           1/2/       Ollie    I think it’s a null pointer.
     3           1/2/3/     Fran     No, I checked for that.
     4           1/4/       Kukla    We need to check valid input.
     5           1/4/5/     Ollie    Yes, that’s a bug.
     6           1/4/6/     Fran     Yes, please add a check
     7           1/4/6/7/   Kukla    That fixed it.

                                                              www.percona.com
Path Enumeration
• Store chain of ancestors in each node
                                              good for
                                              breadcrumbs
     comment_id path        author   comment
     1           1/         Fran     What’s the cause of this bug?
     2           1/2/       Ollie    I think it’s a null pointer.
     3           1/2/3/     Fran     No, I checked for that.
     4           1/4/       Kukla    We need to check valid input.
     5           1/4/5/     Ollie    Yes, that’s a bug.
     6           1/4/6/     Fran     Yes, please add a check
     7           1/4/6/7/   Kukla    That fixed it.

                                                              www.percona.com
Query Ancestors and Subtrees
• Query ancestors of comment #7:
      SELECT * FROM Comments
       WHERE ‘1/4/6/7/’ LIKE path || ‘%’;


• Query descendants of comment #4:
      SELECT * FROM Comments
       WHERE path LIKE ‘1/4/%’;




                                            www.percona.com
Add a New Child of #7
INSERT INTO Comments (author, comment)
  VALUES (‘Ollie’, ‘Good job!’);
SELECT path FROM Comments
  WHERE comment_id = 7;
UPDATE Comments
  SET path = $parent_path || LAST_INSERT_ID() || ‘/’
  WHERE comment_id = LAST_INSERT_ID();




                                         www.percona.com
Nested Sets




              www.percona.com
Nested Sets
• Each comment encodes its descendants
   using two numbers:
    - A comment’s left number is less than all numbers
        used by the comment’s descendants.
    - A comment’s right number is greater than all
        numbers used by the comment’s descendants.
    - A comment’s numbers are between all
        numbers used by the comment’s ancestors.




                                          www.percona.com
What Does This Look Like?

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
What Does This Look Like?

                                        (1) Fran:
                                        What’s the
                                        cause of this
                                        bug?
                                    1                      14
                                                         (4) Kukla:
                     (2) Ollie:
                                                         We need to
                     I think it’s a null
                     pointer.                            check valid
                                                         input.
                 2                         5         6                     13

       (3) Fran:                               (5) Ollie:              (6) Fran:
       No, I checked                           Yes, that’s a           Yes, please add
       for that.                               bug.                    a check.
   3                      4                 7                   8      9                 12

                                                                       (7) Kukla:
                                                                       That fixed it.

                                                                    10                   11


                                                                                          www.percona.com
What Does This Look Like?

comment_id nsleft   nsright   author   comment
1           1       14        Fran     What’s the cause of this bug?
2           2       5         Ollie    I think it’s a null pointer.
3           3       4         Fran     No, I checked for that.
4           6       13        Kukla    We need to check valid input.
5           7       8         Ollie    Yes, that’s a bug.
6           9       12        Fran     Yes, please add a check
7           10      11        Kukla    That fixed it.




                                                        www.percona.com
What Does This Look Like?

comment_id nsleft     nsright    author   comment
1           1         14         Fran     What’s the cause of this bug?
2           2         5          Ollie    I think it’s a null pointer.
3           3         4          Fran     No, I checked for that.
4           6         13         Kukla    We need to check valid input.
5           7         8          Ollie    Yes, that’s a bug.
6           9         12         Fran     Yes, please add a check
7           10        11         Kukla    That fixed it.

                 these are not
                 foreign keys

                                                           www.percona.com
Query Ancestors of #7

                                      (1) Fran:                                       ancestors
                                      What’s the
                                      cause of this
                                      bug?
                                  1                      14
                                                       (4) Kukla:
                   (2) Ollie:
                                                       We need to
                   I think it’s a null
                   pointer.                            check valid
                                                       input.
               2                         5         6                     13                   child
     (3) Fran:                               (5) Ollie:              (6) Fran:
     No, I checked                           Yes, that’s a           Yes, please add
     for that.                               bug.                    a check.
 3                      4                 7                   8      9                 12

                                                                     (7) Kukla:
                                                                     That fixed it.

                                                                  10                   11


                                                                                        www.percona.com
Query Ancestors of #7

SELECT * FROM Comments child
 JOIN Comments ancestor ON child.nsleft
   BETWEEN ancestor.nsleft AND ancestor.nsright
 WHERE child.comment_id = 7;




                                     www.percona.com
Query Subtree Under #4

                                      (1) Fran:
                                      What’s the
                                                                                      parent
                                      cause of this
                                      bug?
                                  1                      14

                   (2) Ollie:                          (4) Kukla:                           descendants
                                                       We need to
                   I think it’s a null
                   pointer.                            check valid
                                                       input.
               2                         5         6                     13

     (3) Fran:                               (5) Ollie:              (6) Fran:
     No, I checked                           Yes, that’s a           Yes, please add
     for that.                               bug.                    a check.
 3                      4                 7                   8      9                 12

                                                                     (7) Kukla:
                                                                     That fixed it.

                                                                  10                   11


                                                                                        www.percona.com
Query Subtree Under #4

SELECT * FROM Comments parent
 JOIN Comments descendant ON descendant.nsleft
   BETWEEN parent.nsleft AND parent.nsright
 WHERE parent.comment_id = 4;




                                    www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                     13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   8      9                 12

                                                                    (7) Kukla:
                                                                    That fixed it.

                                                                 10                   11


                                                                                       www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      16
                                                        14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                   15
                                                                      13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   10 11
                                                             8 9                      14
                                                                                      12

                                                                    (7) Kukla:
                                                                    That fixed it.

                                                                12
                                                                10                    13
                                                                                      11


                                                                                       www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      16
                                                        14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                   15
                                                                      13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   10 11
                                                             8 9                      14
                                                                                      12

                                            (8) Fran:               (7) Kukla:
                                            I agree!                That fixed it.

                                        8                    9 10
                                                               12                     13
                                                                                      11


                                                                                       www.percona.com
Insert New Child of #5
UPDATE Comments
  SET nsleft = CASE WHEN nsleft >= 8 THEN nsleft+2
     ELSE nsleft END,
     nsright = nsright+2
  WHERE nsright >= 7;
INSERT INTO Comments (nsleft, nsright, author, comment)
   VALUES (8, 9, 'Fran', 'I agree!');
• Recalculate left values for all nodes to the right of
   the new child. Recalculate right values for all
   nodes above and to the right.



                                               www.percona.com
Query Immediate Parent of #6

                                         (1) Fran:
                                         What’s the
                                         cause of this
                                         bug?
                                     1                      14
                                                          (4) Kukla:
                      (2) Ollie:
                                                          We need to
                      I think it’s a null
                      pointer.                            check valid
                                                          input.
                  2                         5         6                     13

        (3) Fran:                               (5) Ollie:              (6) Fran:
        No, I checked                           Yes, that’s a           Yes, please add
        for that.                               bug.                    a check.
    3                      4                 7                   8      9                 12

                                                                        (7) Kukla:
                                                                        That fixed it.

                                                                     10                   11


                                                                                           www.percona.com
Query Immediate Parent of #6
• Parent of #6 is an ancestor who has no
   descendant who is also an ancestor of #6.
     SELECT parent.* FROM Comments AS c
       JOIN Comments AS parent
        ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright)
       LEFT OUTER JOIN Comments AS in_between
        ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright
         AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright)
       WHERE c.comment_id = 6 AND in_between.comment_id IS NULL;




                                                           www.percona.com
Query Immediate Parent of #6
• Parent of #6 is an ancestor who has no
   descendant who is also an ancestor of #6.
     SELECT parent.* FROM Comments AS c
       JOIN Comments AS parent
        ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright)
       LEFT OUTER JOIN Comments AS in_between
        ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright
         AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright)
       WHERE c.comment_id = 6 AND in_between.comment_id IS NULL;




                   querying immediate child
                     is a similar problem

                                                           www.percona.com
Closure Table




                www.percona.com
Closure Table
CREATE TABLE TreePaths (
    ancestor    INT NOT NULL,
    descendant INT NOT NULL,
    PRIMARY KEY (ancestor, descendant),
    FOREIGN KEY(ancestor)
      REFERENCES Comments(comment_id),
    FOREIGN KEY(descendant)
      REFERENCES Comments(comment_id)
 );




                              www.percona.com
Closure Table
• Many-to-many table
• Stores every path from each node
    to each of its descendants
• A node even connects to itself




                                     www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
What Does This Look Like?
                                                   ancestor    descendant
comment_id author   comment                           1             1
                                                      1             2
    1      Fran     What’s the cause of this
                    bug?                              1             3
                                                      1             4
    2      Ollie    I think it’s a null pointer.      1             5
    3      Fran     No, I checked for that.           1             6
                                                      1             7
    4      Kukla    We need to check valid
                                                      2             2
                    input.
                                                      2             3
    5      Ollie    Yes, that’s a bug.                3             3
                                                      4             4
    6      Fran     Yes, please add a check
                                                      4             5
    7      Kukla    That fixed it.                    4             6
                                                      4             7
                                                      5             5

requires O(n²) rows                                   6
                                                      6
                                                                    6
                                                                    7
                                                      7             7




                                                              www.percona.com
What Does This Look Like?
                                                   ancestor    descendant
comment_id author   comment                           1             1
                                                      1             2
    1      Fran     What’s the cause of this
                    bug?                              1             3
                                                      1             4
    2      Ollie    I think it’s a null pointer.      1             5
    3      Fran     No, I checked for that.           1             6
                                                      1             7
    4      Kukla    We need to check valid
                                                      2             2
                    input.
                                                      2             3
    5      Ollie    Yes, that’s a bug.                3             3
                                                      4             4
    6      Fran     Yes, please add a check
                                                      4             5
    7      Kukla    That fixed it.                    4             6
                                                      4             7
                                                      5             5

requires O(n²) rows                                   6
                                                      6
                                                                    6
                                                                    7
                                                      7             7
(but far fewer in practice)
                                                              www.percona.com
Query Descendants of #4

SELECT c.* FROM Comments c
 JOIN TreePaths t
  ON (c.comment_id = t.descendant)
 WHERE t.ancestor = 4;




                                     www.percona.com
Paths Starting from #4

                              (1) Fran:
                              What’s the
                              cause of this
                              bug?


                                              (4) Kukla:
             (2) Ollie:
                                              We need to
             I think it’s a null
             pointer.                         check valid
                                              input.



 (3) Fran:                         (5) Ollie:               (6) Fran:
 No, I checked                     Yes, that’s a            Yes, please add
 for that.                         bug.                     a check.




                                                            (7) Kukla:
                                                            That fixed it.




                                                                              www.percona.com
Query Ancestors of #6

SELECT c.* FROM Comments c
 JOIN TreePaths t
  ON (c.comment_id = t.ancestor)
 WHERE t.descendant = 6;




                                   www.percona.com
Paths Terminating at #6

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                               (4) Kukla:
              (2) Ollie:
                                               We need to
              I think it’s a null
              pointer.                         check valid
                                               input.



  (3) Fran:                         (5) Ollie:               (6) Fran:
  No, I checked                     Yes, that’s a            Yes, please add
  for that.                         bug.                     a check.




                                                             (7) Kukla:
                                                             That fixed it.




                                                                               www.percona.com
Insert New Child of #5

INSERT INTO Comments
  VALUES (8, ‘Fran’, ‘I agree!’);


INSERT INTO TreePaths (ancestor, descendant)
  SELECT ancestor, 8 FROM TreePaths
  WHERE descendant = 5
  UNION ALL SELECT 8, 8;



                                    www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Delete Child #7

DELETE FROM TreePaths
  WHERE descendant = 7;




                            www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Subtree Under #4

DELETE FROM TreePaths
  WHERE descendant IN
    (SELECT descendant FROM TreePaths
     WHERE ancestor = 4);




                               www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?



               (2) Ollie:
               I think it’s a null
               pointer.



   (3) Fran:
   No, I checked
   for that.




                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:                       We need to
               I think it’s a null              check valid
               pointer.                         input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Path Length
• Add a length column                      ancestor   descendant   length

• MAX(length) is depth of tree
                                              1           1          0
                                              1           2          1
                                              1           3          2

• Makes it easier to query                    1
                                              1
                                                          4
                                                          5
                                                                     1
                                                                     2
   immediate parent or child:                 1           6          2
                                              1           7          3
     SELECT c.*                               2           2          0

       FROM Comments c                        2           3          1

       JOIN TreePaths t                       3           3          0
                                              4           4          0
        ON (c.comment_id = t.descendant)      4           5          1
       WHERE t.ancestor = 4                   4           6          1
        AND t.length = 1;                     4           7          2
                                              5           5          0
                                              6           6          0
                                              6           7          1
                                              7           7          0




                                                  www.percona.com
Path Length
• Add a length column                      ancestor   descendant   length

• MAX(length) is depth of tree
                                              1           1          0
                                              1           2          1
                                              1           3          2

• Makes it easier to query                    1
                                              1
                                                          4
                                                          5
                                                                     1
                                                                     2
   immediate parent or child:                 1           6          2
                                              1           7          3
     SELECT c.*                               2           2          0

       FROM Comments c                        2           3          1

       JOIN TreePaths t                       3           3          0
                                              4           4          0
        ON (c.comment_id = t.descendant)      4           5          1
       WHERE t.ancestor = 4                   4           6          1
        AND t.length = 1;                     4           7          2
                                              5           5          0
                                              6           6          0
                                              6           7          1
                                              7           7          0




                                                  www.percona.com
Choosing the Right Design

Design        Tables Query Query     Delete   Insert    Move     Referential
                     Child Subtree   Node     Node     Subtree    Integrity
Adjacency       1    Easy   Hard      Easy     Easy     Easy         Yes
List
Path            1    Hard   Easy     Easy     Easy      Easy         No
Enumeration
Nested Sets     1    Hard   Easy     Hard     Hard      Hard         No

Closure         2    Easy   Easy     Easy     Easy      Easy        Yes
Table




                                                           www.percona.com
PHP Demo
of Closure Table



                   www.percona.com
Hierarchical Test Data
• Integrated Taxonomic Information System
  - http://itis.gov/
  - Free authoritative taxonomic information on plants,
     animals, fungi, microbes
  - 518,756 scientific names (as of Feb 2011)




                                             www.percona.com
California Poppy
Kingdom:    Plantae
Division:   Tracheobionta
Class:      Magnoliophyta
Order:      Magnoliopsida
unranked:   Magnoliidae
unranked:   Papaverales
Family:     Papaveraceae
Genus:      Eschscholzia
Species:    Eschscholzia californica




                                       www.percona.com
California Poppy
Kingdom:    Plantae
Division:   Tracheobionta
Class:      Magnoliophyta
Order:      Magnoliopsida
unranked:   Magnoliidae
unranked:   Papaverales
Family:     Papaveraceae
Genus:      Eschscholzia
Species:    Eschscholzia californica


    id=18956

                                       www.percona.com
California Poppy: ITIS Entry

           SELECT * FROM Hierarchy
        WHERE hierarchy_string LIKE ‘%-18956’;

hierarchy_string
202422-564824-18061-18063-18064-18879-18880-18954-18956




                                                  www.percona.com
California Poppy: ITIS Entry

           SELECT * FROM Hierarchy
        WHERE hierarchy_string LIKE ‘%-18956’;

hierarchy_string
202422-564824-18061-18063-18064-18879-18880-18954-18956



    ITIS data uses                  ...but I converted
    path enumeration                it to closure table

                                                  www.percona.com
Hierarchical Data Classes
abstract class ZendX_Db_Table_TreeTable
      extends Zend_Db_Table_Abstract
  {
      public function fetchTreeByRoot($rootId, $expand)
      public function fetchBreadcrumbs($leafId)
  }




                                                 www.percona.com
Hierarchical Data Classes
class ZendX_Db_Table_Row_TreeRow
       extends Zend_Db_Table_Row_Abstract
   {
       public function addChildRow($childRow)
       public function getChildren()
   }
class ZendX_Db_Table_Rowset_TreeRowset
       extends Zend_Db_Table_Rowset_Abstract
   {
       public function append($row)
   }




                                                www.percona.com
Using TreeTable
class ItisTable extends ZendX_Db_Table_TreeTable
   {
       protected $_name = “longnames”;
       protected $_closureName = “treepaths”;
   }
$itis = new ItisTable();




                                               www.percona.com
Breadcrumbs
$breadcrumbs = $itis->fetchBreadcrumbs(18956);
foreach ($breadcrumbs as $crumb) {
      print $crumb->completename . “ > ”;
   }


Plantae > Tracheobionta > Magnoliophyta > Magnoliopsida >
  Magnoliidae > Papaverales > Papaveraceae > Eschscholzia >
  Eschscholzia californica >




                                                 www.percona.com
Breadcrumbs SQL

SELECT a.* FROM longnames AS a
 INNER JOIN treepaths AS c ON a.tsn = c.a
 WHERE (c.d = 18956)
 ORDER BY c.l DESC




                                   www.percona.com
How Does it Perform?
• Query profile = 0.0006 sec
• MySQL EXPLAIN:
table type     key       ref     rows   extra

c     ref      tree_dl   const   9      Using where; Using index

a     eq_ref   primary   c.a     1




                                                    www.percona.com
Dump Tree
$tree = $itis->fetchTreeByRoot(18880); // Papaveraceae
print_tree($tree);


function print_tree($tree, $prefix = ‘’)
  {
    print “{$prefix} {$tree->completename}n”;
    foreach ($tree->getChildren() as $child) {
      print_tree($child, “{$prefix} ”);
    }
  }




                                                 www.percona.com
Dump Tree Result
Papaveraceae                 Romneya
 Platystigma                      Romneya coulteri
   Platystigma linearis           Romneya trichocalyx
 Glaucium                       Dendromecon
   Glaucium corniculatum          Dendromecon harfordii
   Glaucium flavum                Dendromecon rigida
 Chelidonium                    Eschscholzia
   Chelidonium majus              Eschscholzia californica
 Bocconia                         Eschscholzia glyptosperma
   Bocconia frutescens            Eschscholzia hypecoides
 Stylophorum                      Eschscholzia lemmonii
   Stylophorum diphyllum          Eschscholzia lobbii
 Stylomecon                       Eschscholzia minutiflora
   Stylomecon heterophylla        Eschscholzia parishii
 Canbya                           Eschscholzia ramosa
   Canbya aurea                   Eschscholzia rhombipetala
   Canbya candida                 Eschscholzia caespitosa
 Chlidonium                  etc...
   Chlidonium majus
                                                www.percona.com
Dump Tree SQL
SELECT d.*, p.a AS _parent
 FROM treepaths AS c
 INNER JOIN longnames AS d ON c.d = d.tsn
 LEFT JOIN treepaths AS p ON p.d = d.tsn
    AND p.a IN (202422, 564824, 18053, 18020)
    AND p.l = 1
 WHERE (c.a = 202422)
    AND (p.a IS NOT NULL OR d.tsn = 202422)
 ORDER BY c.l, d.completename;




                                        www.percona.com
Dump Tree SQL
                                     show children
SELECT d.*, p.a AS _parent           of these nodes
 FROM treepaths AS c
 INNER JOIN longnames AS d ON c.d = d.tsn
 LEFT JOIN treepaths AS p ON p.d = d.tsn
    AND p.a IN (202422, 564824, 18053, 18020)
    AND p.l = 1
 WHERE (c.a = 202422)
    AND (p.a IS NOT NULL OR d.tsn = 202422)
 ORDER BY c.l, d.completename;




                                        www.percona.com
How Does it Perform?
• Query profile = 0.20 sec on Macbook Pro
• MySQL EXPLAIN:
table type     key        ref     rows     extra

c     ref      tree_adl   const   114240   Using index; Using
                                           temporary; Using filesort
d     eq_ref   primary    c.d     1

p     ref      tree_dl    c.d,    1        Using where; Using index
                          const




                                                        www.percona.com
SHOW CREATE TABLE
CREATE TABLE `treepaths` (
    `a` int(11) NOT NULL DEFAULT '0',
    `d` int(11) NOT NULL DEFAULT '0',
    `l` tinyint(3) unsigned NOT NULL DEFAULT '0',
    PRIMARY KEY (`a`,`d`),
    KEY `tree_adl` (`a`,`d`,`l`),
    KEY `tree_dl` (`d`,`l`),
    CONSTRAINT FOREIGN KEY (`a`)
      REFERENCES `longnames` (`tsn`),
    CONSTRAINT FOREIGN KEY (`d`)
      REFERENCES `longnames` (`tsn`)
  ) ENGINE=InnoDB




                                                    www.percona.com
SHOW TABLE STATUS
Name:             treepaths
 Engine:          InnoDB
 Version:         10
 Row_format:      Compact
 Rows:            4600439
 Avg_row_length: 62
 Data_length:     288276480
 Max_data_length: 0
 Index_length:    273137664
 Data_free:       7340032



                              www.percona.com
Demo Time!




             www.percona.com
SQL Antipatterns




http://www.pragprog.com/titles/bksqla/
                                 www.percona.com
Models for hierarchical data

More Related Content

What's hot

Virtual Flink Forward 2020: A deep dive into Flink SQL - Jark Wu
Virtual Flink Forward 2020: A deep dive into Flink SQL - Jark WuVirtual Flink Forward 2020: A deep dive into Flink SQL - Jark Wu
Virtual Flink Forward 2020: A deep dive into Flink SQL - Jark Wu
Flink Forward
 
More mastering the art of indexing
More mastering the art of indexingMore mastering the art of indexing
More mastering the art of indexing
Yoshinori Matsunobu
 
Dapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any LanguageDapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any Language
Bilgin Ibryam
 
Thrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonThrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased Comparison
Igor Anishchenko
 
How to Design Indexes, Really
How to Design Indexes, ReallyHow to Design Indexes, Really
How to Design Indexes, Really
Karwin Software Solutions LLC
 
Etsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureEtsy Activity Feeds Architecture
Etsy Activity Feeds Architecture
Dan McKinley
 
Neo4j Training Cypher
Neo4j Training CypherNeo4j Training Cypher
Neo4j Training Cypher
Max De Marzi
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
Dvir Volk
 
Redis
RedisRedis
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
Jurriaan Persyn
 
Functional programming with Java 8
Functional programming with Java 8Functional programming with Java 8
Functional programming with Java 8
LivePerson
 
Allyourbase
AllyourbaseAllyourbase
Allyourbase
Alex Scotti
 
Introduction to Apache ZooKeeper
Introduction to Apache ZooKeeperIntroduction to Apache ZooKeeper
Introduction to Apache ZooKeeper
Saurav Haloi
 
MongoDB Aggregation Framework
MongoDB Aggregation FrameworkMongoDB Aggregation Framework
MongoDB Aggregation Framework
Caserta
 
storm at twitter
storm at twitterstorm at twitter
storm at twitter
Krishna Gade
 
Kafka replication apachecon_2013
Kafka replication apachecon_2013Kafka replication apachecon_2013
Kafka replication apachecon_2013
Jun Rao
 
Postman
PostmanPostman
KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...
KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...
KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...
confluent
 
Airflow를 이용한 데이터 Workflow 관리
Airflow를 이용한  데이터 Workflow 관리Airflow를 이용한  데이터 Workflow 관리
Airflow를 이용한 데이터 Workflow 관리
YoungHeon (Roy) Kim
 
Using ClickHouse for Experimentation
Using ClickHouse for ExperimentationUsing ClickHouse for Experimentation
Using ClickHouse for Experimentation
Gleb Kanterov
 

What's hot (20)

Virtual Flink Forward 2020: A deep dive into Flink SQL - Jark Wu
Virtual Flink Forward 2020: A deep dive into Flink SQL - Jark WuVirtual Flink Forward 2020: A deep dive into Flink SQL - Jark Wu
Virtual Flink Forward 2020: A deep dive into Flink SQL - Jark Wu
 
More mastering the art of indexing
More mastering the art of indexingMore mastering the art of indexing
More mastering the art of indexing
 
Dapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any LanguageDapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any Language
 
Thrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonThrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased Comparison
 
How to Design Indexes, Really
How to Design Indexes, ReallyHow to Design Indexes, Really
How to Design Indexes, Really
 
Etsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureEtsy Activity Feeds Architecture
Etsy Activity Feeds Architecture
 
Neo4j Training Cypher
Neo4j Training CypherNeo4j Training Cypher
Neo4j Training Cypher
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Redis
RedisRedis
Redis
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
 
Functional programming with Java 8
Functional programming with Java 8Functional programming with Java 8
Functional programming with Java 8
 
Allyourbase
AllyourbaseAllyourbase
Allyourbase
 
Introduction to Apache ZooKeeper
Introduction to Apache ZooKeeperIntroduction to Apache ZooKeeper
Introduction to Apache ZooKeeper
 
MongoDB Aggregation Framework
MongoDB Aggregation FrameworkMongoDB Aggregation Framework
MongoDB Aggregation Framework
 
storm at twitter
storm at twitterstorm at twitter
storm at twitter
 
Kafka replication apachecon_2013
Kafka replication apachecon_2013Kafka replication apachecon_2013
Kafka replication apachecon_2013
 
Postman
PostmanPostman
Postman
 
KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...
KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...
KafkaConsumer - Decoupling Consumption and Processing for Better Resource Uti...
 
Airflow를 이용한 데이터 Workflow 관리
Airflow를 이용한  데이터 Workflow 관리Airflow를 이용한  데이터 Workflow 관리
Airflow를 이용한 데이터 Workflow 관리
 
Using ClickHouse for Experimentation
Using ClickHouse for ExperimentationUsing ClickHouse for Experimentation
Using ClickHouse for Experimentation
 

Viewers also liked

Trees and Hierarchies in SQL
Trees and Hierarchies in SQLTrees and Hierarchies in SQL
Trees and Hierarchies in SQL
Eduard Hildebrandt
 
SQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォークSQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォーク
ke-m kamekoopa
 
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
gree_tech
 
SQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリーSQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリー
ke-m kamekoopa
 
Database Anti Patterns
Database Anti PatternsDatabase Anti Patterns
Database Anti Patterns
Robert Treat
 
Fax With Sangoma Gateway
Fax With Sangoma GatewayFax With Sangoma Gateway
Fax With Sangoma Gateway
Hossein Yavari
 
10 system.security.cryptography
10 system.security.cryptography10 system.security.cryptography
10 system.security.cryptography
Mohammad Alyan
 
1 first lesson -assemblies
1  first lesson -assemblies1  first lesson -assemblies
1 first lesson -assemblies
Mohammad Alyan
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributes
Mohammad Alyan
 
Introduction To ERP
Introduction To ERPIntroduction To ERP
Introduction To ERP
Mohammad Alyan
 
Login System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysqlLogin System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysql
thesoftwareguy7
 
8 memory managment & pointers
8 memory managment & pointers8 memory managment & pointers
8 memory managment & pointersMohammad Alyan
 
3 third lesson-reflection
3 third lesson-reflection3 third lesson-reflection
3 third lesson-reflectionMohammad Alyan
 
Coursera
CourseraCoursera
Coursera
Mohammad Alyan
 
4 fourth lesson-deployment
4 fourth lesson-deployment4 fourth lesson-deployment
4 fourth lesson-deploymentMohammad Alyan
 

Viewers also liked (18)

Trees and Hierarchies in SQL
Trees and Hierarchies in SQLTrees and Hierarchies in SQL
Trees and Hierarchies in SQL
 
SQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォークSQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォーク
 
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
 
SQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリーSQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリー
 
Database Anti Patterns
Database Anti PatternsDatabase Anti Patterns
Database Anti Patterns
 
Fax With Sangoma Gateway
Fax With Sangoma GatewayFax With Sangoma Gateway
Fax With Sangoma Gateway
 
5 fifth lesson -xml
5 fifth lesson -xml5 fifth lesson -xml
5 fifth lesson -xml
 
7 multi threading
7 multi threading7 multi threading
7 multi threading
 
10 system.security.cryptography
10 system.security.cryptography10 system.security.cryptography
10 system.security.cryptography
 
1 first lesson -assemblies
1  first lesson -assemblies1  first lesson -assemblies
1 first lesson -assemblies
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributes
 
Introduction To ERP
Introduction To ERPIntroduction To ERP
Introduction To ERP
 
Login System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysqlLogin System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysql
 
8 memory managment & pointers
8 memory managment & pointers8 memory managment & pointers
8 memory managment & pointers
 
3 third lesson-reflection
3 third lesson-reflection3 third lesson-reflection
3 third lesson-reflection
 
Coursera
CourseraCoursera
Coursera
 
4 fourth lesson-deployment
4 fourth lesson-deployment4 fourth lesson-deployment
4 fourth lesson-deployment
 
9 networking
9 networking9 networking
9 networking
 

More from Karwin Software Solutions LLC

How to Use JSON in MySQL Wrong
How to Use JSON in MySQL WrongHow to Use JSON in MySQL Wrong
How to Use JSON in MySQL Wrong
Karwin Software Solutions LLC
 
Load Data Fast!
Load Data Fast!Load Data Fast!
InnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick FiguresInnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick Figures
Karwin Software Solutions LLC
 
SQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and ProfitSQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and Profit
Karwin Software Solutions LLC
 
Survey of Percona Toolkit
Survey of Percona ToolkitSurvey of Percona Toolkit
Survey of Percona Toolkit
Karwin Software Solutions LLC
 
Schemadoc
SchemadocSchemadoc
Percona toolkit
Percona toolkitPercona toolkit
MySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB StatusMySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB Status
Karwin Software Solutions LLC
 
Requirements the Last Bottleneck
Requirements the Last BottleneckRequirements the Last Bottleneck
Requirements the Last Bottleneck
Karwin Software Solutions LLC
 
Mentor Your Indexes
Mentor Your IndexesMentor Your Indexes
Mentor Your Indexes
Karwin Software Solutions LLC
 
Sql Injection Myths and Fallacies
Sql Injection Myths and FallaciesSql Injection Myths and Fallacies
Sql Injection Myths and Fallacies
Karwin Software Solutions LLC
 
Full Text Search In PostgreSQL
Full Text Search In PostgreSQLFull Text Search In PostgreSQL
Full Text Search In PostgreSQL
Karwin Software Solutions LLC
 
Practical Object Oriented Models In Sql
Practical Object Oriented Models In SqlPractical Object Oriented Models In Sql
Practical Object Oriented Models In Sql
Karwin Software Solutions LLC
 

More from Karwin Software Solutions LLC (13)

How to Use JSON in MySQL Wrong
How to Use JSON in MySQL WrongHow to Use JSON in MySQL Wrong
How to Use JSON in MySQL Wrong
 
Load Data Fast!
Load Data Fast!Load Data Fast!
Load Data Fast!
 
InnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick FiguresInnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick Figures
 
SQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and ProfitSQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and Profit
 
Survey of Percona Toolkit
Survey of Percona ToolkitSurvey of Percona Toolkit
Survey of Percona Toolkit
 
Schemadoc
SchemadocSchemadoc
Schemadoc
 
Percona toolkit
Percona toolkitPercona toolkit
Percona toolkit
 
MySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB StatusMySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB Status
 
Requirements the Last Bottleneck
Requirements the Last BottleneckRequirements the Last Bottleneck
Requirements the Last Bottleneck
 
Mentor Your Indexes
Mentor Your IndexesMentor Your Indexes
Mentor Your Indexes
 
Sql Injection Myths and Fallacies
Sql Injection Myths and FallaciesSql Injection Myths and Fallacies
Sql Injection Myths and Fallacies
 
Full Text Search In PostgreSQL
Full Text Search In PostgreSQLFull Text Search In PostgreSQL
Full Text Search In PostgreSQL
 
Practical Object Oriented Models In Sql
Practical Object Oriented Models In SqlPractical Object Oriented Models In Sql
Practical Object Oriented Models In Sql
 

Recently uploaded

Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
HarisZaheer8
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
saastr
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
Dinusha Kumarasiri
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Intelisync
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
dbms calicut university B. sc Cs 4th sem.pdf
dbms  calicut university B. sc Cs 4th sem.pdfdbms  calicut university B. sc Cs 4th sem.pdf
dbms calicut university B. sc Cs 4th sem.pdf
Shinana2
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
GDSC PJATK
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
flufftailshop
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 

Recently uploaded (20)

Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
dbms calicut university B. sc Cs 4th sem.pdf
dbms  calicut university B. sc Cs 4th sem.pdfdbms  calicut university B. sc Cs 4th sem.pdf
dbms calicut university B. sc Cs 4th sem.pdf
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 

Models for hierarchical data

  • 1. Models for Hierarchical Data with SQL and PHP Bill Karwin, Percona Inc.
  • 2. Me • Software developer • C, Java, Perl, PHP, Ruby • SQL maven • MySQL Consultant at Percona • Author of SQL Antipatterns: Avoiding the Pitfalls of Database Programming www.percona.com
  • 3. Problem • Store & query hierarchical data - Categories/subcategories - Bill of materials - Threaded discussions www.percona.com
  • 4. Example: Bug Report Comments (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 5. Solutions •Adjacency list •Path enumeration •Nested sets •Closure table www.percona.com
  • 6. Adjacency List www.percona.com
  • 7. Adjacency List • Naive solution nearly everyone uses • Each entry knows its immediate parent comment_id parent_id author comment 1 NULL Fran What’s the cause of this bug? 2 1 Ollie I think it’s a null pointer. 3 2 Fran No, I checked for that. 4 1 Kukla We need to check valid input. 5 4 Ollie Yes, that’s a bug. 6 4 Fran Yes, please add a check 7 6 Kukla That fixed it. www.percona.com
  • 8. Insert a New Node INSERT INTO Comments (parent_id, author, comment) VALUES (5, ‘Fran’, ‘I agree!’); (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 9. Insert a New Node INSERT INTO Comments (parent_id, author, comment) VALUES (5, ‘Fran’, ‘I agree!’); (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 10. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 11. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. www.percona.com
  • 12. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. www.percona.com
  • 13. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. (6) Fran: Yes, please add a check. (7) Kukla: That fixed it. www.percona.com
  • 14. Query Immediate Child/Parent • Query a node’s children: SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id); • Query a node’s parent: SELECT * FROM Comments c1 JOIN Comments c2 ON (c1.parent_id = c2.comment_id); www.percona.com
  • 15. Can’t Handle Deep Trees SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id) LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id) LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id) LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id) LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id) LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id) LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id) LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id) LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id) ... www.percona.com
  • 16. Can’t Handle Deep Trees SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id) LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id) LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id) LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id) LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id) LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id) LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id) LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id) LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id) ... it still doesn’t support unlimited depth! www.percona.com
  • 17. SQL-99 recursive syntax WITH [RECURSIVE] CommentTree (comment_id, bug_id, parent_id, author, comment, depth) AS ( SELECT *, 0 AS depth FROM Comments WHERE parent_id IS NULL UNION ALL SELECT c.*, ct.depth+1 AS depth FROM CommentTree ct JOIN Comments c ON (ct.comment_id = c.parent_id) ) SELECT * FROM CommentTree WHERE bug_id = 1234; ✓ PostgreSQL, Oracle 11g, IBM DB2, Microsoft SQL Server, Apache Derby ✗ MySQL, SQLite, Informix, Firebird,etc. www.percona.com
  • 18. Path Enumeration www.percona.com
  • 19. Path Enumeration • Store chain of ancestors in each node comment_id path author comment 1 1/ Fran What’s the cause of this bug? 2 1/2/ Ollie I think it’s a null pointer. 3 1/2/3/ Fran No, I checked for that. 4 1/4/ Kukla We need to check valid input. 5 1/4/5/ Ollie Yes, that’s a bug. 6 1/4/6/ Fran Yes, please add a check 7 1/4/6/7/ Kukla That fixed it. www.percona.com
  • 20. Path Enumeration • Store chain of ancestors in each node good for breadcrumbs comment_id path author comment 1 1/ Fran What’s the cause of this bug? 2 1/2/ Ollie I think it’s a null pointer. 3 1/2/3/ Fran No, I checked for that. 4 1/4/ Kukla We need to check valid input. 5 1/4/5/ Ollie Yes, that’s a bug. 6 1/4/6/ Fran Yes, please add a check 7 1/4/6/7/ Kukla That fixed it. www.percona.com
  • 21. Query Ancestors and Subtrees • Query ancestors of comment #7: SELECT * FROM Comments WHERE ‘1/4/6/7/’ LIKE path || ‘%’; • Query descendants of comment #4: SELECT * FROM Comments WHERE path LIKE ‘1/4/%’; www.percona.com
  • 22. Add a New Child of #7 INSERT INTO Comments (author, comment) VALUES (‘Ollie’, ‘Good job!’); SELECT path FROM Comments WHERE comment_id = 7; UPDATE Comments SET path = $parent_path || LAST_INSERT_ID() || ‘/’ WHERE comment_id = LAST_INSERT_ID(); www.percona.com
  • 23. Nested Sets www.percona.com
  • 24. Nested Sets • Each comment encodes its descendants using two numbers: - A comment’s left number is less than all numbers used by the comment’s descendants. - A comment’s right number is greater than all numbers used by the comment’s descendants. - A comment’s numbers are between all numbers used by the comment’s ancestors. www.percona.com
  • 25. What Does This Look Like? (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 26. What Does This Look Like? (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 27. What Does This Look Like? comment_id nsleft nsright author comment 1 1 14 Fran What’s the cause of this bug? 2 2 5 Ollie I think it’s a null pointer. 3 3 4 Fran No, I checked for that. 4 6 13 Kukla We need to check valid input. 5 7 8 Ollie Yes, that’s a bug. 6 9 12 Fran Yes, please add a check 7 10 11 Kukla That fixed it. www.percona.com
  • 28. What Does This Look Like? comment_id nsleft nsright author comment 1 1 14 Fran What’s the cause of this bug? 2 2 5 Ollie I think it’s a null pointer. 3 3 4 Fran No, I checked for that. 4 6 13 Kukla We need to check valid input. 5 7 8 Ollie Yes, that’s a bug. 6 9 12 Fran Yes, please add a check 7 10 11 Kukla That fixed it. these are not foreign keys www.percona.com
  • 29. Query Ancestors of #7 (1) Fran: ancestors What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 child (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 30. Query Ancestors of #7 SELECT * FROM Comments child JOIN Comments ancestor ON child.nsleft BETWEEN ancestor.nsleft AND ancestor.nsright WHERE child.comment_id = 7; www.percona.com
  • 31. Query Subtree Under #4 (1) Fran: What’s the parent cause of this bug? 1 14 (2) Ollie: (4) Kukla: descendants We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 32. Query Subtree Under #4 SELECT * FROM Comments parent JOIN Comments descendant ON descendant.nsleft BETWEEN parent.nsleft AND parent.nsright WHERE parent.comment_id = 4; www.percona.com
  • 33. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 34. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 16 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 15 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 10 11 8 9 14 12 (7) Kukla: That fixed it. 12 10 13 11 www.percona.com
  • 35. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 16 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 15 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 10 11 8 9 14 12 (8) Fran: (7) Kukla: I agree! That fixed it. 8 9 10 12 13 11 www.percona.com
  • 36. Insert New Child of #5 UPDATE Comments SET nsleft = CASE WHEN nsleft >= 8 THEN nsleft+2 ELSE nsleft END, nsright = nsright+2 WHERE nsright >= 7; INSERT INTO Comments (nsleft, nsright, author, comment) VALUES (8, 9, 'Fran', 'I agree!'); • Recalculate left values for all nodes to the right of the new child. Recalculate right values for all nodes above and to the right. www.percona.com
  • 37. Query Immediate Parent of #6 (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 38. Query Immediate Parent of #6 • Parent of #6 is an ancestor who has no descendant who is also an ancestor of #6. SELECT parent.* FROM Comments AS c JOIN Comments AS parent ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright) LEFT OUTER JOIN Comments AS in_between ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright) WHERE c.comment_id = 6 AND in_between.comment_id IS NULL; www.percona.com
  • 39. Query Immediate Parent of #6 • Parent of #6 is an ancestor who has no descendant who is also an ancestor of #6. SELECT parent.* FROM Comments AS c JOIN Comments AS parent ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright) LEFT OUTER JOIN Comments AS in_between ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright) WHERE c.comment_id = 6 AND in_between.comment_id IS NULL; querying immediate child is a similar problem www.percona.com
  • 40. Closure Table www.percona.com
  • 41. Closure Table CREATE TABLE TreePaths ( ancestor INT NOT NULL, descendant INT NOT NULL, PRIMARY KEY (ancestor, descendant), FOREIGN KEY(ancestor) REFERENCES Comments(comment_id), FOREIGN KEY(descendant) REFERENCES Comments(comment_id) ); www.percona.com
  • 42. Closure Table • Many-to-many table • Stores every path from each node to each of its descendants • A node even connects to itself www.percona.com
  • 43. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 44. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 45. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 46. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 47. What Does This Look Like? ancestor descendant comment_id author comment 1 1 1 2 1 Fran What’s the cause of this bug? 1 3 1 4 2 Ollie I think it’s a null pointer. 1 5 3 Fran No, I checked for that. 1 6 1 7 4 Kukla We need to check valid 2 2 input. 2 3 5 Ollie Yes, that’s a bug. 3 3 4 4 6 Fran Yes, please add a check 4 5 7 Kukla That fixed it. 4 6 4 7 5 5 requires O(n²) rows 6 6 6 7 7 7 www.percona.com
  • 48. What Does This Look Like? ancestor descendant comment_id author comment 1 1 1 2 1 Fran What’s the cause of this bug? 1 3 1 4 2 Ollie I think it’s a null pointer. 1 5 3 Fran No, I checked for that. 1 6 1 7 4 Kukla We need to check valid 2 2 input. 2 3 5 Ollie Yes, that’s a bug. 3 3 4 4 6 Fran Yes, please add a check 4 5 7 Kukla That fixed it. 4 6 4 7 5 5 requires O(n²) rows 6 6 6 7 7 7 (but far fewer in practice) www.percona.com
  • 49. Query Descendants of #4 SELECT c.* FROM Comments c JOIN TreePaths t ON (c.comment_id = t.descendant) WHERE t.ancestor = 4; www.percona.com
  • 50. Paths Starting from #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 51. Query Ancestors of #6 SELECT c.* FROM Comments c JOIN TreePaths t ON (c.comment_id = t.ancestor) WHERE t.descendant = 6; www.percona.com
  • 52. Paths Terminating at #6 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 53. Insert New Child of #5 INSERT INTO Comments VALUES (8, ‘Fran’, ‘I agree!’); INSERT INTO TreePaths (ancestor, descendant) SELECT ancestor, 8 FROM TreePaths WHERE descendant = 5 UNION ALL SELECT 8, 8; www.percona.com
  • 54. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 55. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 56. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 57. Delete Child #7 DELETE FROM TreePaths WHERE descendant = 7; www.percona.com
  • 58. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 59. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 60. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. www.percona.com
  • 61. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 62. Delete Subtree Under #4 DELETE FROM TreePaths WHERE descendant IN (SELECT descendant FROM TreePaths WHERE ancestor = 4); www.percona.com
  • 63. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 64. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 65. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (2) Ollie: I think it’s a null pointer. (3) Fran: No, I checked for that. www.percona.com
  • 66. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null check valid pointer. input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 67. Path Length • Add a length column ancestor descendant length • MAX(length) is depth of tree 1 1 0 1 2 1 1 3 2 • Makes it easier to query 1 1 4 5 1 2 immediate parent or child: 1 6 2 1 7 3 SELECT c.* 2 2 0 FROM Comments c 2 3 1 JOIN TreePaths t 3 3 0 4 4 0 ON (c.comment_id = t.descendant) 4 5 1 WHERE t.ancestor = 4 4 6 1 AND t.length = 1; 4 7 2 5 5 0 6 6 0 6 7 1 7 7 0 www.percona.com
  • 68. Path Length • Add a length column ancestor descendant length • MAX(length) is depth of tree 1 1 0 1 2 1 1 3 2 • Makes it easier to query 1 1 4 5 1 2 immediate parent or child: 1 6 2 1 7 3 SELECT c.* 2 2 0 FROM Comments c 2 3 1 JOIN TreePaths t 3 3 0 4 4 0 ON (c.comment_id = t.descendant) 4 5 1 WHERE t.ancestor = 4 4 6 1 AND t.length = 1; 4 7 2 5 5 0 6 6 0 6 7 1 7 7 0 www.percona.com
  • 69. Choosing the Right Design Design Tables Query Query Delete Insert Move Referential Child Subtree Node Node Subtree Integrity Adjacency 1 Easy Hard Easy Easy Easy Yes List Path 1 Hard Easy Easy Easy Easy No Enumeration Nested Sets 1 Hard Easy Hard Hard Hard No Closure 2 Easy Easy Easy Easy Easy Yes Table www.percona.com
  • 70. PHP Demo of Closure Table www.percona.com
  • 71. Hierarchical Test Data • Integrated Taxonomic Information System - http://itis.gov/ - Free authoritative taxonomic information on plants, animals, fungi, microbes - 518,756 scientific names (as of Feb 2011) www.percona.com
  • 72. California Poppy Kingdom: Plantae Division: Tracheobionta Class: Magnoliophyta Order: Magnoliopsida unranked: Magnoliidae unranked: Papaverales Family: Papaveraceae Genus: Eschscholzia Species: Eschscholzia californica www.percona.com
  • 73. California Poppy Kingdom: Plantae Division: Tracheobionta Class: Magnoliophyta Order: Magnoliopsida unranked: Magnoliidae unranked: Papaverales Family: Papaveraceae Genus: Eschscholzia Species: Eschscholzia californica id=18956 www.percona.com
  • 74. California Poppy: ITIS Entry SELECT * FROM Hierarchy WHERE hierarchy_string LIKE ‘%-18956’; hierarchy_string 202422-564824-18061-18063-18064-18879-18880-18954-18956 www.percona.com
  • 75. California Poppy: ITIS Entry SELECT * FROM Hierarchy WHERE hierarchy_string LIKE ‘%-18956’; hierarchy_string 202422-564824-18061-18063-18064-18879-18880-18954-18956 ITIS data uses ...but I converted path enumeration it to closure table www.percona.com
  • 76. Hierarchical Data Classes abstract class ZendX_Db_Table_TreeTable extends Zend_Db_Table_Abstract { public function fetchTreeByRoot($rootId, $expand) public function fetchBreadcrumbs($leafId) } www.percona.com
  • 77. Hierarchical Data Classes class ZendX_Db_Table_Row_TreeRow extends Zend_Db_Table_Row_Abstract { public function addChildRow($childRow) public function getChildren() } class ZendX_Db_Table_Rowset_TreeRowset extends Zend_Db_Table_Rowset_Abstract { public function append($row) } www.percona.com
  • 78. Using TreeTable class ItisTable extends ZendX_Db_Table_TreeTable { protected $_name = “longnames”; protected $_closureName = “treepaths”; } $itis = new ItisTable(); www.percona.com
  • 79. Breadcrumbs $breadcrumbs = $itis->fetchBreadcrumbs(18956); foreach ($breadcrumbs as $crumb) { print $crumb->completename . “ > ”; } Plantae > Tracheobionta > Magnoliophyta > Magnoliopsida > Magnoliidae > Papaverales > Papaveraceae > Eschscholzia > Eschscholzia californica > www.percona.com
  • 80. Breadcrumbs SQL SELECT a.* FROM longnames AS a INNER JOIN treepaths AS c ON a.tsn = c.a WHERE (c.d = 18956) ORDER BY c.l DESC www.percona.com
  • 81. How Does it Perform? • Query profile = 0.0006 sec • MySQL EXPLAIN: table type key ref rows extra c ref tree_dl const 9 Using where; Using index a eq_ref primary c.a 1 www.percona.com
  • 82. Dump Tree $tree = $itis->fetchTreeByRoot(18880); // Papaveraceae print_tree($tree); function print_tree($tree, $prefix = ‘’) { print “{$prefix} {$tree->completename}n”; foreach ($tree->getChildren() as $child) { print_tree($child, “{$prefix} ”); } } www.percona.com
  • 83. Dump Tree Result Papaveraceae Romneya Platystigma Romneya coulteri Platystigma linearis Romneya trichocalyx Glaucium Dendromecon Glaucium corniculatum Dendromecon harfordii Glaucium flavum Dendromecon rigida Chelidonium Eschscholzia Chelidonium majus Eschscholzia californica Bocconia Eschscholzia glyptosperma Bocconia frutescens Eschscholzia hypecoides Stylophorum Eschscholzia lemmonii Stylophorum diphyllum Eschscholzia lobbii Stylomecon Eschscholzia minutiflora Stylomecon heterophylla Eschscholzia parishii Canbya Eschscholzia ramosa Canbya aurea Eschscholzia rhombipetala Canbya candida Eschscholzia caespitosa Chlidonium etc... Chlidonium majus www.percona.com
  • 84. Dump Tree SQL SELECT d.*, p.a AS _parent FROM treepaths AS c INNER JOIN longnames AS d ON c.d = d.tsn LEFT JOIN treepaths AS p ON p.d = d.tsn AND p.a IN (202422, 564824, 18053, 18020) AND p.l = 1 WHERE (c.a = 202422) AND (p.a IS NOT NULL OR d.tsn = 202422) ORDER BY c.l, d.completename; www.percona.com
  • 85. Dump Tree SQL show children SELECT d.*, p.a AS _parent of these nodes FROM treepaths AS c INNER JOIN longnames AS d ON c.d = d.tsn LEFT JOIN treepaths AS p ON p.d = d.tsn AND p.a IN (202422, 564824, 18053, 18020) AND p.l = 1 WHERE (c.a = 202422) AND (p.a IS NOT NULL OR d.tsn = 202422) ORDER BY c.l, d.completename; www.percona.com
  • 86. How Does it Perform? • Query profile = 0.20 sec on Macbook Pro • MySQL EXPLAIN: table type key ref rows extra c ref tree_adl const 114240 Using index; Using temporary; Using filesort d eq_ref primary c.d 1 p ref tree_dl c.d, 1 Using where; Using index const www.percona.com
  • 87. SHOW CREATE TABLE CREATE TABLE `treepaths` ( `a` int(11) NOT NULL DEFAULT '0', `d` int(11) NOT NULL DEFAULT '0', `l` tinyint(3) unsigned NOT NULL DEFAULT '0', PRIMARY KEY (`a`,`d`), KEY `tree_adl` (`a`,`d`,`l`), KEY `tree_dl` (`d`,`l`), CONSTRAINT FOREIGN KEY (`a`) REFERENCES `longnames` (`tsn`), CONSTRAINT FOREIGN KEY (`d`) REFERENCES `longnames` (`tsn`) ) ENGINE=InnoDB www.percona.com
  • 88. SHOW TABLE STATUS Name: treepaths Engine: InnoDB Version: 10 Row_format: Compact Rows: 4600439 Avg_row_length: 62 Data_length: 288276480 Max_data_length: 0 Index_length: 273137664 Data_free: 7340032 www.percona.com
  • 89. Demo Time! www.percona.com