SlideShare a Scribd company logo
Models for Hierarchical Data
         with SQL and PHP
             Bill Karwin, Percona Inc.
Me
• Software developer
• C, Java, Perl, PHP, Ruby
• SQL maven
• MySQL Consultant at Percona
• Author of SQL Antipatterns:
   Avoiding the Pitfalls of
   Database Programming




                                www.percona.com
Problem
• Store & query hierarchical data
  - Categories/subcategories
  - Bill of materials
  - Threaded discussions




                                    www.percona.com
Example: Bug Report
       Comments
                               (1) Fran:
                               What’s the cause
                               of this bug?



             (2) Ollie:                      (4) Kukla:
             I think it’s a null             We need to check
             pointer.                        valid input.



(3) Fran:                                                 (6) Fran:
                                    (5) Ollie:
No, I checked for                                         Yes, please add a
                                    Yes, that’s a bug.
that.                                                     check.




                                                          (7) Kukla:
                                                          That fixed it.



                                                                    www.percona.com
Solutions

•Adjacency list
•Path enumeration
•Nested sets
•Closure table


                      www.percona.com
Adjacency List




                 www.percona.com
Adjacency List
• Naive solution nearly everyone uses
• Each entry knows its immediate parent

     comment_id parent_id author   comment
     1           NULL     Fran     What’s the cause of this bug?
     2           1        Ollie    I think it’s a null pointer.
     3           2        Fran     No, I checked for that.
     4           1        Kukla    We need to check valid input.
     5           4        Ollie    Yes, that’s a bug.
     6           4        Fran     Yes, please add a check
     7           6        Kukla    That fixed it.

                                                            www.percona.com
Insert a New Node
INSERT INTO Comments (parent_id, author, comment)
  VALUES (5, ‘Fran’, ‘I agree!’);
                                                 (1) Fran:
                                                 What’s the cause of
                                                 this bug?




                           (2) Ollie:                            (4) Kukla:
                           I think it’s a null                   We need to check
                           pointer.                              valid input.




            (3) Fran:                                                          (6) Fran:
                                                       (5) Ollie:
            No, I checked for                                                  Yes, please add a
                                                       Yes, that’s a bug.
            that.                                                              check.




                                                                               (7) Kukla:
                                                                               That fixed it.




                                                                                                   www.percona.com
Insert a New Node
INSERT INTO Comments (parent_id, author, comment)
  VALUES (5, ‘Fran’, ‘I agree!’);
                                                 (1) Fran:
                                                 What’s the cause of
                                                 this bug?




                           (2) Ollie:                              (4) Kukla:
                           I think it’s a null                     We need to check
                           pointer.                                valid input.




            (3) Fran:                                                            (6) Fran:
                                                       (5) Ollie:
            No, I checked for                                                    Yes, please add a
                                                       Yes, that’s a bug.
            that.                                                                check.




                                                       (8) Fran:                 (7) Kukla:
                                                       I agree!                  That fixed it.




                                                                                                     www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:                                                          (6) Fran:
                                                  (5) Ollie:
       No, I checked for                                                  Yes, please add a
                                                  Yes, that’s a bug.
       that.                                                              check.




                                                                          (7) Kukla:
                                                                          That fixed it.




                                                                                              www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:
                                                  (5) Ollie:
       No, I checked for
                                                  Yes, that’s a bug.
       that.




                                                                               www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                            (1) Fran:
                                            What’s the cause of
                                            this bug?




                      (2) Ollie:                            (4) Kukla:
                      I think it’s a null                   We need to check
                      pointer.                              valid input.




       (3) Fran:
                                                  (5) Ollie:
       No, I checked for
                                                  Yes, that’s a bug.
       that.




                                                                               www.percona.com
Move a Node or Subtree
UPDATE Comments SET parent_id = 3
 WHERE comment_id = 6;
                                              (1) Fran:
                                              What’s the cause of
                                              this bug?




                        (2) Ollie:                            (4) Kukla:
                        I think it’s a null                   We need to check
                        pointer.                              valid input.




       (3) Fran:
                                                    (5) Ollie:
       No, I checked for
                                                    Yes, that’s a bug.
       that.




       (6) Fran:
       Yes, please add a
       check.




       (7) Kukla:
       That fixed it.                                                            www.percona.com
Query Immediate Child/Parent
• Query a node’s children:
       SELECT * FROM Comments c1
        LEFT JOIN Comments c2
         ON (c2.parent_id = c1.comment_id);


• Query a node’s parent:
      SELECT * FROM Comments c1
       JOIN Comments c2
        ON (c1.parent_id = c2.comment_id);


                                        www.percona.com
Can’t Handle Deep Trees
SELECT * FROM Comments c1
  LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id)
  LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id)
  LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id)
  LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id)
  LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id)
  LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id)
  LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id)
  LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id)
  LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id)
  ...




                                              www.percona.com
Can’t Handle Deep Trees
SELECT * FROM Comments c1
  LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id)
  LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id)
  LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id)
  LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id)
  LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id)
  LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id)
  LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id)
  LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id)
  LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id)
  ...
                it still doesn’t support
                unlimited depth!



                                              www.percona.com
SQL-99 recursive syntax
WITH [RECURSIVE] CommentTree
       (comment_id, bug_id, parent_id, author, comment, depth)
  AS (
       SELECT *, 0 AS depth FROM Comments
       WHERE parent_id IS NULL
    UNION ALL
       SELECT c.*, ct.depth+1 AS depth FROM CommentTree ct
       JOIN Comments c ON (ct.comment_id = c.parent_id)
  )
  SELECT * FROM CommentTree WHERE bug_id = 1234;




   ✓
       PostgreSQL, Oracle 11g,
       IBM DB2, Microsoft SQL
       Server, Apache Derby            ✗   MySQL, SQLite, Informix,
                                           Firebird,etc.



                                                        www.percona.com
Path Enumeration




                   www.percona.com
Path Enumeration
• Store chain of ancestors in each node


     comment_id path        author   comment
     1           1/         Fran     What’s the cause of this bug?
     2           1/2/       Ollie    I think it’s a null pointer.
     3           1/2/3/     Fran     No, I checked for that.
     4           1/4/       Kukla    We need to check valid input.
     5           1/4/5/     Ollie    Yes, that’s a bug.
     6           1/4/6/     Fran     Yes, please add a check
     7           1/4/6/7/   Kukla    That fixed it.

                                                              www.percona.com
Path Enumeration
• Store chain of ancestors in each node
                                              good for
                                              breadcrumbs
     comment_id path        author   comment
     1           1/         Fran     What’s the cause of this bug?
     2           1/2/       Ollie    I think it’s a null pointer.
     3           1/2/3/     Fran     No, I checked for that.
     4           1/4/       Kukla    We need to check valid input.
     5           1/4/5/     Ollie    Yes, that’s a bug.
     6           1/4/6/     Fran     Yes, please add a check
     7           1/4/6/7/   Kukla    That fixed it.

                                                              www.percona.com
Query Ancestors and Subtrees
• Query ancestors of comment #7:
      SELECT * FROM Comments
       WHERE ‘1/4/6/7/’ LIKE path || ‘%’;


• Query descendants of comment #4:
      SELECT * FROM Comments
       WHERE path LIKE ‘1/4/%’;




                                            www.percona.com
Add a New Child of #7
INSERT INTO Comments (author, comment)
  VALUES (‘Ollie’, ‘Good job!’);
SELECT path FROM Comments
  WHERE comment_id = 7;
UPDATE Comments
  SET path = $parent_path || LAST_INSERT_ID() || ‘/’
  WHERE comment_id = LAST_INSERT_ID();




                                         www.percona.com
Nested Sets




              www.percona.com
Nested Sets
• Each comment encodes its descendants
   using two numbers:
    - A comment’s left number is less than all numbers
        used by the comment’s descendants.
    - A comment’s right number is greater than all
        numbers used by the comment’s descendants.
    - A comment’s numbers are between all
        numbers used by the comment’s ancestors.




                                          www.percona.com
What Does This Look Like?

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
What Does This Look Like?

                                        (1) Fran:
                                        What’s the
                                        cause of this
                                        bug?
                                    1                      14
                                                         (4) Kukla:
                     (2) Ollie:
                                                         We need to
                     I think it’s a null
                     pointer.                            check valid
                                                         input.
                 2                         5         6                     13

       (3) Fran:                               (5) Ollie:              (6) Fran:
       No, I checked                           Yes, that’s a           Yes, please add
       for that.                               bug.                    a check.
   3                      4                 7                   8      9                 12

                                                                       (7) Kukla:
                                                                       That fixed it.

                                                                    10                   11


                                                                                          www.percona.com
What Does This Look Like?

comment_id nsleft   nsright   author   comment
1           1       14        Fran     What’s the cause of this bug?
2           2       5         Ollie    I think it’s a null pointer.
3           3       4         Fran     No, I checked for that.
4           6       13        Kukla    We need to check valid input.
5           7       8         Ollie    Yes, that’s a bug.
6           9       12        Fran     Yes, please add a check
7           10      11        Kukla    That fixed it.




                                                        www.percona.com
What Does This Look Like?

comment_id nsleft     nsright    author   comment
1           1         14         Fran     What’s the cause of this bug?
2           2         5          Ollie    I think it’s a null pointer.
3           3         4          Fran     No, I checked for that.
4           6         13         Kukla    We need to check valid input.
5           7         8          Ollie    Yes, that’s a bug.
6           9         12         Fran     Yes, please add a check
7           10        11         Kukla    That fixed it.

                 these are not
                 foreign keys

                                                           www.percona.com
Query Ancestors of #7

                                      (1) Fran:                                       ancestors
                                      What’s the
                                      cause of this
                                      bug?
                                  1                      14
                                                       (4) Kukla:
                   (2) Ollie:
                                                       We need to
                   I think it’s a null
                   pointer.                            check valid
                                                       input.
               2                         5         6                     13                   child
     (3) Fran:                               (5) Ollie:              (6) Fran:
     No, I checked                           Yes, that’s a           Yes, please add
     for that.                               bug.                    a check.
 3                      4                 7                   8      9                 12

                                                                     (7) Kukla:
                                                                     That fixed it.

                                                                  10                   11


                                                                                        www.percona.com
Query Ancestors of #7

SELECT * FROM Comments child
 JOIN Comments ancestor ON child.nsleft
   BETWEEN ancestor.nsleft AND ancestor.nsright
 WHERE child.comment_id = 7;




                                     www.percona.com
Query Subtree Under #4

                                      (1) Fran:
                                      What’s the
                                                                                      parent
                                      cause of this
                                      bug?
                                  1                      14

                   (2) Ollie:                          (4) Kukla:                           descendants
                                                       We need to
                   I think it’s a null
                   pointer.                            check valid
                                                       input.
               2                         5         6                     13

     (3) Fran:                               (5) Ollie:              (6) Fran:
     No, I checked                           Yes, that’s a           Yes, please add
     for that.                               bug.                    a check.
 3                      4                 7                   8      9                 12

                                                                     (7) Kukla:
                                                                     That fixed it.

                                                                  10                   11


                                                                                        www.percona.com
Query Subtree Under #4

SELECT * FROM Comments parent
 JOIN Comments descendant ON descendant.nsleft
   BETWEEN parent.nsleft AND parent.nsright
 WHERE parent.comment_id = 4;




                                    www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                     13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   8      9                 12

                                                                    (7) Kukla:
                                                                    That fixed it.

                                                                 10                   11


                                                                                       www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      16
                                                        14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                   15
                                                                      13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   10 11
                                                             8 9                      14
                                                                                      12

                                                                    (7) Kukla:
                                                                    That fixed it.

                                                                12
                                                                10                    13
                                                                                      11


                                                                                       www.percona.com
Insert New Child of #5

                                     (1) Fran:
                                     What’s the
                                     cause of this
                                     bug?
                                 1                      16
                                                        14
                                                      (4) Kukla:
                  (2) Ollie:
                                                      We need to
                  I think it’s a null
                  pointer.                            check valid
                                                      input.
              2                         5         6                   15
                                                                      13

    (3) Fran:                               (5) Ollie:              (6) Fran:
    No, I checked                           Yes, that’s a           Yes, please add
    for that.                               bug.                    a check.
3                      4                 7                   10 11
                                                             8 9                      14
                                                                                      12

                                            (8) Fran:               (7) Kukla:
                                            I agree!                That fixed it.

                                        8                    9 10
                                                               12                     13
                                                                                      11


                                                                                       www.percona.com
Insert New Child of #5
UPDATE Comments
  SET nsleft = CASE WHEN nsleft >= 8 THEN nsleft+2
     ELSE nsleft END,
     nsright = nsright+2
  WHERE nsright >= 7;
INSERT INTO Comments (nsleft, nsright, author, comment)
   VALUES (8, 9, 'Fran', 'I agree!');
• Recalculate left values for all nodes to the right of
   the new child. Recalculate right values for all
   nodes above and to the right.



                                               www.percona.com
Query Immediate Parent of #6

                                         (1) Fran:
                                         What’s the
                                         cause of this
                                         bug?
                                     1                      14
                                                          (4) Kukla:
                      (2) Ollie:
                                                          We need to
                      I think it’s a null
                      pointer.                            check valid
                                                          input.
                  2                         5         6                     13

        (3) Fran:                               (5) Ollie:              (6) Fran:
        No, I checked                           Yes, that’s a           Yes, please add
        for that.                               bug.                    a check.
    3                      4                 7                   8      9                 12

                                                                        (7) Kukla:
                                                                        That fixed it.

                                                                     10                   11


                                                                                           www.percona.com
Query Immediate Parent of #6
• Parent of #6 is an ancestor who has no
   descendant who is also an ancestor of #6.
     SELECT parent.* FROM Comments AS c
       JOIN Comments AS parent
        ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright)
       LEFT OUTER JOIN Comments AS in_between
        ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright
         AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright)
       WHERE c.comment_id = 6 AND in_between.comment_id IS NULL;




                                                           www.percona.com
Query Immediate Parent of #6
• Parent of #6 is an ancestor who has no
   descendant who is also an ancestor of #6.
     SELECT parent.* FROM Comments AS c
       JOIN Comments AS parent
        ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright)
       LEFT OUTER JOIN Comments AS in_between
        ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright
         AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright)
       WHERE c.comment_id = 6 AND in_between.comment_id IS NULL;




                   querying immediate child
                     is a similar problem

                                                           www.percona.com
Closure Table




                www.percona.com
Closure Table
CREATE TABLE TreePaths (
    ancestor    INT NOT NULL,
    descendant INT NOT NULL,
    PRIMARY KEY (ancestor, descendant),
    FOREIGN KEY(ancestor)
      REFERENCES Comments(comment_id),
    FOREIGN KEY(descendant)
      REFERENCES Comments(comment_id)
 );




                              www.percona.com
Closure Table
• Many-to-many table
• Stores every path from each node
    to each of its descendants
• A node even connects to itself




                                     www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Closure Table illustration

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
What Does This Look Like?
                                                   ancestor    descendant
comment_id author   comment                           1             1
                                                      1             2
    1      Fran     What’s the cause of this
                    bug?                              1             3
                                                      1             4
    2      Ollie    I think it’s a null pointer.      1             5
    3      Fran     No, I checked for that.           1             6
                                                      1             7
    4      Kukla    We need to check valid
                                                      2             2
                    input.
                                                      2             3
    5      Ollie    Yes, that’s a bug.                3             3
                                                      4             4
    6      Fran     Yes, please add a check
                                                      4             5
    7      Kukla    That fixed it.                    4             6
                                                      4             7
                                                      5             5

requires O(n²) rows                                   6
                                                      6
                                                                    6
                                                                    7
                                                      7             7




                                                              www.percona.com
What Does This Look Like?
                                                   ancestor    descendant
comment_id author   comment                           1             1
                                                      1             2
    1      Fran     What’s the cause of this
                    bug?                              1             3
                                                      1             4
    2      Ollie    I think it’s a null pointer.      1             5
    3      Fran     No, I checked for that.           1             6
                                                      1             7
    4      Kukla    We need to check valid
                                                      2             2
                    input.
                                                      2             3
    5      Ollie    Yes, that’s a bug.                3             3
                                                      4             4
    6      Fran     Yes, please add a check
                                                      4             5
    7      Kukla    That fixed it.                    4             6
                                                      4             7
                                                      5             5

requires O(n²) rows                                   6
                                                      6
                                                                    6
                                                                    7
                                                      7             7
(but far fewer in practice)
                                                              www.percona.com
Query Descendants of #4

SELECT c.* FROM Comments c
 JOIN TreePaths t
  ON (c.comment_id = t.descendant)
 WHERE t.ancestor = 4;




                                     www.percona.com
Paths Starting from #4

                              (1) Fran:
                              What’s the
                              cause of this
                              bug?


                                              (4) Kukla:
             (2) Ollie:
                                              We need to
             I think it’s a null
             pointer.                         check valid
                                              input.



 (3) Fran:                         (5) Ollie:               (6) Fran:
 No, I checked                     Yes, that’s a            Yes, please add
 for that.                         bug.                     a check.




                                                            (7) Kukla:
                                                            That fixed it.




                                                                              www.percona.com
Query Ancestors of #6

SELECT c.* FROM Comments c
 JOIN TreePaths t
  ON (c.comment_id = t.ancestor)
 WHERE t.descendant = 6;




                                   www.percona.com
Paths Terminating at #6

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                               (4) Kukla:
              (2) Ollie:
                                               We need to
              I think it’s a null
              pointer.                         check valid
                                               input.



  (3) Fran:                         (5) Ollie:               (6) Fran:
  No, I checked                     Yes, that’s a            Yes, please add
  for that.                         bug.                     a check.




                                                             (7) Kukla:
                                                             That fixed it.




                                                                               www.percona.com
Insert New Child of #5

INSERT INTO Comments
  VALUES (8, ‘Fran’, ‘I agree!’);


INSERT INTO TreePaths (ancestor, descendant)
  SELECT ancestor, 8 FROM TreePaths
  WHERE descendant = 5
  UNION ALL SELECT 8, 8;



                                    www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Copy Paths from Parent

                               (1) Fran:
                               What’s the
                               cause of this
                               bug?


                                                (4) Kukla:
              (2) Ollie:
                                                We need to
              I think it’s a null
              pointer.                          check valid
                                                input.



  (3) Fran:                         (5) Ollie:                (6) Fran:
  No, I checked                     Yes, that’s a             Yes, please add
  for that.                         bug.                      a check.




                                    (8) Fran:                 (7) Kukla:
                                    I agree!                  That fixed it.




                                                                                www.percona.com
Delete Child #7

DELETE FROM TreePaths
  WHERE descendant = 7;




                            www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                                  www.percona.com
Delete Paths Terminating at #7

                                  (1) Fran:
                                  What’s the
                                  cause of this
                                  bug?


                                                  (4) Kukla:
                 (2) Ollie:
                                                  We need to
                 I think it’s a null
                 pointer.                         check valid
                                                  input.



     (3) Fran:                         (5) Ollie:               (6) Fran:
     No, I checked                     Yes, that’s a            Yes, please add
     for that.                         bug.                     a check.




                                                                (7) Kukla:
                                                                That fixed it.




                                                                                  www.percona.com
Delete Subtree Under #4

DELETE FROM TreePaths
  WHERE descendant IN
    (SELECT descendant FROM TreePaths
     WHERE ancestor = 4);




                               www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:
                                                We need to
               I think it’s a null
               pointer.                         check valid
                                                input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?



               (2) Ollie:
               I think it’s a null
               pointer.



   (3) Fran:
   No, I checked
   for that.




                                                www.percona.com
Delete Any Paths Under #4

                                (1) Fran:
                                What’s the
                                cause of this
                                bug?


                                                (4) Kukla:
               (2) Ollie:                       We need to
               I think it’s a null              check valid
               pointer.                         input.



   (3) Fran:                         (5) Ollie:               (6) Fran:
   No, I checked                     Yes, that’s a            Yes, please add
   for that.                         bug.                     a check.




                                                              (7) Kukla:
                                                              That fixed it.




                                                                                www.percona.com
Path Length
• Add a length column                      ancestor   descendant   length

• MAX(length) is depth of tree
                                              1           1          0
                                              1           2          1
                                              1           3          2

• Makes it easier to query                    1
                                              1
                                                          4
                                                          5
                                                                     1
                                                                     2
   immediate parent or child:                 1           6          2
                                              1           7          3
     SELECT c.*                               2           2          0

       FROM Comments c                        2           3          1

       JOIN TreePaths t                       3           3          0
                                              4           4          0
        ON (c.comment_id = t.descendant)      4           5          1
       WHERE t.ancestor = 4                   4           6          1
        AND t.length = 1;                     4           7          2
                                              5           5          0
                                              6           6          0
                                              6           7          1
                                              7           7          0




                                                  www.percona.com
Path Length
• Add a length column                      ancestor   descendant   length

• MAX(length) is depth of tree
                                              1           1          0
                                              1           2          1
                                              1           3          2

• Makes it easier to query                    1
                                              1
                                                          4
                                                          5
                                                                     1
                                                                     2
   immediate parent or child:                 1           6          2
                                              1           7          3
     SELECT c.*                               2           2          0

       FROM Comments c                        2           3          1

       JOIN TreePaths t                       3           3          0
                                              4           4          0
        ON (c.comment_id = t.descendant)      4           5          1
       WHERE t.ancestor = 4                   4           6          1
        AND t.length = 1;                     4           7          2
                                              5           5          0
                                              6           6          0
                                              6           7          1
                                              7           7          0




                                                  www.percona.com
Choosing the Right Design

Design        Tables Query Query     Delete   Insert    Move     Referential
                     Child Subtree   Node     Node     Subtree    Integrity
Adjacency       1    Easy   Hard      Easy     Easy     Easy         Yes
List
Path            1    Hard   Easy     Easy     Easy      Easy         No
Enumeration
Nested Sets     1    Hard   Easy     Hard     Hard      Hard         No

Closure         2    Easy   Easy     Easy     Easy      Easy        Yes
Table




                                                           www.percona.com
PHP Demo
of Closure Table



                   www.percona.com
Hierarchical Test Data
• Integrated Taxonomic Information System
  - http://itis.gov/
  - Free authoritative taxonomic information on plants,
     animals, fungi, microbes
  - 518,756 scientific names (as of Feb 2011)




                                             www.percona.com
California Poppy
Kingdom:    Plantae
Division:   Tracheobionta
Class:      Magnoliophyta
Order:      Magnoliopsida
unranked:   Magnoliidae
unranked:   Papaverales
Family:     Papaveraceae
Genus:      Eschscholzia
Species:    Eschscholzia californica




                                       www.percona.com
California Poppy
Kingdom:    Plantae
Division:   Tracheobionta
Class:      Magnoliophyta
Order:      Magnoliopsida
unranked:   Magnoliidae
unranked:   Papaverales
Family:     Papaveraceae
Genus:      Eschscholzia
Species:    Eschscholzia californica


    id=18956

                                       www.percona.com
California Poppy: ITIS Entry

           SELECT * FROM Hierarchy
        WHERE hierarchy_string LIKE ‘%-18956’;

hierarchy_string
202422-564824-18061-18063-18064-18879-18880-18954-18956




                                                  www.percona.com
California Poppy: ITIS Entry

           SELECT * FROM Hierarchy
        WHERE hierarchy_string LIKE ‘%-18956’;

hierarchy_string
202422-564824-18061-18063-18064-18879-18880-18954-18956



    ITIS data uses                  ...but I converted
    path enumeration                it to closure table

                                                  www.percona.com
Hierarchical Data Classes
abstract class ZendX_Db_Table_TreeTable
      extends Zend_Db_Table_Abstract
  {
      public function fetchTreeByRoot($rootId, $expand)
      public function fetchBreadcrumbs($leafId)
  }




                                                 www.percona.com
Hierarchical Data Classes
class ZendX_Db_Table_Row_TreeRow
       extends Zend_Db_Table_Row_Abstract
   {
       public function addChildRow($childRow)
       public function getChildren()
   }
class ZendX_Db_Table_Rowset_TreeRowset
       extends Zend_Db_Table_Rowset_Abstract
   {
       public function append($row)
   }




                                                www.percona.com
Using TreeTable
class ItisTable extends ZendX_Db_Table_TreeTable
   {
       protected $_name = “longnames”;
       protected $_closureName = “treepaths”;
   }
$itis = new ItisTable();




                                               www.percona.com
Breadcrumbs
$breadcrumbs = $itis->fetchBreadcrumbs(18956);
foreach ($breadcrumbs as $crumb) {
      print $crumb->completename . “ > ”;
   }


Plantae > Tracheobionta > Magnoliophyta > Magnoliopsida >
  Magnoliidae > Papaverales > Papaveraceae > Eschscholzia >
  Eschscholzia californica >




                                                 www.percona.com
Breadcrumbs SQL

SELECT a.* FROM longnames AS a
 INNER JOIN treepaths AS c ON a.tsn = c.a
 WHERE (c.d = 18956)
 ORDER BY c.l DESC




                                   www.percona.com
How Does it Perform?
• Query profile = 0.0006 sec
• MySQL EXPLAIN:
table type     key       ref     rows   extra

c     ref      tree_dl   const   9      Using where; Using index

a     eq_ref   primary   c.a     1




                                                    www.percona.com
Dump Tree
$tree = $itis->fetchTreeByRoot(18880); // Papaveraceae
print_tree($tree);


function print_tree($tree, $prefix = ‘’)
  {
    print “{$prefix} {$tree->completename}n”;
    foreach ($tree->getChildren() as $child) {
      print_tree($child, “{$prefix} ”);
    }
  }




                                                 www.percona.com
Dump Tree Result
Papaveraceae                 Romneya
 Platystigma                      Romneya coulteri
   Platystigma linearis           Romneya trichocalyx
 Glaucium                       Dendromecon
   Glaucium corniculatum          Dendromecon harfordii
   Glaucium flavum                Dendromecon rigida
 Chelidonium                    Eschscholzia
   Chelidonium majus              Eschscholzia californica
 Bocconia                         Eschscholzia glyptosperma
   Bocconia frutescens            Eschscholzia hypecoides
 Stylophorum                      Eschscholzia lemmonii
   Stylophorum diphyllum          Eschscholzia lobbii
 Stylomecon                       Eschscholzia minutiflora
   Stylomecon heterophylla        Eschscholzia parishii
 Canbya                           Eschscholzia ramosa
   Canbya aurea                   Eschscholzia rhombipetala
   Canbya candida                 Eschscholzia caespitosa
 Chlidonium                  etc...
   Chlidonium majus
                                                www.percona.com
Dump Tree SQL
SELECT d.*, p.a AS _parent
 FROM treepaths AS c
 INNER JOIN longnames AS d ON c.d = d.tsn
 LEFT JOIN treepaths AS p ON p.d = d.tsn
    AND p.a IN (202422, 564824, 18053, 18020)
    AND p.l = 1
 WHERE (c.a = 202422)
    AND (p.a IS NOT NULL OR d.tsn = 202422)
 ORDER BY c.l, d.completename;




                                        www.percona.com
Dump Tree SQL
                                     show children
SELECT d.*, p.a AS _parent           of these nodes
 FROM treepaths AS c
 INNER JOIN longnames AS d ON c.d = d.tsn
 LEFT JOIN treepaths AS p ON p.d = d.tsn
    AND p.a IN (202422, 564824, 18053, 18020)
    AND p.l = 1
 WHERE (c.a = 202422)
    AND (p.a IS NOT NULL OR d.tsn = 202422)
 ORDER BY c.l, d.completename;




                                        www.percona.com
How Does it Perform?
• Query profile = 0.20 sec on Macbook Pro
• MySQL EXPLAIN:
table type     key        ref     rows     extra

c     ref      tree_adl   const   114240   Using index; Using
                                           temporary; Using filesort
d     eq_ref   primary    c.d     1

p     ref      tree_dl    c.d,    1        Using where; Using index
                          const




                                                        www.percona.com
SHOW CREATE TABLE
CREATE TABLE `treepaths` (
    `a` int(11) NOT NULL DEFAULT '0',
    `d` int(11) NOT NULL DEFAULT '0',
    `l` tinyint(3) unsigned NOT NULL DEFAULT '0',
    PRIMARY KEY (`a`,`d`),
    KEY `tree_adl` (`a`,`d`,`l`),
    KEY `tree_dl` (`d`,`l`),
    CONSTRAINT FOREIGN KEY (`a`)
      REFERENCES `longnames` (`tsn`),
    CONSTRAINT FOREIGN KEY (`d`)
      REFERENCES `longnames` (`tsn`)
  ) ENGINE=InnoDB




                                                    www.percona.com
SHOW TABLE STATUS
Name:             treepaths
 Engine:          InnoDB
 Version:         10
 Row_format:      Compact
 Rows:            4600439
 Avg_row_length: 62
 Data_length:     288276480
 Max_data_length: 0
 Index_length:    273137664
 Data_free:       7340032



                              www.percona.com
Demo Time!




             www.percona.com
SQL Antipatterns




http://www.pragprog.com/titles/bksqla/
                                 www.percona.com
Models for hierarchical data

More Related Content

What's hot

Elasticsearch in Netflix
Elasticsearch in NetflixElasticsearch in Netflix
Elasticsearch in Netflix
Danny Yuan
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
Dvir Volk
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
Mike Dirolf
 
이것이 레디스다.
이것이 레디스다.이것이 레디스다.
이것이 레디스다.
Kris Jeong
 
InnoDB Internal
InnoDB InternalInnoDB Internal
InnoDB Internal
mysqlops
 
Polyglot persistence @ netflix (CDE Meetup)
Polyglot persistence @ netflix (CDE Meetup) Polyglot persistence @ netflix (CDE Meetup)
Polyglot persistence @ netflix (CDE Meetup)
Roopa Tangirala
 
Workshop on CIFS / SMB Protocol Performance Analysis
Workshop on CIFS / SMB Protocol Performance AnalysisWorkshop on CIFS / SMB Protocol Performance Analysis
Workshop on CIFS / SMB Protocol Performance Analysis
PerformanceVision (previously SecurActive)
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
Chandler Huang
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
Jonas Bonér
 
Load Data Fast!
Load Data Fast!Load Data Fast!
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei Milovidov
Altinity Ltd
 
Graphs in the Database: Rdbms In The Social Networks Age
Graphs in the Database: Rdbms In The Social Networks AgeGraphs in the Database: Rdbms In The Social Networks Age
Graphs in the Database: Rdbms In The Social Networks Age
Lorenzo Alberton
 
Rest api 테스트 수행가이드
Rest api 테스트 수행가이드Rest api 테스트 수행가이드
Rest api 테스트 수행가이드
SangIn Choung
 
[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화
NAVER D2
 
Introduction to Apache ZooKeeper
Introduction to Apache ZooKeeperIntroduction to Apache ZooKeeper
Introduction to Apache ZooKeeper
Saurav Haloi
 
REST API 설계
REST API 설계REST API 설계
REST API 설계
Terry Cho
 
Neo4j Training Cypher
Neo4j Training CypherNeo4j Training Cypher
Neo4j Training Cypher
Max De Marzi
 
NoSQL databases - An introduction
NoSQL databases - An introductionNoSQL databases - An introduction
NoSQL databases - An introduction
Pooyan Mehrparvar
 
Scaling Twitter
Scaling TwitterScaling Twitter
Scaling Twitter
Blaine
 
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdfDeep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Altinity Ltd
 

What's hot (20)

Elasticsearch in Netflix
Elasticsearch in NetflixElasticsearch in Netflix
Elasticsearch in Netflix
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
이것이 레디스다.
이것이 레디스다.이것이 레디스다.
이것이 레디스다.
 
InnoDB Internal
InnoDB InternalInnoDB Internal
InnoDB Internal
 
Polyglot persistence @ netflix (CDE Meetup)
Polyglot persistence @ netflix (CDE Meetup) Polyglot persistence @ netflix (CDE Meetup)
Polyglot persistence @ netflix (CDE Meetup)
 
Workshop on CIFS / SMB Protocol Performance Analysis
Workshop on CIFS / SMB Protocol Performance AnalysisWorkshop on CIFS / SMB Protocol Performance Analysis
Workshop on CIFS / SMB Protocol Performance Analysis
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
Load Data Fast!
Load Data Fast!Load Data Fast!
Load Data Fast!
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei Milovidov
 
Graphs in the Database: Rdbms In The Social Networks Age
Graphs in the Database: Rdbms In The Social Networks AgeGraphs in the Database: Rdbms In The Social Networks Age
Graphs in the Database: Rdbms In The Social Networks Age
 
Rest api 테스트 수행가이드
Rest api 테스트 수행가이드Rest api 테스트 수행가이드
Rest api 테스트 수행가이드
 
[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화
 
Introduction to Apache ZooKeeper
Introduction to Apache ZooKeeperIntroduction to Apache ZooKeeper
Introduction to Apache ZooKeeper
 
REST API 설계
REST API 설계REST API 설계
REST API 설계
 
Neo4j Training Cypher
Neo4j Training CypherNeo4j Training Cypher
Neo4j Training Cypher
 
NoSQL databases - An introduction
NoSQL databases - An introductionNoSQL databases - An introduction
NoSQL databases - An introduction
 
Scaling Twitter
Scaling TwitterScaling Twitter
Scaling Twitter
 
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdfDeep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
 

Viewers also liked

Trees and Hierarchies in SQL
Trees and Hierarchies in SQLTrees and Hierarchies in SQL
Trees and Hierarchies in SQL
Eduard Hildebrandt
 
Recursive Query Throwdown
Recursive Query ThrowdownRecursive Query Throwdown
Recursive Query Throwdown
Karwin Software Solutions LLC
 
SQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォークSQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォーク
ke-m kamekoopa
 
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
gree_tech
 
SQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリーSQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリー
ke-m kamekoopa
 
Fax With Sangoma Gateway
Fax With Sangoma GatewayFax With Sangoma Gateway
Fax With Sangoma Gateway
Hossein Yavari
 
10 system.security.cryptography
10 system.security.cryptography10 system.security.cryptography
10 system.security.cryptography
Mohammad Alyan
 
1 first lesson -assemblies
1  first lesson -assemblies1  first lesson -assemblies
1 first lesson -assemblies
Mohammad Alyan
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributes
Mohammad Alyan
 
Introduction To ERP
Introduction To ERPIntroduction To ERP
Introduction To ERP
Mohammad Alyan
 
Login System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysqlLogin System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysql
thesoftwareguy7
 
8 memory managment & pointers
8 memory managment & pointers8 memory managment & pointers
8 memory managment & pointersMohammad Alyan
 
3 third lesson-reflection
3 third lesson-reflection3 third lesson-reflection
3 third lesson-reflectionMohammad Alyan
 
Coursera
CourseraCoursera
Coursera
Mohammad Alyan
 
4 fourth lesson-deployment
4 fourth lesson-deployment4 fourth lesson-deployment
4 fourth lesson-deploymentMohammad Alyan
 

Viewers also liked (18)

Trees and Hierarchies in SQL
Trees and Hierarchies in SQLTrees and Hierarchies in SQL
Trees and Hierarchies in SQL
 
Recursive Query Throwdown
Recursive Query ThrowdownRecursive Query Throwdown
Recursive Query Throwdown
 
SQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォークSQLアンチパターン - ジェイウォーク
SQLアンチパターン - ジェイウォーク
 
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
アナザーエデンにおける非同期オートセーブを用いた通信待ちストレスのないゲーム体験の実現
 
SQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリーSQLアンチパターン - ナイーブツリー
SQLアンチパターン - ナイーブツリー
 
Fax With Sangoma Gateway
Fax With Sangoma GatewayFax With Sangoma Gateway
Fax With Sangoma Gateway
 
5 fifth lesson -xml
5 fifth lesson -xml5 fifth lesson -xml
5 fifth lesson -xml
 
7 multi threading
7 multi threading7 multi threading
7 multi threading
 
10 system.security.cryptography
10 system.security.cryptography10 system.security.cryptography
10 system.security.cryptography
 
1 first lesson -assemblies
1  first lesson -assemblies1  first lesson -assemblies
1 first lesson -assemblies
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributes
 
Introduction To ERP
Introduction To ERPIntroduction To ERP
Introduction To ERP
 
Login System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysqlLogin System with Windows/Microsoft Live using OAuth php and mysql
Login System with Windows/Microsoft Live using OAuth php and mysql
 
8 memory managment & pointers
8 memory managment & pointers8 memory managment & pointers
8 memory managment & pointers
 
3 third lesson-reflection
3 third lesson-reflection3 third lesson-reflection
3 third lesson-reflection
 
Coursera
CourseraCoursera
Coursera
 
4 fourth lesson-deployment
4 fourth lesson-deployment4 fourth lesson-deployment
4 fourth lesson-deployment
 
9 networking
9 networking9 networking
9 networking
 

More from Karwin Software Solutions LLC

InnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick FiguresInnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick Figures
Karwin Software Solutions LLC
 
SQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and ProfitSQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and Profit
Karwin Software Solutions LLC
 
Extensible Data Modeling
Extensible Data ModelingExtensible Data Modeling
Extensible Data Modeling
Karwin Software Solutions LLC
 
Sql query patterns, optimized
Sql query patterns, optimizedSql query patterns, optimized
Sql query patterns, optimized
Karwin Software Solutions LLC
 
Survey of Percona Toolkit
Survey of Percona ToolkitSurvey of Percona Toolkit
Survey of Percona Toolkit
Karwin Software Solutions LLC
 
How to Design Indexes, Really
How to Design Indexes, ReallyHow to Design Indexes, Really
How to Design Indexes, Really
Karwin Software Solutions LLC
 
Schemadoc
SchemadocSchemadoc
Percona toolkit
Percona toolkitPercona toolkit
MySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB StatusMySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB Status
Karwin Software Solutions LLC
 
Requirements the Last Bottleneck
Requirements the Last BottleneckRequirements the Last Bottleneck
Requirements the Last Bottleneck
Karwin Software Solutions LLC
 
Mentor Your Indexes
Mentor Your IndexesMentor Your Indexes
Mentor Your Indexes
Karwin Software Solutions LLC
 
Sql Injection Myths and Fallacies
Sql Injection Myths and FallaciesSql Injection Myths and Fallacies
Sql Injection Myths and Fallacies
Karwin Software Solutions LLC
 
Full Text Search In PostgreSQL
Full Text Search In PostgreSQLFull Text Search In PostgreSQL
Full Text Search In PostgreSQL
Karwin Software Solutions LLC
 
Sql Antipatterns Strike Back
Sql Antipatterns Strike BackSql Antipatterns Strike Back
Sql Antipatterns Strike Back
Karwin Software Solutions LLC
 

More from Karwin Software Solutions LLC (14)

InnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick FiguresInnoDB Locking Explained with Stick Figures
InnoDB Locking Explained with Stick Figures
 
SQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and ProfitSQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and Profit
 
Extensible Data Modeling
Extensible Data ModelingExtensible Data Modeling
Extensible Data Modeling
 
Sql query patterns, optimized
Sql query patterns, optimizedSql query patterns, optimized
Sql query patterns, optimized
 
Survey of Percona Toolkit
Survey of Percona ToolkitSurvey of Percona Toolkit
Survey of Percona Toolkit
 
How to Design Indexes, Really
How to Design Indexes, ReallyHow to Design Indexes, Really
How to Design Indexes, Really
 
Schemadoc
SchemadocSchemadoc
Schemadoc
 
Percona toolkit
Percona toolkitPercona toolkit
Percona toolkit
 
MySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB StatusMySQL 5.5 Guide to InnoDB Status
MySQL 5.5 Guide to InnoDB Status
 
Requirements the Last Bottleneck
Requirements the Last BottleneckRequirements the Last Bottleneck
Requirements the Last Bottleneck
 
Mentor Your Indexes
Mentor Your IndexesMentor Your Indexes
Mentor Your Indexes
 
Sql Injection Myths and Fallacies
Sql Injection Myths and FallaciesSql Injection Myths and Fallacies
Sql Injection Myths and Fallacies
 
Full Text Search In PostgreSQL
Full Text Search In PostgreSQLFull Text Search In PostgreSQL
Full Text Search In PostgreSQL
 
Sql Antipatterns Strike Back
Sql Antipatterns Strike BackSql Antipatterns Strike Back
Sql Antipatterns Strike Back
 

Recently uploaded

Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 

Recently uploaded (20)

Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 

Models for hierarchical data

  • 1. Models for Hierarchical Data with SQL and PHP Bill Karwin, Percona Inc.
  • 2. Me • Software developer • C, Java, Perl, PHP, Ruby • SQL maven • MySQL Consultant at Percona • Author of SQL Antipatterns: Avoiding the Pitfalls of Database Programming www.percona.com
  • 3. Problem • Store & query hierarchical data - Categories/subcategories - Bill of materials - Threaded discussions www.percona.com
  • 4. Example: Bug Report Comments (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 5. Solutions •Adjacency list •Path enumeration •Nested sets •Closure table www.percona.com
  • 6. Adjacency List www.percona.com
  • 7. Adjacency List • Naive solution nearly everyone uses • Each entry knows its immediate parent comment_id parent_id author comment 1 NULL Fran What’s the cause of this bug? 2 1 Ollie I think it’s a null pointer. 3 2 Fran No, I checked for that. 4 1 Kukla We need to check valid input. 5 4 Ollie Yes, that’s a bug. 6 4 Fran Yes, please add a check 7 6 Kukla That fixed it. www.percona.com
  • 8. Insert a New Node INSERT INTO Comments (parent_id, author, comment) VALUES (5, ‘Fran’, ‘I agree!’); (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 9. Insert a New Node INSERT INTO Comments (parent_id, author, comment) VALUES (5, ‘Fran’, ‘I agree!’); (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 10. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (6) Fran: (5) Ollie: No, I checked for Yes, please add a Yes, that’s a bug. that. check. (7) Kukla: That fixed it. www.percona.com
  • 11. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. www.percona.com
  • 12. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. www.percona.com
  • 13. Move a Node or Subtree UPDATE Comments SET parent_id = 3 WHERE comment_id = 6; (1) Fran: What’s the cause of this bug? (2) Ollie: (4) Kukla: I think it’s a null We need to check pointer. valid input. (3) Fran: (5) Ollie: No, I checked for Yes, that’s a bug. that. (6) Fran: Yes, please add a check. (7) Kukla: That fixed it. www.percona.com
  • 14. Query Immediate Child/Parent • Query a node’s children: SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id); • Query a node’s parent: SELECT * FROM Comments c1 JOIN Comments c2 ON (c1.parent_id = c2.comment_id); www.percona.com
  • 15. Can’t Handle Deep Trees SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id) LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id) LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id) LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id) LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id) LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id) LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id) LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id) LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id) ... www.percona.com
  • 16. Can’t Handle Deep Trees SELECT * FROM Comments c1 LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id) LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id) LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id) LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id) LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id) LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id) LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id) LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id) LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id) ... it still doesn’t support unlimited depth! www.percona.com
  • 17. SQL-99 recursive syntax WITH [RECURSIVE] CommentTree (comment_id, bug_id, parent_id, author, comment, depth) AS ( SELECT *, 0 AS depth FROM Comments WHERE parent_id IS NULL UNION ALL SELECT c.*, ct.depth+1 AS depth FROM CommentTree ct JOIN Comments c ON (ct.comment_id = c.parent_id) ) SELECT * FROM CommentTree WHERE bug_id = 1234; ✓ PostgreSQL, Oracle 11g, IBM DB2, Microsoft SQL Server, Apache Derby ✗ MySQL, SQLite, Informix, Firebird,etc. www.percona.com
  • 18. Path Enumeration www.percona.com
  • 19. Path Enumeration • Store chain of ancestors in each node comment_id path author comment 1 1/ Fran What’s the cause of this bug? 2 1/2/ Ollie I think it’s a null pointer. 3 1/2/3/ Fran No, I checked for that. 4 1/4/ Kukla We need to check valid input. 5 1/4/5/ Ollie Yes, that’s a bug. 6 1/4/6/ Fran Yes, please add a check 7 1/4/6/7/ Kukla That fixed it. www.percona.com
  • 20. Path Enumeration • Store chain of ancestors in each node good for breadcrumbs comment_id path author comment 1 1/ Fran What’s the cause of this bug? 2 1/2/ Ollie I think it’s a null pointer. 3 1/2/3/ Fran No, I checked for that. 4 1/4/ Kukla We need to check valid input. 5 1/4/5/ Ollie Yes, that’s a bug. 6 1/4/6/ Fran Yes, please add a check 7 1/4/6/7/ Kukla That fixed it. www.percona.com
  • 21. Query Ancestors and Subtrees • Query ancestors of comment #7: SELECT * FROM Comments WHERE ‘1/4/6/7/’ LIKE path || ‘%’; • Query descendants of comment #4: SELECT * FROM Comments WHERE path LIKE ‘1/4/%’; www.percona.com
  • 22. Add a New Child of #7 INSERT INTO Comments (author, comment) VALUES (‘Ollie’, ‘Good job!’); SELECT path FROM Comments WHERE comment_id = 7; UPDATE Comments SET path = $parent_path || LAST_INSERT_ID() || ‘/’ WHERE comment_id = LAST_INSERT_ID(); www.percona.com
  • 23. Nested Sets www.percona.com
  • 24. Nested Sets • Each comment encodes its descendants using two numbers: - A comment’s left number is less than all numbers used by the comment’s descendants. - A comment’s right number is greater than all numbers used by the comment’s descendants. - A comment’s numbers are between all numbers used by the comment’s ancestors. www.percona.com
  • 25. What Does This Look Like? (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 26. What Does This Look Like? (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 27. What Does This Look Like? comment_id nsleft nsright author comment 1 1 14 Fran What’s the cause of this bug? 2 2 5 Ollie I think it’s a null pointer. 3 3 4 Fran No, I checked for that. 4 6 13 Kukla We need to check valid input. 5 7 8 Ollie Yes, that’s a bug. 6 9 12 Fran Yes, please add a check 7 10 11 Kukla That fixed it. www.percona.com
  • 28. What Does This Look Like? comment_id nsleft nsright author comment 1 1 14 Fran What’s the cause of this bug? 2 2 5 Ollie I think it’s a null pointer. 3 3 4 Fran No, I checked for that. 4 6 13 Kukla We need to check valid input. 5 7 8 Ollie Yes, that’s a bug. 6 9 12 Fran Yes, please add a check 7 10 11 Kukla That fixed it. these are not foreign keys www.percona.com
  • 29. Query Ancestors of #7 (1) Fran: ancestors What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 child (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 30. Query Ancestors of #7 SELECT * FROM Comments child JOIN Comments ancestor ON child.nsleft BETWEEN ancestor.nsleft AND ancestor.nsright WHERE child.comment_id = 7; www.percona.com
  • 31. Query Subtree Under #4 (1) Fran: What’s the parent cause of this bug? 1 14 (2) Ollie: (4) Kukla: descendants We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 32. Query Subtree Under #4 SELECT * FROM Comments parent JOIN Comments descendant ON descendant.nsleft BETWEEN parent.nsleft AND parent.nsright WHERE parent.comment_id = 4; www.percona.com
  • 33. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 34. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 16 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 15 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 10 11 8 9 14 12 (7) Kukla: That fixed it. 12 10 13 11 www.percona.com
  • 35. Insert New Child of #5 (1) Fran: What’s the cause of this bug? 1 16 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 15 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 10 11 8 9 14 12 (8) Fran: (7) Kukla: I agree! That fixed it. 8 9 10 12 13 11 www.percona.com
  • 36. Insert New Child of #5 UPDATE Comments SET nsleft = CASE WHEN nsleft >= 8 THEN nsleft+2 ELSE nsleft END, nsright = nsright+2 WHERE nsright >= 7; INSERT INTO Comments (nsleft, nsright, author, comment) VALUES (8, 9, 'Fran', 'I agree!'); • Recalculate left values for all nodes to the right of the new child. Recalculate right values for all nodes above and to the right. www.percona.com
  • 37. Query Immediate Parent of #6 (1) Fran: What’s the cause of this bug? 1 14 (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. 2 5 6 13 (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. 3 4 7 8 9 12 (7) Kukla: That fixed it. 10 11 www.percona.com
  • 38. Query Immediate Parent of #6 • Parent of #6 is an ancestor who has no descendant who is also an ancestor of #6. SELECT parent.* FROM Comments AS c JOIN Comments AS parent ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright) LEFT OUTER JOIN Comments AS in_between ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright) WHERE c.comment_id = 6 AND in_between.comment_id IS NULL; www.percona.com
  • 39. Query Immediate Parent of #6 • Parent of #6 is an ancestor who has no descendant who is also an ancestor of #6. SELECT parent.* FROM Comments AS c JOIN Comments AS parent ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright) LEFT OUTER JOIN Comments AS in_between ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright) WHERE c.comment_id = 6 AND in_between.comment_id IS NULL; querying immediate child is a similar problem www.percona.com
  • 40. Closure Table www.percona.com
  • 41. Closure Table CREATE TABLE TreePaths ( ancestor INT NOT NULL, descendant INT NOT NULL, PRIMARY KEY (ancestor, descendant), FOREIGN KEY(ancestor) REFERENCES Comments(comment_id), FOREIGN KEY(descendant) REFERENCES Comments(comment_id) ); www.percona.com
  • 42. Closure Table • Many-to-many table • Stores every path from each node to each of its descendants • A node even connects to itself www.percona.com
  • 43. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 44. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 45. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 46. Closure Table illustration (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 47. What Does This Look Like? ancestor descendant comment_id author comment 1 1 1 2 1 Fran What’s the cause of this bug? 1 3 1 4 2 Ollie I think it’s a null pointer. 1 5 3 Fran No, I checked for that. 1 6 1 7 4 Kukla We need to check valid 2 2 input. 2 3 5 Ollie Yes, that’s a bug. 3 3 4 4 6 Fran Yes, please add a check 4 5 7 Kukla That fixed it. 4 6 4 7 5 5 requires O(n²) rows 6 6 6 7 7 7 www.percona.com
  • 48. What Does This Look Like? ancestor descendant comment_id author comment 1 1 1 2 1 Fran What’s the cause of this bug? 1 3 1 4 2 Ollie I think it’s a null pointer. 1 5 3 Fran No, I checked for that. 1 6 1 7 4 Kukla We need to check valid 2 2 input. 2 3 5 Ollie Yes, that’s a bug. 3 3 4 4 6 Fran Yes, please add a check 4 5 7 Kukla That fixed it. 4 6 4 7 5 5 requires O(n²) rows 6 6 6 7 7 7 (but far fewer in practice) www.percona.com
  • 49. Query Descendants of #4 SELECT c.* FROM Comments c JOIN TreePaths t ON (c.comment_id = t.descendant) WHERE t.ancestor = 4; www.percona.com
  • 50. Paths Starting from #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 51. Query Ancestors of #6 SELECT c.* FROM Comments c JOIN TreePaths t ON (c.comment_id = t.ancestor) WHERE t.descendant = 6; www.percona.com
  • 52. Paths Terminating at #6 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 53. Insert New Child of #5 INSERT INTO Comments VALUES (8, ‘Fran’, ‘I agree!’); INSERT INTO TreePaths (ancestor, descendant) SELECT ancestor, 8 FROM TreePaths WHERE descendant = 5 UNION ALL SELECT 8, 8; www.percona.com
  • 54. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 55. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 56. Copy Paths from Parent (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (8) Fran: (7) Kukla: I agree! That fixed it. www.percona.com
  • 57. Delete Child #7 DELETE FROM TreePaths WHERE descendant = 7; www.percona.com
  • 58. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 59. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 60. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. www.percona.com
  • 61. Delete Paths Terminating at #7 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 62. Delete Subtree Under #4 DELETE FROM TreePaths WHERE descendant IN (SELECT descendant FROM TreePaths WHERE ancestor = 4); www.percona.com
  • 63. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 64. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null pointer. check valid input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 65. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (2) Ollie: I think it’s a null pointer. (3) Fran: No, I checked for that. www.percona.com
  • 66. Delete Any Paths Under #4 (1) Fran: What’s the cause of this bug? (4) Kukla: (2) Ollie: We need to I think it’s a null check valid pointer. input. (3) Fran: (5) Ollie: (6) Fran: No, I checked Yes, that’s a Yes, please add for that. bug. a check. (7) Kukla: That fixed it. www.percona.com
  • 67. Path Length • Add a length column ancestor descendant length • MAX(length) is depth of tree 1 1 0 1 2 1 1 3 2 • Makes it easier to query 1 1 4 5 1 2 immediate parent or child: 1 6 2 1 7 3 SELECT c.* 2 2 0 FROM Comments c 2 3 1 JOIN TreePaths t 3 3 0 4 4 0 ON (c.comment_id = t.descendant) 4 5 1 WHERE t.ancestor = 4 4 6 1 AND t.length = 1; 4 7 2 5 5 0 6 6 0 6 7 1 7 7 0 www.percona.com
  • 68. Path Length • Add a length column ancestor descendant length • MAX(length) is depth of tree 1 1 0 1 2 1 1 3 2 • Makes it easier to query 1 1 4 5 1 2 immediate parent or child: 1 6 2 1 7 3 SELECT c.* 2 2 0 FROM Comments c 2 3 1 JOIN TreePaths t 3 3 0 4 4 0 ON (c.comment_id = t.descendant) 4 5 1 WHERE t.ancestor = 4 4 6 1 AND t.length = 1; 4 7 2 5 5 0 6 6 0 6 7 1 7 7 0 www.percona.com
  • 69. Choosing the Right Design Design Tables Query Query Delete Insert Move Referential Child Subtree Node Node Subtree Integrity Adjacency 1 Easy Hard Easy Easy Easy Yes List Path 1 Hard Easy Easy Easy Easy No Enumeration Nested Sets 1 Hard Easy Hard Hard Hard No Closure 2 Easy Easy Easy Easy Easy Yes Table www.percona.com
  • 70. PHP Demo of Closure Table www.percona.com
  • 71. Hierarchical Test Data • Integrated Taxonomic Information System - http://itis.gov/ - Free authoritative taxonomic information on plants, animals, fungi, microbes - 518,756 scientific names (as of Feb 2011) www.percona.com
  • 72. California Poppy Kingdom: Plantae Division: Tracheobionta Class: Magnoliophyta Order: Magnoliopsida unranked: Magnoliidae unranked: Papaverales Family: Papaveraceae Genus: Eschscholzia Species: Eschscholzia californica www.percona.com
  • 73. California Poppy Kingdom: Plantae Division: Tracheobionta Class: Magnoliophyta Order: Magnoliopsida unranked: Magnoliidae unranked: Papaverales Family: Papaveraceae Genus: Eschscholzia Species: Eschscholzia californica id=18956 www.percona.com
  • 74. California Poppy: ITIS Entry SELECT * FROM Hierarchy WHERE hierarchy_string LIKE ‘%-18956’; hierarchy_string 202422-564824-18061-18063-18064-18879-18880-18954-18956 www.percona.com
  • 75. California Poppy: ITIS Entry SELECT * FROM Hierarchy WHERE hierarchy_string LIKE ‘%-18956’; hierarchy_string 202422-564824-18061-18063-18064-18879-18880-18954-18956 ITIS data uses ...but I converted path enumeration it to closure table www.percona.com
  • 76. Hierarchical Data Classes abstract class ZendX_Db_Table_TreeTable extends Zend_Db_Table_Abstract { public function fetchTreeByRoot($rootId, $expand) public function fetchBreadcrumbs($leafId) } www.percona.com
  • 77. Hierarchical Data Classes class ZendX_Db_Table_Row_TreeRow extends Zend_Db_Table_Row_Abstract { public function addChildRow($childRow) public function getChildren() } class ZendX_Db_Table_Rowset_TreeRowset extends Zend_Db_Table_Rowset_Abstract { public function append($row) } www.percona.com
  • 78. Using TreeTable class ItisTable extends ZendX_Db_Table_TreeTable { protected $_name = “longnames”; protected $_closureName = “treepaths”; } $itis = new ItisTable(); www.percona.com
  • 79. Breadcrumbs $breadcrumbs = $itis->fetchBreadcrumbs(18956); foreach ($breadcrumbs as $crumb) { print $crumb->completename . “ > ”; } Plantae > Tracheobionta > Magnoliophyta > Magnoliopsida > Magnoliidae > Papaverales > Papaveraceae > Eschscholzia > Eschscholzia californica > www.percona.com
  • 80. Breadcrumbs SQL SELECT a.* FROM longnames AS a INNER JOIN treepaths AS c ON a.tsn = c.a WHERE (c.d = 18956) ORDER BY c.l DESC www.percona.com
  • 81. How Does it Perform? • Query profile = 0.0006 sec • MySQL EXPLAIN: table type key ref rows extra c ref tree_dl const 9 Using where; Using index a eq_ref primary c.a 1 www.percona.com
  • 82. Dump Tree $tree = $itis->fetchTreeByRoot(18880); // Papaveraceae print_tree($tree); function print_tree($tree, $prefix = ‘’) { print “{$prefix} {$tree->completename}n”; foreach ($tree->getChildren() as $child) { print_tree($child, “{$prefix} ”); } } www.percona.com
  • 83. Dump Tree Result Papaveraceae Romneya Platystigma Romneya coulteri Platystigma linearis Romneya trichocalyx Glaucium Dendromecon Glaucium corniculatum Dendromecon harfordii Glaucium flavum Dendromecon rigida Chelidonium Eschscholzia Chelidonium majus Eschscholzia californica Bocconia Eschscholzia glyptosperma Bocconia frutescens Eschscholzia hypecoides Stylophorum Eschscholzia lemmonii Stylophorum diphyllum Eschscholzia lobbii Stylomecon Eschscholzia minutiflora Stylomecon heterophylla Eschscholzia parishii Canbya Eschscholzia ramosa Canbya aurea Eschscholzia rhombipetala Canbya candida Eschscholzia caespitosa Chlidonium etc... Chlidonium majus www.percona.com
  • 84. Dump Tree SQL SELECT d.*, p.a AS _parent FROM treepaths AS c INNER JOIN longnames AS d ON c.d = d.tsn LEFT JOIN treepaths AS p ON p.d = d.tsn AND p.a IN (202422, 564824, 18053, 18020) AND p.l = 1 WHERE (c.a = 202422) AND (p.a IS NOT NULL OR d.tsn = 202422) ORDER BY c.l, d.completename; www.percona.com
  • 85. Dump Tree SQL show children SELECT d.*, p.a AS _parent of these nodes FROM treepaths AS c INNER JOIN longnames AS d ON c.d = d.tsn LEFT JOIN treepaths AS p ON p.d = d.tsn AND p.a IN (202422, 564824, 18053, 18020) AND p.l = 1 WHERE (c.a = 202422) AND (p.a IS NOT NULL OR d.tsn = 202422) ORDER BY c.l, d.completename; www.percona.com
  • 86. How Does it Perform? • Query profile = 0.20 sec on Macbook Pro • MySQL EXPLAIN: table type key ref rows extra c ref tree_adl const 114240 Using index; Using temporary; Using filesort d eq_ref primary c.d 1 p ref tree_dl c.d, 1 Using where; Using index const www.percona.com
  • 87. SHOW CREATE TABLE CREATE TABLE `treepaths` ( `a` int(11) NOT NULL DEFAULT '0', `d` int(11) NOT NULL DEFAULT '0', `l` tinyint(3) unsigned NOT NULL DEFAULT '0', PRIMARY KEY (`a`,`d`), KEY `tree_adl` (`a`,`d`,`l`), KEY `tree_dl` (`d`,`l`), CONSTRAINT FOREIGN KEY (`a`) REFERENCES `longnames` (`tsn`), CONSTRAINT FOREIGN KEY (`d`) REFERENCES `longnames` (`tsn`) ) ENGINE=InnoDB www.percona.com
  • 88. SHOW TABLE STATUS Name: treepaths Engine: InnoDB Version: 10 Row_format: Compact Rows: 4600439 Avg_row_length: 62 Data_length: 288276480 Max_data_length: 0 Index_length: 273137664 Data_free: 7340032 www.percona.com
  • 89. Demo Time! www.percona.com