© 2010 Hal Stern
Some Rights Reserved
Parsing Strange:
URL to SQL to HTML
Hal Stern
snowmanonfire.com
slideshare.net/freeh...
© 2010 Hal Stern
Some Rights Reserved
Why Do You Care?
• Database performance = user experience
• A little database expert...
© 2010 Hal Stern
Some Rights Reserved
Flow of Control
• Web server URL manipulation
> Real file or permalink URL?
• URL to...
© 2010 Hal Stern
Some Rights Reserved
Whose File Is This?
• User URL request passed to web server
• Web server checks
.hta...
© 2010 Hal Stern
Some Rights Reserved
Example Meta Fail: 404 Not Found
• Access broken image URLs for
unintended results: ...
© 2010 Hal Stern
Some Rights Reserved
What Happens Before The Loop
• Parse URL into a query
• Set conditionals & select te...
© 2010 Hal Stern
Some Rights Reserved
Examining the Query String
• SQL passed to MySQL in WP_Query
object’s request elemen...
© 2010 Hal Stern
Some Rights Reserved
“Home Page” Query Deconstruction
WordCamp Philly 2010 8
SELECT SQL_CALC_FOUND_ROWS w...
© 2010 Hal Stern
Some Rights Reserved
Query Parsing
• parse_request() method of WP_Query
extracts query variables from URL...
© 2010 Hal Stern
Some Rights Reserved
Query Variables to SQL
• Query type: post by title, posts by category
or tag, posts ...
© 2010 Hal Stern
Some Rights Reserved
Simple Title Slug Parsing
• Rewrite matches root of permalink,
extracts tail of URL ...
© 2010 Hal Stern
Some Rights Reserved
CPT Query Variables
• Register CPT with a custom query variable
'query_var' => 'ebay...
© 2010 Hal Stern
Some Rights Reserved
WordPress Meta Data
• Common DB mechanics for all meta data
> Categories, tags, cust...
© 2010 Hal Stern
Some Rights Reserved
Graphs and JOIN Operations
• WordPress maps tags and categories 1:N
to posts (each t...
© 2010 Hal Stern
Some Rights Reserved
WordPress Taxonomy Tables
• Term relationships table maps
N terms to each post
• Ter...
© 2010 Hal Stern
Some Rights Reserved
SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts
INNER JOIN wp_term_relationships...
© 2010 Hal Stern
Some Rights Reserved
More on Canonical URLs
• Canonical URLs improve SEO
• WordPress is really good about...
© 2010 Hal Stern
Some Rights Reserved
Modifying the Query
• Brute force isn’t necessarily good
> Using query_posts() ignor...
© 2010 Hal Stern
Some Rights Reserved
SQL Generation Filters
• posts_where
> More explicit control over query variable to
...
© 2010 Hal Stern
Some Rights Reserved
Custom Post Types
• Change SQL WHERE clause on post type
> wp_posts.post_type=‘ebay’...
© 2010 Hal Stern
Some Rights Reserved
Applications
• Stylized listings
> Category sorted alphabetically
> Use posts as lis...
© 2010 Hal Stern
Some Rights Reserved
Template File Selection
• is_x() conditionals set in query parsing
• Used to drive t...
© 2010 Hal Stern
Some Rights Reserved
HTML Generation
• Done in the_post() method
• Raw content retrieved from MySQL
> Sho...
© 2010 Hal Stern
Some Rights Reserved
More Complex Example
• For category or tag templates, intersperse
custom post types ...
© 2010 Hal Stern
Some Rights Reserved
Matching Tags
• What if you want to match tags from user-
selected posts to tagged C...
© 2010 Hal Stern
Some Rights Reserved
Why Do You Care?
• User experience improvement
> JOINS are expensive
> Large post ta...
© 2010 Hal Stern
Some Rights Reserved
Resources
• Core files where SQL stuff happens
> query.php
> post.php
> canonical.ph...
© 2010 Hal Stern
Some Rights Reserved
Contact (Shameless Plug)
Hal Stern
freeholdhal@gmail.com
@freeholdhal
snowmanonfire....
Upcoming SlideShare
Loading in …5
×

Parsing strange v4

2,262 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,262
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • yesterday – UX, URLs are your currency
    it’s how the world sees us, and gives us incredible flexibility to direct/guide users
  • SQL_CALC_FOUND_ROWS limits the number of returned rows via the LIMIT clause, and ensures that you don’t tax MySQL,
    Perform immense queries (SMOF has 600 post entries)

    WHERE 1=1 is for building compound where clauses; ensures there’s no degenerate case

    Type=post versus revision; status publish/private versus draft, trash
  • Look at rewrite.php, and canonical.php (more on that later)
    Default terms of “tag” and “category” can be changed in the Settings/Permalinks section of the Dashboard
  • You can
  • Separate namespaces for pages and posts
    What about parent pages?
    In this example the permalink structure is %year%/%title%
  • Three joins needed to build the full cartesian product of related tables.

    Get all of the terms that have a slug of “premio”, and find out what taxonomies they’re in
    Get the taxonomies that are post tags, and find all taxonomy object ids (that are post tags of slug “premio”)
    Get all of the posts that have this object id associated with them from term_relationships
    Order the final table by post date,starting with the most recent (0) and getting 10 of them.
  • Don’t want multiple URLs pointing to the same page, so canonical parsing cleans them up
  • Parsing strange v4

    1. 1. © 2010 Hal Stern Some Rights Reserved Parsing Strange: URL to SQL to HTML Hal Stern snowmanonfire.com slideshare.net/freeholdhal headshot by Richard Stevens http://dieselsweeties.com
    2. 2. © 2010 Hal Stern Some Rights Reserved Why Do You Care? • Database performance = user experience • A little database expertise goes a long way • Taxonomies for more than sidebar lists • Custom post types • WordPress as a powerful CMS >> blog > Change default behaviors > Defy the common wisdom > Integrate other content sources/filters WordCamp Philly 2010 2
    3. 3. © 2010 Hal Stern Some Rights Reserved Flow of Control • Web server URL manipulation > Real file or permalink URL? • URL to query variables > What to display? Tag? Post? Category? • Query variables to SQL generation > How exactly to get that content? • Template file selection > How will content be displayed? • Content manipulation 3WordCamp Philly 2010
    4. 4. © 2010 Hal Stern Some Rights Reserved Whose File Is This? • User URL request passed to web server • Web server checks .htaccess file > WP install root > Other .htaccess files may interfere • Basic rewriting rules: If file or directory URL doesn’t exist, start WordPress via index.php WordCamp Philly 2010 4 <IfModule mod_rewrite.c> RewriteEngine On RewriteBase /whereyouputWordPress/ RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L] </IfModule>
    5. 5. © 2010 Hal Stern Some Rights Reserved Example Meta Fail: 404 Not Found • Access broken image URLs for unintended results: no 404 pages! myblog/images/not-a-pic.jpg • Web server can’t find file, assumes it’s a permalink, hands to WP • WP can’t interpret it, so defaults to home WordCamp Philly 2010 5 myblog/ myblog/wp-content (etc) myblog/images
    6. 6. © 2010 Hal Stern Some Rights Reserved What Happens Before The Loop • Parse URL into a query • Set conditionals & select templates • Execute the query & cache results • Run the Loop: <?php if (have_posts()) : while (have_posts()) : the_post(); //loop content endwhile; endif; ?> WordCamp Philly 2010 6
    7. 7. © 2010 Hal Stern Some Rights Reserved Examining the Query String • SQL passed to MySQL in WP_Query object’s request element • Brute force: edit theme footer.php to see main loop’s query for displayed page WordCamp Philly 2010 7 <?php global $wp_query; echo ”SQL for this page "; echo $wp_query->request; echo "<br>"; ?>
    8. 8. © 2010 Hal Stern Some Rights Reserved “Home Page” Query Deconstruction WordCamp Philly 2010 8 SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts WHERE 1=1 AND wp_posts.post_type = 'post’ AND (wp_posts.post_status = 'publish' OR wp_posts.post_status = 'private’) ORDER BY wp_posts.post_date DESC LIMIT 0, 10 Get all fields from posts table, but limit number of returned rows Only get posts, and those that are published or private to the user Sort the results by date in descending order Start results starting with record 0 and up to 10 more results
    9. 9. © 2010 Hal Stern Some Rights Reserved Query Parsing • parse_request() method of WP_Query extracts query variables from URL • Execute rewrite rules > Pick off ?p=67 style http GET variables > Match permalink structure > Match keywords like “author” and “tag” > Match custom post type slugs WordCamp Philly 2010 9
    10. 10. © 2010 Hal Stern Some Rights Reserved Query Variables to SQL • Query type: post by title, posts by category or tag, posts by date • Variables for the query > Slug values for category/tags > Month/day numbers > Explicit variable values • post_type variable has been around for a while; CPT queries fill in new values WordCamp Philly 2010 10
    11. 11. © 2010 Hal Stern Some Rights Reserved Simple Title Slug Parsing • Rewrite matches root of permalink, extracts tail of URL as a title slug WordCamp Philly 2010 11 SELECT wp_posts.* FROM wp_posts WHERE 1=1 AND YEAR(wp_posts.post_date)='2010' AND wp_posts.post_name = 'premio- sausage' AND wp_posts.post_type = 'post' ORDER BY wp_posts.post_date DESC /2010/premio-sausage
    12. 12. © 2010 Hal Stern Some Rights Reserved CPT Query Variables • Register CPT with a custom query variable 'query_var' => 'ebay' • Variable works in URLs like built-ins myblog.com/?ebay=current_items myblog.com/?ebay=cool_searches • Variable value matches CPT title slug WordCamp Philly 2010 12
    13. 13. © 2010 Hal Stern Some Rights Reserved WordPress Meta Data • Common DB mechanics for all meta data > Categories, tags, custom taxonomies • Normalized down to 3 tables > Terms: word strings and their slugs > Taxonomies: collections of terms > Relationships: terms attached to posts • It’s so simple it gets really complex. Really. WordCamp Philly 2010 13
    14. 14. © 2010 Hal Stern Some Rights Reserved Graphs and JOIN Operations • WordPress maps tags and categories 1:N to posts (each term in many posts) • You need to punch MySQL to handle hierarchical, multi-valued relations > INNER JOIN builds intermediate tables on common key values • Following link in a graph is equivalent to an INNER JOIN on tables of linked items WordCamp Philly 2010 14
    15. 15. © 2010 Hal Stern Some Rights Reserved WordPress Taxonomy Tables • Term relationships table maps N terms to each post • Term taxonomy maps N terms to each taxonomy • Term table has slugs for URL mapping WordCamp Philly 2010 15 wp_term_relationships object_id term_taxonomy_id wp_posts post_id …. post_date … post_content wp_term_taxonomy term_taxonomy_id term_id taxonomy description wp_terms term_id name slug
    16. 16. © 2010 Hal Stern Some Rights Reserved SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts INNER JOIN wp_term_relationships ON (wp_posts.ID = wp_term_relationships.object_id) INNER JOIN wp_term_taxonomy ON (wp_term_relationships.term_taxonomy_id = wp_term_taxonomy.term_taxonomy_id) INNER JOIN wp_terms ON (wp_term_taxonomy.term_id = wp_terms.term_id) WHERE 1=1 AND wp_term_taxonomy.taxonomy = 'post_tag' AND wp_terms.slug IN ('premio') AND wp_posts.post_type = 'post' AND (wp_posts.post_status = 'publish' OR wp_posts.post_status = 'private') GROUP BY wp_posts.ID ORDER BY wp_posts.post_date DESC LIMIT 0, 10 Taxonomy Lookup WordCamp Philly 2010 16 /tag/premio
    17. 17. © 2010 Hal Stern Some Rights Reserved More on Canonical URLs • Canonical URLs improve SEO • WordPress is really good about generating 301 Redirects for non-standard URLs • Example: URL doesn’t appear to match a permalink, WordPress does prediction > Use “LIKE title%” in WHERE clause > Matches “title” as initial substring with % wildcard WordCamp Philly 2010 17
    18. 18. © 2010 Hal Stern Some Rights Reserved Modifying the Query • Brute force isn’t necessarily good > Using query_posts() ignores all previous parsing, runs a new SQL query • Filter query_vars > Change default parsing (convert any day to a week’s worth of posts, for example) • Actions parse_query & parse_request > Access WP_Query object before execution > is_xx() conditionals are already set WordCamp Philly 2010 18
    19. 19. © 2010 Hal Stern Some Rights Reserved SQL Generation Filters • posts_where > More explicit control over query variable to SQL grammar mapping • posts_join > Add or modify JOIN operations for other graph relationships • Many other filters > Change grouping of results > Change ordering of results WordCamp Philly 2010 19
    20. 20. © 2010 Hal Stern Some Rights Reserved Custom Post Types • Change SQL WHERE clause on post type > wp_posts.post_type=‘ebay’ • Add new rewrite rules for URL parsing similar to category & tag > Set slug in CPT registration array 'rewrite' => array ("slug" => “ebay”) • Watch out for competing, overwritten or unflushed rewrite entries <?php echo "<pre>”; print_r(get_option('rewrite_rules')); echo "</pre>”; ?> WordCamp Philly 2010 20
    21. 21. © 2010 Hal Stern Some Rights Reserved Applications • Stylized listings > Category sorted alphabetically > Use posts as listings of resources (jobs, clients, events) – use CPT for more control • Custom URL slugs > Add rewrite rules to match slug and set query variables • Joining other social graphs > Suggested/related content WordCamp Philly 2010 21
    22. 22. © 2010 Hal Stern Some Rights Reserved Template File Selection • is_x() conditionals set in query parsing • Used to drive template selection > is_tag() looks for tag-slug, tag-id, then tag > Full search hierarchy in Codex • template_redirect action > Called in the template loader > Add actions to override defaults WordCamp Philly 2010 22
    23. 23. © 2010 Hal Stern Some Rights Reserved HTML Generation • Done in the_post() method • Raw content retrieved from MySQL > Short codes interpreted > CSS applied • Some caching plugins generate and store HTML, so YMMV WordCamp Philly 2010 23
    24. 24. © 2010 Hal Stern Some Rights Reserved More Complex Example • For category or tag templates, intersperse custom post types into the results, sorting by post date or alphabetically by title. • Attach filter to posts_where: if (query is for a category or tag) $where = ‘(‘ . $where . ‘)’ . “OR post_type=‘extras’” ; • This changes the database query….. WordCamp Philly 2010 24
    25. 25. © 2010 Hal Stern Some Rights Reserved Matching Tags • What if you want to match tags from user- selected posts to tagged CPT posts? • Need to get the list of tags from the main loop query • In template, create a new query looking for post_type=‘cpt type’ • Use JOIN to match collected tags to tags in the CPT (or other custom) taxonomy WordCamp Philly 2010 25
    26. 26. © 2010 Hal Stern Some Rights Reserved Why Do You Care? • User experience improvement > JOINS are expensive > Large post table & repetitive SELECTs = slow > Running query once keeps cache warm > Category, permalink, title slug choices matter • More CMS, less “blog” > Alphabetical sort > Adding taxonomy/social graph elements WordCamp Philly 2010 26
    27. 27. © 2010 Hal Stern Some Rights Reserved Resources • Core files where SQL stuff happens > query.php > post.php > canonical.php > rewrite.php • Template loader search path > http://codex.wordpress.org/Template_Hierarchy WordCamp Philly 2010 27
    28. 28. © 2010 Hal Stern Some Rights Reserved Contact (Shameless Plug) Hal Stern freeholdhal@gmail.com @freeholdhal snowmanonfire.com facebook.com/hal.stern slideshare.net/freeholdhal WordCamp Philly 2010 28

    ×