Barcelona DrupalCon An Update on Drupal Searching

(INSERT PICTURE OF BARCELONA) (maybe Gaudi House on Catalunya) By Doug Green [email_address] [email_address] An Update on Drupal Searching

Search in General Indexing Searching – in this talk User Interface – in this talk An Update on Drupal Searching

Search in General

Indexing

Searching – in this talk

User Interface – in this talk

Overview of this Talk An Update on Drupal Searching Start with the easy & fun stuff Views based UI improvements How does Core Search work How does Core Search SQL work Three patches to improve Core Search SQL VFS proof of concept Other Search Options – not in this talk SoC Fuzzy Search Semantec Search, Faceted Search

Overview of this Talk

Start with the easy & fun stuff

Views based UI improvements

How does Core Search work

How does Core Search SQL work

Three patches to improve Core Search SQL

VFS proof of concept

Other Search Options – not in this talk

SoC Fuzzy Search

Semantec Search, Faceted Search

Views based searching An Update on Drupal Searching Presented at OS/CMS modules - VFB, VFS, daterange demos filter block override search box views search plugin d.o #130714

Views based searching

Presented at OS/CMS

modules - VFB, VFS, daterange

demos

filter block

override search box

views search plugin

d.o #130714

Core Search Overview Need to understand to talk about performance and link referencing indexing UI node searches / user searches extending with Search Implementors credit An Update on Drupal Searching

Core Search Overview

Need to understand to talk about

performance and link referencing

indexing

UI

node searches / user searches

extending with Search Implementors

credit

Search Node Ranking Keyword Relevance / Node Linking Recently Posted (newness) Number of Comments An Update on Drupal Searching d.o #146466

Search Node Ranking

Keyword Relevance / Node Linking

Recently Posted (newness)

Number of Comments

d.o #146466

Introduction to Queries AND OR Exclude Phrase Advanced Tab An Update on Drupal Searching dogs cats dogs OR cat dogs or cats dogs -cats “ sheep dog” type:forum

Introduction to Queries

AND

OR

Exclude

Phrase

Advanced Tab

dogs cats

dogs OR cat

dogs or cats

dogs -cats

“ sheep dog”

type:forum

AND SQL SELECT sid from {search_index} WHERE (word = 'cat' OR word = 'dog') GROUP BY sid HAVING COUNT(*) >= 2 An Update on Drupal Searching SELECT sid from {search_dataset} WHERE (data LIKE '% cat %' AND data LIKE '% dog %') Why >= and why the second query?

AND SQL

SELECT sid from {search_index}

WHERE (word = 'cat' OR word = 'dog')

GROUP BY sid HAVING COUNT(*) >= 2

SELECT sid from {search_dataset}

WHERE (data LIKE '% cat %'

AND data LIKE '% dog %')

Why >= and why the second query?

OR SQL SELECT sid from {search_index} WHERE (word = 'cat' OR word = 'dog') GROUP BY sid HAVING COUNT(*) >= 1 An Update on Drupal Searching SELECT sid from {search_dataset} WHERE (data LIKE '% cat %' OR data LIKE '% dog %') Why the second query?

OR SQL

SELECT sid from {search_index}

WHERE (word = 'cat' OR word = 'dog')

GROUP BY sid HAVING COUNT(*) >= 1

SELECT sid from {search_dataset}

WHERE (data LIKE '% cat %'

OR data LIKE '% dog %')

Why the second query?

Exclude SQL same as previous (AND or OR depending) An Update on Drupal Searching SELECT sid from {search_dataset} WHERE (data LIKE '% cat %' AND data NOT LIKE '% dog %') Why the second query? Mysql 3 doesn't support Subqueries d.o #143888

Exclude SQL

same as previous (AND or OR depending)

SELECT sid from {search_dataset}

WHERE (data LIKE '% cat %'

AND data NOT LIKE '% dog %')

Why the second query?

Mysql 3 doesn't support Subqueries

d.o #143888

AND: SELECT sid from {search_index} WHERE (word = 'cat' OR word = 'dog') GROUP BY sid HAVING COUNT(*) = 2 OR: SELECT sid from {search_index} WHERE (word = 'cat' OR word = 'dog') GROUP BY sid One Solution – Unique Index An Update on Drupal Searching d.o #143160

AND: SELECT sid from {search_index}

WHERE (word = 'cat' OR word = 'dog')

GROUP BY sid HAVING COUNT(*) = 2

OR: SELECT sid from {search_index}

WHERE (word = 'cat' OR word = 'dog')

GROUP BY sid

One Solution – Unique Index

d.o #143160

Exclude: SELECT ... WHERE NOT IN (...) Yes, this is kinda slow But, I bet 99% of users never use it And, all remaining queries aren't penalized for something hardly used Second Solution – Subqueries An Update on Drupal Searching d.o #151910 and #143888 (views)

Exclude:

SELECT ... WHERE NOT IN (...)

Yes, this is kinda slow

But, I bet 99% of users never use it

And, all remaining queries aren't

penalized for something hardly used

Second Solution – Subqueries

d.o #151910 and #143888 (views)

Necessary to {search_index} AND queries Drastically smaller {search_index} table Added benefit {search_node_links} Third Solution – Node Link Tracker An Update on Drupal Searching d.o #146466

Necessary to {search_index} AND queries

Drastically smaller {search_index} table

Added benefit {search_node_links}

Third Solution – Node Link Tracker

d.o #146466

views_fastsearch Includes All of above Plus Views Argument handler for links My Playground - VFS An Update on Drupal Searching d.o/project/views_fastsearch

views_fastsearch

Includes All of above

Plus Views Argument handler for links

My Playground - VFS

d.o/project/views_fastsearch

http://www.civicactions.com/blog/search d.o #146466 d.o #143888 d.o #151910 d.o #143160 d.o #146466 g.d.o #4102 (Search group) More References An Update on Drupal Searching

http://www.civicactions.com/blog/search

d.o #146466

d.o #143888

d.o #151910

d.o #143160

d.o #146466

g.d.o #4102 (Search group)

More References

[email_address] views_fastsearch An Update on Drupal Searching