Using full text search in OpenERP. Holger Brunn, Therp

2,704 views
2,517 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,704
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
74
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Using full text search in OpenERP. Holger Brunn, Therp

  1. 1. Using full text search in OpenERP Holger Brunn Therp BV 03 july 2013
  2. 2. Intro PostgreSQL full text search capabilities OpenObject FTS Outline 1 Intro 2 PostgreSQL full text search capabilities 3 OpenObject FTS Holger Brunn Therp BV Using full text search in OpenERP
  3. 3. Intro PostgreSQL full text search capabilities OpenObject FTS Searching content of text fields is very slow for large amounts of data finds only fields containing the exact string you filled in restricted to just this string Full text search gives us fast searches lexeme level fuzzy search logical operators Holger Brunn Therp BV Using full text search in OpenERP
  4. 4. Intro PostgreSQL full text search capabilities OpenObject FTS Demo imported part of the project gutenberg database as documents 10.000 books, every one as its own document, resulting in a lot of text in different languages doing several searches there with standard OpenERP search makes you a serious coffee addict Holger Brunn Therp BV Using full text search in OpenERP
  5. 5. Intro PostgreSQL full text search capabilities OpenObject FTS Basics documents are the lump of text you are searching through tokens are parts of a document: words, but also i.e. urls, numbers lexemes are roughly the core of a word: ‘searching’, ‘search’, ‘searched’ are all derived from the lexeme SEARCH stop words are words that don’t make sense to store and are to be ignored, think of ‘the’ dictionaries define lexemes, synonyms and stop words of a language more info http://www.postgresql.org/docs/current/ static/textsearch.html Holger Brunn Therp BV Using full text search in OpenERP
  6. 6. Intro PostgreSQL full text search capabilities OpenObject FTS Basics II tsvector is the datatype postgres uses to represent a tokenized, lexemized text tsquery is the datatype to represent a query against a tsvector, with operators ‘&’ and ‘|’ or ‘!’ not ‘(’, ‘)’ brackets :* substring search @@ is the operator to match a tsquery against a tsvector Holger Brunn Therp BV Using full text search in OpenERP
  7. 7. Intro PostgreSQL full text search capabilities OpenObject FTS Examples select to tsvector(’english’, ’This is an english text. Pay attention how it is broken up’); select to tsvector(’french’, ’This is an english text. Pay attention how it is broken up’); select plainto tsquery(’english’, ’this’); select plainto tsquery(’french’, ’this’); Holger Brunn Therp BV Using full text search in OpenERP
  8. 8. Intro PostgreSQL full text search capabilities OpenObject FTS Untouched topics ranking highlighting proximity search parser tuning dictionary tuning configurations index tuning Holger Brunn Therp BV Using full text search in OpenERP
  9. 9. Intro PostgreSQL full text search capabilities OpenObject FTS Full text search support for OpenERP is available on launchpad: https://launchpad.net/openobject-fts is a collection of addons that enable full text search for several models takes care of creating appropriate indexes and triggers for underlying database tables and columns can be misused to provide a unified search over several models Holger Brunn Therp BV Using full text search in OpenERP
  10. 10. Intro PostgreSQL full text search capabilities OpenObject FTS Installation / Configuration on installation, a cronjob is created to create the initial index watch your logs, there is an info on when index creation starts and when it ends only then, this model is searchable via full text search this can take hours if you have enough data the initial configuration is ‘simple’, this won’t get you good results set the parameter fts base tsconfig to english, french, etc. to change that then go to Configuration / Fulltextsearch configuration to recreate the index Holger Brunn Therp BV Using full text search in OpenERP
  11. 11. Intro PostgreSQL full text search capabilities OpenObject FTS Hacking it depend on fts base inherit fts base set properties model and indexed column the class fts base is extensively documented, read that note that this are not ORM models to change the combined search view, inherit fts.proxy’s search view Holger Brunn Therp BV Using full text search in OpenERP
  12. 12. Intro PostgreSQL full text search capabilities OpenObject FTS Contributions welcome hook into search methods of full text indexed models to also use the full text search in their own searches hook into fts.proxy’s treeview and override edit/form actions to get rid of the awkward extra buttons currently used hack 7.0’s search widget to propose lexemes as you type dig deeper into caching mechanisms for smoother scrolling query available configurations from postgres and offer then in the confguration wizard Holger Brunn Therp BV Using full text search in OpenERP

×