MariaDB with SphinxSE          Colin Charles, Monty Program Ab             colin@montyprogram.comhttp://www.montyprogram.c...
whoami• MariaDB guy at Monty Program• Formerly at MySQL AB/Sun Microsystems• Past lives include FESCO (Fedora Project),  O...
MariaDB/MySQL usedinterchangeably in this talk
The old days• Download MySQL, including sources• Download SphinxSE for compiling• Download Sphinx to compile with MySQL  s...
Today• Install sphinx from your distribution• Install MariaDB 5.5 from your distribution  or from http://mariadb.org/• Get...
Getting startedmysql> INSTALL PLUGIN sphinxSONAME ha_sphinx.so;Query OK, 0 rows affected(0.01 sec)
Another engine appears
What is SphinxSE?• SphinxSE is just the storage engine that still  depends on the Sphinx daemon• It doesn’t store any data...
Configure sphinx!• /usr/local/sphinx/sphinx.conf• Source (multiple, include mysql, with  connection info)• Setup indexer (e...
Use case scenarios• Already have an existing application that  makes use of full-text-search in MyISAM?  Porting should be...
Use case scenarios• Results from Sphinx itself almost always  require additional work involving MariaDB  • Say to pull out...
An exampleCREATE TABLE t1(    id             INTEGER UNSIGNED NOT NULL,    weight         INTEGER NOT NULL,    query      ...
Sphinx search tables• 1st column: INTEGER UNSIGNED or  BIGINT (document ID)• 2nd column: match weight• 3rd column: VARCHAR...
What actually happens• SELECT passes a Sphinx query as the query  column in the WHERE clause• searchd returns the results•...
SHOW ENGINE             SPHINX STATUS•   Per-query & per-word statistics that searchd returns are accessible via SHOW STAT...
What queries are          supported?•   Most of the Sphinx API is exposed to SphinxSE•   query, mode, sort, offset, limit,...
Efficiency•   Allow Sphinx to perform sorting, filtering, and    slicing of result set    •   ... as opposed to using WHERE,...
JOINs•   Perform JOINs on a SphinxSE search table using tables from other engines    SELECT content, date_added FROM test....
Why MariaDB?• We keep up to date with Sphinx releases• In MariaDB 5.5.21 we upgraded to 2.0.4,  the latest upstream releas...
Why MariaDB II?• Engineering and furthering MySQL happens  with MariaDB• Benefit from a better-built in optimizer  (that ca...
Warning• If sphinx is itself not setup, SphinxSE will  accept doing things like CREATE TABLE• Try doing a SELECT and you’l...
We have extensive    documentation• http://kb.askmonty.org/en/sphinx-storage-  engine• http://sphinxsearch.com/docs/1.10/ ...
Q&A?       email: colin@montyprogram.comhttp://montyprogram.com/ | http://mariadb.org/twitter: @bytebot / url: http://byte...
Upcoming SlideShare
Loading in …5
×

MariaDB with SphinxSE

5,163 views
4,901 views

Published on

Published in: Technology
0 Comments
13 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,163
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
31
Comments
0
Likes
13
Embeds 0
No embeds

No notes for slide

MariaDB with SphinxSE

  1. 1. MariaDB with SphinxSE Colin Charles, Monty Program Ab colin@montyprogram.comhttp://www.montyprogram.com / http://mariadb.org http://bytebot.net/blog / @bytebot on Twitter Sphinx Search Day 2012, Santa Clara, CA, USA 13 April 2012
  2. 2. whoami• MariaDB guy at Monty Program• Formerly at MySQL AB/Sun Microsystems• Past lives include FESCO (Fedora Project), OpenOffice.org
  3. 3. MariaDB/MySQL usedinterchangeably in this talk
  4. 4. The old days• Download MySQL, including sources• Download SphinxSE for compiling• Download Sphinx to compile with MySQL support• Documented: http://www.howtoforge.com/ sphinx-as-mysql-storage-engine-sphinxse
  5. 5. Today• Install sphinx from your distribution• Install MariaDB 5.5 from your distribution or from http://mariadb.org/• Get started!
  6. 6. Getting startedmysql> INSTALL PLUGIN sphinxSONAME ha_sphinx.so;Query OK, 0 rows affected(0.01 sec)
  7. 7. Another engine appears
  8. 8. What is SphinxSE?• SphinxSE is just the storage engine that still depends on the Sphinx daemon• It doesn’t store any data itself• Its just a built-in client to allow MariaDB to talk to Sphinx searchd, run queries, obtain results• Indexing, searching is performed on Sphinx
  9. 9. Configure sphinx!• /usr/local/sphinx/sphinx.conf• Source (multiple, include mysql, with connection info)• Setup indexer (esp. if its on localhost) - mem_limit, max_iops, max_iosize• Setup searchd (where to listen to, query log, etc.)
  10. 10. Use case scenarios• Already have an existing application that makes use of full-text-search in MyISAM? Porting should be easier• Have a programming language without a native API for Sphinx? Surely there’s a connector for MariaDB ;-)
  11. 11. Use case scenarios• Results from Sphinx itself almost always require additional work involving MariaDB • Say to pull out text column that Sphinx index doesn’t store • JOIN with another table (using a different engine)
  12. 12. An exampleCREATE TABLE t1( id INTEGER UNSIGNED NOT NULL, weight INTEGER NOT NULL, query VARCHAR(3072) NOT NULL, group_id INTEGER, INDEX(query)) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/test";SELECT * FROM t1 WHERE query=test it;mode=any;
  13. 13. Sphinx search tables• 1st column: INTEGER UNSIGNED or BIGINT (document ID)• 2nd column: match weight• 3rd column: VARCHAR or TEXT (your query)• Query column needs indexing, no other column needs to be
  14. 14. What actually happens• SELECT passes a Sphinx query as the query column in the WHERE clause• searchd returns the results• SphinxSE translates and returns the results to MariaDB
  15. 15. SHOW ENGINE SPHINX STATUS• Per-query & per-word statistics that searchd returns are accessible via SHOW STATUS mysql> SHOW ENGINE SPHINX STATUS; +--------+-------+-------------------------------------------------+ | Type | Name | Status | +--------+-------+-------------------------------------------------+ | SPHINX | stats | total: 25, total found: 25, time: 126, words: 2 | | SPHINX | words | sphinx:591:1256 soft:11076:15945 | +--------+-------+-------------------------------------------------+ 2 rows in set (0.00 sec)
  16. 16. What queries are supported?• Most of the Sphinx API is exposed to SphinxSE• query, mode, sort, offset, limit, index, minid, maxid, weights, filter, !filter, range, !range, maxmatches, groupby, groupsort, indexweights, comment, select• Sphinx search modes can also be supported via _sph attributes • obtain value of @groupby? use ‘_sph_groupby’
  17. 17. Efficiency• Allow Sphinx to perform sorting, filtering, and slicing of result set • ... as opposed to using WHERE, ORDER BY, LIMIT clauses on MariaDB• Why? • Sphinx optimises and performs better on these tasks • Less data packed by searchd, and transferred and unpacked by SphinxSE
  18. 18. JOINs• Perform JOINs on a SphinxSE search table using tables from other engines SELECT content, date_added FROM test.documents docs -> JOIN t1 ON (docs.id=t1.id) -> WHERE query="one document;mode=any"; +-------------------------------------+---------------------+ | content | docdate | +-------------------------------------+---------------------+ | this is my test document number two | 2006-06-17 14:04:28 | | this is my test document number one | 2006-06-17 14:04:28 | +-------------------------------------+---------------------+ 2 rows in set (0.00 sec)
  19. 19. Why MariaDB?• We keep up to date with Sphinx releases• In MariaDB 5.5.21 we upgraded to 2.0.4, the latest upstream release• MariaDB 5.5.23 is GA and ready for use today
  20. 20. Why MariaDB II?• Engineering and furthering MySQL happens with MariaDB• Benefit from a better-built in optimizer (that can materialize subqueries), XtraDB, microsecond precision, more statistics, NoSQL-like features (dynamic columns), GIS functionality (which works for geo- distance type searches in Sphinx)
  21. 21. Warning• If sphinx is itself not setup, SphinxSE will accept doing things like CREATE TABLE• Try doing a SELECT and you’ll see it fail though
  22. 22. We have extensive documentation• http://kb.askmonty.org/en/sphinx-storage- engine• http://sphinxsearch.com/docs/1.10/ sphinxse-using.html• Introduction to Search with Sphinx by Andrew Aksyonoff (O’Reilly)
  23. 23. Q&A? email: colin@montyprogram.comhttp://montyprogram.com/ | http://mariadb.org/twitter: @bytebot / url: http://bytebot.net/blog/

×