Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sqlite virtual-tables written in Perl


Published on

SQLite has this extensibility feature called "virtual tables", which is cool, except that it has to be written in C code. Now the same can be done in Perl code :-).

Published in: Software
  • Be the first to comment

  • Be the first to like this

Sqlite virtual-tables written in Perl

  1. 1. 05.09.2014 - Page 1 Département Office Sqlite Virtual Tables written in Perl Swiss Perl Workshop 2014 Etat de Genève, Pouvoir Judiciaire Département Office
  2. 2. What is SQLite • Relational Database Engine in a C software library –  SQL support, including foreign keys constraints, transactions, etc. – Also fulltext support – Self-contained • File-based –  No server, no sockets, zero configuration • Extensible –  can add user-defined functions, aggregate functions, etc. Great for tests Great for embedded devices  "the most widely deployed database engine in the world" (smartphones, Firefox, etc.)
  3. 3. SQLite and Perl • DBD::SQLite driver – Contains the whole SQLite source (no external dependency) – Regularly updated • Usage : use DBI; my $dsn = "dbi:SQLite:dbname=$dbfile"; my $dbh = DBI->connect($dsn,"","", %options);
  4. 4. Tied variables in Perl my %regular_hash Perl Core implementation tie my %foo_hash, 'Tie::Hash::Foo'; Tie::Hash::Foo +TIEHASH +FETCH +STORE ...
  5. 5. Virtual tables in SQLite CREATE TABLE Foo(...) SQLite core implementation CREATE VIRTUAL TABLE Bar USING foobar (....) Foobar.c +xOpen +xNext +xEof +xColumn ... C code 
  6. 6. Virtual tables in DBD::Sqlite '1.43_08' # register the virtual table module within sqlite $dbh->sqlite_create_module(mod_name => "DBD::SQLite::VirtualTable::Subclass"); # create a virtual table $dbh->do("CREATE VIRTUAL TABLE vtbl " "USING mod_name(arg1, arg2, ...)") Will be passed to the CREATE() method # use it as any regular table my $sth = $dbh->prepare("SELECT * FROM vtbl ...");
  7. 7. Implementing a virtual table DBD::SQLite::VirtualTable DBD::SQLite::VirtualTable::Cursor MyVTab MyVTab::Cursor +CREATE +BEST_INDEX ... +FILTER +NEXT +EOF +COLUMN ...
  8. 8. Builtin 1: VirtualTable::FileContent CREATE VIRTUAL TABLE tbl USING fcontent( source = src_table, content_col = content, path_col = path, expose = "path, col1, col2, col3", --or "*" root = "/foo/bar", get_content = Foo::Bar::read_from_file ); SELECT col1, path, content FROM tbl WHERE ...; CREATE VIRTUAL TABLE fts USING fts4( content="tbl", tokenize=unicode61);
  9. 9. FileContent + FTS4 Virtual (FTS4, C code) Virtual (FileContent, Perl) C1 Content C1 C2 Path get_content() C1 Content Regular table FTS4 internals Filesystem
  10. 10. Builtin 2: VirtualTable::PerlData CREATE VIRTUAL TABLE atbl USING perl( foo, bar, etc, arrayrefs="some::global::var::aref") CREATE VIRTUAL TABLE htbl USING perl( foo, bar, etc, hashrefs="some::global::var::href") CREATE VIRTUAL TABLE ctbl USING perl( single_col, colref="some::global::var::ref")
  11. 11. Hashrefs example use Unicode::UCD 'charinfo'; our $chars = [map {charinfo($_)} 0x300..0x400]; CREATE VIRTUAL TABLE charinfo USING perl( code, name, block, script, category, hashrefs="main::chars"); SELECT * FROM charinfo WHERE script='Greek' AND name LIKE '%SIGMA%'
  12. 12. Colref example # perl our $values = @ARGV; # SQL CREATE VIRTUAL TABLE iarray USING perl(i INT, colref="main::values"); SELECT * FROM some_table WHERE some_col IN iarray; # BEWARE! INSERT INTO iarray VALUES (99); # pushes in @ARGV
  13. 13. Go and try • 1.43_08 •