SQLite has this extensibility feature called "virtual tables", which is cool, except that it has to be written in C code. Now the same can be done in Perl code :-).

1. 05.09.2014 - Page 1
Département
Office
Sqlite Virtual Tables
written in Perl
Swiss Perl Workshop 2014
laurent.dami@justice.ge.ch
Etat de Genève, Pouvoir Judiciaire
Département
Office

2. What is SQLite
• Relational Database Engine in a C software library
–  SQL support, including foreign keys constraints, transactions, etc.
– Also fulltext support
– Self-contained
• File-based
–  No server, no sockets, zero configuration
• Extensible
–  can add user-defined functions, aggregate functions, etc.
Great for tests
Great for embedded devices
 "the most widely deployed database engine in the world"
(smartphones, Firefox, etc.) http://www.sqlite.org

3. SQLite and Perl
• DBD::SQLite driver
– Contains the whole SQLite source (no external dependency)
– Regularly updated
• Usage :
use DBI;
my $dsn = "dbi:SQLite:dbname=$dbfile";
my $dbh = DBI->connect($dsn,"","", %options);

4. Tied variables in Perl
my %regular_hash
Perl Core
implementation
tie my %foo_hash, 'Tie::Hash::Foo';
Tie::Hash::Foo
+TIEHASH
+FETCH
+STORE
...

5. Virtual tables in SQLite
CREATE TABLE Foo(...)
SQLite core
implementation
CREATE VIRTUAL TABLE Bar
USING foobar (....)
Foobar.c
+xOpen
+xNext
+xEof
+xColumn
...
C code 

6. Virtual tables in DBD::Sqlite '1.43_08'
# register the virtual table module within sqlite
$dbh->sqlite_create_module(mod_name =>
"DBD::SQLite::VirtualTable::Subclass");
# create a virtual table
$dbh->do("CREATE VIRTUAL TABLE vtbl "
"USING mod_name(arg1, arg2, ...)")
Will be passed to
the CREATE()
method
# use it as any regular table
my $sth = $dbh->prepare("SELECT * FROM vtbl ...");

7. Implementing a virtual table
DBD::SQLite::VirtualTable DBD::SQLite::VirtualTable::Cursor
MyVTab MyVTab::Cursor
+CREATE
+BEST_INDEX
...
+FILTER
+NEXT
+EOF
+COLUMN
...

8. Builtin 1: VirtualTable::FileContent
CREATE VIRTUAL TABLE tbl USING fcontent(
source = src_table,
content_col = content,
path_col = path,
expose = "path, col1, col2, col3", --or "*"
root = "/foo/bar",
get_content = Foo::Bar::read_from_file
);
SELECT col1, path, content FROM tbl WHERE ...;
CREATE VIRTUAL TABLE fts USING fts4(
content="tbl", tokenize=unicode61);

9. FileContent + FTS4
Virtual (FTS4, C code) Virtual (FileContent, Perl)
C1 Content
C1 C2 Path
get_content()
C1 Content
Regular table
FTS4 internals
Filesystem

10. Builtin 2: VirtualTable::PerlData
CREATE VIRTUAL TABLE atbl USING perl(
foo, bar, etc,
arrayrefs="some::global::var::aref")
CREATE VIRTUAL TABLE htbl USING perl(
foo, bar, etc,
hashrefs="some::global::var::href")
CREATE VIRTUAL TABLE ctbl USING perl(
single_col,
colref="some::global::var::ref")

11. Hashrefs example
use Unicode::UCD 'charinfo';
our $chars = [map {charinfo($_)} 0x300..0x400];
CREATE VIRTUAL TABLE charinfo USING perl(
code, name, block, script, category,
hashrefs="main::chars");
SELECT * FROM charinfo
WHERE script='Greek'
AND name LIKE '%SIGMA%'

12. Colref example
# perl
our $values = @ARGV;
# SQL
CREATE VIRTUAL TABLE iarray USING perl(i INT,
colref="main::values");
SELECT * FROM some_table WHERE some_col IN iarray;
# BEWARE!
INSERT INTO iarray VALUES (99); # pushes in @ARGV

13. Go and try
• https://metacpan.org/source/ISHIGAKI/DBD-SQLite-
1.43_08
• https://github.com/DBD-SQLite/DBD-SQLite