The document discusses SQLite, a relational database engine, highlighting its features such as being file-based, extensible, and the most widely deployed database engine. It also details the integration and usage of SQLite with Perl through the dbd::sqlite driver, including the creation and implementation of virtual tables within SQLite. Various examples illustrate the creation of virtual tables using Perl and the manipulation of data using SQL queries.

1. 05.09.2014 - Page 1
Département
Office
Sqlite Virtual Tables
written in Perl
Swiss Perl Workshop 2014
laurent.dami@justice.ge.ch
Etat de Genève, Pouvoir Judiciaire
Département
Office

2. What is SQLite
• Relational Database Engine in a C software library
–  SQL support, including foreign keys constraints, transactions, etc.
– Also fulltext support
– Self-contained
• File-based
–  No server, no sockets, zero configuration
• Extensible
–  can add user-defined functions, aggregate functions, etc.
Great for tests
Great for embedded devices
 "the most widely deployed database engine in the world"
(smartphones, Firefox, etc.) http://www.sqlite.org

3. SQLite and Perl
• DBD::SQLite driver
– Contains the whole SQLite source (no external dependency)
– Regularly updated
• Usage :
use DBI;
my $dsn = "dbi:SQLite:dbname=$dbfile";
my $dbh = DBI->connect($dsn,"","", %options);

4. Tied variables in Perl
my %regular_hash
Perl Core
implementation
tie my %foo_hash, 'Tie::Hash::Foo';
Tie::Hash::Foo
+TIEHASH
+FETCH
+STORE
...

5. Virtual tables in SQLite
CREATE TABLE Foo(...)
SQLite core
implementation
CREATE VIRTUAL TABLE Bar
USING foobar (....)
Foobar.c
+xOpen
+xNext
+xEof
+xColumn
...
C code 

6. Virtual tables in DBD::Sqlite '1.43_08'
# register the virtual table module within sqlite
$dbh->sqlite_create_module(mod_name =>
"DBD::SQLite::VirtualTable::Subclass");
# create a virtual table
$dbh->do("CREATE VIRTUAL TABLE vtbl "
"USING mod_name(arg1, arg2, ...)")
Will be passed to
the CREATE()
method
# use it as any regular table
my $sth = $dbh->prepare("SELECT * FROM vtbl ...");

7. Implementing a virtual table
DBD::SQLite::VirtualTable DBD::SQLite::VirtualTable::Cursor
MyVTab MyVTab::Cursor
+CREATE
+BEST_INDEX
...
+FILTER
+NEXT
+EOF
+COLUMN
...

8. Builtin 1: VirtualTable::FileContent
CREATE VIRTUAL TABLE tbl USING fcontent(
source = src_table,
content_col = content,
path_col = path,
expose = "path, col1, col2, col3", --or "*"
root = "/foo/bar",
get_content = Foo::Bar::read_from_file
);
SELECT col1, path, content FROM tbl WHERE ...;
CREATE VIRTUAL TABLE fts USING fts4(
content="tbl", tokenize=unicode61);

9. FileContent + FTS4
Virtual (FTS4, C code) Virtual (FileContent, Perl)
C1 Content
C1 C2 Path
get_content()
C1 Content
Regular table
FTS4 internals
Filesystem

10. Builtin 2: VirtualTable::PerlData
CREATE VIRTUAL TABLE atbl USING perl(
foo, bar, etc,
arrayrefs="some::global::var::aref")
CREATE VIRTUAL TABLE htbl USING perl(
foo, bar, etc,
hashrefs="some::global::var::href")
CREATE VIRTUAL TABLE ctbl USING perl(
single_col,
colref="some::global::var::ref")

11. Hashrefs example
use Unicode::UCD 'charinfo';
our $chars = [map {charinfo($_)} 0x300..0x400];
CREATE VIRTUAL TABLE charinfo USING perl(
code, name, block, script, category,
hashrefs="main::chars");
SELECT * FROM charinfo
WHERE script='Greek'
AND name LIKE '%SIGMA%'

12. Colref example
# perl
our $values = @ARGV;
# SQL
CREATE VIRTUAL TABLE iarray USING perl(i INT,
colref="main::values");
SELECT * FROM some_table WHERE some_col IN iarray;
# BEWARE!
INSERT INTO iarray VALUES (99); # pushes in @ARGV

13. Go and try
• https://metacpan.org/source/ISHIGAKI/DBD-SQLite-
1.43_08
• https://github.com/DBD-SQLite/DBD-SQLite