HandlerSocket

HANDLERSOCKET
A NOSQL APPROACH TO MYSQL

HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
ABOUT ME
Lukasz Barulski
• backend dev AKA IAmDoingEverythingExceptFrontend dev
• been with DocPlanner since dinosaurs
• kicks butts in MK X
/lbarulski /in/lukaszbarulski

FETCHING DATA - THE MYSQL WAY

SAMPLE QUERIES
— OR —

SO, WHAT’S WRONG WITH THAT?

BUT…LET’S TEST THE SPEED
• PHP 7.0.4 • MariaDB 10.0.22
500 000 records with random data
+

SQL WAY | SPEED TEST
$resultsInMs = []; 
$statement = $pdo->prepare('SELECT value FROM t WHERE id=:id LIMIT 1'); 
for ($i = 1; $i <= 500000; ++$i) 
{ 
$before = microtime(true);
 
$statement->execute(['id' => $i]); 
$data = $statement->fetchColumn();
 
$after = microtime(true); 
$resultsInMs[] = ($after-$before)*1000; 
} 
$resultInMs = array_sum($resultsInMs)/count($resultsInMs); 
echo round($resultInMs, 3) . ' ms.' . PHP_EOL; 
echo round(stats_standard_deviation($resultsInMs, true), 3) . ' ms.' .
PHP_EOL;

SQL WAY | TEST RESULTS
Average time: 0.131 ms.
Standard deviation: 0.012 ms.

SQL WAY… WITHOUT STEROIDS| SPEED TEST
for ($i = 1; $i <= 500000; ++$i) 
{ 
$before = microtime(true); 
 
$data = $pdo 
->query('SELECT value FROM t WHERE id='.$i.' LIMIT 1') 
->fetchColumn(); 
 
} 
$resultInMs = array_sum($resultsInMs)/count($resultsInMs); 
PHP_EOL;

SQL WAY… WITHOUT STEROIDS| TEST RESULTS

SQL WAY | EXAMPLES
SQL Prepared Statements AVG: 0.131 ms.
SQL AVG: 0.257 ms.
Wordpress 4.4.2 homepage: 21 SELECT
21 x 0.257 ms. = 5.397 ms.
Where that (sometimes) matters?
• API
• High frequency operations
• Complicated (long running) reports
• Microservices

SIMPLE SOLUTION?

SQL WAY | SOLUTION OR A HALF-MEASURE?
• Warmup
• Invalidation
Problems:

HANDLERSOCKET TO THE RESCUE!

HANDLERSOCKET | THEORY
•MySQL based SQL servers plugin
•Text-like binary protocol - not SQL queries
•TCP connection
•Single thread for modifying data
•Multiple threads for reading data
•Coexists with standard way of using MySQL

HANDLERSOCKET WAY | SPEED TEST
$indexId = $hs->getIndexId('test', 't', '', 'id,value'); 
for ($i = 1; $i <= 500000; ++$i) 
{ 
$before = microtime(true); 
 
$hs->select($indexId, '=', [$i]); 
$data = $hs->readResponse(); 
 
} 
$resultInMs = array_sum($resultsInMs)/(float)count($resultsInMs); 
PHP_EOL;

HANDLERSOCKET WAY | TEST RESULTS

SQL VS. HANDLERSOCKET

SQL VS. HANDLERSOCKET | AVG(FETCHING TIME) BOXING NIGHT
SQL
SQL PS
HS
0 0,065 0,13 0,195 0,26
0,073MS
0,131MS
0,257MS

SQL VS. HANDLERSOCKET | STDEV(FETCHING TIME) BOXING NIGHT
SQL
SQL PS
HS
0 0,02 0,04 0,06 0,08
0,009MS
0,012MS
0,074MS

OK, BUT WHY IS IT FASTER?

SQL VS. HANDLERSOCKET | LEO, WHY?
Few steps less
Few bytes less
• No query parsing
• No query analysis
• No query execution plan
• ~40% smaller request size
• ~40% smaller response size
Less CPU usage
Less network traffic
Based on our examples

COMMUNICATION

COMMUNICATION | THEORY
• Port 9998
• Port 9999
• Read only
• Multi threaded
• Write allowed
• Single threaded
• One-line request
• One-line response
• Many requests on single connection

PROTOCOL

PROTOCOL BASICS | THEORY
• Text-like binary protocol
• Message delimiter -> n, 0x0a
• Column delimiter -> t, 0x09
• Null -> 0, 0x00
• Empty string -> tt, tn
• Characters [0x00 - 0x0f] -> prefixed by 0x01 and shifted by 0x40

OPEN INDEX | THEORY
P <index_id> <db> <table> <index> <columns> [<fcolumns>]
<index_id>
Opened index identifier, integer, opened until the client connection
is closed
<db> Database name
<table> Table name
<index> Index name or "PRIMARY" to use primary key
<columns> List of columns to return in response, delimited by ","
[<fcolumns>] Optional argument, list of columns to filter result on

FIND DATA | THEORY
<index_id> <op> <vlen> <v1> … <vn> [LIM] [IN] [FILTER …]
<index_id> Opened index identifier
<op> Comparison operation to use: =, >, >=, <, <=
<vlen> Number parameters to compare, equal or less than number of
columns in opened index
<v1> Value to compare using <op> with corresponding column from
index
[LIM] <limit> <offset> default values: limit=1, offset=0
[IN] @ <icol> <ivlen> <iv1> ... <ivn>
[FILTER …] <ftyp> <fop> <fcol> <fval>

RESPONSE | THEORY
<errorcode> <numcolumns> <r1> ... <rn>
<errorcode>
Request has successfully executed or not.
'0' means success. Non-zero means an error
<numcolumns> Number of columns of the result set
<r1> … <rn> Result set, n equals <numcolumns>

RESPONSE | THEORY
„If <errorcode> is non-zero, <numcolumns>
is always 1 and <r1> indicates a
human-readable error message, though
sometimes <r1> is not provided.”

SQL TO HANDLERSOCKET | THEORY
SELECT value FROM t WHERE id=1 LIMIT 1
P 1 test t PRIMARY value
= 0 1
1 = 1 1 0 1 600a5841de

SUMMARY

PROS | SUMMARY
• Stable
• High performance
• Data consistency
• Replication support
• Not dependent on data storage engine
• Shipped with MariaDB, Percona Server
• Still allows to use SQL client/server protocol

CONS | SUMMARY
• Filtering only using indexes
• Non-standard protocol
• Very simple authentication (password per port)
• Lack of support for transactions
• Small community
• Rarely used on production

KNOWN ISSUES | SUMMARY
• Persistent connections
• Too many indexes (> 1024?)
• Concurrent structure changes via SQL and data
modification via HandlerSocket

Q’N’A
Questions?

THANK YOU!
docplanner.com/career
Join us!

HandlerSocket

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to HandlerSocket

Similar to HandlerSocket (20)

Recently uploaded

Recently uploaded (20)

HandlerSocket