HandlerSocket - A NoSQL plugin for MySQL

HANDLERSOCKET - A NOSQL PLUGIN FOR MYSQL Jui-Nan Lin

About this talk
URL of this Slide:
http://0rz.tw/jqQqY
QRCode:

What is HandlerSocket?
“ A NoSQL interface for MySQL”
Only supports queries which use index
Developed by DeNA (a Japanese social gaming provider)
https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL

Architecture libmysql libhsclient Application (Ex. PHP, Ruby, Perl …) Listener for libmysql SQL Layer Handler Interface InnoDB MyISAM Other storage engines HandlerSocket Plugin application mysqld Reference: http://www.slideshare.net/akirahiguchi/handlersocket-plugin-for-mysql-4664154

NoSQL for Web Application
Most operations in web application are simple CRUD.
RDBMS is too powerful for web application.
Typical solution:
Memcached + MySQL
Cassandra
Apache CouchDB
Amazon SimpleDB

NoSQL for Web Application
Pros:
Scalable
Fast
Cons:
Unstable
Twitter drops Cassandra in 2010: http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html

HandlerSocket for MySQL
MySQL is very stable
HandlerSocket provides a NoSQL interface for MySQL
HandlerSocket is very fast. Faster than MySQL.

How fast is it
750,000 qps on a commodity server (tested by original developers)
MySQL 5.1 + E5540 2.53Ghz + 32 GB RAM
All data fits in memory
All Clients are written in C
100,000 qps with Original MySQL
About 7.5x faster than original MySQL!

How fast is it
1m PK SELECT queries each test on 1 table contains 1m rows. Client is written in PHP.
SELECT * from table1 WHERE pk = ?
Concurrent Connections 1 10 50 100 HandlerSocket Time (sec) 256 28 15 13 QPS 3906 35714 66667 76923 MySQL Time (sec) 372 83 87 85 QPS 2688 12048 11494 11765

Why HandlerSocket is fast from http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html

Why HandlerSocket is fast
HandlerSocket do not need to do:
Parsing SQL Statements
Opening, locking table
Making SQL execution plans
Index usage
Unlocking, closing tables

Why HandlerSocket is fast
HandlerSocket does do:
Plain text-based protocol
Keep table open for reuse
Event based (use epoll)
Many Connection per MySQL thread
Accept only index queries
Support range query, LIMIT, multi_get operations

Language Bindings
PHP: php-handlersocket
Java: hs4j
Python: python-handler-socket
Ruby: ruby-handlersocket
Perl: Net::HanderSocket

Programming Example
PHP
Read Operation
Port used for write operation is different

Configuration (my.cnf)
handlersocket_port = 9998
Port for reading
handlersocket_port_wr = 9999
Port for writing
handlersocket_threads = 16
No. of reader threads
Recommended Value: # of CPUs
handlersocket_threads_wr = 1
No. of writer threads
open_files_limit = 65535

Pros and Cons
Pros
Stable
High Performance
Support a lot of query types
No duplicate cache (compared with MySQL + memcached)
Invalidation Problem
Flexible
Support SQL interface. We can still use SQL statements! (ex. report generating)
Write binlog and support MySQL replication.
Included in Percona Server

Pros and Cons
Cons
No Security
Without username and password
Do not update MySQL Query Cache
Fixed in recent commits
No “INSERT … ON DUPLICATED KEY UPDATE …” Feature
Do not support auto increment column while inserting
Fixed in recent commits

Q&A
Thank you!

http://jnlin.org	337
http://infuture.pixnet.net	206
http://webcache.googleusercontent.com	6
http://b.hatena.ne.jp	2
http://translate.googleusercontent.com	1

HandlerSocket - A NoSQL plugin for MySQL

by Jui-Nan Lin

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 552

Statistics