HandlerSocket - A NoSQL plugin for MySQL
Upcoming SlideShare
Loading in...5
×
 

HandlerSocket - A NoSQL plugin for MySQL

on

  • 6,631 views

Presented on osdc.tw, 2011

Presented on osdc.tw, 2011

Statistics

Views

Total Views
6,631
Views on SlideShare
5,992
Embed Views
639

Actions

Likes
11
Downloads
111
Comments
0

6 Embeds 639

http://jnlin.org 365
http://infuture.pixnet.net 252
http://www.scoop.it 13
http://webcache.googleusercontent.com 6
http://b.hatena.ne.jp 2
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

HandlerSocket - A NoSQL plugin for MySQL HandlerSocket - A NoSQL plugin for MySQL Presentation Transcript

  • HANDLERSOCKET - A NOSQL PLUGIN FOR MYSQL Jui-Nan Lin
  • About this talk
    • URL of this Slide:
      • http://0rz.tw/jqQqY
    • QRCode:
  • What is HandlerSocket?
    • “ A NoSQL interface for MySQL”
    • Only supports queries which use index
    • Developed by DeNA (a Japanese social gaming provider)
    • https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL
  • Architecture libmysql libhsclient Application (Ex. PHP, Ruby, Perl …) Listener for libmysql SQL Layer Handler Interface InnoDB MyISAM Other storage engines HandlerSocket Plugin application mysqld Reference: http://www.slideshare.net/akirahiguchi/handlersocket-plugin-for-mysql-4664154
  • NoSQL for Web Application
    • Most operations in web application are simple CRUD.
    • RDBMS is too powerful for web application.
    • Typical solution:
      • Memcached + MySQL
      • Cassandra
      • Apache CouchDB
      • Amazon SimpleDB
  • NoSQL for Web Application
    • Pros:
      • Scalable
      • Fast
    • Cons:
      • Unstable
        • Twitter drops Cassandra in 2010: http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html
  • HandlerSocket for MySQL
    • MySQL is very stable
    • HandlerSocket provides a NoSQL interface for MySQL
    • HandlerSocket is very fast. Faster than MySQL.
  • How fast is it
    • 750,000 qps on a commodity server (tested by original developers)
      • MySQL 5.1 + E5540 2.53Ghz + 32 GB RAM
      • All data fits in memory
      • All Clients are written in C
      • 100,000 qps with Original MySQL
    • About 7.5x faster than original MySQL!
  • How fast is it
    • 1m PK SELECT queries each test on 1 table contains 1m rows. Client is written in PHP.
      • SELECT * from table1 WHERE pk = ?
    Concurrent Connections 1 10 50 100 HandlerSocket Time (sec) 256 28 15 13 QPS 3906 35714 66667 76923 MySQL Time (sec) 372 83 87 85 QPS 2688 12048 11494 11765
  • Why HandlerSocket is fast from http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html
  • Why HandlerSocket is fast
    • HandlerSocket do not need to do:
      • Parsing SQL Statements
      • Opening, locking table
      • Making SQL execution plans
        • Index usage
      • Unlocking, closing tables
  • Why HandlerSocket is fast
    • HandlerSocket does do:
      • Plain text-based protocol
      • Keep table open for reuse
      • Event based (use epoll)
        • Many Connection per MySQL thread
      • Accept only index queries
        • Support range query, LIMIT, multi_get operations
  • Language Bindings
    • PHP: php-handlersocket
    • Java: hs4j
    • Python: python-handler-socket
    • Ruby: ruby-handlersocket
    • Perl: Net::HanderSocket
  • Programming Example
    • PHP
    • Read Operation
    • Port used for write operation is different
  • Configuration (my.cnf)
    • handlersocket_port = 9998
      • Port for reading
    • handlersocket_port_wr = 9999
      • Port for writing
    • handlersocket_threads = 16
      • No. of reader threads
      • Recommended Value: # of CPUs
    • handlersocket_threads_wr = 1
      • No. of writer threads
    • open_files_limit = 65535
  • Pros and Cons
    • Pros
      • Stable
      • High Performance
      • Support a lot of query types
      • No duplicate cache (compared with MySQL + memcached)
        • Invalidation Problem
      • Flexible
        • Support SQL interface. We can still use SQL statements! (ex. report generating)
        • Write binlog and support MySQL replication.
      • Included in Percona Server
  • Pros and Cons
    • Cons
      • No Security
        • Without username and password
      • Do not update MySQL Query Cache
        • Fixed in recent commits
      • No “INSERT … ON DUPLICATED KEY UPDATE …” Feature
      • Do not support auto increment column while inserting
        • Fixed in recent commits
  • Q&A
    • Thank you!