                CTO2011 7 16         2
 Palmscape       1997               Xiino       Palm OS                        (   )             IBM WorkPad, Sony CLI...
(2)	                :        2005 8              2010 11        Japanize             Web UI                    	      ...
(3)	                :        2010 11                             IT        2011 5              CTO 	2011 7 16         ...
              DeNA                       OSS   DeNA                      OSS               	2011 7 16                 ...
2011 7 16        7
Q4M	2011 7 16           8
What is Q4M?	   A message queue       runs as a storage plugin of MySQL 5.1   Why is it a MySQL plugin?       accessib...
Design Goals of Q4M	   Robust       Does not lose data on OS crash or power failure             necessary for Tokyo wo....
Users of Q4M	   Many leading web services in Japan       DeNA Co., Ltd.       livedoor Co., Ltd.       mixi, Inc.     ...
What is a Message Queue?	2011 7 16                                12
What is a Message Queue?	   Middleware for persistent asynchronous    communication       communicate between fixed pair...
Minimal Configuration of a Message Queue	   Senders and receivers access a single    queue	             Sender           ...
Using MQ as a Relay	   Separate queue for sender and receiver   Messages relayed between queues	                        ...
Merits of using Message Relays	   Destination can be changed easily       Relays may transfer messages to different     ...
Message Brokers	   Publish / subscribe model       Separation between components and their        integration       Com...
What about Q4M?	   Q4M itself is a message queue   Can connect Q4M instances to create a    message relay   Provides AP...
Performance of Q4M	   over 7,000 mess/sec.       message size: avg. 512 bytes       syncing to disk   Outperforming mo...
Q4M in DeNA	   uses Q4M for sending notifications    asynchronously                    http://engineer.dena.jp/2010/03/de...
Q4M in DeNA	   Q4M        2,000   /                    DeNA           2011 7 16                   21
HandlerSocket Plugin	2011 7 16                            22
Memcached                       	           RDBMS                  /                   RDBMS                       ...
Memcached             	   RDBMS                                   +                        :                       ...
SQL            (   )	2011 7 16               25
Mycached	   2009 8   mysqld       memcached                      memcached   get                            get       ...
Mycached –                  	                                                                             mycached	       ...
Mycached –           	   1,200   JSON, msgpack,   Picoev               (I/O           )       MySQL                   ...
Mycached –           	   SQL      40,000qps        	2011 7 16                          29
Mycached   	           2011 7 16           30
HandlerSocket Plugin          	   DeNA       2010 8              MySQL Contributor Award 2011   MySQL           SQL  ...
…   	2011 7 16            32
…2011 7 16       33
…            …	2011 7 16         34
…	            …	            …	2011 7 16         35
2011 7 16        36
                                    2011 7 16        37
HandlerSocket Plugin   	   mycached                   FastEther   2011 7 16                                  38
2011 7 16        39
2011 7 16        40
–   	                                      ?	2011 7 16                  41
–        	           Server::Starter –        Parallel::Prefork –        Starlet –                  PSGI2011 7 16     ...
–                       	           fork                  : mod_bumpy_life (Apache)        by hirose31++        MaxRe...
–              	               MaxRequestsPerChild                     fork            …	2011 7 16                      ...
–            : Slow Restart	                                                   fork                       	             ...
–          OSS      	              Perl               github            CPAN                             …	2011 7 16   ...
 Q.                         ?  $SIG{HUP} = sub { $got_sig = 1 };  …  while (! $got_sig) {    sleep(1000); //  }2011 7 16 ...
 Q.                         ?  $SIG{HUP} = sub { $got_sig = 1 };  …  while (! $got_sig) {    //                          ...
                              Parallel::Prefork                                 …                :                    ...
OSS   DeNA   	2011 7 16                 50
 Perl             …        2009              Plack             Perl       WSGI or Rack        Furl (                  ...
                                 MySQL        YAPC        MySQL Conference                         : MySQL Casual  ...
2011 7 16        53
 OSS                            OSS                         	2011 7 16                   54
オープンソース開発者がDeNAを選ぶ理由
Upcoming SlideShare
Loading in...5
×

オープンソース開発者がDeNAを選ぶ理由

13,072

Published on

Published in: Technology

オープンソース開発者がDeNAを選ぶ理由

  1. 1.     CTO2011 7 16 2
  2. 2.  Palmscape  1997 Xiino  Palm OS ( )  IBM WorkPad, Sony CLIE  NTT DoCoMo  M.I.T. TR100/2002    Web  IPA 2004 –2011 7 16 3
  3. 3. (2)   :  2005 8 2010 11  Japanize  Web UI      Q4M   (MySQL ) 2011 7 16 4
  4. 4. (3)   :  2010 11   IT  2011 5 CTO 2011 7 16 5
  5. 5.   DeNA   OSS  DeNA OSS   2011 7 16 6
  6. 6. 2011 7 16 7
  7. 7. Q4M 2011 7 16 8
  8. 8. What is Q4M?  A message queue  runs as a storage plugin of MySQL 5.1  Why is it a MySQL plugin?  accessible by using existing MySQL clients  no need for a new client library  administrable by using SQL  friendly to DB admins  First release: Jan 2008 2011 7 16 9
  9. 9. Design Goals of Q4M  Robust  Does not lose data on OS crash or power failure  necessary for Tokyo wo. nuclear power plants… orz  Fast  Transfer thousands of messages per second  Easy to Use  Use SQL for access / maintenance  Integration into MySQL  no more separate daemons to take care of 2011 7 16 10
  10. 10. Users of Q4M  Many leading web services in Japan  DeNA Co., Ltd.  livedoor Co., Ltd.  mixi, Inc.  Zynga Japan (formerly Unoh, Inc.)2011 7 16 11
  11. 11. What is a Message Queue? 2011 7 16 12
  12. 12. What is a Message Queue?  Middleware for persistent asynchronous communication  communicate between fixed pairs (parties)  a.k.a. Message Oriented Middleware  MQ is intermediate storage  RDBMS is persistent storage  Senders / receivers may go down2011 7 16 13
  13. 13. Minimal Configuration of a Message Queue  Senders and receivers access a single queue Sender Receiver Queue2011 7 16 14
  14. 14. Using MQ as a Relay  Separate queue for sender and receiver  Messages relayed between queues Relay Sender Receiver Queue Queue2011 7 16 15
  15. 15. Merits of using Message Relays  Destination can be changed easily  Relays may transfer messages to different locations depending on their headers  Robustness against network failure  no loss or duplicates when the relay fails  Logging and Multicasting, etc. 2011 7 16 16
  16. 16. Message Brokers  Publish / subscribe model  Separation between components and their integration  Components read / write to predefined queues  Integration is definition of routing rules between the message queues  Messages are often transformed (filtered) within the relay agent2011 7 16 17
  17. 17. What about Q4M?  Q4M itself is a message queue  Can connect Q4M instances to create a message relay  Provides API for creating message relays and brokers2011 7 16 18
  18. 18. Performance of Q4M  over 7,000 mess/sec.  message size: avg. 512 bytes  syncing to disk  Outperforming most needs  if you need more, just scale out  Can coexist with other storage engines without sacrificing their performance see http://labs.cybozu.co.jp/blog/kazuhoatwork/2008/06/q4m_06_release_and_benchmarks.php2011 7 16 19
  19. 19. Q4M in DeNA  uses Q4M for sending notifications asynchronously http://engineer.dena.jp/2010/03/dena-technical- seminar-1-2.html2011 7 16 20
  20. 20. Q4M in DeNA  Q4M  2,000 /   DeNA    2011 7 16 21
  21. 21. HandlerSocket Plugin 2011 7 16 22
  22. 22. Memcached    RDBMS /      RDBMS      Memcached RDBMS    primary storage2011 7 16 23
  23. 23. Memcached  RDBMS       +   :     SQL2011 7 16 24
  24. 24. SQL ( ) 2011 7 16 25
  25. 25. Mycached  2009 8  mysqld memcached   memcached get   get   SQL 2011 7 16 26
  26. 26. Mycached – mycached http://dev.mysql.com/doc/refman/5.1/en/pluggable-storage-overview.html 2011 7 16 27
  27. 27. Mycached –  1,200  JSON, msgpack,  Picoev (I/O )  MySQL ( )    get 2011 7 16 28
  28. 28. Mycached –  SQL 40,000qps 2011 7 16 29
  29. 29. Mycached    2011 7 16 30
  30. 30. HandlerSocket Plugin  DeNA  2010 8    MySQL Contributor Award 2011  MySQL SQL   qps   mycached2011 7 16 31
  31. 31. … 2011 7 16 32
  32. 32. …2011 7 16 33
  33. 33. … … 2011 7 16 34
  34. 34. … … … 2011 7 16 35
  35. 35. 2011 7 16 36
  36. 36.            2011 7 16 37
  37. 37. HandlerSocket Plugin  mycached      FastEther  2011 7 16 38
  38. 38. 2011 7 16 39
  39. 39. 2011 7 16 40
  40. 40. –         ? 2011 7 16 41
  41. 41. –    Server::Starter –  Parallel::Prefork –  Starlet – PSGI2011 7 16 42
  42. 42. –    fork   : mod_bumpy_life (Apache)  by hirose31++  MaxRequestsPerChild     Parallel::Prefork2011 7 16 43
  43. 43. –   MaxRequestsPerChild   fork … 2011 7 16 44
  44. 44. – : Slow Restart     fork listen accept fork & exec SIGUSR1 accept logic app. spawn app. servers fork & exec app. logic SIGHUP 2011 7 16 45
  45. 45. – OSS   Perl github CPAN     … 2011 7 16 46
  46. 46.  Q. ? $SIG{HUP} = sub { $got_sig = 1 }; … while (! $got_sig) { sleep(1000); // }2011 7 16 47
  47. 47.  Q. ? $SIG{HUP} = sub { $got_sig = 1 }; … while (! $got_sig) { // 1000 sleep(1000); // }2011 7 16 48
  48. 48.   Parallel::Prefork     …   : pselect   FAQ   : Perl pselect   XS (Perl C )   Perl XS 2011 7 16 49
  49. 49. OSS DeNA 2011 7 16 50
  50. 50.  Perl …  2009 Plack  Perl WSGI or Rack  Furl ( HTTP )   DeNA id:xaicron    MySQL2011 7 16 51
  51. 51.     MySQL  YAPC  MySQL Conference       : MySQL Casual    2011 7 16 52
  52. 52. 2011 7 16 53
  53. 53.  OSS     OSS     2011 7 16 54
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×