Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

オープンソース開発者がDeNAを選ぶ理由

15,778 views

Published on

Published in: Technology
  • Be the first to comment

オープンソース開発者がDeNAを選ぶ理由

  1. 1.     CTO2011 7 16 2
  2. 2.  Palmscape  1997 Xiino  Palm OS ( )  IBM WorkPad, Sony CLIE  NTT DoCoMo  M.I.T. TR100/2002    Web  IPA 2004 –2011 7 16 3
  3. 3. (2)   :  2005 8 2010 11  Japanize  Web UI      Q4M   (MySQL ) 2011 7 16 4
  4. 4. (3)   :  2010 11   IT  2011 5 CTO 2011 7 16 5
  5. 5.   DeNA   OSS  DeNA OSS   2011 7 16 6
  6. 6. 2011 7 16 7
  7. 7. Q4M 2011 7 16 8
  8. 8. What is Q4M?  A message queue  runs as a storage plugin of MySQL 5.1  Why is it a MySQL plugin?  accessible by using existing MySQL clients  no need for a new client library  administrable by using SQL  friendly to DB admins  First release: Jan 2008 2011 7 16 9
  9. 9. Design Goals of Q4M  Robust  Does not lose data on OS crash or power failure  necessary for Tokyo wo. nuclear power plants… orz  Fast  Transfer thousands of messages per second  Easy to Use  Use SQL for access / maintenance  Integration into MySQL  no more separate daemons to take care of 2011 7 16 10
  10. 10. Users of Q4M  Many leading web services in Japan  DeNA Co., Ltd.  livedoor Co., Ltd.  mixi, Inc.  Zynga Japan (formerly Unoh, Inc.)2011 7 16 11
  11. 11. What is a Message Queue? 2011 7 16 12
  12. 12. What is a Message Queue?  Middleware for persistent asynchronous communication  communicate between fixed pairs (parties)  a.k.a. Message Oriented Middleware  MQ is intermediate storage  RDBMS is persistent storage  Senders / receivers may go down2011 7 16 13
  13. 13. Minimal Configuration of a Message Queue  Senders and receivers access a single queue Sender Receiver Queue2011 7 16 14
  14. 14. Using MQ as a Relay  Separate queue for sender and receiver  Messages relayed between queues Relay Sender Receiver Queue Queue2011 7 16 15
  15. 15. Merits of using Message Relays  Destination can be changed easily  Relays may transfer messages to different locations depending on their headers  Robustness against network failure  no loss or duplicates when the relay fails  Logging and Multicasting, etc. 2011 7 16 16
  16. 16. Message Brokers  Publish / subscribe model  Separation between components and their integration  Components read / write to predefined queues  Integration is definition of routing rules between the message queues  Messages are often transformed (filtered) within the relay agent2011 7 16 17
  17. 17. What about Q4M?  Q4M itself is a message queue  Can connect Q4M instances to create a message relay  Provides API for creating message relays and brokers2011 7 16 18
  18. 18. Performance of Q4M  over 7,000 mess/sec.  message size: avg. 512 bytes  syncing to disk  Outperforming most needs  if you need more, just scale out  Can coexist with other storage engines without sacrificing their performance see http://labs.cybozu.co.jp/blog/kazuhoatwork/2008/06/q4m_06_release_and_benchmarks.php2011 7 16 19
  19. 19. Q4M in DeNA  uses Q4M for sending notifications asynchronously http://engineer.dena.jp/2010/03/dena-technical- seminar-1-2.html2011 7 16 20
  20. 20. Q4M in DeNA  Q4M  2,000 /   DeNA    2011 7 16 21
  21. 21. HandlerSocket Plugin 2011 7 16 22
  22. 22. Memcached    RDBMS /      RDBMS      Memcached RDBMS    primary storage2011 7 16 23
  23. 23. Memcached  RDBMS       +   :     SQL2011 7 16 24
  24. 24. SQL ( ) 2011 7 16 25
  25. 25. Mycached  2009 8  mysqld memcached   memcached get   get   SQL 2011 7 16 26
  26. 26. Mycached – mycached http://dev.mysql.com/doc/refman/5.1/en/pluggable-storage-overview.html 2011 7 16 27
  27. 27. Mycached –  1,200  JSON, msgpack,  Picoev (I/O )  MySQL ( )    get 2011 7 16 28
  28. 28. Mycached –  SQL 40,000qps 2011 7 16 29
  29. 29. Mycached    2011 7 16 30
  30. 30. HandlerSocket Plugin  DeNA  2010 8    MySQL Contributor Award 2011  MySQL SQL   qps   mycached2011 7 16 31
  31. 31. … 2011 7 16 32
  32. 32. …2011 7 16 33
  33. 33. … … 2011 7 16 34
  34. 34. … … … 2011 7 16 35
  35. 35. 2011 7 16 36
  36. 36.            2011 7 16 37
  37. 37. HandlerSocket Plugin  mycached      FastEther  2011 7 16 38
  38. 38. 2011 7 16 39
  39. 39. 2011 7 16 40
  40. 40. –         ? 2011 7 16 41
  41. 41. –    Server::Starter –  Parallel::Prefork –  Starlet – PSGI2011 7 16 42
  42. 42. –    fork   : mod_bumpy_life (Apache)  by hirose31++  MaxRequestsPerChild     Parallel::Prefork2011 7 16 43
  43. 43. –   MaxRequestsPerChild   fork … 2011 7 16 44
  44. 44. – : Slow Restart     fork listen accept fork & exec SIGUSR1 accept logic app. spawn app. servers fork & exec app. logic SIGHUP 2011 7 16 45
  45. 45. – OSS   Perl github CPAN     … 2011 7 16 46
  46. 46.  Q. ? $SIG{HUP} = sub { $got_sig = 1 }; … while (! $got_sig) { sleep(1000); // }2011 7 16 47
  47. 47.  Q. ? $SIG{HUP} = sub { $got_sig = 1 }; … while (! $got_sig) { // 1000 sleep(1000); // }2011 7 16 48
  48. 48.   Parallel::Prefork     …   : pselect   FAQ   : Perl pselect   XS (Perl C )   Perl XS 2011 7 16 49
  49. 49. OSS DeNA 2011 7 16 50
  50. 50.  Perl …  2009 Plack  Perl WSGI or Rack  Furl ( HTTP )   DeNA id:xaicron    MySQL2011 7 16 51
  51. 51.     MySQL  YAPC  MySQL Conference       : MySQL Casual    2011 7 16 52
  52. 52. 2011 7 16 53
  53. 53.  OSS     OSS     2011 7 16 54

×