Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MySQL       NoSQL            NoSQLMySQL Casual Talks #2
•   id: oinume (   :   )
•   id: oinume (   :   )•   2010   3
•   id: oinume (   :   )•   2010   3•
•   id: oinume (     :   )•   2010   3••              SQL           MySQL
•   id: oinume (     :   )•   2010   3••              SQL             MySQL•   ver 3.23                 MySQL
• MySQL   NoSQL(   )
• MySQL   NoSQL(   )•                  MySQL
• MySQL NoSQL( )•             MySQL• NoSQL(KVS)
• MySQL NoSQL( )•             MySQL• NoSQL(KVS)• NoSQL(KVS)
•   Fusion-IO
•   Fusion-IO•           (Java)
•    Fusion-IO•            (Java)•   MySQL
•        Fusion-IO•                   (Java)•       MySQL    •      Oracle
•        Fusion-IO•                   (Java)•       MySQL    •      Oracle•
•
••   2009 2
••   2009 2•            800   ↑ (2011   7 )
••   2009 2•            800   ↑ (2011   7 )•
••   2009 2•            800   ↑ (2011   7 )•    •
••   2009 2•            800   ↑ (2011   7 )•    •    •
••   2009 2•            800      ↑ (2011   7 )•    •    •    •   (          ) ← NEW
Browser          Web x 35                        Memcached x 4 MySQL x 6                                       with FIO   ...
requestBrowser                    Web x 35                                  Memcached x 4 MySQL x 6                       ...
requestBrowser           swf                    Web x 35                                  Memcached x 4 MySQL x 6         ...
requestBrowser           swf                    Web x 35                                  Memcached x 4 MySQL x 6         ...
requestBrowser           swf                    Web x 35            TCP Socket                                   Memcached...
requestBrowser           swf                    Web x 35            TCP Socket                                   Memcached...
requestBrowser           swf                    Web x 35            TCP Socket                                   Memcached...
requestBrowser           swf                    Web x 35            TCP Socket                                   Memcached...
requestBrowser           swf                    Web x 35            TCP Socket                                   Memcached...
•   : 2.1Gbps
•     : 2.1Gbps•   : 100,000
•               : 2.1Gbps•            : 100,000• Web: 36,000 req / s
•                : 2.1Gbps•             : 100,000• Web: 36,000 req / s• Socket: 160,000 req / s
•                : 2.1Gbps•             : 100,000• Web: 36,000 req / s• Socket: 160,000 req / s• Database: 52000 qps (read...
MySQL
MySQL
MySQL•   RDBMS
MySQL•   RDBMS    •           KVS + MySQL
MySQL•   RDBMS    •              KVS + MySQL    •   KVS      MySQL
MySQL•   RDBMS    •                     KVS + MySQL    •      KVS          MySQL•   IndexPersister              MySQL   KVS
IndexPersister ?
IndexPersister•                      Java      MySQL• key / value•                      SQL•
DDLCREATE TABLE `user` (   `id` varbinary(255) NOT NULL,   `data` longblob NOT NULL,   PRIMARY KEY (`id`)) ENGINE=InnoDB D...
Java@Persistable("user")public class User {  @IndexKey  @Store(index=0)  public int getUserId() {      return userId;  }  ...
Java                          Java  @Persistable("user")  public class User {    @IndexKey    @Store(index=0)    public in...
Java                          Java  @Persistable("user")  public class User {    @IndexKey    @Store(index=0)    public in...
Java                          Java  @Persistable("user")  public class User {    @IndexKey    @Store(index=0)    public in...
select hex(id), hex(data) from user order by id limit 1 ¥Ghex(id) : 00000001hex(data):070000000600000006010300000002000000...
load - 1       ID: 1User user = persister.load(1, User.class);String nickname = user.getNickname();...
fetch -                             10IndexQuery query = IndexQuery.all();query.limit(10);List<User> users =   persister.f...
fetch -                                      (2)      ID:2User user = persister.load(2, User.class);IndexQuery query =  In...
fetch -                                                (2)@Persistable("user_item")public class UserItem {                ...
fetch -                     (2)          userId   itemId            1        1            1        2            1        3...
fetch -                                     (2)User user = persister.load(2, User.class);IndexQuery query =  IndexQuery.st...
• count• exists• remove• save (insert ... on duplicate key update)
•   MySQL   slow query
•   MySQL   slow query•
•   MySQL   slow query••                        slow query
•   MySQL   slow query••                        slow query•               Java
•   MySQL   slow query••                        slow query•               Java•
•   load
•          load    •   MySQL     context switch
•             load    •   MySQL               context switch    •   select ... where id in (...)
•                 load    •       MySQL               context switch    •       select ... where id in (...)•       1     ...
•                 load    •       MySQL               context switch    •       select ... where id in (...)•       1     ...
•                 load    •       MySQL               context switch    •       select ... where id in (...)•       1     ...
•                 load    •       MySQL               context switch    •       select ... where id in (...)•       1     ...
MySQL   Key Value Store
KVS
KVS•   SQL
KVS•        SQL    •   JOIN
KVS•           SQL    •   JOIN    •   (         )INDEX
KVS•           SQL    •   JOIN    •   (             )INDEX    •   (         )
KVS•             SQL    •     JOIN    •     (             )INDEX    •     (         )•       ALTER TABLE
KVS•             SQL    •     JOIN    •     (             )INDEX    •     (         )•       ALTER TABLE    •
KVS•             SQL    •     JOIN    •     (             )INDEX    •     (         )•       ALTER TABLE    •    •
KVS•             SQL    •     JOIN    •     (             )INDEX    •     (         )•       ALTER TABLE    •    ••       ...
KVS•             SQL    •     JOIN    •     (             )INDEX    •     (         )•       ALTER TABLE    •    ••       ...
•   Column
•       Column    •
•       Column    •    •            Column
•       Column    •    •            Column•
•       Column    •    •            Column•    •
•                Column    •    •                     Column•    ••       Column       (ALTER TABLE)
•                  Column    •    •                        Column•    ••       Column          (ALTER TABLE)    •    short...
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
MySQLでNoSQL - アメーバピグでのNoSQLの実例
Upcoming SlideShare
Loading in …5
×

MySQLでNoSQL - アメーバピグでのNoSQLの実例

20,280 views

Published on

NoSQL with MySQL

Published in: Technology
  • Be the first to comment

MySQLでNoSQL - アメーバピグでのNoSQLの実例

  1. 1. MySQL NoSQL NoSQLMySQL Casual Talks #2
  2. 2. • id: oinume ( : )
  3. 3. • id: oinume ( : )• 2010 3
  4. 4. • id: oinume ( : )• 2010 3•
  5. 5. • id: oinume ( : )• 2010 3•• SQL MySQL
  6. 6. • id: oinume ( : )• 2010 3•• SQL MySQL• ver 3.23 MySQL
  7. 7. • MySQL NoSQL( )
  8. 8. • MySQL NoSQL( )• MySQL
  9. 9. • MySQL NoSQL( )• MySQL• NoSQL(KVS)
  10. 10. • MySQL NoSQL( )• MySQL• NoSQL(KVS)• NoSQL(KVS)
  11. 11. • Fusion-IO
  12. 12. • Fusion-IO• (Java)
  13. 13. • Fusion-IO• (Java)• MySQL
  14. 14. • Fusion-IO• (Java)• MySQL • Oracle
  15. 15. • Fusion-IO• (Java)• MySQL • Oracle•
  16. 16.
  17. 17. •• 2009 2
  18. 18. •• 2009 2• 800 ↑ (2011 7 )
  19. 19. •• 2009 2• 800 ↑ (2011 7 )•
  20. 20. •• 2009 2• 800 ↑ (2011 7 )• •
  21. 21. •• 2009 2• 800 ↑ (2011 7 )• • •
  22. 22. •• 2009 2• 800 ↑ (2011 7 )• • • • ( ) ← NEW
  23. 23. Browser Web x 35 Memcached x 4 MySQL x 6 with FIO Socket x 70
  24. 24. requestBrowser Web x 35 Memcached x 4 MySQL x 6 with FIO Socket x 70
  25. 25. requestBrowser swf Web x 35 Memcached x 4 MySQL x 6 with FIO Socket x 70
  26. 26. requestBrowser swf Web x 35 Memcached x 4 MySQL x 6 with FIO Socket x 70
  27. 27. requestBrowser swf Web x 35 TCP Socket Memcached x 4 MySQL x 6 with FIO Socket x 70
  28. 28. requestBrowser swf Web x 35 TCP Socket Memcached x 4 MySQL x 6 with FIO Socket x 70 Java
  29. 29. requestBrowser swf Web x 35 TCP Socket Memcached x 4 MySQL x 6 with FIO Socket x 70 Java
  30. 30. requestBrowser swf Web x 35 TCP Socket Memcached x 4 MySQL x 6 with FIO Socket x 70 Java
  31. 31. requestBrowser swf Web x 35 TCP Socket Memcached x 4 MySQL x 6 with FIO Socket x 70 Java
  32. 32. • : 2.1Gbps
  33. 33. • : 2.1Gbps• : 100,000
  34. 34. • : 2.1Gbps• : 100,000• Web: 36,000 req / s
  35. 35. • : 2.1Gbps• : 100,000• Web: 36,000 req / s• Socket: 160,000 req / s
  36. 36. • : 2.1Gbps• : 100,000• Web: 36,000 req / s• Socket: 160,000 req / s• Database: 52000 qps (read: 40000, write: 12000)
  37. 37. MySQL
  38. 38. MySQL
  39. 39. MySQL• RDBMS
  40. 40. MySQL• RDBMS • KVS + MySQL
  41. 41. MySQL• RDBMS • KVS + MySQL • KVS MySQL
  42. 42. MySQL• RDBMS • KVS + MySQL • KVS MySQL• IndexPersister MySQL KVS
  43. 43. IndexPersister ?
  44. 44. IndexPersister• Java MySQL• key / value• SQL•
  45. 45. DDLCREATE TABLE `user` ( `id` varbinary(255) NOT NULL, `data` longblob NOT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8
  46. 46. Java@Persistable("user")public class User { @IndexKey @Store(index=0) public int getUserId() { return userId; } @Store(index=1) public String getNickname() { return nickname; } ....}
  47. 47. Java Java @Persistable("user") public class User { @IndexKey @Store(index=0) public int getUserId() { return userId; } @Store(index=1) public String getNickname() { return nickname; } .... }
  48. 48. Java Java @Persistable("user") public class User { @IndexKey @Store(index=0) public int getUserId() { return userId; } @Store(index=1) public String getNickname() { return nickname; } .... }
  49. 49. Java Java @Persistable("user") public class User { @IndexKey @Store(index=0) public int getUserId() { DB return userId; } index=0 0 @Store(index=1) public String getNickname() { return nickname; } .... }
  50. 50. select hex(id), hex(data) from user order by id limit 1 ¥Ghex(id) : 00000001hex(data):0700000006000000060103000000020000000E02000B797573616B752D3030303600000019020016E38286E38186E38195E3818F36E58FB7E381A7E381990000000301010100000003010101FFFFFFFF
  51. 51. load - 1 ID: 1User user = persister.load(1, User.class);String nickname = user.getNickname();...
  52. 52. fetch - 10IndexQuery query = IndexQuery.all();query.limit(10);List<User> users = persister.fetch(query, User.class);for (User user : users) ...
  53. 53. fetch - (2) ID:2User user = persister.load(2, User.class);IndexQuery query = IndexQuery.startWith(new IndexValue(user.getUserId()));List<UserItem> userItems = persister.fetch(query, UserItem.class);
  54. 54. fetch - (2)@Persistable("user_item")public class UserItem { userId, itemId @IndexKey public IndexKey getKey() { KeyCreator.toValue(userId, itemId); } @Store(index=0) public int getUserId() { return userId; } @Store(index=1) public int getItemId() { return itemId; } ....}
  55. 55. fetch - (2) userId itemId 1 1 1 2 1 3 2 10 2 20
  56. 56. fetch - (2)User user = persister.load(2, User.class);IndexQuery query = IndexQuery.startWith(new IndexValue(user.getUserId()));List<UserItem> userItems = persister.fetch(query, UserItem.class);◆ SQLselect id, data from user_item where id >= {v1} and id < {v2} v1: 00000002 v2: 00000003
  57. 57. • count• exists• remove• save (insert ... on duplicate key update)
  58. 58. • MySQL slow query
  59. 59. • MySQL slow query•
  60. 60. • MySQL slow query•• slow query
  61. 61. • MySQL slow query•• slow query• Java
  62. 62. • MySQL slow query•• slow query• Java•
  63. 63. • load
  64. 64. • load • MySQL context switch
  65. 65. • load • MySQL context switch • select ... where id in (...)
  66. 66. • load • MySQL context switch • select ... where id in (...)• 1 fetch ( )
  67. 67. • load • MySQL context switch • select ... where id in (...)• 1 fetch ( ) • key Column
  68. 68. • load • MySQL context switch • select ... where id in (...)• 1 fetch ( ) • key Column •
  69. 69. • load • MySQL context switch • select ... where id in (...)• 1 fetch ( ) • key Column • • 1 fetch 1000
  70. 70. MySQL Key Value Store
  71. 71. KVS
  72. 72. KVS• SQL
  73. 73. KVS• SQL • JOIN
  74. 74. KVS• SQL • JOIN • ( )INDEX
  75. 75. KVS• SQL • JOIN • ( )INDEX • ( )
  76. 76. KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE
  77. 77. KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE •
  78. 78. KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE • •
  79. 79. KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE • •• mongoDB KVS
  80. 80. KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE • •• mongoDB KVS • Ameba Pico mongoDB
  81. 81. • Column
  82. 82. • Column •
  83. 83. • Column • • Column
  84. 84. • Column • • Column•
  85. 85. • Column • • Column• •
  86. 86. • Column • • Column• •• Column (ALTER TABLE)
  87. 87. • Column • • Column• •• Column (ALTER TABLE) • short -> int Column

×