MySQLでNoSQL - アメーバピグでのNoSQLの実例

MySQL NoSQL NoSQLMySQL Casual Talks #2

• id: oinume ( : )

• id: oinume ( : )• 2010 3

• id: oinume ( : )• 2010 3•

• id: oinume ( : )• 2010 3•• SQL MySQL

• id: oinume ( : )• 2010 3•• SQL MySQL• ver 3.23 MySQL

• MySQL NoSQL( )

• MySQL NoSQL( )• MySQL

• MySQL NoSQL( )• MySQL• NoSQL(KVS)

• MySQL NoSQL( )• MySQL• NoSQL(KVS)• NoSQL(KVS)

• Fusion-IO

• Fusion-IO• (Java)

• Fusion-IO• (Java)• MySQL

• Fusion-IO• (Java)• MySQL • Oracle

• Fusion-IO• (Java)• MySQL • Oracle•

•• 2009 2

•• 2009 2• 800 ↑ (2011 7 )

•• 2009 2• 800 ↑ (2011 7 )•

•• 2009 2• 800 ↑ (2011 7 )• •

•• 2009 2• 800 ↑ (2011 7 )• • •

•• 2009 2• 800 ↑ (2011 7 )• • • • ( ) ← NEW

Browser Web x 35 Memcached x 4 MySQL x 6 with FIO Socket x 70

requestBrowser Web x 35 Memcached x 4 MySQL x 6 with FIO Socket x 70

requestBrowser swf Web x 35 Memcached x 4 MySQL x 6 with FIO Socket x 70

requestBrowser swf Web x 35 TCP Socket Memcached x 4 MySQL x 6 with FIO Socket x 70

requestBrowser swf Web x 35 TCP Socket Memcached x 4 MySQL x 6 with FIO Socket x 70 Java

• : 2.1Gbps

• : 2.1Gbps• : 100,000

• : 2.1Gbps• : 100,000• Web: 36,000 req / s

• : 2.1Gbps• : 100,000• Web: 36,000 req / s• Socket: 160,000 req / s

• : 2.1Gbps• : 100,000• Web: 36,000 req / s• Socket: 160,000 req / s• Database: 52000 qps (read: 40000, write: 12000)

MySQL• RDBMS

MySQL• RDBMS • KVS + MySQL

MySQL• RDBMS • KVS + MySQL • KVS MySQL

MySQL• RDBMS • KVS + MySQL • KVS MySQL• IndexPersister MySQL KVS

IndexPersister ?

IndexPersister• Java MySQL• key / value• SQL•

DDLCREATE TABLE ùser` ( ìd` varbinary(255) NOT NULL, `data` longblob NOT NULL, PRIMARY KEY (ìd`)) ENGINE=InnoDB DEFAULT CHARSET=utf8

Java@Persistable("user")public class User { @IndexKey @Store(index=0) public int getUserId() { return userId; } @Store(index=1) public String getNickname() { return nickname; } ....}

Java Java @Persistable("user") public class User { @IndexKey @Store(index=0) public int getUserId() { return userId; } @Store(index=1) public String getNickname() { return nickname; } .... }

Java Java @Persistable("user") public class User { @IndexKey @Store(index=0) public int getUserId() { DB return userId; } index=0 0 @Store(index=1) public String getNickname() { return nickname; } .... }

select hex(id), hex(data) from user order by id limit 1 ¥Ghex(id) : 00000001hex(data):0700000006000000060103000000020000000E02000B797573616B752D3030303600000019020016E38286E38186E38195E3818F36E58FB7E381A7E381990000000301010100000003010101FFFFFFFF

load - 1 ID: 1User user = persister.load(1, User.class);String nickname = user.getNickname();...

fetch - 10IndexQuery query = IndexQuery.all();query.limit(10);List<User> users = persister.fetch(query, User.class);for (User user : users) ...

fetch - (2) ID:2User user = persister.load(2, User.class);IndexQuery query = IndexQuery.startWith(new IndexValue(user.getUserId()));List<UserItem> userItems = persister.fetch(query, UserItem.class);

fetch - (2)@Persistable("user_item")public class UserItem { userId, itemId @IndexKey public IndexKey getKey() { KeyCreator.toValue(userId, itemId); } @Store(index=0) public int getUserId() { return userId; } @Store(index=1) public int getItemId() { return itemId; } ....}

fetch - (2) userId itemId 1 1 1 2 1 3 2 10 2 20

fetch - (2)User user = persister.load(2, User.class);IndexQuery query = IndexQuery.startWith(new IndexValue(user.getUserId()));List<UserItem> userItems = persister.fetch(query, UserItem.class);◆ SQLselect id, data from user_item where id >= {v1} and id < {v2} v1: 00000002 v2: 00000003

• count• exists• remove• save (insert ... on duplicate key update)

• MySQL slow query

• MySQL slow query•

• MySQL slow query•• slow query

• MySQL slow query•• slow query• Java

• MySQL slow query•• slow query• Java•

• load

• load • MySQL context switch

• load • MySQL context switch • select ... where id in (...)

• load • MySQL context switch • select ... where id in (...)• 1 fetch ( )

• load • MySQL context switch • select ... where id in (...)• 1 fetch ( ) • key Column

• load • MySQL context switch • select ... where id in (...)• 1 fetch ( ) • key Column •

• load • MySQL context switch • select ... where id in (...)• 1 fetch ( ) • key Column • • 1 fetch 1000

MySQL Key Value Store

KVS• SQL

KVS• SQL • JOIN

KVS• SQL • JOIN • ( )INDEX

KVS• SQL • JOIN • ( )INDEX • ( )

KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE

KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE •

KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE • •

KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE • •• mongoDB KVS

KVS• SQL • JOIN • ( )INDEX • ( )• ALTER TABLE • •• mongoDB KVS • Ameba Pico mongoDB

• Column

• Column •

• Column • • Column

• Column • • Column•

• Column • • Column• •

• Column • • Column• •• Column (ALTER TABLE)

• Column • • Column• •• Column (ALTER TABLE) • short -> int Column

http://mysql-casual.org	2869
http://tech.lampetty.net	1177
http://d.hatena.ne.jp	806
http://snip-snap.posterous.com	225
http://ameblo.jp	55
http://webcache.googleusercontent.com	35
http://feeds2.feedburner.com	17
http://ist-dev.private.i-studio.co.jp	12
http://infra.rrdtool.net	11
http://s.deeeki.com	3
http://us-w1.rockmelt.com	2
http://feeds.feedburner.com	1
http://rss.ameba.jp	1
http://www.slideshare.net	1
http://twitter.com	1
https://twitter.com	1
http://feeds.feedburner.jp	1

MySQLでNoSQL - アメーバピグでのNoSQLの実例

by Kazuhiro Oinuma on Aug 10, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

17 Embeds 5,218

Statistics

MySQLでNoSQL - アメーバピグでのNoSQLの実例 — Presentation Transcript