ISUCONで学ぶ Webアプリケーションのパフォーマンス向上のコツ 実践編 完全版

40,528 views
40,444 views

Published on

Published in: Internet
1 Comment
132 Likes
Statistics
Notes
  • Like And share plessz
    https://www.facebook.com/pages/QNet-Sweden/680155715393723
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
40,528
On SlideShare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
110
Comments
1
Likes
132
Embeds 0
No embeds

No notes for slide

ISUCONで学ぶ Webアプリケーションのパフォーマンス向上のコツ 実践編 完全版

  1. 1. ISUCONで学ぶ Webアプリケーションの パフォーマンス向上のコツ 実践編 完全版 ISUCON夏期講習 2014/8/20 Masahiro Nagano
  2. 2. この資料を読む前に 以下の記事をお読みください http://blog.nomadscafe.jp/2014/08/isucon-2014-ami.html
  3. 3. チューニングにあたり@acidlemon さんの blog記事を参考にしています 「ざっくりと #isucon 2013年予選問題の 解き方教えます」 http://isucon.net/archives/32976287.html
  4. 4. 挑戦してみました
  5. 5. 最終スコア 9079
  6. 6. やってみたことを 紹介します
  7. 7. 初期スコア 1664 ruby実装にて
  8. 8. (1) 環境整備
  9. 9. 静的コンテンツを Reverse Proxy で配信 Reverse Proxy: クライアントからの接続を 受け、Applicationサーバに処理を中継す る。画像,js,css などの静的コンテンツを返す 役割もある Application Server: ユーザからのリクエス トを受けて適切なページを構築・レスポン スを行う
  10. 10. /etc/httpd/conf.d/isucon.conf <VirtualHost *:80> DocumentRoot /home/isu-user/isucon/webapp/public RewriteEngine on RewriteCond REQUEST_URI !^/favicon.ico$ RewriteCond REQUEST_URI !^/(img|css|js)/ RewriteRule /(.*)$ http://localhost:5000/$1 [P] </VirtualHost>
  11. 11. スコア 1664 => 1719
  12. 12. Nginx 化 • オープンソースのWebサーバ。高速に動 作し、メモリ使用量がすくないなどの 特徴があります
  13. 13. Apache vs. Nginx リクエスト worker worker worker worker worker worker worker worker worker コンテキストスイッチが 大量発生 リクエスト worker 1個のプロセスで 効率よく通信を処理
  14. 14. command $ sudo yum install nginx $ sudo service httpd stop run.ini [program:nginx] directory=/ command=/usr/sbin/nginx -c /home/isu-user/isucon/ nginx.conf autostart = true nginx.conf: https://gist.github.com/kazeburo/7b0385cce1b0a4565581
  15. 15. スコア 1719 => 1764
  16. 16. (2) Perl にします ワタシハパールチョットデキル
  17. 17. Perl の起動方法 run.ini TCPではなくUNIX domain socketを使う command=/home/../isucon/env.sh carton exec -- start_server --path /tmp/app.sock -- plackup -s Starlet --max-workers 4 プロセスはあげすぎない --max-reqs-per-child 50000 -E production -a app.psgi プロセスを長生きさせる
  18. 18. TCPの接続は高コスト Reverse Proxy リクエスト毎に three way handshake App Server
  19. 19. スコア 1764 => 1891
  20. 20. (3) アプリをみよう
  21. 21. “/” “/recent/xxx” “/memo/xxxx” “/mypage”
  22. 22. “/” “/recent/xxx” DBへの問い合わせが重い “/memo/xxxx” “/mypage” markdown の変換に プロセス起動 DBへの問い合わせが 若干重い
  23. 23. (4) 外部プロセス起動
  24. 24. webapp/perl/lib/Isucon3/Web.pm “/memo/xxxx” +use Text::Markdown::Hoedown qw//; sub markdown { my $content = shift; - my ($fh, $filename) = tempfile(); - $fh->print(encode_utf8($content)); - $fh->close; - my $html = qx{ ../bin/markdown $filename }; - unlink $filename; - return $html; + Text::Markdown::Hoedown::markdown($content) } ここがmarkdownコマンドを 起動している XS(C)で高速にmarkdownを 処理するモジュール
  25. 25. スコア 1891 => 2233
  26. 26. (5) N+1 クエリ
  27. 27. webapp/perl/lib/Isucon3/Web.pm “/” my $memos = $self->dbh->select_all( 'SELECT * FROM memos WHERE is_private=0 ORDER BY created_at DESC, id DESC LIMIT 100' ); for my $memo (@$memos) { $memo->{username} = $self->dbh->select_one( 'SELECT username FROM users WHERE id=?', $memo->{user}, ); } 100回ルーーーープ
  28. 28. use the join, luke
  29. 29. memosテーブルusersテーブル id user_id id name memos JOIN users ON memos.user_id = user.id id user_id name
  30. 30. webapp/perl/lib/Isucon3/Web.pm my $memos = $self->dbh->select_all( 'SELECT memos.*,users.username FROM memos JOIN users ON memos.user = users.id WHERE memos.is_private=0 ORDER BY memos.created_at DESC, memos.id DESC LIMIT 100' ); “/”, “/recent”
  31. 31. スコア 2233 => 2398
  32. 32. (6) インデックス
  33. 33. indexがないと SELECT * FROM memos WHERE is_private=0 ORDER BY created_at DESC LIMIT 100 memosテーブル id is_priv ate ... 0 0 1 0 1 id is_priv ate ... 0 0 0 ソート webapp/perl/lib/Isucon3/Web.pm 抽出 CPU負荷高い
  34. 34. indexをつくる init.sh cat <<'EOF' | mysql -u isucon isucon ALTER TABLE memos ADD INDEX (is_private,created_at); EOF
  35. 35. B-Tree is_private 0 1 created_at older newer older newer
  36. 36. B-Tree is_private 0 1 created_at older newer older newer
  37. 37. B-Tree is_private 0 1 created_at older newer older newer
  38. 38. B-Tree is_private 0 1 created_at older newer older newer
  39. 39. B-Tree is_private 0 1 created_at older newer older newer 順に取得するだけ
  40. 40. スコア 2398 => 2668
  41. 41. (7) タイトル事前生成
  42. 42. これ
  43. 43. mysql mysql> show create table memosG *************************** 1. row *************************** Table: memos Create Table: CREATE TABLE `memos` ( `id` int(11) NOT NULL AUTO_INCREMENT, `user` int(11) NOT NULL, `content` text, `is_private` tinyint(4) NOT NULL DEFAULT '0', `created_at` datetime NOT NULL, `updated_at` timestamp NOT NULL DEFAULT, PRIMARY KEY (`id`), ) ENGINE=InnoDB AUTO_INCREMENT=41311 DEFAULT CHARSET=utf8 1 row in set (0.00 sec) titleカラムが存在しない!
  44. 44. タイトルは本文から 都度生成 webapp/perl/views/index.tx <: $memo.content.split('r?n').first() :> contentの転送で通信splitでCPU使用
  45. 45. titleカラムの追加し、 init.sh 事前生成 cat <<'EOF' | mysql -u isucon isucon ALTER TABLE memos ADD COLUMN title text; UPDATE memos SET title = substring_index(content,"n",1); EOF
  46. 46. POST時にも生成 webapp/perl/lib/Isucon3/Web.pm $self->dbh->query(   'INSERT INTO memos (user, title, content, is_private, created_at) VALUES (?, ?, ?, ?, now()) ', $user_id, (split /r?n/, $content)[0], $content, $is_private, );
  47. 47. webapp/perl/lib/Isucon3/Web.pm my $memos = $self->dbh->select_all( 'SELECT memos.id, memos.title, memos.is_private, memos.created_at, users.username FROM memos JOIN users ON memos.user = users.id WHERE memos.is_private=0 ORDER BY memos.created_at DESC, memos.id DESC LIMIT 100' ); “/”, “/recent” memos.* だと contentを 取ってしまう
  48. 48. スコア 2668 => 3060
  49. 49. (8) OFFSET = 破棄
  50. 50. ”/recent/100” 100ページ目 SELECT * FROM memos ORDER BY created_at LIMIT 100 OFFSET 10000 とても大きなOFFSET
  51. 51. MySQLのOFFSET処理のイメージ 1 2 3 4 id title user ... . id title user ... . id title user ... . id title user ... . 5 6 7 8 id title user ... . id title user ... . id title user ... . id title user ... . 9 10 11 12 id title user ... . id title user ... . id title user ... . id title user ... . 13 id title user ... . 10000 id title user ... . 10001 10002 10003 10004 id title user ... . id title user ... . id title user ... . id title user ... .
  52. 52. MySQLのOFFSET処理のイメージ 頑張ってソート 1 2 3 4 id title user ... . id title user ... . id title user ... . id title user ... . 5 6 7 8 id title user ... . id title user ... . id title user ... . id title user ... . 9 10 11 12 id title user ... . id title user ... . id title user ... . id title user ... . 13 id title user ... . 10000 id title user ... . 10001 10002 10003 10004 id title user ... . id title user ... . id title user ... . id title user ... . 必要な個数まで到達
  53. 53. MySQLのOFFSET処理のイメージ 頑張ってソート 1 2 3 4 id title user ... . id title user ... . id title user ... . id title user ... . 5 6 7 8 id title user ... . id title user ... . id title user ... . id title user ... . 9 10 11 12 id title user ... . id title user ... . id title user ... . id title user ... . 13 id title user ... . 10000 id title user ... . 10001 10002 10003 10004 id title user ... . id title user ... . id title user ... . id title user ... . 必要な個数まで到達 廃棄
  54. 54. MOTTAINAI
  55. 55. 捨てるデータを減らす
  56. 56. 取得するデータを制限 SELECT id FROM memos ORDER BY created_at LIMIT 100 OFFSET 10000
  57. 57. MySQLのOFFSET処理のイメージ 1 2 3 4 5 6 7 8 9 10 11 12 13 id id id id id id id id id id id id id ・・・・・ ・・・・・ 9999 10000 id id 10001 10002 10003 10004 id id id id
  58. 58. MySQLのOFFSET処理のイメージ 1 2 3 4 5 6 7 8 9 10 11 12 13 id id id id id id id id id id id id id ・・・・・ 10001 10002 10003 10004 廃棄 ・・・・・ 9999 10000 id id id id id id 読むデータも、捨て るデータも少ない
  59. 59. “id” だけにすると 高速になるもう一つの理由 “Covering Index”
  60. 60. MySQLのインデックスと データの持ち方 title user ... title user ... user ... PRIMARY title user ... title user ... title user ... title user ... title user ... title KEY CLUSTERED INDEX リーフノードに データを含む small large id id id id id id id id
  61. 61. MySQLのインデックスと データの持ち方 SECONDARY KEY primary keyじゃないkey リーフノードに PRIMARY KEYが含まれ、 データはCLUSTERED INDEX から取得 is_private id id id id id id id id created_at older newer older newer
  62. 62. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  63. 63. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  64. 64. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  65. 65. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  66. 66. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  67. 67. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  68. 68. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  69. 69. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  70. 70. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  71. 71. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  72. 72. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  73. 73. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  74. 74. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  75. 75. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  76. 76. SELECT * の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at 何度も繰り返す
  77. 77. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  78. 78. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  79. 79. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  80. 80. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  81. 81. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  82. 82. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  83. 83. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  84. 84. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at
  85. 85. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at indexだけで 探索が終わる
  86. 86. SELECT id の場合 title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . title user ... . PRIMARY KEY id id id id id id id id SECONDARY KEY id id id id id id id id is_private created_at indexだけで= “Covering Index” 探索が終わる
  87. 87. Covering Indexで高速に 絞り込んだidの titleなど、他のデータを 取得する方法
  88. 88. (1) IN 句 クエリ1 SELECT id FROM memos WHERE is_private = 0 ORDER BY created_at DESC, id DESC LIMIT 100 OFFSET 100000 クエリ2 ID羅列 SELECT * FROM memos WHERE id IN (10000,10001,10002,1003,....) ORDER BY created_at DESC, id DESC
  89. 89. (2) SELF JOIN クエリ SELECT memos.id, memos.title, memos.is_private, memos.created_at, users.username FROM memos, users, (SELECT id FROM memos WHERE is_private = 0 ORDER BY created_at DESC, id DESC LIMIT 100) AS t WHERE t.id = memos.id AND users.id = memos.user サブクエリーを使用し 派生テーブル”t”と派生テーブル”t”を作成 元のテーブルをJOIN
  90. 90. スコア 3060 => 4234 よりクエリの少ないSELF JOINを使いました
  91. 91. (9) その他インデックス
  92. 92. init.sh cat <<'EOF' | mysql -u isucon isucon ALTER TABLE memos ADD INDEX (is_private,created_at), ADD INDEX mypage(user,created_at), ADD INDEX memo_private(user,is_private,created_at) EOF
  93. 93. webapp/perl/lib/Isucon3/Web.pm my $memos = $self->dbh->select_all( "SELECT id FROM memos WHERE user=? $cond ORDER BY created_at", $memo->{user}, ); “/memo/xxx” 元は”*”だが、Covering Indexを 狙って”id”に変更
  94. 94. スコア 4234 => 5309
  95. 95. (10) OFFSET殲滅 データ構造を変更
  96. 96. 予めソート済みのmemoの リストがあり、BETWEEN句で アクセスができれば OFFSETで破棄される データはいなくなり、エコ
  97. 97. public_memos テーブル id memo 1 4 2 6 3 8 ... ... ... ... 10525 21274 10526 21277 10527 21280 ... ... 10626 21477 10627 21480 ... ... 20627 41345 is_private=0 のmemoの idリストolder OFFSET 10000の代わりに BETWEEN 10001 AND 10100 memoの 個数にもなる! newer
  98. 98. B-Treeでイメージ PRIMARY KEY older newer id id id id id id id id memo memo memo memo memo memo memo memo BETWEEN 10001 AND 10100
  99. 99. init.sh cat <<'EOF' | mysql -u isucon isucon DROP TABLE IF EXISTS public_memos; CREATE TABLE public_memos ( id INT NOT NULL AUTO_INCREMENT, memo int DEFAULT NULL, PRIMARY KEY (id) ) ENGINE=MyISAM DEFAULT CHARSET=utf8; INSERT INTO public_memos (memo) SELECT id FROM memos WHERE is_private=0 ORDER BY created_at ASC, id ASC; EOF * innodb_autoinc_lock_mode の影響で InnoDBではauto increment が連続した値にならない可能性がある
  100. 100. webapp/perl/lib/Isucon3/Web.pm my $total = $self->dbh->select_one( 'SELECT MAX(id) FROM public_memos' ); my $memos = $self->dbh->select_all( 'SELECT memos.id, memos.title, memos.is_private, memos.created_at, users.username FROM memos,users, (SELECT memo FROM public_memos WHERE id BETWEEN ? AND ? ORDER BY id DESC) AS t WHERE t.memo = memos.id AND users.id=memos.user', $total-99, $total ); “/” or “/recent/xxx”
  101. 101. post “/memo” webapp/perl/lib/Isucon3/Web.pm my $memo_id = $self->dbh->last_insert_id; if ( ! scalar($c->req->param('is_private')) ) { $self->dbh->query('INSERT INTO public_memos (memo) VALUES (?)',$memo_id); } is_private = 0 なら public_memosにもinsert
  102. 102. スコア 5309 => 8720
  103. 103. あと、セッション周りの クエリを減らしたりすると
  104. 104. スコア 8720 => 9079
  105. 105. Cache がなくても SQL や インデックスのチューニングで ここまで変わる、この問題は 面白いなぁと思いました。 出題の@fujiwaraさん、@acidlemonさん をはじめKAYACの皆様にあらためて感謝

×