Using ngx_lua / lua-nginx-module in pixiv
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Using ngx_lua / lua-nginx-module in pixiv

  • 5,056 views
Uploaded on

ドリコム、リブセンスさんとの合同勉強会で話したスライドです。

ドリコム、リブセンスさんとの合同勉強会で話したスライドです。

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
5,056
On Slideshare
4,611
From Embeds
445
Number of Embeds
10

Actions

Shares
Downloads
38
Comments
0
Likes
17

Embeds 445

http://techhey.hatenablog.com 257
http://tech.akat.info 96
https://twitter.com 78
http://192.168.0.100 5
https://translate.googleusercontent.com 2
http://webcache.googleusercontent.com 2
https://duckduckgo.com 2
http://translate.googleusercontent.com 1
http://54.64.11.67 1
http://www.google.co.jp 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. / 30 Using ngx_lua in pixiv Harukasan a.k.a MICHII Shunsuke 2013-06-24
  • 2. / 30 はるかさん インフラ部所属 2年目 Nginx / Apache / MySQL / MongoDB Kyototycoon / Solr / Traffic Server / Fluentd Capistrano / Ruby on Rails / Macbook Air / Vim 2
  • 3. / 303 • about pixiv Image Cluster • ngx_lua • directives in nginx.conf • nginx api in lua scripts • ex. code from pixiv Image Cluster Agenda
  • 4. / 304 pixiv Image Cluster
  • 5. / 305  Front Cache Cluster Image Store  SSD-RAID0     HDD SSD Thumbnailbig size Memory DEVICEpeak: 1k req/sec 10Gbps x20 x20
  • 6. / 306  Front Cache Cluster Image Store PRODUCT Nginx with tmpfs Traffic Server Apache / NginxNginx
  • 7. / 307 ngx_lua github.com/chaoslawful/lua-nginx-module
  • 8. / 308 • embeds Lua/LuaJIT into Nginx • has 100% non-blocking API • has many libraries provided by OpenResty • memcached, mysql, redis, etc. ngx_lua
  • 9. / 309 # load from /etc/nginx/scripts/?.lua and default root lua_package_path "/etc/nginx/scripts/?.lua;;"; server { listen 80; # GET /fact?a=4 -> 24 (4! = 4 * 3 * 2 * 1) location /fact { content_by_lua " local val = 1 for i = 2, ngx.var.arg_a do val *= i; end "; } # same code in file location /fact_script { content_by_lua_file "fact.lua"; } }
  • 10. / 3010 Directives in nginx.conf • *_by_lua, *_by_lua_file • init_by_lua • rewrite_by_lua • access_by_lua • content_by_lua 設定読み込み rewrite処理 access処理 bodyの生成処理
  • 11. / 3011 init_by_lua <code> masterが設定をロードするタイミングで実行する init_by_lua " memcached = require 'resty.memcached' "; server { ! listen 8888; ! default_type 'text/plain'; ! location /init_by { ! ! content_by_lua " ! ! local hello = memcached::get('Hello'); ... ! ! "; ! } }
  • 12. / 3012 server { listen 80; # GET /sum?a=2&b=3 -> 5 location /sum { set_by_lua $sum " return tonumber(ngx.var.arg_a) + tonumber(ngx.var.arg_b) "; echo $sum; } } set_by_lua $res <code> 実行したコードの返値を$resにセットする
  • 13. / 3013 server rewrite find config rewrite post rewrite pre-access access post-access try files content logging Request rewrite/access/content_by_lua サーバーコンテキストのrewrite 該当するlocation configを探す locationのrewrite処理 ファイルへのアクセス try_fileの処理 bodyの生成 ログ書き込み access_by_lua content_by_lua rewrite_by_lua rewriteに関連する処理 アクセス制御 bodyの操作
  • 14. / 3014 server { listen 8888; location /rewrite_by { rewrite_by_lua "ngx.var.toggle = true"; if ($toggle) { rewrite / /index_second.html; } } } rewrite_by_lua <code> rewriteの前に指定したコードを実行する
  • 15. / 3015 server { listen 80; # GET /access_by?status=404 => 404 Not Found location /access_by { access_by_lua "ngx.exit(ngx.var.arg_status)"; } } access_by_lua <code> access phaseに指定したコードを実行する
  • 16. / 3016 # GET /hello location /hello { content_by_lua " ngx.say('Hello, from lua') "; } content_by_lua <code> responseを生成するコードを実行する
  • 17. / 3017 Directives in nginx.conf • LuaスクリプトでNginxの各動作を制御できる • HTTPで必要な各フェーズでフックできる • rewrite • access control • content generation ここまで8分
  • 18. / 3018 Nginx API in lua scripts • generate content • variables and shared tables • access memcached / MySQL
  • 19. / 3019 location /var { content_by_lua " ngx.say('Hello, from lua!') "; } ngx.say content bodyに文字列を出力する location /var { access_by_lua " ngx.exit(418) "; # i'm a teapot } ngx.exit 指定したステータスコードを返す
  • 20. / 3020 location /var { set $a "alpha"; rewrite_by_lua "ngx.var.b = ngx.var.a .. 'beta'"; echo $b; } ngx.var.* ngx.ctx.* location /ctx { access_by_lua "ngx.ctx.foo = 74"; rewrite_by_lua "ngx.ctx.foo = ngx.ctx.foo + 5"; content_by_lua "ngx.say('foo: ' .. ngx.ctx.foo)"; } Nginxの変数にアクセスする リクエスト中に共有されるテーブル
  • 21. / 3021 ngx.shared.* shared memoryテーブル http { lua_shared_dict foo 10m; server { location /shared-set { content_by_lua " local foo = ngx.shared.foo foo:set(ngx.var.arg_key, ngx.var.arg_val) "; } location /shared-get { content_by_lua " local foo = ngx.shared.foo ngx.say(foo:get(ngx.var.arg_key)) "; } } }
  • 22. / 3022 Regular Expressions • ngx.re.match • ngx.re.gmatch • ngx.re.sub • ngx.re.gsub
  • 23. / 3023 local matches, err = ngx.re.match("key=val", "([^=]+)=(.+)") ngx.re.match ngx.re.sub matches[0] => "key=val" matches[1] => "key" matches[2] => "val" local newstr,n,err = ngx.re.sub("Hello,lua","lua", "nginx") newstr = "Hello,nginx"
  • 24. / 3024 • OpenResty • lua-resty-memcached github.com/agentzh/lua-resty-memcached • lua-resty-mysql github.com/agentzh/lua-resty-mysql • ngx.socketを使って実装されている Access to memcached/MySQL
  • 25. / 3025 Nginx API in lua scripts • 簡単な処理を書くには十分な関数が 提供されている • 正規表現 • ハッシュ(md5, sha1) • ソケット • OpenRestyを使うとmemcachedへのアクセス とかも簡単
  • 26. / 3026 Example code from pixiv Image Cluster
  • 27. / 3027 Kyototycoon Logical Delete Nginx 200 404 /img01.png 200 /img02.png 200 /img03.png 404 /img04.png 200   GET /img01.png GET /img03.png 404
  • 28. / 3028 Logical Delete local memcached = require "resty.memcached" local uri = ngx.var.request_uri local memc = memcached:new() local val, flags, err = memc:get(request_uri) if val and val ~= "200" then exit(tonumber(val)) end logical_delete.lua location / { access_by_lua_file logical_delete.lua; }
  • 29. / 3029 Conclusion • Nginxだけで簡単な処理を実現できる • アクセス制御とかインプレッション計測 とかには便利そう • 高速で省メモリ • メンテナンスコストを抑えられる
  • 30. / 3030 References HTTPLuaModule - Nginx Wiki http://wiki.nginx.org/HttpLuaModule chaoslawful/lua-nginx-module - Github https://github.com/chaoslawful/lua-nginx-module