Using ngx_lua / lua-nginx-module in pixiv
Upcoming SlideShare
Loading in...5
×
 

Using ngx_lua / lua-nginx-module in pixiv

on

  • 4,305 views

ドリコム、リブセンスさんとの合同勉強会で話したスライドです。

ドリコム、リブセンスさんとの合同勉強会で話したスライドです。

Statistics

Views

Total Views
4,305
Views on SlideShare
3,965
Embed Views
340

Actions

Likes
15
Downloads
36
Comments
0

6 Embeds 340

http://techhey.hatenablog.com 218
https://twitter.com 74
http://tech.akat.info 43
https://duckduckgo.com 2
https://translate.googleusercontent.com 2
http://webcache.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Using ngx_lua / lua-nginx-module in pixiv Using ngx_lua / lua-nginx-module in pixiv Presentation Transcript

  • / 30 Using ngx_lua in pixiv Harukasan a.k.a MICHII Shunsuke 2013-06-24
  • / 30 はるかさん インフラ部所属 2年目 Nginx / Apache / MySQL / MongoDB Kyototycoon / Solr / Traffic Server / Fluentd Capistrano / Ruby on Rails / Macbook Air / Vim 2
  • / 303 • about pixiv Image Cluster • ngx_lua • directives in nginx.conf • nginx api in lua scripts • ex. code from pixiv Image Cluster Agenda
  • / 304 pixiv Image Cluster
  • / 305  Front Cache Cluster Image Store  SSD-RAID0     HDD SSD Thumbnailbig size Memory DEVICEpeak: 1k req/sec 10Gbps x20 x20
  • / 306  Front Cache Cluster Image Store PRODUCT Nginx with tmpfs Traffic Server Apache / NginxNginx
  • / 307 ngx_lua github.com/chaoslawful/lua-nginx-module
  • / 308 • embeds Lua/LuaJIT into Nginx • has 100% non-blocking API • has many libraries provided by OpenResty • memcached, mysql, redis, etc. ngx_lua
  • / 309 # load from /etc/nginx/scripts/?.lua and default root lua_package_path "/etc/nginx/scripts/?.lua;;"; server { listen 80; # GET /fact?a=4 -> 24 (4! = 4 * 3 * 2 * 1) location /fact { content_by_lua " local val = 1 for i = 2, ngx.var.arg_a do val *= i; end "; } # same code in file location /fact_script { content_by_lua_file "fact.lua"; } }
  • / 3010 Directives in nginx.conf • *_by_lua, *_by_lua_file • init_by_lua • rewrite_by_lua • access_by_lua • content_by_lua 設定読み込み rewrite処理 access処理 bodyの生成処理
  • / 3011 init_by_lua <code> masterが設定をロードするタイミングで実行する init_by_lua " memcached = require 'resty.memcached' "; server { ! listen 8888; ! default_type 'text/plain'; ! location /init_by { ! ! content_by_lua " ! ! local hello = memcached::get('Hello'); ... ! ! "; ! } }
  • / 3012 server { listen 80; # GET /sum?a=2&b=3 -> 5 location /sum { set_by_lua $sum " return tonumber(ngx.var.arg_a) + tonumber(ngx.var.arg_b) "; echo $sum; } } set_by_lua $res <code> 実行したコードの返値を$resにセットする
  • / 3013 server rewrite find config rewrite post rewrite pre-access access post-access try files content logging Request rewrite/access/content_by_lua サーバーコンテキストのrewrite 該当するlocation configを探す locationのrewrite処理 ファイルへのアクセス try_fileの処理 bodyの生成 ログ書き込み access_by_lua content_by_lua rewrite_by_lua rewriteに関連する処理 アクセス制御 bodyの操作
  • / 3014 server { listen 8888; location /rewrite_by { rewrite_by_lua "ngx.var.toggle = true"; if ($toggle) { rewrite / /index_second.html; } } } rewrite_by_lua <code> rewriteの前に指定したコードを実行する
  • / 3015 server { listen 80; # GET /access_by?status=404 => 404 Not Found location /access_by { access_by_lua "ngx.exit(ngx.var.arg_status)"; } } access_by_lua <code> access phaseに指定したコードを実行する
  • / 3016 # GET /hello location /hello { content_by_lua " ngx.say('Hello, from lua') "; } content_by_lua <code> responseを生成するコードを実行する
  • / 3017 Directives in nginx.conf • LuaスクリプトでNginxの各動作を制御できる • HTTPで必要な各フェーズでフックできる • rewrite • access control • content generation ここまで8分
  • / 3018 Nginx API in lua scripts • generate content • variables and shared tables • access memcached / MySQL
  • / 3019 location /var { content_by_lua " ngx.say('Hello, from lua!') "; } ngx.say content bodyに文字列を出力する location /var { access_by_lua " ngx.exit(418) "; # i'm a teapot } ngx.exit 指定したステータスコードを返す
  • / 3020 location /var { set $a "alpha"; rewrite_by_lua "ngx.var.b = ngx.var.a .. 'beta'"; echo $b; } ngx.var.* ngx.ctx.* location /ctx { access_by_lua "ngx.ctx.foo = 74"; rewrite_by_lua "ngx.ctx.foo = ngx.ctx.foo + 5"; content_by_lua "ngx.say('foo: ' .. ngx.ctx.foo)"; } Nginxの変数にアクセスする リクエスト中に共有されるテーブル
  • / 3021 ngx.shared.* shared memoryテーブル http { lua_shared_dict foo 10m; server { location /shared-set { content_by_lua " local foo = ngx.shared.foo foo:set(ngx.var.arg_key, ngx.var.arg_val) "; } location /shared-get { content_by_lua " local foo = ngx.shared.foo ngx.say(foo:get(ngx.var.arg_key)) "; } } }
  • / 3022 Regular Expressions • ngx.re.match • ngx.re.gmatch • ngx.re.sub • ngx.re.gsub
  • / 3023 local matches, err = ngx.re.match("key=val", "([^=]+)=(.+)") ngx.re.match ngx.re.sub matches[0] => "key=val" matches[1] => "key" matches[2] => "val" local newstr,n,err = ngx.re.sub("Hello,lua","lua", "nginx") newstr = "Hello,nginx"
  • / 3024 • OpenResty • lua-resty-memcached github.com/agentzh/lua-resty-memcached • lua-resty-mysql github.com/agentzh/lua-resty-mysql • ngx.socketを使って実装されている Access to memcached/MySQL
  • / 3025 Nginx API in lua scripts • 簡単な処理を書くには十分な関数が 提供されている • 正規表現 • ハッシュ(md5, sha1) • ソケット • OpenRestyを使うとmemcachedへのアクセス とかも簡単
  • / 3026 Example code from pixiv Image Cluster
  • / 3027 Kyototycoon Logical Delete Nginx 200 404 /img01.png 200 /img02.png 200 /img03.png 404 /img04.png 200   GET /img01.png GET /img03.png 404
  • / 3028 Logical Delete local memcached = require "resty.memcached" local uri = ngx.var.request_uri local memc = memcached:new() local val, flags, err = memc:get(request_uri) if val and val ~= "200" then exit(tonumber(val)) end logical_delete.lua location / { access_by_lua_file logical_delete.lua; }
  • / 3029 Conclusion • Nginxだけで簡単な処理を実現できる • アクセス制御とかインプレッション計測 とかには便利そう • 高速で省メモリ • メンテナンスコストを抑えられる
  • / 3030 References HTTPLuaModule - Nginx Wiki http://wiki.nginx.org/HttpLuaModule chaoslawful/lua-nginx-module - Github https://github.com/chaoslawful/lua-nginx-module