Your SlideShare is downloading. ×
0
Inside Sqale’s Backend     RubyConf Taiwan 2012       Gosuke Miyashita   paperboy&co., Inc.,Japan
cpan:mizzygithub.com/mizzy     mizzy.org  @gosukenator
Kentaro KuribayashiLiving on the Edge Rails
Kensuke NagaeIntroducing nonopaste-cli
Shinya Tsunematsu     Building production serverenvironment of ruby in modern way
Hiroshi SHIBATAHow to discover the Rubys defects      with web application
Gosuke MiyashitaInside Sqale’s Backend
We are frompaperboy&co., Japan
What is Sqale?
Cloud ApplicationPlatform like Heroku
Architecture Overview
SFTP       Git over SSH         HTTP/HTTPSAWS    SSH                             Web Proxy      SSH Router                ...
Containers
SFTP       Git over SSH         HTTP/HTTPSAWS    SSH                             Web Proxy      SSH Router                ...
Virtual Environments  Assigned To Users
Similar to Dynos of      Heroku
Containers made byLXC (Linux Containers)
EC2 Instance (1 Virtual Machine)Container   Container   Container   Container   Container   for         for         for   ...
Nginx     Unicorn      sshd   supervisrodon each container
Amazon Linux          +Patched kernel(3.2.16)
grsecurity kernel patchfor various restrictions
original kernel patches  to restrict tcp port bind and fork bomb
Anti fork bomb patchmakes some changes tocgroup and fork process
Seepaperboy-sqale/sqale-patches         on GitHub
Web Proxy
SFTP       Git over SSH         HTTP/HTTPSAWS    SSH                             Web Proxy      SSH Router                ...
HTTP/HTTPS                               ELB                nginx                           nginxContainer   Container   C...
nginx  lua-nginx-moduleredis2-nginx-module
http://www.i4pc.jp/                        Which containers?           Redis                                      nginx   ...
nginx.conf (excerpt)location / {    set $container "";    set $next_containers "";    error_page 502 = @failover;    rewri...
dynamic-proxy.lua (excerpt)local reply = ngx.location.capture("/redis")if reply.status ~= ngx.HTTP_OK then  ngx.exit(503)e...
dynamic-proxy.lua (excerpt)while #containers > 0 do  tmp = table.remove(    containers,    math.random(#containers))  if n...
dynamic-proxy.lua (excerpt)ngx.var.container = containerngx.var.next_containers  = luabins.save(containers)
nginx.conf (again)location / {    set $container "";    set $next_containers "";    error_page 502 = @failover;    rewrite...
nginx.conf (excerpt)location @failover {    error_page 502 = @failover;    rewrite_by_lua_file failover.lua;    proxy_pass...
failover.lua (excerpt)local downed_container = ngx.var.containerif downed_container then  ngx.shared.downed_containers:set...
failover.lua (excerpt)while #containers > 0 do  tmp = table.remove(    containers,    math.random(#containers))  if ngx.sh...
failover.lua (excerpt)if not container then  ngx.exit(503)endngx.var.container = containerngx.var.next_containers  = luabi...
nginx.conf (agin)location @failover {    error_page 502 = @failover;    rewrite_by_lua_file failover.lua;    proxy_pass ht...
SSH Router
SFTP       Git over SSH         HTTP/HTTPSAWS    SSH                             Web Proxy      SSH Router                ...
Git           SFTP            SSH Login               SSH Router    File            FileRepositories   Repositories       ...
How implement this     routing?
OpenSSH with scriptauthentication patch
Seemizzy/openssh-script-auth       on GitHub
Change routes bySSH_ORIGNAL_COMMAND
In case ofSSH_ORIGINAL_COMMAND        is “git-*”
git push        (ssh sqale@gateway.sqale.jp git-recieve-pack        ‘/mizzy/lokka.git’)                Run AuthorizedKeys ...
In case ofSSH_ORIGINAL_COMMAND     is “sftp-server”
sftp sqale@gateway.sqale.jp        (ssh sqale@gateway.sqale.jp sftp-server)                Run AuthorizedKeys             ...
In case ofSSH_ORIGINAL_COMMAND        is empty
ssh sqale@gateway.sqale.jp             Run AuthorizedKeys             ScriptSSH Router                                 MyS...
Ruby On Rails Puppet/Chef   Unicorn    ResqueAnd many gems
多謝
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Inside Sqale's Backend at RubyConf Taiwan 2012
Upcoming SlideShare
Loading in...5
×

Inside Sqale's Backend at RubyConf Taiwan 2012

3,107

Published on

Published in: Technology

Transcript of "Inside Sqale's Backend at RubyConf Taiwan 2012"

  1. 1. Inside Sqale’s Backend RubyConf Taiwan 2012 Gosuke Miyashita paperboy&co., Inc.,Japan
  2. 2. cpan:mizzygithub.com/mizzy mizzy.org @gosukenator
  3. 3. Kentaro KuribayashiLiving on the Edge Rails
  4. 4. Kensuke NagaeIntroducing nonopaste-cli
  5. 5. Shinya Tsunematsu Building production serverenvironment of ruby in modern way
  6. 6. Hiroshi SHIBATAHow to discover the Rubys defects with web application
  7. 7. Gosuke MiyashitaInside Sqale’s Backend
  8. 8. We are frompaperboy&co., Japan
  9. 9. What is Sqale?
  10. 10. Cloud ApplicationPlatform like Heroku
  11. 11. Architecture Overview
  12. 12. SFTP Git over SSH HTTP/HTTPSAWS SSH Web Proxy SSH Router to Containers File Deploy Containers Repositories Servers
  13. 13. Containers
  14. 14. SFTP Git over SSH HTTP/HTTPSAWS SSH Web Proxy SSH Router to Containers File Deploy Containers Repositories Servers
  15. 15. Virtual Environments Assigned To Users
  16. 16. Similar to Dynos of Heroku
  17. 17. Containers made byLXC (Linux Containers)
  18. 18. EC2 Instance (1 Virtual Machine)Container Container Container Container Container for for for for for user A user A user B user B user BContainer Container Container Container Container for for for for for user C user D user D user E user EContainer Container Container Container Container for for for for for user E user F user F user F user F
  19. 19. Nginx Unicorn sshd supervisrodon each container
  20. 20. Amazon Linux +Patched kernel(3.2.16)
  21. 21. grsecurity kernel patchfor various restrictions
  22. 22. original kernel patches to restrict tcp port bind and fork bomb
  23. 23. Anti fork bomb patchmakes some changes tocgroup and fork process
  24. 24. Seepaperboy-sqale/sqale-patches on GitHub
  25. 25. Web Proxy
  26. 26. SFTP Git over SSH HTTP/HTTPSAWS SSH Web Proxy SSH Router to Containers File Deploy Containers Repositories Servers
  27. 27. HTTP/HTTPS ELB nginx nginxContainer Container Container Container Container Container for for for for for for user A user B user B user C user C user C
  28. 28. nginx lua-nginx-moduleredis2-nginx-module
  29. 29. http://www.i4pc.jp/ Which containers? Redis nginx host001:8083, host001:8084 orhost001 nginx port 8081 nginx port 8082 nginx port 8083 nginx port 8084 Container Container Container Container for for for for lokka-mizzy lokka-mizzy i4pc-mizzy i4pc-mizzy
  30. 30. nginx.conf (excerpt)location / { set $container ""; set $next_containers ""; error_page 502 = @failover; rewrite_by_lua_file dynamic-proxy.lua; proxy_pass http://$container;}
  31. 31. dynamic-proxy.lua (excerpt)local reply = ngx.location.capture("/redis")if reply.status ~= ngx.HTTP_OK then ngx.exit(503)endlocal containers, type = parser.parse_reply(reply.body)
  32. 32. dynamic-proxy.lua (excerpt)while #containers > 0 do tmp = table.remove( containers, math.random(#containers)) if ngx.shared.downed_containers:get(tmp) then ngx.log(ngx.DEBUG, tmp .. " is down") else container = tmp break endend
  33. 33. dynamic-proxy.lua (excerpt)ngx.var.container = containerngx.var.next_containers = luabins.save(containers)
  34. 34. nginx.conf (again)location / { set $container ""; set $next_containers ""; error_page 502 = @failover; rewrite_by_lua_file dynamic-proxy.lua; proxy_pass http://$container;}
  35. 35. nginx.conf (excerpt)location @failover { error_page 502 = @failover; rewrite_by_lua_file failover.lua; proxy_pass http://$container;}
  36. 36. failover.lua (excerpt)local downed_container = ngx.var.containerif downed_container then ngx.shared.downed_containers:set( downed_container, 1, sqale.NEGATIVE_CACHE_SECONDS )end
  37. 37. failover.lua (excerpt)while #containers > 0 do tmp = table.remove( containers, math.random(#containers)) if ngx.shared.downed_containers:get(tmp) then ngx.log(ngx.DEBUG, tmp .. " is down") else container = tmp break endend
  38. 38. failover.lua (excerpt)if not container then ngx.exit(503)endngx.var.container = containerngx.var.next_containers = luabins.save(containers)
  39. 39. nginx.conf (agin)location @failover { error_page 502 = @failover; rewrite_by_lua_file failover.lua; proxy_pass http://$container;}
  40. 40. SSH Router
  41. 41. SFTP Git over SSH HTTP/HTTPSAWS SSH Web Proxy SSH Router to Containers File Deploy Containers Repositories Servers
  42. 42. Git SFTP SSH Login SSH Router File FileRepositories Repositories Containers(Git Server) (File Server)
  43. 43. How implement this routing?
  44. 44. OpenSSH with scriptauthentication patch
  45. 45. Seemizzy/openssh-script-auth on GitHub
  46. 46. Change routes bySSH_ORIGNAL_COMMAND
  47. 47. In case ofSSH_ORIGINAL_COMMAND is “git-*”
  48. 48. git push (ssh sqale@gateway.sqale.jp git-recieve-pack ‘/mizzy/lokka.git’) Run AuthorizedKeys ScriptSSH Router MySQL Verify the public key and get the user’s git server command=“ssh sqale@git001.sqale.lan git-recieve-pack File ‘/var/repos/mizzy/lokka.git’”Repository(Git Server)
  49. 49. In case ofSSH_ORIGINAL_COMMAND is “sftp-server”
  50. 50. sftp sqale@gateway.sqale.jp (ssh sqale@gateway.sqale.jp sftp-server) Run AuthorizedKeys ScriptSSH Router MySQL Verify the public key and get the user’s file server command=“ssh sqale@file001.sqale.lan sftp-server” File git push File Repository Repository(File Server) (Git Server)
  51. 51. In case ofSSH_ORIGINAL_COMMAND is empty
  52. 52. ssh sqale@gateway.sqale.jp Run AuthorizedKeys ScriptSSH Router MySQL Verify the public key and get the user’s cotainers list Display the user’s containers list and wait the user’s selection command=“ssh sqale@Container users001.sqale.lan -p 8081”
  53. 53. Ruby On Rails Puppet/Chef Unicorn ResqueAnd many gems
  54. 54. 多謝
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×