Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
How	 to	 create	 multiprocess	 server	 	 
on	 windows	 with	 ruby
Rubykaigi	 2016
Ritta Narita
@naritta @narittan
Treasure Data, Inc.
Treasure Data Service Architecture
Treasure Data Service Architecture
As one of solutions for importing data, we use fluentd
Fluentd is an open source data collector for unified logging layer.
For example…
<source>
@type http
port 8000
bind 0.0.0.0
</source>
in_http out_stdout
<match pattern>
@type stdout
</match>...
in_http → out_stdout
For example…
<source>
@type forward
port 24224
bind 0.0.0.0
</source>
in_forward out_stdout
<match pattern>
@type stdout
<...
in_forward → out_stdout
fluentd uses multiprocess server framework
github.com/fluent/serverengine
ServerEngine
What’s ServerEngine?
A framework to implement robust multiprocess servers like Unicorn
Dynamic reconfig

live restart
Heart...
Auto Restart
Heartbeat via pipe
auto restart
serversupervisor
worker
worker
worker
Heartbeat via pipe
auto restart
serversupervisor
worker
worker
worker
if worker died unexpectedly…
It’s detected by heartbeat and server launch worker again.
Heartbeat via pipe
auto restart
serversupervisor
worker
worker
...
How to use ServerEngine
1. write server module and worker module
serversupervisor worker
module MyServer
def before_run
@s...
2. write configuration and run
How to use ServerEngine
serversupervisor worker
se = ServerEngine.create(MyServer, MyWorker,...
Worker type
thread
spawnprocess
server worker
server worker
thread
process
Worker type
ServerEngine.create(MyServer, MyWorker, {
worker_type: 'thread'
})
...
server worker
spawn
Worker type
module MyWorker
def spawn(process_manager)
process_manager.spawn(env, config[:spawn_cmd])
...
in_multiprocess plugin
you have to use multi sockets and assign each ports
port:
24224
port:
24226
port:
24225server
worke...
Multiprocess model with ServerEngine
port:
24224
using Socket Manager, share one listening socket
→can use multicore witho...
What differs from other prefork server like unicorn or nginx
Ordinal prefork model
can know which port before fork
only have to pass the socket when fork
server workersocket
In ServerEngine,
For example, in fluentd
Assuming that you can know which port after fork
port information is written in co...
serverengine prefork model
After fork, have to send request from worker firstly,
As a response, send socket to worker
reque...
server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
ht...
server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
ht...
To create worker
There is no fork in Windows, then I used spawn
executes specified command and return its pid
pid = spawn(R...
Create Daemon with spawn
serversupervisor
spawn(config[:daemon_cmdline])
serversupervisor
ruby daemon.rb
execute script for daemon from application side
:
ServerEngine::Daemon.run_server(
:
spawn...
server worker
socket request
socket
manager
server
socket
manager
client
socket socket
create worker
socket
http
How to se...
There is a problem about File descriptor
In unix
it can share file descriptor for other process.
server worker
FD FD
FD
In Windows
FD is independent for processes.
you cannot share file descriptor
server worker
FD FD
FD
Then, How do you send a socket in windows?
process
pid:1000
process
pid:2000
Windows API : WSADuplicatesocket
you can get duplicated socket only for assigned PID
soc...
But this API is only for C
Almost all of Windows API is for C.
It’s important to call API from Others.
How do you call this API from Ruby?
1. C extension
The most general way
There is no C extension code in ServerEngine.
If possible, I want to write with only r...
2. ffi
Foreign function interface (FFI) for Ruby
require 'ffi'
module MyLib
extend FFI::Library
ffi_lib 'c'
attach_function...
2. ffi
typedef :ulong, :dword
typedef :uintptr_t, :socket
typedef :pointer, :ptr
ffi_lib :ws2_32
attach_function :WSADuplic...
This will increase gem dependency.
I want to reduce dependency as possible.
2. ffi
3. fiddle
This is built-in library, then you can call without gem dependency
require "fiddle/import"
module M
extend Fiddle...
how to use fiddle
you can just define using extern and c sentence.
extern "int WSADuplicateSocketA(int, DWORD, void *)"
WSAD...
how to use fiddle
you can get protocol of duplicated socket
extern "int WSADuplicateSocketA(int, DWORD, void *)"
WSADuplica...
how to use fiddle
extern "int WSADuplicateSocketA(int, DWORD, void *)"
proto = WSAPROTOCOL_INFO.malloc
WSADuplicateSocketA(...
WSAPROTOCOL_INFO = struct([
         "DWORD dwServiceFlags1",
"DWORD dwServiceFlags2”,
・・・
])
proto = WSAPROTOCOL_INFO.mal...
server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
ht...
worker
server
bin = proto.to_ptr.to_s
proto = WSAPROTOCOL_INFO.malloc
proto.to_ptr.ref.ptr[0, size] = bin
send binary
How ...
server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
ht...
Get socket from protocol
extern "int WSASocketA(int, int, int, void *, int, DWORD)”
socket = WSASocketA(Socket::AF_INET, S...
This socket WSASocket is handle
what’s handle?
Handle is like FD, but we can’t duplicate handle like FD.
For UNIX compatib...
This handle is C layer
we can’t useTCPServer class
it’s inconvenient!
we have to use ffi for everything
like WSASend or WSA...
Ruby API: rb_w32_wrap_io_handle
It can convert handle to FD
socklist
sock
ioinfo
sock
ioinfo
sock
ioinfo
handle
you can de...
Use fiddle to call rb_w32_wrap_io_handle
ruby_dll_paths = File.dirname(ruby_bin_path) + '/*msvcr*ruby*.dll'
ruby_dll_path =...
you can deal with socket in Ruby layer
server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
ht...
There is no UDS in windows, then I used drb as first idea
server
worker
worker
drb
drb
drb
socket
method call
method
create...
This has race condition problem
server
workerA
workerB
drb
drb
drbmethod call
create socket for workerA
socketA
send_io
This has race condition problem
server
workerA
workerB
drb
drb
drb
create socket for workerB
socketA
send_io
method call
s...
This has race condition problem
server
workerA
workerB
drb
drb
drb
create socket for workerB
socketB
send_io
socketAcall r...
This has race condition problem
server
workerA
workerB
drb
drb
drb
create socket for workerB
socketBcall receive_io
This r...
server
worker
worker
uds
uds
uds
socket
request
Against this problem, I used UDS andTCPServer
Create several UDS for each ...
To send request/result
In Unix, can use Unix domain socket
There is no UDS in windows,
Then I usedTCPServer
for port in 10...
server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
ht...
server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
ht...
There is no read_nonblock in windows.
Used readpartial
read_nonblock
readpartial
with O_NONBLOCK
raise an exception if there is no data
block if there is no data
without O_NONBL...
readpartial(n)
read
difference between read and readpartial
read less than 2048 bytes
read less than n bytes
return nil if...
There is no thundering herd problem on Windows?
But I implemented accept mutex at first
As a result of benchmark test,
I fo...
Accept Mutex
get
mutex
accept
mutex between processes
read and send data
to buffer/output
server socket
get
mutex
accept
r...
rotation in order by accept mutex
Better way using IOCP?
completion port
thread threadthread
socket socketsocket
CreateIoCompletionPort
Using WindowAPI for ...
Better way using IOCP?
completion port
thread threadthread
socket socketsocket
GetQueued
CompletionStatus
GetQueued
Comple...
Better way using IOCP?
completion port
thread threadthread
socket socketsocket
GetQueued
CompletionStatus
GetQueued
Comple...
Better way using IOCP?
・Create IOCP in Server process and share socket?
CreateIoCompletionPort
server worker
socket socket...
Benchmark result
AWS Microsoft Windows Server 2012 R2 m4.xlarge
RPS IO
conventional
model
1834.01
/sec
385.07
kb/s
new mod...
Conclusion
We need to care about some problems using windows API.
There is very helpful and niche function to deal with
th...
How to create multiprocess server on windows with ruby - rubykaigi2016 Ritta Narita-
How to create multiprocess server on windows with ruby - rubykaigi2016 Ritta Narita-
How to create multiprocess server on windows with ruby - rubykaigi2016 Ritta Narita-
Upcoming SlideShare
Loading in …5
×

How to create multiprocess server on windows with ruby - rubykaigi2016 Ritta Narita-

269 views

Published on

rubykaigi2016 Ritta Narita

Published in: Software
  • Be the first to comment

  • Be the first to like this

How to create multiprocess server on windows with ruby - rubykaigi2016 Ritta Narita-

  1. 1. How to create multiprocess server on windows with ruby Rubykaigi 2016
  2. 2. Ritta Narita @naritta @narittan Treasure Data, Inc.
  3. 3. Treasure Data Service Architecture
  4. 4. Treasure Data Service Architecture As one of solutions for importing data, we use fluentd
  5. 5. Fluentd is an open source data collector for unified logging layer.
  6. 6. For example… <source> @type http port 8000 bind 0.0.0.0 </source> in_http out_stdout <match pattern> @type stdout </match> http:8000
  7. 7. in_http → out_stdout
  8. 8. For example… <source> @type forward port 24224 bind 0.0.0.0 </source> in_forward out_stdout <match pattern> @type stdout </match> 24224 fluent-cat: send json to the port
  9. 9. in_forward → out_stdout
  10. 10. fluentd uses multiprocess server framework github.com/fluent/serverengine ServerEngine
  11. 11. What’s ServerEngine? A framework to implement robust multiprocess servers like Unicorn Dynamic reconfig
 live restart Heartbeat via pipe auto restart serversupervisor worker worker worker
  12. 12. Auto Restart Heartbeat via pipe auto restart serversupervisor worker worker worker
  13. 13. Heartbeat via pipe auto restart serversupervisor worker worker worker if worker died unexpectedly…
  14. 14. It’s detected by heartbeat and server launch worker again. Heartbeat via pipe auto restart serversupervisor worker worker worker if worker died unexpectedly…
  15. 15. How to use ServerEngine 1. write server module and worker module serversupervisor worker module MyServer def before_run @sock = TCPServer.new( config[:bind], config[:port]) end attr_reader :sock end module MyWorker def run c = server.sock.accept c.write "work" end def stop c.close end end
  16. 16. 2. write configuration and run How to use ServerEngine serversupervisor worker se = ServerEngine.create(MyServer, MyWorker, { daemonize: true, log: 'myserver.log', pid_path: 'myserver.pid', worker_type: 'process', workers: 4, bind: '0.0.0.0', port: 9071, }) se.run
  17. 17. Worker type thread spawnprocess
  18. 18. server worker server worker thread process Worker type ServerEngine.create(MyServer, MyWorker, { worker_type: 'thread' }) ServerEngine.create(MyServer, MyWorker, { worker_type: 'process' })
  19. 19. server worker spawn Worker type module MyWorker def spawn(process_manager) process_manager.spawn(env, config[:spawn_cmd]) end end ServerEngine.create(MyServer, MyWorker, { worker_type: 'spawn', spawn_cmd: cmd })
  20. 20. in_multiprocess plugin you have to use multi sockets and assign each ports port: 24224 port: 24226 port: 24225server worker worker worker conventional multiprocess model on fluentd
  21. 21. Multiprocess model with ServerEngine port: 24224 using Socket Manager, share one listening socket →can use multicore without any assignment port: 24224 port: 24224 server worker socket manager server socket manager client worker socket manager client worker socket manager client
  22. 22. What differs from other prefork server like unicorn or nginx
  23. 23. Ordinal prefork model can know which port before fork only have to pass the socket when fork server workersocket
  24. 24. In ServerEngine, For example, in fluentd Assuming that you can know which port after fork port information is written in config, it parses config in worker after fork port information server worker
  25. 25. serverengine prefork model After fork, have to send request from worker firstly, As a response, send socket to worker request socket server worker
  26. 26. server worker socket request socket manager server socket manager client socket socket information create worker socket http How do you create multiprocess server for windows with ruby? I’ll introduce ruby function and tips to develop for windows.
  27. 27. server worker socket request socket manager server socket manager client socket socket information create worker socket http To create worker
  28. 28. To create worker There is no fork in Windows, then I used spawn executes specified command and return its pid pid = spawn(RbConfig.ruby, "-e puts 'Hello, world!'") Process.wait pid
  29. 29. Create Daemon with spawn serversupervisor spawn(config[:daemon_cmdline])
  30. 30. serversupervisor ruby daemon.rb execute script for daemon from application side : ServerEngine::Daemon.run_server( : spawn(config[:daemon_cmdline]) Create Daemon with spawn
  31. 31. server worker socket request socket manager server socket manager client socket socket create worker socket http How to send a socket
  32. 32. There is a problem about File descriptor
  33. 33. In unix it can share file descriptor for other process. server worker FD FD FD
  34. 34. In Windows FD is independent for processes. you cannot share file descriptor server worker FD FD FD
  35. 35. Then, How do you send a socket in windows?
  36. 36. process pid:1000 process pid:2000 Windows API : WSADuplicatesocket you can get duplicated socket only for assigned PID socket dup socket for 2000 socket for 2000 pass
  37. 37. But this API is only for C Almost all of Windows API is for C. It’s important to call API from Others.
  38. 38. How do you call this API from Ruby?
  39. 39. 1. C extension The most general way There is no C extension code in ServerEngine. If possible, I want to write with only ruby. void Init_test() { VALUE module; module = rb_define_module("Test"); rb_define_module_function(module, "hello", wrap_hello, 0); }
  40. 40. 2. ffi Foreign function interface (FFI) for Ruby require 'ffi' module MyLib extend FFI::Library ffi_lib 'c' attach_function :puts, [ :string ], :int end MyLib.puts 'Hello, World using libc!’
  41. 41. 2. ffi typedef :ulong, :dword typedef :uintptr_t, :socket typedef :pointer, :ptr ffi_lib :ws2_32 attach_function :WSADuplicateSocketA, [:socket, :dword, :ptr], :int class WSAPROTOCOL_INFO < FFI::Struct layout( :dwServiceFlags1, :dword, ) end proto = WinSock::WSAPROTOCOL_INFO.new WSADuplicateSocketA(sock, pid, proto) Definition Use
  42. 42. This will increase gem dependency. I want to reduce dependency as possible. 2. ffi
  43. 43. 3. fiddle This is built-in library, then you can call without gem dependency require "fiddle/import" module M extend Fiddle::Importer dlload "libc.so.6","libm.so.6" extern "int strlen(char*)" end p M.strlen('abc') #=> 3 the library to deal with shared library
  44. 44. how to use fiddle you can just define using extern and c sentence. extern "int WSADuplicateSocketA(int, DWORD, void *)" WSADuplicateSocketA(sock.handle, pid, proto) Definition Use
  45. 45. how to use fiddle you can get protocol of duplicated socket extern "int WSADuplicateSocketA(int, DWORD, void *)" WSADuplicateSocketA(sock.handle, pid, proto) Definition Use
  46. 46. how to use fiddle extern "int WSADuplicateSocketA(int, DWORD, void *)" proto = WSAPROTOCOL_INFO.malloc WSADuplicateSocketA(sock.handle, pid, proto) Definition Use
  47. 47. WSAPROTOCOL_INFO = struct([          "DWORD dwServiceFlags1", "DWORD dwServiceFlags2”, ・・・ ]) proto = WSAPROTOCOL_INFO.malloc you can use struct you can get struct pointer by calling malloc
  48. 48. server worker socket request socket manager server socket manager client socket socket information create worker socket http To share this protocol with worker
  49. 49. worker server bin = proto.to_ptr.to_s proto = WSAPROTOCOL_INFO.malloc proto.to_ptr.ref.ptr[0, size] = bin send binary How to share this protocol with worker protocol → binary binary → protocol
  50. 50. server worker socket request socket manager server socket manager client socket socket information create worker socket http To create socket from sent information
  51. 51. Get socket from protocol extern "int WSASocketA(int, int, int, void *, int, DWORD)” socket = WSASocketA(Socket::AF_INET, Socket::SOCK_STREAM, 0, proto, 0, 1) Definition Use
  52. 52. This socket WSASocket is handle what’s handle? Handle is like FD, but we can’t duplicate handle like FD. For UNIX compatibility, There is function to convert handle to FD in Windows CRT
  53. 53. This handle is C layer we can’t useTCPServer class it’s inconvenient! we have to use ffi for everything like WSASend or WSAReceive for write or read I want to deal with socket in ruby layer
  54. 54. Ruby API: rb_w32_wrap_io_handle It can convert handle to FD socklist sock ioinfo sock ioinfo sock ioinfo handle you can deal with socket in ruby layer handle fd(7) wrap _open_osfhandle(assign FD) and create io object rb_w32_wrap_io_handle
  55. 55. Use fiddle to call rb_w32_wrap_io_handle ruby_dll_paths = File.dirname(ruby_bin_path) + '/*msvcr*ruby*.dll' ruby_dll_path = Dir.glob(ruby_dll_paths).first dlload ruby_dll_path extern "int rb_w32_wrap_io_handle(int, int)" you can call with loading ruby dll
  56. 56. you can deal with socket in Ruby layer
  57. 57. server worker socket request socket manager server socket manager client socket socket information create worker socket http To send request/response
  58. 58. There is no UDS in windows, then I used drb as first idea server worker worker drb drb drb socket method call method create socket socket send_io receive_io
  59. 59. This has race condition problem server workerA workerB drb drb drbmethod call create socket for workerA socketA send_io
  60. 60. This has race condition problem server workerA workerB drb drb drb create socket for workerB socketA send_io method call socketB send_io
  61. 61. This has race condition problem server workerA workerB drb drb drb create socket for workerB socketB send_io socketAcall receive_io
  62. 62. This has race condition problem server workerA workerB drb drb drb create socket for workerB socketBcall receive_io This rarely happens but its impact is significant if it happens.
  63. 63. server worker worker uds uds uds socket request Against this problem, I used UDS andTCPServer Create several UDS for each worker uds
  64. 64. To send request/result In Unix, can use Unix domain socket There is no UDS in windows, Then I usedTCPServer for port in 10000..65535 if `netstat -na | find "#{port}"`.length == 0 return port end end Use netstat to find unused port If there is NamedPipe in ruby, I’ll use it.
  65. 65. server worker socket request socket manager server socket manager client socket socket information create worker socket http Then you can share socket with worker, it can listen with multiple worker
  66. 66. server worker socket request socket manager server socket manager client socket socket information create worker socket http To deal with http request
  67. 67. There is no read_nonblock in windows. Used readpartial
  68. 68. read_nonblock readpartial with O_NONBLOCK raise an exception if there is no data block if there is no data without O_NONBLOCK difference between read_nonblock and readpartial
  69. 69. readpartial(n) read difference between read and readpartial read less than 2048 bytes read less than n bytes return nil if EOF raise an exception if EOF
  70. 70. There is no thundering herd problem on Windows? But I implemented accept mutex at first As a result of benchmark test, I found that thundering herd doesn’t occur. worker worker request accept! accept!
  71. 71. Accept Mutex get mutex accept mutex between processes read and send data to buffer/output server socket get mutex accept read and send data to buffer/output deal with post processing other process can listen while this process is dealing with data detach release mutex attach listening socket to event loop detach release mutex attach listening socket to event loop worker worker
  72. 72. rotation in order by accept mutex
  73. 73. Better way using IOCP? completion port thread threadthread socket socketsocket CreateIoCompletionPort Using WindowAPI for IOCP(I/O Completion Port), can implement asynchronous IO in Windows.
  74. 74. Better way using IOCP? completion port thread threadthread socket socketsocket GetQueued CompletionStatus GetQueued CompletionStatus listen accept create worker thread pool and IOCP manage it
  75. 75. Better way using IOCP? completion port thread threadthread socket socketsocket GetQueued CompletionStatus GetQueued CompletionStatus listen GetQueuedCompletionStatus accept
  76. 76. Better way using IOCP? ・Create IOCP in Server process and share socket? CreateIoCompletionPort server worker socket socket ・Or create it in only Worker process? CreateIoCompletionPort
  77. 77. Benchmark result AWS Microsoft Windows Server 2012 R2 m4.xlarge RPS IO conventional model 1834.01 /sec 385.07 kb/s new model (4 workers) 3513.31 /sec 737.66 kb/s in_http → out_forward
  78. 78. Conclusion We need to care about some problems using windows API. There is very helpful and niche function to deal with the difference between OS in ruby.

×