How to create multiprocess server
on windows with ruby
Rubykaigi 2016

Ritta Narita
@naritta @narittan
Treasure Data, Inc.

Treasure Data Service Architecture

Treasure Data Service Architecture
As one of solutions for importing data, we use fluentd

Fluentd is an open source data collector for unified logging layer.

For example…
<source>
@type http
port 8000
bind 0.0.0.0
</source>
in_http out_stdout
<match pattern>
@type stdout
</match>
http:8000

in_http → out_stdout

For example…
<source>
@type forward
port 24224
bind 0.0.0.0
</source>
in_forward out_stdout
<match pattern>
@type stdout
</match>
24224
fluent-cat:
send json to the port

in_forward → out_stdout

fluentd uses multiprocess server framework
github.com/fluent/serverengine
ServerEngine

What’s ServerEngine?
A framework to implement robust multiprocess servers like Unicorn
Dynamic reconfig
live restart
Heartbeat via pipe
auto restart
serversupervisor
worker
worker
worker

Auto Restart
Heartbeat via pipe
auto restart
serversupervisor
worker
worker
worker

Heartbeat via pipe
auto restart
serversupervisor
worker
worker
worker
if worker died unexpectedly…

It’s detected by heartbeat and server launch worker again.
Heartbeat via pipe
auto restart
serversupervisor
worker
worker
worker
if worker died unexpectedly…

How to use ServerEngine
1. write server module and worker module
serversupervisor worker
module MyServer
def before_run
@sock = TCPServer.new(
config[:bind],
config[:port])
end
attr_reader :sock
end
module MyWorker
def run
c = server.sock.accept
c.write "work"
end
def stop
c.close
end
end

2. write configuration and run
How to use ServerEngine
serversupervisor worker
se = ServerEngine.create(MyServer, MyWorker, {
daemonize: true,
log: 'myserver.log',
pid_path: 'myserver.pid',
worker_type: 'process',
workers: 4,
bind: '0.0.0.0',
port: 9071,
})
se.run

Worker type
thread
spawnprocess

server worker
server worker
thread
process
Worker type
ServerEngine.create(MyServer, MyWorker, {
worker_type: 'thread'
})
ServerEngine.create(MyServer, MyWorker, {
worker_type: 'process'
})

server worker
spawn
Worker type
module MyWorker
def spawn(process_manager)
process_manager.spawn(env, config[:spawn_cmd])
end
end
ServerEngine.create(MyServer, MyWorker, {
worker_type: 'spawn',
spawn_cmd: cmd
})

in_multiprocess plugin
you have to use multi sockets and assign each ports
port:
24224
port:
24226
port:
24225server
worker
worker
worker
conventional multiprocess model on fluentd

Multiprocess model with ServerEngine
port:
24224
using Socket Manager, share one listening socket
→can use multicore without any assignment
port:
24224
port:
24224
server
worker
socket
manager
server
socket
manager
client
worker
socket
manager
client
worker
socket
manager
client

What differs from other prefork server like unicorn or nginx

Ordinal prefork model
can know which port before fork
only have to pass the socket when fork
server workersocket

In ServerEngine,
For example, in fluentd
Assuming that you can know which port after fork
port information is written in config,
it parses config in worker after fork
port
information
server worker

serverengine prefork model
After fork, have to send request from worker firstly,
As a response, send socket to worker
request
socket
server worker

server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
http
How do you create multiprocess server for windows with ruby?
I’ll introduce ruby function and tips to develop for windows.

server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
http
To create worker

To create worker
There is no fork in Windows, then I used spawn
executes specified command and return its pid
pid = spawn(RbConfig.ruby, "-e puts 'Hello, world!'")
Process.wait pid

Create Daemon with spawn
serversupervisor
spawn(config[:daemon_cmdline])

serversupervisor
ruby daemon.rb
execute script for daemon from application side
:
ServerEngine::Daemon.run_server(
:
spawn(config[:daemon_cmdline])
Create Daemon with spawn

server worker
socket request
socket
manager
server
socket
manager
client
socket socket
create worker
socket
http
How to send a socket

There is a problem about File descriptor

In unix
it can share file descriptor for other process.
server worker
FD FD
FD

In Windows
FD is independent for processes.
you cannot share file descriptor
server worker
FD FD
FD

Then, How do you send a socket in windows?

process
pid:1000
process
pid:2000
Windows API : WSADuplicatesocket
you can get duplicated socket only for assigned PID
socket
dup socket
for 2000
socket
for 2000
pass

But this API is only for C
Almost all of Windows API is for C.
It’s important to call API from Others.

How do you call this API from Ruby?

1. C extension
The most general way
There is no C extension code in ServerEngine.
If possible, I want to write with only ruby.
void Init_test()
{
VALUE module;
module = rb_define_module("Test");
rb_define_module_function(module, "hello", wrap_hello,
0);
}

2. ffi
Foreign function interface (FFI) for Ruby
require 'ffi'
module MyLib
extend FFI::Library
ffi_lib 'c'
attach_function :puts, [ :string ], :int
end
MyLib.puts 'Hello, World using libc!’

2. ffi
typedef :ulong, :dword
typedef :uintptr_t, :socket
typedef :pointer, :ptr
ffi_lib :ws2_32
attach_function :WSADuplicateSocketA,
[:socket, :dword, :ptr], :int
class WSAPROTOCOL_INFO < FFI::Struct
layout(
:dwServiceFlags1, :dword,
)
end
proto = WinSock::WSAPROTOCOL_INFO.new
WSADuplicateSocketA(sock, pid, proto)
Definition
Use

This will increase gem dependency.
I want to reduce dependency as possible.
2. ffi

3. fiddle
This is built-in library, then you can call without gem dependency
require "fiddle/import"
module M
extend Fiddle::Importer
dlload "libc.so.6","libm.so.6"
extern "int strlen(char*)"
end
p M.strlen('abc') #=> 3
the library to deal with shared library

how to use fiddle
you can just define using extern and c sentence.
extern "int WSADuplicateSocketA(int, DWORD, void *)"
WSADuplicateSocketA(sock.handle, pid, proto)
Definition
Use

how to use fiddle
you can get protocol of duplicated socket
extern "int WSADuplicateSocketA(int, DWORD, void *)"
WSADuplicateSocketA(sock.handle, pid, proto)
Definition
Use

how to use fiddle
extern "int WSADuplicateSocketA(int, DWORD, void *)"
proto = WSAPROTOCOL_INFO.malloc
WSADuplicateSocketA(sock.handle, pid, proto)
Definition
Use

WSAPROTOCOL_INFO = struct([
"DWORD dwServiceFlags1",
"DWORD dwServiceFlags2”,
・・・
])
proto = WSAPROTOCOL_INFO.malloc
you can use struct
you can get struct pointer by calling malloc

server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
http
To share this protocol with worker

worker
server
bin = proto.to_ptr.to_s
proto = WSAPROTOCOL_INFO.malloc
proto.to_ptr.ref.ptr[0, size] = bin
send binary
How to share this protocol with worker
protocol → binary
binary → protocol

server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
http
To create socket from sent information

Get socket from protocol
extern "int WSASocketA(int, int, int, void *, int, DWORD)”
socket = WSASocketA(Socket::AF_INET, Socket::SOCK_STREAM, 0, proto, 0, 1)
Definition
Use

This socket WSASocket is handle
what’s handle?
Handle is like FD, but we can’t duplicate handle like FD.
For UNIX compatibility,
There is function to convert handle to FD in Windows CRT

This handle is C layer
we can’t useTCPServer class
it’s inconvenient!
we have to use ffi for everything
like WSASend or WSAReceive for write or read
I want to deal with socket in ruby layer

Ruby API: rb_w32_wrap_io_handle
It can convert handle to FD
socklist
sock
ioinfo
sock
ioinfo
sock
ioinfo
handle
you can deal with socket in ruby layer
handle
fd(7)
wrap _open_osfhandle(assign FD)
and create io object
rb_w32_wrap_io_handle

Use fiddle to call rb_w32_wrap_io_handle
ruby_dll_paths = File.dirname(ruby_bin_path) + '/*msvcr*ruby*.dll'
ruby_dll_path = Dir.glob(ruby_dll_paths).first
dlload ruby_dll_path
extern "int rb_w32_wrap_io_handle(int, int)"
you can call with loading ruby dll

you can deal with socket in Ruby layer

server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
http
To send request/response

There is no UDS in windows, then I used drb as first idea
server
worker
worker
drb
drb
drb
socket
method call
method
create socket
socket
send_io
receive_io

This has race condition problem
server
workerA
workerB
drb
drb
drbmethod call
create socket for workerA
socketA
send_io

This has race condition problem
server
workerA
workerB
drb
drb
drb
create socket for workerB
socketA
send_io
method call
socketB
send_io

This has race condition problem
server
workerA
workerB
drb
drb
drb
create socket for workerB
socketB
send_io
socketAcall receive_io

This has race condition problem
server
workerA
workerB
drb
drb
drb
create socket for workerB
socketBcall receive_io
This rarely happens but its impact is significant if it happens.

server
worker
worker
uds
uds
uds
socket
request
Against this problem, I used UDS andTCPServer
Create several UDS for each worker
uds

To send request/result
In Unix, can use Unix domain socket
There is no UDS in windows,
Then I usedTCPServer
for port in 10000..65535
if `netstat -na | find "#{port}"`.length == 0
return port
end
end
Use netstat to find unused port
If there is NamedPipe in ruby, I’ll use it.

server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
http
Then you can share socket with worker,
it can listen with multiple worker

server worker
socket request
socket
manager
server
socket
manager
client
socket socket information
create worker
socket
http
To deal with http request

There is no read_nonblock in windows.
Used readpartial

read_nonblock
readpartial
with O_NONBLOCK
raise an exception if there is no data
block if there is no data
without O_NONBLOCK
difference between read_nonblock and readpartial

readpartial(n)
read
difference between read and readpartial
read less than 2048 bytes
read less than n bytes
return nil if EOF
raise an exception if EOF

There is no thundering herd problem on Windows?
But I implemented accept mutex at first
As a result of benchmark test,
I found that thundering herd doesn’t occur.
worker
worker
request
accept!
accept!

Accept Mutex
get
mutex
accept
mutex between processes
read and send data
to buffer/output
server socket
get
mutex
accept
read and send data
to buffer/output
deal with post processing
other process can listen while this process is dealing with data
detach
release
mutex
attach
listening socket
to event loop
detach
release
mutex
attach
listening socket
to event loop
worker
worker

rotation in order by accept mutex

Better way using IOCP?
completion port
thread threadthread
socket socketsocket
CreateIoCompletionPort
Using WindowAPI for IOCP(I/O Completion Port),
can implement asynchronous IO in Windows.

Better way using IOCP?
completion port
thread threadthread
socket socketsocket
GetQueued
CompletionStatus
GetQueued
CompletionStatus
listen
accept
create worker thread pool and IOCP manage it

Better way using IOCP?
completion port
thread threadthread
socket socketsocket
GetQueued
CompletionStatus
GetQueued
CompletionStatus
listen
GetQueuedCompletionStatus
accept

Better way using IOCP?
・Create IOCP in Server process and share socket?
CreateIoCompletionPort
server worker
socket socket
・Or create it in only Worker process?
CreateIoCompletionPort

Benchmark result
AWS Microsoft Windows Server 2012 R2 m4.xlarge
RPS IO
conventional
model
1834.01
/sec
385.07
kb/s
new model
(4 workers)
3513.31
/sec
737.66
kb/s
in_http → out_forward

Conclusion
We need to care about some problems using windows API.
There is very helpful and niche function to deal with
the difference between OS in ruby.