ServerEngine Integration & Windows support
Ritta Narita
Fluentd Meetup 2016

About me.
Ritta Narita
joined Treasure Data after internship in 2015
2
@naritta
@narittan

Fluentd v0.14
ServerEngine is integrated

What’s ServerEngine?
A framework to implement robust multiprocess servers like Unicorn
worker
worker
worker
super
visor
Server
Dynamic reconfig
live restart
Heartbeat via pipe
auto restart
multi
process/thread

how to use ServerEngine
worker
super
visor
Server
module MyServer
def before_run
@sock = TCPServer.new(
config[:bind], config[:port])
end
attr_reader :sock
end
module MyWorker
def run
c = server.sock.accept
c.write "Awesome work!"
end
def stop
c.close
end
end
1. write server module and worker module

how to use ServerEngine
worker
super
visor
Server
se = ServerEngine.create(MyServer, MyWorker, {
daemonize: true,
log: 'myserver.log',
pid_path: 'myserver.pid',
worker_type: 'process',
workers: 4,
bind: '0.0.0.0',
port: 9071,
})
se.run
2. write configuration and run

module MyWorker
def spawn(process_manager)
process_manager.spawn(env, config[:spawn_cmd])
end
end
ServerEngine.create(MyServer, MyWorker, {
worker_type:‘spawn’,
spawn_cmd: cmd
})
ServerEngine.create(MyServer, MyWorker, {
worker_type:‘thread'
})
workerServer
workerServer
workerServer
spawn
thread
processServerEngine.create(MyServer, MyWorker, {
worker_type:‘process'
})
worker type

What’s the advantages for integrating serverengine into fluentd?

serverengine advantages
Auto restart
live restart
socket manager
signal handler
log rotation

Auto restart
live restart
socket manager
signal handler
log rotation
serverengine advantages

if worker died unexpectedly…
worker
worker
super
visor
Server
worker

if worker died unexpectedly…
worker
worker
worker
super
visor
Server

if worker died unexpectedly…
worker
worker
super
visor
Server
It’s detected by heartbeat and server launch worker again.
worker

Auto restart
live restart
socket manager
signal handler
log rotation
serverengine advantages

worker
super
visor
Server
tcp tcp
send SIGHUP
live restart

super
visor
Server worker
tcp
kill the worker, leaving server’s condition
live restart

worker
super
visor
Server worker
tcp
start new worker and read config again
live restart

worker
super
visor
Server worker
tcp
tcp
share the TCP with living zero downtime
live restart

Signal and RPC server
api/config.reloadlive restart/config reload HUP
/api/processes
.interruptWorkers
INT
/api/processes
.killWorkers
/api/plugins.
flushBuffers
system config reload USR2
TERM
USR1flush buffers
shutdown

worker
super
visor
Server
you can reload system config without restart
system config reload
send USR2
check mtime and previous load time,
if there is new one, reload!
serverengine
config cash
new config

Auto restart
live restart
socket manager
signal handler
log rotation
serverengine advantages

Socket Manager
worker
Socket
Manager
client
Socket
Manager
server
Server
26
①socket listen request②find or create listening socket
③share the socket FD(unix)/
duplicated socket info(windows)
UDS(unix)/
TCP(windows)
④get socket and accept in worker

how to use Socket Manager
worker
Socket
Manager
client
Socket
Manager
server
Server
27
#generate path for UDS/TCP for server connection
socket_manager_path =
ServerEngine::SocketManager::Server.generate_path
#open socket manager server
socket_manager_server =
ServerEngine::SocketManager::Server.open(socket_manager_path)
In server

worker
Socket
Manager
client
Socket
Manager
server
Server
28
#get socket manager client from socket manager path
socket_manager =
ServerEngine::SocketManager::Client.new(socket_manager_path)
#get listening socket from socket manager
lsock = socket_manager.listen_tcp(bind, port)
In worker
how to use Socket Manager

What’s the merit of Socket Manager?
worker
worker
worker
Socket
Manager
client
Socket
Manager
client
Socket
Manager
client
Socket
Manager
server
super
visor
Server
29
we can use multicore easily in future

easy to use multicore power fully
30
with SocketManagerwith in_multiprocess plugin
<source>
type multiprocess
<process>
cmdline -c /etc/td-agent/td-agent-child1.conf
</process>
<process>
cmdline -c /etc/td-agent/td-agent-child2.conf
</process>
</source>
#/etc/td-agent/td-agent-child1.conf
<source>
type forward
port 24224
</source>
#/etc/td-agent/td-agent-child2.conf
<source>
type forward
port 24225
</source>
<source>
type forward
port 24224
</source>
setting when using 2 core

Auto restart
live restart
socket manager
signal handler
log rotation
serverengine advantages

Queue based signal handling
When user sent USR1 and QUIT…
signal handler thread
user
USR1
send USR1

Queue based signal handling
When user sent USR1 and QUIT…
signal handler thread
user
USR1
USR1

Queue based signal handling
When user sent USR1 and QUIT…
signal handler thread
user
QUIT
USR1
send QUIT

Queue based signal handling
When user sent USR1 and QUIT…
signal handler thread
user
USR1
QUIT

Queue based signal handling
When user sent USR1 and QUIT…
signal handler thread
user
USR1
QUIT
run!

Queue based signal handling
When user sent USR1 and QUIT…
signal handler thread
user
QUIT

Queue based signal handling
When user sent USR1 and QUIT…
signal handler thread
user
QUIT run!

Auto restart
live restart
socket manager
signal handler
log rotation
serverengine advantages

support new features
(These are already ported to ruby core.)
・multiprocess aware log rotation
・reopening of log file
・'trace' level, which is lower level than ‘debug'
ServerEngine.create(MyServer, MyWorker, {
log: 'myserver.log',
log_level: 'debug',
log_rotate_age: 5,
log_rotate_size: 1*1024*1024,
})

summary
windows
・signal → rpc handler
・no socket FD and UDS
・fork based → spawn based
・live & auto restart
・multicore with socket manager
・sophisticated signal handler
・easier log rotation

Thank you!

appendix

unix→fork based damon
windows→spawn based damon
damonize
fluentd
server
engine
①set windows_daemon_cmdline
in configuration
②run windows_daemon_cmdline
when damonize
③windows_daemon_cmdline
runs daemon.rb
④run_server with
servermodule and workermodule