Fune is a Go program that acts as a Docker monitoring agent. It listens to Docker events through the Docker events API and emits actions in response to events, such as notifying Slack or updating a service registry. The agent communicates with the Docker daemon via its socket. It allows emitting pluggable actions when container events like create, start, stop and die are received.

1. Writing Docker
monitoring agent with Go
ainoya

2. About me
• Naoki Ainoya( ainoya)
• iOS(new!)/Server/Infra engineer
• Swift/Scala/Go/Docker
• Recruit Marketing Partners Co, Ltd.

3. A tiny deployment pipeline tool
Written in Go
walter-cd/walter
Open source project

4. Introduction of Fune
• An event emitter triggered by Docker Event API
• The agent process (fune-agent) communicates with docker
daemon via its socket
• https://github.com/ainoya/fune
Docker daemon
Funeagent
ConttainerA
ConttainerB
ConttainerC
Nginx/Redis
vulcand(etcd)
Slack notification etc..
Listens docker
events API
notifies each containers are
created/died/stopped/started
Emits pluggable
actions
Your
arbitrary docker ecosystem
ECS/k8s/fleet/etc..
Docker Host
Deploy containers

5. Overview of Fune
• Docker Events API provides a lifecycle of container
as JSON format
• create/start/die/stop

6. Overview of Fune
fune-agent listens these events
Docker daemon
Funeagent
ConttainerA
ConttainerB
ConttainerC
Listens docker
events API
notifies each containers are
created/died/stopped/started
Your
arbitrary docker ecosystem
ECS/k8s/fleet/etc..
Docker Host
Deploy containers

7. Overview of Fune
fune-agent emits Action
when received Docker events
from event
Docker daemon
Funeagent
ConttainerA
ConttainerB
Nginx/Redis
vulcand(etcd)
Slack notification etc..
Listens docker
events API
notifies each containers are
created/died/stopped/started
Emits pluggable
actions
Your
arbitrary docker ecosystem
ECS/k8s/fleet/etc..
Docker Host
Deploy containers

8. Overview of Fune
Docker daemon
Funeagent
ConttainerA
ConttainerB
Nginx/Redis
vulcand(etcd)
Slack notification etc..
Listens docker
events API
notifies each containers are
created/died/stopped/started
Emits pluggable
actions
Your
arbitrary docker ecosystem
ECS/k8s/fleet/etc..
Docker Host
Deploy containers
• Action, for example;
• Set/Del a container information to Redis as
FQDN/IP:Port pair for proxy
• Slack notification
• More details are next time :)

9. Today I talk about:
• Some tips I learned with writing fune
• Studied a lot of stuffs from coreos/etcd

10. Setup project
• Standard structure for
working with Godeps correctly
• /src/github.com/ainoya/fune
./${GOPATH}/src
!"" 9fans.net
# %"" go
!"" code.google.com
# %"" p
!"" github.com
# !"" ainoya/fune
# !"" barakmich
# !"" coreos etc...

11. Godeps for the package
dependencies
• Using Godeps once, all dependencies copied into
directory and rewrite import statement
• Then you need Godeps command only if package
dependency is added
import "github.com/fsouza/go-dockerclient"
// after `Godep save -r ./…`
import "github.com/ainoya/fune/Godeps/_workspace/
src/github.com/fsouza/go-dockerclient"

12. Godeps for the package
dependencies
• All dependencies are included in git repos
• But it makes dirty commit log!!
• Keep off its dirtiness by split commits :)

13. How about vendor feature in
go1.5?
• go1.5 vendor feature have some problems yet
• ex) go
tool
"./…" doesn't works well
• https://github.com/golang/go/issues/11659
• See also ) http://deeeet.com/writing/2015/06/26/golang-dependency-
vendoring/

14. Write tests
• Rerun automatically if files are changed
• rerun is awesome! (but ruby tool gem)
rerun -x -c -d directory1, directory2,...
—pattern '**/*.{go}' -- go test ./...

15. Be careful about race condition
• Test with "-­‐race" option if you often use goroutine in your code
• "-­‐cpu
N" option may reveal race condition you didn't expected
• I use "go
test
./…
-­‐race
-­‐cpu
1,2,4" inside test script
WARNING: DATA RACE
Write by goroutine 8:
sync.raceWrite()
/usr/local/Cellar/go/1.5/libexec/src/sync/race.go:41 +0x2e
sync.(*WaitGroup).Wait()
/usr/local/Cellar/go/1.5/libexec/src/sync/waitgroup.go:124 +0xf9
github.com/ainoya/fune/emitter.TestBroadCast()
/Users/ainoya/.go/src/github.com/ainoya/fune/gopath/src/github.com/
ainoya/fune/emitter/emitter_test.go:57 +0x444
testing.tRunner()
/usr/local/Cellar/go/1.5/libexec/src/testing/testing.go:456 +0xdc

16. Is Go1.5 compilation time
slower than 1.4?
• "go
test" obviously takes time much longer
• It's probably because of GoGo compiler
performance
Ref) Performance section of https://golang.org/doc/go1.5
• It should be improved with Go1.6

17. Channel tips
• coreos/etcd implements a shutdown feature with
using close(channel)
type
Server
struct
{
name
string
channel
chan
struct{}
done
chan
struct{}
stop
chan
struct{}
}
func
main()
{
done
:=
make(chan
struct{})
stop
:=
make(chan
struct{})
s
:=
&Server{done:
done,
stop:
stop,
hoge:
hoge}
go
s.run()
s.channel
<-­‐
struct{}{}
osutil.RegisterInterrputHandler(s.Stop)
<-­‐s.done
}
func
(s
*Server)
run()
{
defer
func()
{
close(s.done)
}()
for
{
select
{
case
<-­‐s.channel:
fmt.Println("do
something")
case
<-­‐s.stop:
return
}
}
}
func
(s
*S)
Stop()
{
select
{
case
s.stop
<-­‐
struct{}{}:
case
<-­‐s.done:
return
}
<-­‐s.done
}

18. Channel tips
• coreos/etcd implements a shutdown feature with
using close(channel)
type
Server
struct
{
name
string
channel
chan
struct{}
done
chan
struct{}
stop
chan
struct{}
}
func
main()
{
done
:=
make(chan
struct{})
stop
:=
make(chan
struct{})
s
:=
&Server{done:
done,
stop:
stop,
hoge:
hoge}
go
s.run()
s.channel
<-­‐
struct{}{}
osutil.RegisterInterrputHandler(s.Stop)
<-­‐s.done
}
func
(s
*Server)
run()
{
defer
func()
{
close(s.done)
}()
for
{
select
{
case
<-­‐s.channel:
fmt.Println("do
something")
case
<-­‐s.stop:
return
}
}
}
func
(s
*S)
Stop()
{
select
{
case
s.stop
<-­‐
struct{}{}:
case
<-­‐s.done:
return
}
<-­‐s.done
}

19. Channel tips
• coreos/etcd implements a shutdown feature with
using close(channel)
type
Server
struct
{
name
string
channel
chan
struct{}
done
chan
struct{}
stop
chan
struct{}
}
func
main()
{
done
:=
make(chan
struct{})
stop
:=
make(chan
struct{})
s
:=
&Server{done:
done,
stop:
stop,
hoge:
hoge}
go
s.run()
s.channel
<-­‐
struct{}{}
osutil.RegisterInterrputHandler(s.stop)
<-­‐s.done
}
func
(s
*Server)
run()
{
defer
func()
{
close(s.done)
}()
for
{
select
{
case
<-­‐s.channel:
fmt.Println("do
something")
case
<-­‐s.stop:
return
}
}
}
func
(s
*S)
Stop()
{
select
{
case
s.stop
<-­‐
struct{}{}:
case
<-­‐s.done:
return
}
<-­‐s.done
}

20. Channel tips
• coreos/etcd implements a shutdown feature with
using close(channel)
type
Server
struct
{
name
string
channel
chan
struct{}
done
chan
struct{}
stop
chan
struct{}
}
func
main()
{
done
:=
make(chan
struct{})
stop
:=
make(chan
struct{})
s
:=
&Server{done:
done,
stop:
stop,
hoge:
hoge}
go
s.run()
s.channel
<-­‐
struct{}{}
osutil.RegisterInterrputHandler(s.Stop)
<-­‐s.done
}
func
(s
*Server)
run()
{
defer
func()
{
close(s.done)
}()
for
{
select
{
case
<-­‐s.channel:
fmt.Println("do
something")
case
<-­‐s.stop:
return
}
}
}
func
(s
*S)
Stop()
{
select
{
case
s.stop
<-­‐
struct{}{}:
case
<-­‐s.done:
return
}
<-­‐s.done
}

21. Channel tips
• coreos/etcd implements a shutdown feature with
using close(channel)
type
Server
struct
{
name
string
channel
chan
struct{}
done
chan
struct{}
stop
chan
struct{}
}
func
main()
{
done
:=
make(chan
struct{})
stop
:=
make(chan
struct{})
s
:=
&Server{done:
done,
stop:
stop,
hoge:
hoge}
go
s.run()
s.channel
<-­‐
struct{}{}
osutil.RegisterInterrputHandler(s.Stop)
<-­‐s.done
}
func
(s
*Server)
run()
{
defer
func()
{
close(s.done)
}()
for
{
select
{
case
<-­‐s.channel:
fmt.Println("do
something")
case
<-­‐s.stop:
return
}
}
}
func
(s
*S)
Stop()
{
select
{
case
s.stop
<-­‐
struct{}{}:
case
<-­‐s.done:
return
}
<-­‐s.done
}

22. Test goroutine easily
"s.done" channel makes testing goroutine easily
//
https://github.com/coreos/etcd/blob/master/etcdserver/
server_test.go#L1031
//
TestPublishRetry
tests
that
publish
will
keep
retry
until
success.
func
TestPublishRetry(t
*testing.T)
{
n
:=
&nodeRecorder{}
srv
:=
&EtcdServer{
cfg:
&ServerConfig{TickMs:
1},
r:
raftNode{Node:
n},
w:
&waitRecorder{},
done:
make(chan
struct{}),
reqIDGen:
idutil.NewGenerator(0,
time.Time{}),
}
time.AfterFunc(500*time.Microsecond,
func()
{
close(srv.done)
})
srv.publish(10
*
time.Nanosecond)

23. Run go as Docker container
• Go single binary with "SCRATCH"
• You can run a go stuff inside extremely simple container!
FROM
scratch
EXPOSE
8080
COPY
your-­‐golang-­‐app
/app/your-­‐golang-­‐app
ENV
PATH=/app:$PATH
ENTRYPOINT
["/app/your-­‐golang-­‐app"]
CMD
["-­‐some-­‐option=haha"]

24. Round up
• Learned a lot of stuffs from well-knowned products
(coreos/etcd)
• project structure, test, golang-way

25. See you next kyobashi.*!
• 9/7: kyobashi.dex
• 9/16: potatotips
• ???: kyobashi.???