This document discusses GoHakai, an HTTP load testing tool written in Go. It summarizes Hideo Hattori's work developing GoHakai and several other Go utilities. GoHakai can generate load using configurable scenarios defined in YAML and supports features like HTTP/2, gzip compression, and distributed load generation across multiple servers. Benchmark results show that GoHakai uses CPU and memory more efficiently than an equivalent Python tool, especially at high concurrency levels. Future work may integrate the fasthttp package to improve performance further.

1. Go [gohakai]
3 golang
17 April 2016
Hideo Hattori
Software Engineer, KLab Inc.

2. Hideo Hattori
Python / Go / C / Rust
KLab
Go 2 Go1.1
Go
Go gocloc(https://github.com/hhatto/gocloc), gorst(https://github.com/hhatto/gorst), jc(https://github.com/hhatto/jc),
goroo(https://github.com/hhatto/goroo), ftcat(https://github.com/hhatto/ftcat), ...
Go goz(http://goz.hexacosa.net/)

3. Go
gofmt, golint, go-unused, etc...

4. Go

5. Go

6. Go
https://github.com/KLab/gohakai(https://github.com/KLab/gohakai)
HTTP
$ gohakai -h
Usage of gohakai:
-c=0: max concurrency requests
-d=0: total duration
-f=1: fork num (set GOMAXPROCS)
-n=1: scenario exec N-loop
-s=1: max scenario
-verbose=false: verbose mode

7. $ gohakai sample.yml
request count:2, concurrency:1, time:0.12560[s], 15.923309[req/s]
SUCCESS 2
FAILED 0
Average response time[ms]: 62.54169900000001
$ gohakai sample.yml
x
request count:2, concurrency:1, time:0.12560[s], 15.923309[req/s]
SUCCESS 1
FAILED 1
Average response time[ms]: 62.54169900000001

8. 2011 [internethakai] (http://internethakai.rubyforge.org/)(Ruby)
2012 [green-hakai] (https://github.com/KLab/green-hakai)(Python)
2015 Go [gohakai] (https://github.com/KLab/gohakai)(Go) New!!

9. Go
HTTP
ApacheBench, JMeter(http://jmeter.apache.org/), Tsung(http://tsung.erlang-projects.org/), vegeta(https://github.com/tsenart/vegeta),
etc...
(YAML)
HTTP/2
(Python )

10. Go
,

12. (YAML)
domain: http://localhost:8889
# 選 変数
vars:
- name: var
file: var.txt
consts:
c1: c1val
c2: c2val
actions:
- path: /
- path: /hello
method: POST
scan: "Hello (?P<word>w+)"
content: '{"v":"%(var)%","c2":"%(c2)%"}'
content_type: 'application/json'

13. $ gohakai -c 10 -s 200 sample.yml
....
request count:400, concurrency:10, time:0.0.9423[s], 4244.830624[req/s]
SUCCESS 400
FAILED 0
Average response time[ms]: 2.29882956
-c :
-s :

14. gzip & HTTP/2
gzip
gzip: true
HTTP/2
domain: https://localhost:8000
http_version: 2 # for HTTP/2
actions:
- path: /
- path: /hello

15. 1 .
.
gohakai
OS
gohakai

16. domain: https://localhost:8000
nodes:
- host: vagrant@192.168.1.100
proc: 2
ssh_key: ~/.ssh/id_rsa
- host: vagrant@192.168.1.101
proc: 2
ssh_key: ~/.ssh/id_rsa
- host: localhost
proc: 1
actions:
- path: /
proc

17. (https://github.com/KLab/green-hakai/blob/master/example/sample.yml)

19. attacker.go - 本体( 生成, 攻撃)
config.go - 設定 (YAML読 出 Gob形式 読 書 )
indicator.go - .....
main.go - ( )
remote.go - 攻撃用処理
statistics.go - 統計情報収集

20. gopkg.in/yaml.v2(https://github.com/go-yaml/yaml)
golang.org/x/net/http2(https://godoc.org/golang.org/x/net/http2)
golang.org/x/crypto/ssh(https://godoc.org/golang.org/x/crypto/ssh)

21. gzip
if atk.Gzip {
req.Header.Set("Accept-Encoding", "gzip")
} else {
req.Header.Set("Accept-Encoding", "")
}
var reader io.ReadCloser
switch res.Header.Get("Content-Encoding") {
case "gzip", "deflate":
reader, _ = gzip.NewReader(res.Body)
defer reader.Close()
default:
reader = res.Body
}
body, _ := ioutil.ReadAll(reader)

22. HTTP/2
if config.HTTPVersion == 2 {
client = http.Client{
Transport: &http2.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: false,
},
},
CheckRedirect: redirectFunc,
}
} else {
client = http.Client{
Transport: &http.Transport{
MaxIdleConnsPerHost: maxRequest, // default is 2
},
Timeout: time.Duration(config.Timeout) * time.Second, // default is 30
CheckRedirect: redirectFunc,
}
}

23. Gob
scp
gohakai scp
ssh & Attack!!
Gob

25. Goroutines
Worker x
x1
x1
Gob
gohakai

26. fan-out
http.Client
http.Client MaxIdleConnsPerHost (-c, -s )
regexp.MustCompile ( )

27. fan-out
func worker(id int, wg *sync.WaitGroup, limiter chan Worker) {
for {
ret := <-limiter
hakai(ret.Client, ret.Config, ret.ExVarOffset)
wg.Done()
}
}
// exec worker
for num := 0; num < maxRequest; num++ {
go worker(num, &wg, limiter)
}
// attack
for i := 0; i < loop*maxScenario; i++ {
wg.Add(1)
w := Worker{Client: client, Config: config, ExVarOffset: offset}
limiter <- w
}

28. fan-out

30. Python -f Go Go

32. iMac 21.5-inch, Late 2012
MacOSX El Capitan 10.11.4
CPU 3.1GHz Intel Core i7 (4core)
Memory 16GB 1600MHz DDR3
Go
Go1.6.1
Python
2.7.11

33. ps
$ ps aux | grep hakai | grep -v grep
CPU

35. concurrency Python (cpu %) Python (RSS KB) Python-s (RSS KB) Go (cpu %) Go (RSS KB)
1 62.2 233336 16160 50.9 10072
2 93.9 294880 16196 58.8 10144
4 97.4 304296 16288 60.9 10284
8 98.4 309184 16744 64.4 10488
16 99.1 310244 17796 63.6 10648
32 100 317272 18960 65.6 10992
64 100 318216 20656 69.1 12012
128 100 333144 25032 72.3 14888
gohakai
ghakai 1 CPU -f
.
CPU .

37. Go UP !!
Go !!
!!

38. TODO
(fasthttp )

39. Thank you
Hideo Hattori
Software Engineer, KLab Inc.
@hhatto(http://twitter.com/hhatto)