1. KAI CHU CHUNG
Cloud GDE
GDG Cloud Taipei co-organizers
@CageChung
https://kaichu.io
如何透過 Go-kit 快速搭建微服務架
構應用程式實戰

2. KAI CHU CHUNG
Cloud GDE
GDG Cloud Taipei co-organizers
QNAP
@CageChung
https://kaichu.io

3. https://www.facebook.com/groups/GCPUG.TW

4. Agenda
● Go-kit
● Layout
● Test
● Toolchain

5. Toolkit for microservices
Go-kit

6. package main
import (
"fmt"
"net/http"
)
func hello(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "Hello Gophercon TW 2020")
}
func main() {
http.HandleFunc("/", hello)
http.ListenAndServe(":8888", nil)
}
Scenario 1

7. type Addsvc interface {
sum(a, b int64) (int64, error)
}
type SumRequest struct {
A int64 `json:"a"`
B int64 `json:"b"`
}
type addService struct{}
func (s *addService) sum(a, b int64) (int64, error) {
return a + b, nil
}
Scenario 2
Service
Define method and
implementation

8. func (s *addService) ServeHTTP(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodPost:
var req SumRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if res, err := s.sum(req.A, req.B); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
} else {
fmt.Fprint(w, res)
}
}
}
func main() {
http.ListenAndServe(":8888", &addService{})
}
Scenario 2

9. type Endpoint func(request) (response)
type Endpoint func(ctx context.Context, request interface{}) (response interface{}, err error)
func MakePostSumEndpoint(s addService) endpoint.Endpoint {
return func(_ context.Context, request interface{}) (response interface{}, err error) {
p := request.(SumRequest)
return s.sum(p.A, p.B)
}
}
Endpoint
Endpoint is the fundamental building block of servers and clients. It represents a single RPC
method.

10. func EncodeJSONRequest(c context.Context, r *http.Request, request interface{}) error
func EncodeJSONResponse(_ context.Context, w http.ResponseWriter, response interface{}) error
func decodeHTTPSquareRequest(_ context.Context, r *http.Request) (interface{}, error) {
var req SumRequest
err := json.NewDecoder(r.Body).Decode(&req)
return req, err
}
func encodeJSONResponse(_ context.Context, w http.ResponseWriter, response interface{}) error {
return json.NewEncoder(w).Encode(response)
}
Transport
DecodeRequestFunc / EncodeResponseFunc
● EncodeJSONRequest is an EncodeRequestFunc that serializes the request as a JSON object to the
Request body
● EncodeJSONResponse is a EncodeResponseFunc that serializes the response as a JSON object to the
ResponseWriter

11. func NewServer(
e endpoint.Endpoint,
dec DecodeRequestFunc,
enc EncodeResponseFunc,
options ...ServerOption,
) *Server { ... }
postSum := httptransport.NewServer(e, decodeHTTPSumRequest, httptransport.EncodeJSONResponse)
r := mux.NewRouter()
r.Methods(http.MethodPost).Handler(postSum)
log.Printf("listening on :8888")
log.Fatal(http.ListenAndServe(":8888", r))
Transport
Go kit comes with handy HTTP transport.NewServer constructs a new server, which
implements http.Handler and wraps the provided endpoint.

12. type Server struct {
e Endpoint
dec DecodeRequestFunc
enc EncodeResponseFunc
}
func (s Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
request, err := s.dec(ctx, r)
if err != nil {
return
}
response, err := s.e(ctx, request)
if err != nil {
return
}
if err := s.enc(ctx, w, response); err != nil {
return
}
}
Scenario 3

13. How is a Go-kit microservice modeled?
Transport
Endpoint
Service
https://gokit.io/faq/#design-mdash-how-is-a-go-kit-microservice-modeled

14. Go-kit Endpoint
Go + microservices = Go kit - Speaker Deck - https://speakerdeck.com/peterbourgon/go-plus-microservices-equals-go-kit?slide=78

15. .
├── auth
├── circuitbreaker
├── cmd
├── endpoint
├── examples
├── log
├── metrics
├── ratelimit
├── sd
├── tracing
├── transport
├── util
├── .build.yml
├── .gitignore
├── .travis.yml
Go-kit components
Authentication: Basic, casbin, JWT.
Circuit Breaker: Hystrix, GoBreaker, and HandyBreaker.
Logging: Provide an interface for structured logging. Recognizes that logs
are data. They need context and semantics to be useful for analysis.
Supported formats are logfmt and JSON.
Metrics: Provides uniform interfaces for service instrumentation. Comes
with counters, gauges, and histograms. Has adapters for CloudWatch,
Prometheus, Graphite, DogStatsD, StatsD, expvar, and more.
Rate Limit: Uses Go's token bucket implementation.
Service Discovery: Consul, DNS SRV, etcd, Eureka, ZooKeeper, and more.
Tracing: OpenCensus, OpenTracing, and Zipkin.
Transport: AMQP, AWS Lambda, gRPC, HTTP, NATS, Thrift.

16. Go-kit microservice
Golang UK Conference 2015 - Peter Bourgon - Go Kit A Toolkit for Microservices - https://youtu.be/aL6sd4d4hxk?t=1022

17. Go-kit microservice + Istio
Golang UK Conference 2015 - Peter Bourgon - Go Kit A Toolkit for Microservices - https://youtu.be/aL6sd4d4hxk?t=1022

18. Go-kit simple
workshop
https://github.com/cage1016/gokit-workshop
- Simple tutorial from http/net
to Gokit skeleton

19. Project layout
Layout

20. .
├── cmd
│ ├── add
│ ├── ...
│ └── tictac
├── internal
│ ├── app
│ │ ├── add
│ │ ├── ...
│ │ └── tictac
│ └── pkg
│ ├── account
│ ├── ...
│ ├── errors
│ ├── jwt
│ ├── responses
│ └── version
├── pb
│ ├── add
│ └── tictac
Layout
● Every service put at internal/app/<service-name>
● cmd/<service-name> is service entry point
● internal/app/<shared-pkg> is shared pkg for other service
● pb/<service-name> for external usage

21. ├── internal
│ ├── app
│ │ ├── add
│ │ │ ├── endpoints
│ │ │ │ ├── endpoints.go
│ │ │ │ ├── middleware.go
│ │ │ │ ├── requests.go
│ │ │ │ └── responses.go
│ │ │ ├── service
│ │ │ │ ├── logging.go
│ │ │ │ ├── service.go
│ │ │ │ └── version.go
│ │ │ └── transports
│ │ │ ├── grpc
│ │ │ └── http
│ │ └── tictac
│ │ ├── endpoints
│ │ ├── ...
│ │ ├── model
│ │ │ └── count.go
│ │ ├── postgres
│ │ │ ├── init.go
│ │ │ └── postgres.go
│ │ ├── service
│ │ └── transports
Layout
Single service
● Each service at least contain Endpoints, Service,
Transports
● Service: provide expose service method for serving
● Endpoints: handle decode/encode request between service
and transport
● Transports: implement support transports for service
● Other associated request components

22. ├── deployments
│ ├── docker
│ └── helm
├── Makefile
├── README.md
├── CHANGELOG.md
├── cloudbuild.yaml
├── gk.json
├── go.mod
├── go.sum
└── skaffold.yaml
Layout
● Basic workflow components
○ Skaffold.yaml
○ Cloudbuild.yaml
● README.md, CHANGELOG.md etc documentation.
● Deployment workflow like docker, helm put at
deployments

23. Go-kit Microservice
demo
https://github.com/cage1016/ms-demo
- Go-kit base
- GRPC/HTTP
- Service
- Add
- TicTac
- Kubernetes/Istio
- Sakffold support

24. A little copying is better than a little
dependency.
Proverbs from @rob_pike

25. Test

26. func setupXTestCase(_ *testing.T) (xTestSuit, func(t *testing.T)) {
s := xTestSuit{}
return s, func(t *testing.T) { }
}
func TestAddOrganization(t *testing.T) {
_, teardownTestCase := setupXTestCase(t)
defer teardownTestCase(t)
cases := []struct {
...
}{
{
desc: "A",
setupSubTest: func(t *testing.T) func(t *testing.T) {
return func(t *testing.T) { }
},
},
}
for _, tc := range cases {
t.Run(tc.desc, func(t *testing.T) {
teardownSubTest := tc.setupSubTest(t)
defer teardownSubTest(t)
// test code
})
}
Unitest skeleton
=== RUN TestAddOrganization
test start
=== RUN
TestAddOrganization/A
sub test start
sub test teardown
test teardown
--- PASS: TestAddOrganization
(0.00s)
--- PASS:
TestAddOrganization/A (0.00s)
PASS

27. ● Cloudbuild pipeline does not support docker in docker,
mock required interface object before test
○ // +build !integration unit
○ // +build integration
● Data layer
○ postgres
● Services
● Transports
○ HTTP/GRPC
Unitest
.
├── mocks
│ ├── account.go
│ ├── authz.go
│ ├── commons.go
│ ├── context.go
│ ├── fixture.go
│ ├── idp.go
│ ├── organization.go
│ └── users.go
├── postgres
│ ├── database.go
│ ├── init.go
│ ├── users.go
│ └── users_test.go
├── service
│ ├── ...
│ ├── service.go
│ ├── service_test.go
│ └── version.go
└── transports
├── grpc.go
├── grpc_test.go
├── http.go
├── http_test.go
└── nats.go

28. Toolchain

29. A generator for go-kit that helps you create/update boilerplate code
Usage:
gk [flags]
gk [command]
Available Commands:
add Use to add additional transports to a service
help Help about any command
init Initiates a service
new A set of generators used to create new
cmd/services/transports/middlewares
update Use to update service
Flags:
-d, --debug If you want to se the debug logs.
--folder string If you want to specify the base folder of the
project.
-f, --force Force overwrite existing files without asking.
-h, --help help for gk
--testing If testing the generator.
Use "gk [command] --help" for more information about a command.
gk
https://github.com/cage1016
/gk/tree/feature/gokitconsul
k8sistio
Go-kit generator

30. hello:
curl localhost:8180/hello | jq
generate:
go mod init github.com/cage1016/gophercon-tw
gk n s hellogophercon
sed -i "" 's|Foo(ctx context.Context, s string) (res string, err error)|Hello(ctx context.Context) (res
string, err error)|g' internal/app/hellogophercon/service/service.go
sed -i "" 's|method=post,expose=true|method=get,expose=true|g'
internal/app/hellogophercon/service/service.go
gk init hellogophercon
sed -i "" 's|return res, err|return "Hello Gophercon TW 2020", err|g'
internal/app/hellogophercon/service/service.go
gk add grpc hellogophercon
cd pb/hellogophercon && ./compile.sh
gk init grpc hellogophercon
gk new cmd hellogophercon
go run cmd/hellogophercon/main.go
Makefile

31. Build/Test Store Deploy Monitor
workflow

32. KAI CHU CHUNG
GDE Cloud
GDG Cloud Taipei co-organizers
@CageChung
https://kaichu.io
Q & A