The document discusses building a concurrent events collector microservice in Go. It describes collecting events from clients over HTTP, buffering them, and passing batches to a processor. It presents three versions of the collector: (1) a naive synchronous approach, (2) an improved concurrent version using worker goroutines, and (3) a final version that forces flushing buffers periodically using a timer. The final collector implementation provides good concurrency while avoiding blocking the collection of new events.

1. Go on!
3 June 2017
Vadim Petrov
Software engineer, Juno

2. Events collector
The microservice receives events from mobile clients by HTTP, collects them in a batch of
events and pass it to some 3rd-party system.

3. Microservice composition

4. Let's write some code
All further code is real and runnable.
You can nd it on GitHub.
github.com/Barberrrry/go-on-presentation(https://github.com/Barberrrry/go-on-presentation)

5. Processor

6. Fake processor
Create fake processor with random processing time up to 200 ms.
package processor
type Batch []string
type Fake struct{}
func (f *Fake) Process(batch Batch) {
if len(batch) == 0 {
return
}
// Fake process time
time.Sleep(time.Duration(rand.Intn(200)) * time.Millisecond)
}
That's enough for processor.

7. Collector

8. Collector v1: naive plan
Use a slice as a bu er
Append incoming data to the bu er
If bu er size reaches N, ush the bu er to Processor

9. Collector v1: types
package collector
type Processor interface {
Process(processor.Batch)
}
type Collector struct {
Processor Processor
MaxBatchSize int
buffer processor.Batch
}

10. Collector v1: collect
func (c *Collector) Collect(payload string) {
c.buffer = append(c.buffer, payload)
log.Printf("collected: %s", payload)
if len(c.buffer) >= c.MaxBatchSize {
c.flush(c.buffer)
c.buffer = nil
}
}
func (c *Collector) flush(batch processor.Batch) {
t := time.Now()
c.Processor.Process(batch)
log.Printf("flushed %d payloads in %s", len(batch), time.Since(t))
}

11. Collector v1: run
func main() {
collector := &collector.Collector{
Processor: &processor.Fake{},
MaxBatchSize: 2,
}
for i := 1; i <= 5; i++ {
collector.Collect(fmt.Sprintf("event_%d", i))
}
} Run

12. Collector v1: conclusion
Resolution: bad :(
Problem:
Function Collect() waits for Processor each N calls
To be xed:
Processor shouldn't block Collect()
Let's make Collector concurrent.

13. Concurrency

14. What is concurrency?
Concurrency is the composition of independently executing computations.
Concurrency is a way to design software.
Concurrency is not parallelism, although it enables parallelism.
Concurrency is about structure, parallelism is about execution.
Go has rich support for concurrency using goroutines and channels.

15. Goroutines
Goroutine is an independently executing function, launched by a go statement.
go func() {
fmt.Println("Hi!")
}()
It's very cheap.
It's practical to have thousands, even hundreds of thousands of goroutines.
Gouroutine is not OS thread.

16. Scheduler
Scheduler runs N contexts. N is con gurable. It may be 1 context.
Each context is running as OS thread.
Each context has own queue of goroutines.
Context stops goroutine execution on any blocking operation and starts next one from the
queue.

17. Communication
Goroutines without any input or output are useless.
Multiple goroutines require communication.
Don't communicate by sharing memory, share memory by communicating.
Instead of using locks to mediate access to shared data, use channels to pass data between
goroutines.

18. Channels
A channel in Go provides a connection between two goroutines, allowing them to
communicate.
Channel values may have any type, even other channel.
A channel is a blocking queue with de ned length.

19. Channel bu er
Bu ered channel
c := make(chan string, 5)
Unbu ered channel
c := make(chan string)

20. Collector v2

21. Collector v2: plan
Implement worker
Use channel to split incoming payloads among workers
Worker has own bu er and ush it when ready
Run several concurrent workers

22. Collector v2: collector
Add collector attributes and use channel as queue of incoming payloads.
type (
Collector struct {
Processor Processor
MaxBatchSize int
WorkersCount int
QueueSize int
payloadsQueue chan string
}
)

23. Collector v2: worker
Create worker() function with own bu er inside. Worker will ush bu er when it's ready.
func (c *Collector) worker(id int) {
var buffer processor.Batch
log.Printf("worker_%d start", id)
for payload := range c.payloadsQueue {
buffer = append(buffer, payload)
if len(buffer) >= c.MaxBatchSize {
c.flush(id, buffer)
buffer = nil
}
}
}

24. Collector v2: run workers
Collect() just writes payload to queue.
Create Run() function which will init queue channel and start N workers.
func (c *Collector) Collect(payload string) {
c.payloadsQueue <- payload
log.Printf("collected: %s", payload)
}
func (c *Collector) Run() {
log.Print("collector start")
c.payloadsQueue = make(chan string, c.QueueSize)
for i := 0; i < c.WorkersCount; i++ {
go func(id int) {
c.worker(id)
}(i)
}
}

25. Collector v2: run
func main() {
collector := &collector.Collector{
Processor: &processor.Fake{},
MaxBatchSize: 2,
WorkersCount: 2,
QueueSize: 1000,
}
collector.Run()
for i := 1; i <= 5; i++ {
collector.Collect(fmt.Sprintf("event_%d", i))
}
time.Sleep(200 * time.Millisecond)
} Run

26. Collector v2: conclusion
Resolution: better, but still not good
Fixed:
Function Collect() is not blocked anymore by Processor
Workers are concurrent
Problems:
Bu er is ushed only when reaches max batch size
To be xed:
Flush bu er after some timeout since last ush

27. Select

28. Select
The select statement waits on multiple communication operations.
A select blocks until one of its cases can run, then it executes that case.
It chooses one at random if multiple are ready.
If all cases are not ready, default happens.
values := make(chan int, 10)
quit := make(chan struct{})
for {
select {
case v := <-values:
fmt.Println(v)
case <-quit:
return
default:
time.Sleep(time.Second)
}
}

29. Collector v3. Final.

30. Collector v3: plan
Add ush interval as Collector parameter
Force ush periodically

31. Collector v3: collector
Add ush interval as Collector parameter.
type (
Collector struct {
Processor Processor
WorkersCount int
MaxBatchSize int
QueueSize int
FlushInterval time.Duration
payloadsQueue chan string
}
)

32. Collector v3: ush timer
func (c *Collector) worker(id int) {
var buffer processor.Batch
timer := time.NewTimer(c.FlushInterval)
for {
select {
case payload := <-c.payloadsQueue:
buffer = append(buffer, payload)
if len(buffer) >= c.MaxBatchSize {
c.flush(id, buffer, "size")
buffer = nil
timer.Reset(c.FlushInterval)
}
case <-timer.C:
c.flush(id, buffer, "timer")
buffer = nil
timer.Reset(c.FlushInterval)
}
}
}

33. Collector v3: run
func main() {
collector := &collector.Collector{
Processor: &processor.Fake{},
MaxBatchSize: 2,
WorkersCount: 2,
QueueSize: 1000,
FlushInterval: 200 * time.Millisecond,
}
collector.Run()
for i := 1; i <= 5; i++ {
time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond) // Fake delay
collector.Collect(fmt.Sprintf("event_%d", i))
}
time.Sleep(300 * time.Millisecond)
} Run

34. Collector v3: conclusion
Resolution: good enough

35. HTTP server

36. HTTP handler
Read HTTP request body and collect it by collector.
func NewCollectorHandler(d *collector.Collector) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
payload, err := ioutil.ReadAll(req.Body)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
d.Collect(string(payload))
}
}

37. Start server
func main() {
collector := &collector.Collector{
Processor: &processor.Fake{},
MaxBatchSize: 5,
WorkersCount: 3,
QueueSize: 10000,
FlushInterval: 5 * time.Second,
}
collector.Run()
http.ListenAndServe("localhost:9090", server.NewCollectorHandler(collector))
}

38. Go on!
Let's run microservice and see how it works.

39. Conclusions
Go provides instruments to make complex things easy.
Concurrent code enables runtime parallelism.
Goroutines and channels are easy to use.
But don't overuse these ideas. Always use the right tool for the job.

40. Questions?
This presentation slides and code:
github.com/Barberrrry/go-on-presentation(https://github.com/Barberrrry/go-on-presentation)
Concurrency in Go
www.golang-book.com/books/intro/10(https://www.golang-book.com/books/intro/10)
golang.org/doc/e ective_go.html#concurrency(https://golang.org/doc/e ective_go.html#concurrency)
"Go Concurrency Patterns" by Rob Pike
www.youtube.com/watch?v=f6kdp27TYZs(https://www.youtube.com/watch?v=f6kdp27TYZs)
"Concurrency is not parallelism" by Rob Pike
blog.golang.org/concurrency-is-not-parallelism(https://blog.golang.org/concurrency-is-not-parallelism)

41. Thank you
Vadim Petrov
Software engineer, Juno
vadim.petrov@gmail.com(mailto:vadim.petrov@gmail.com)
https://www.facebook.com/barberry(https://www.facebook.com/barberry)