Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Tokopedia - How Tokopedia Became one of Indonesia’s Most Promising Startups
Next
Download to read offline and view in fullscreen.

4

Share

Download to read offline

Golang @ Tokopedia

Download to read offline

Talk at Sequoia CTO Summit on programming with Go language

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Golang @ Tokopedia

  1. 1. Go @ Tokopedia Qasim Zaidi
  2. 2. About Tokopedia • Launched in 2009 • 300K Active merchants • 8 Million Products • Indonesia’s biggest online marketplace • Everyone pays shipping charges • Shipping is a function of distance • No discounting or cash-backs, neutral marketplace. • Most payments are offline
  3. 3. nginx ssl termination and proxy pass nginx-mod_perl nginx-mod_perl nginx-mod_perl The Stack postgres mongo redis proxy_pass
  4. 4. What we already use go for • Search & Discovery • Image Uploads • Analytics
  5. 5. 60% of the time, it works every time.
  6. 6. nginx ssl termination and proxy pass nginx-mod_perl nginx-mod_perl nginx-mod_perl The Stack postgres mongo redis proxy_pass mod_perl blocks nginx isolation is hard cancellation is harder
  7. 7. Challenges of scaling with mod_perl • Each request is synchronously processed in a worker • If there is one bad request, it ends up queuing everything else. • Cancellation is not easy - so even when the user moves away, the request continues to run.
  8. 8. In 2012, we bet on node.js
  9. 9. scaled well too - from 50K transactions in 2012 to over a million in 2015
  10. 10. node.js challenges • Too easy to mess up (dynamic, no type checking) • linting, code reviews • Unbounded Concurrency is hard. • async, promises • Scale issues with http client for c10k • even dns latencies can have big impact
  11. 11. Picking a new language in 2015.
  12. 12. the no QA philosophy QA is not a different role. Developers are responsible for testing their code.
  13. 13. Testing in Go • Easy to write unit tests func TestFoo(t *testing.T) { ... } • run as go test • No separate frameworks • Benchmarking is as easy as testing
  14. 14. the no Documentation philosophy Code should be self explanatory. Documentation outside of code is never updated.
  15. 15. Documentation with GoDoc • Comments in code - e.g. preceding function definition • It extracts the comments and presents them: • $ godoc strings Join
 
 func Join(a []string, sep string) string
 
 Join concatenates the elements of a to create a single string. The separator string sep is placed between elements in the resulting string. // Join concatenates the elements of a to create a single string.
 // The separator string sep is placed between elements in the resulting string.
 
 func Join(a []string, sep string) string {
 //self explanatory blah blah blah
 //more of the same }
  16. 16. godoc - examples and testing func ExampleJoin() { s := []string{"foo", "bar", "baz"} fmt.Println(strings.Join(s, ", ")) // Output: foo, bar, baz } • Also integrated with the testing framework to provide testable example functions.
  17. 17. godoc - webview
  18. 18. Concurrency should be easy And not yet forced. Too much concurrency spoils the code.
  19. 19. Code should be hard to mess with A compiler that catches bugs and is not too slow statically compiled binaries - no library mess
  20. 20. Institutional knowledge Go brings google’s institutional knowledge to you, for free - Groupcache - Expvars - Single-flights and Cancellations
  21. 21. Object Caching • In node.js - we built in an object caching solution in house, that can easily make a function called cached without changing much code. • var wrapper = cache.wrap(original); • Can chose whether to cache in memory or redis. • Can avoid thundering herds - single flighting
  22. 22. Enter group cache • In memory • distributed • single flight ELB app groupcache app app groupcachegroupcache http http
  23. 23. expvars - server metrics over http • Export variables over http • http://localhost:8123/debug/vars
 {
 "cmdline": ["/var/folders/bc/27hv15_d2zvcc3n3s9dxmfg00000gn/T/go-build421732654/command- line-arguments/_obj/exe/main","-l","debug","-s","i.sock","-c","realtest"],
 "counters": {"a": 10, "b": 10},
 "memstats": {"Alloc":1076016,"TotalAlloc":1801544,"Sys":5966072,"Lookups":209,"Mallocs": 7986,"Frees":4528,"HeapAlloc":1076016,"HeapSys":2097152,"HeapIdle":327680,"HeapInuse": 1769472,"HeapReleased":0,"HeapObjects":3458,"StackInuse":212992,"StackSys": 917504,"MSpanInuse":21504,"MSpanSys":32768,"MCacheInuse":8800,"MCacheSys": 16384,"BuckHashSys":1441160,"GCSys":1183744,"OtherSys":277360,"NextGC": 1436032,"LastGC":1418102095002592201,"PauseTotalNs":2744531,"PauseNs": [480149,171430,603839,288381,494934,522995,182803,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 ,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 ,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"NumGC":7,"EnableGC":true,"DebugGC":false}
 }
  24. 24. Cancellations • User moves away, nginx sends 499, but queries keep running. • Not Abandoning work when user has moved away is often wasteful • Go’s has advisory cancellations - cancel across go- routines • facilitated by contexts
  25. 25. moving from ‘dynamic’ to ‘compiled’ • Deployment changes • from git clone to compile, build & deploy • have a way of knowing binary version that is automatic BUILDHASH=$(shell cd src/$(SRCDIR) && git rev-parse --verify HEAD | cut -c 1-7)
 go build —ldflags -X $BUILDHASH appname —version • config and code is separate • make sure everything you may ever need to configure is in config/flags, can’t change binary appname —port appname —configtest appname —debug • have good debugging (we have debug() statements that are turned on by —debug flag)
  26. 26. The go workflow jenkins dpkg- buildpackagetriggers build build dpkg using go get ansible srv1 srv1 srv1 elb nginx binary, run via upstart deploy dpkg restart 2 1 triggers deploy 3
  27. 27. No templates, no exceptions, weird and sometimes inconsistent. Go is an opinionated language “Instructions, registers, and assembler directives are always in UPPER CASE to remind you that assembly programming is a fraught endeavor.” - Rob Pike
  28. 28. -Dick Gabriel, from the golang FAQ Who would have guessed sophistication bought such noise?
  29. 29. Golang - missing a package manager? • go get - but its not complete • No dependency management, unlike npm, no easy hermetic builds • gopkg.in is a solution - allows versioning • if you have to change versions, edit every single import
  30. 30. Deployment Challenges • No Fork() - forking with subroutines ain’t easy • Facebook grace (https://github.com/facebookgo/grace) • PID changes - no relationship • problems with upstart / job monitor • log rotation
  31. 31. Moving from a script to binary • Deployment changes (from git clone to build, package and deploy) • Need to know which version of code is running • appname —version • Need to debug easily • appname —debug - enables debug() output • Keep configuration flexible, because in case of issues, you can’t just edit and deploy • appname —configtest • appname —port (over ride config)
  32. 32. Go @ Tokopedia • Go is a modern language, very suitable for web. • Go is Performant, unless you are doing C. • Go is mostly google still, and not 100% complete. • Go is opinionated, like Plan9, and there will be a learning curve. • Go is slower for prototyping, e.g. when compared to node.js
  33. 33. Questions?
  • GattigagaHayyutaDewa

    May. 17, 2017
  • AndrewEfendi

    Apr. 22, 2017
  • edysalim

    Nov. 1, 2016
  • RidhaRamadhansyah

    Aug. 25, 2016

Talk at Sequoia CTO Summit on programming with Go language

Views

Total views

2,414

On Slideshare

0

From embeds

0

Number of embeds

360

Actions

Downloads

71

Shares

0

Comments

0

Likes

4

×