Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Gophers Riding Elephants: Writing PostgreSQL tools in Go

354 views

Published on



This talk will start with an overview of Go, then dive into some examples of using it to work with Postgres. We'll show basics like running queries, then demonstrate how Go makes difficult things easy, such as inspecting Postgres's TCP wire protocol and providing retry mechanisms and monitoring for restores (including fun stories, tips and tricks).

Join us and see why Go - with it's powerful concurrency primitives and unbeatable performance - can be one of the most powerful tools in your toolbox.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Gophers Riding Elephants: Writing PostgreSQL tools in Go

  1. 1. Gophers Riding Elephants: Writing PostgreSQL Tools in Go by AJ Bahnken, Senior Engineer @ Procore
  2. 2. Who am I? Senior Engineer @ Procore Work on availability, performance, and (now mostly) security Been writing Go code actively for 2 years Twitter: Email: aj.bahnken@procore.com @ajvbahnken
  3. 3. Who is this talk for?
  4. 4. Overview of Go
  5. 5. Go was created by Google
  6. 6. It was built for Google as well
  7. 7. Reliable Good for teams Good for building the kind of things they need to build
  8. 8. "Less is exponentially more" by Rob Pike Go is purposefully lacking certain features. Link: https://commandcenter.blogspot.com/2012/06/less-is- exponentially-more.html
  9. 9. Statically Typed Compiled Garbage Collected
  10. 10. package main  import "fmt"  func main() {      fmt.Println("Hello, 世界")  } 
  11. 11. Why Go with Postgres?
  12. 12. 1. Performance
  13. 13. It's pretty fast
  14. 14. Garbage collector is pretty solid
  15. 15. Concurrency is a breeze package main  import (    "fmt"    "time"  )  func say(s string) {    for i := 0; i < 5; i++ {      time.Sleep(100 * time.Millisecond)      fmt.Println(s)    }  }  func main() {    go say("world")    say("hello") // Allows for the goroutine to run, by blocking.  }
  16. 16. $ go run main.go  world  hello  hello  world  world  hello  hello  world  world  hello 
  17. 17. 2. Reliability
  18. 18. Statically Typed (yay!) bool  string  int  int8  int16  int32  int64  uint uint8 uint16 uint32 uint64 uintptr  byte // alias for uint8  rune // alias for int32       // represents a Unicode code point  float32 float64  complex64 complex128 
  19. 19. Simple type EventProcessor struct {      eventQueue chan Event  }  func (ep *EventProcessor) Add(event Event) {      ep.eventQueue <­ event  }  func (ep *EventProcessor) Start() {    for {      event := <­ep.eventQueue      go event.Process()    }  } 
  20. 20. Testing is simple and built in + race detector $ ls  processing.go       processing_test.go  utils.go  $ go test  PASS  ok      ~/pgnetdetective/processing        0.165s  $ go test ­­race  PASS  ok      ~/pgnetdetective/processing        2.133s 
  21. 21. Error handling instead of exceptions func MyFunc() (string, error) {    str, err := run()    if err != nil {      return "", err    }    return str, nil  }  func MustMyFunc() string {    str, err := run()    if err != nil {      panic("run() returned an err: "+err.String())    }    return str  } 
  22. 22. 3. Ease of Use
  23. 23. Tooling (Gofmt, testing, godocs, go build/run, vim-go)
  24. 24. Familiarity
  25. 25. Library support and ease of installation $ go get github.com/urfave/cli 
  26. 26. Distribute a single binary anywhere $ go build  $ file dbduke  dbduke: Mach­O 64­bit executable x86_64  $ GOOS=linux go build  $ file dbduke  dbduke: ELF 64­bit LSB executable, x86­64, version 1 (SYSV),        statically linked, not stripped  $ GOOS=linux GOARCH=386 go build  $ file dbduke  dbduke: ELF 32­bit LSB executable, Intel 80386, version 1       (SYSV), statically linked, not stripped 
  27. 27. Performance Reliability Ease of Use
  28. 28. Interacting with Postgres in Go
  29. 29. database/sql Docs: https://golang.org/pkg/database/sql/
  30. 30. Provides core interface for interacting with SQL databases Open / Close Begin / Rollback / Commit Exec / Query / QueryRow Ping / Connection Pooling
  31. 31. go get github.com/lib/pq
  32. 32. package main  import (    "database/sql"    "fmt"    _ "github.com/lib/pq"  )  func main() {    dbUrl := "postgres://postgres@localhost:5432/postgres"    db, err := sql.Open("postgres", dbUrl)    if err != nil {      panic(err.String())    }    var result int    err = db.QueryRow('SELECT 1').Scan(&result)    if err != nil {      panic(err.String())    }    fmt.Printf("1 == %d", result)  } 
  33. 33. http://go-database-sql.org/
  34. 34. Example #1 pgnetdetective https://github.com/procore/pgnetdetective
  35. 35. ?????
  36. 36. tcpdump ­n ­w ~/pg.cap ­i any port 5432  ~1GB every 10 seconds
  37. 37. We needed something faster, so I decided to rewrite it in Go
  38. 38. https://github.com/google/gopacket Provides packet processing capabilities for Go
  39. 39. // If the destination port is 5432...  if tcp.DstPort == 5432 {    // And the packet payload starts with P or Q...    raw = fmt.Sprintf("%s", tcp.Payload)    if strings.HasPrefix(raw, "P") || strings.HasPrefix(raw, "Q") {      // It is a Parse or Query packet, therefore it contains a Query      combinedQueryMetrics.Add(        metrics.New(          NormalizeQuery(raw),          1,          ip.SrcIP,          tcp.Seq,        ),      )    }  } else if tcp.SrcPort == 5432 && tcp.ACK {    responses = append(responses, &ResponsePacket{     DstIP: ip.DstIP,      Ack:   tcp.Ack,      Size:  uint64(len(tcp.Payload)),    })  } 
  40. 40. So I got some output like this: ******* Query *******  Query: SELECT attr.attname FROM pg_attribute attr      INNER JOIN pg_constraint cons ON attr.attrelid = cons.conrelid      AND attr.attnum = any(cons.conkey) WHERE cons.contype = p      AND cons.conrelid = "drawing_log_imports"::regclass  TotalNetBytes: 170 MB  TotalResponsePackets: 64041  TotalQueryPackets: 63 
  41. 41. ummm, catalog queries?
  42. 42. Introducing: Resque https://github.com/resque/resque http://resque.github.io/
  43. 43. ~10,000 jobs per hour 1-8 tables being touched per job average of 20 columns per table.
  44. 44. During spikes, this can get up to 120MB per second. On to Sidekiq we go...
  45. 45. Where Go won with pgnetdetective: Performance Community (Ease of Use)
  46. 46. Example #2 dbduke (not open source + still under active development)
  47. 47. Context: 1. We restore staging/qa/testing databases frequently 2. It's important that they successfully restore
  48. 48. Problem: 1. When restores fail, productivity dies 2. The process of kicking restores off by hand is faulty
  49. 49. Further Context for a Solution: 1. Restores sometimes fail from easily recoverable errors
  50. 50. A tool for making restores of Postgres databases manageable and fault tolerant.
  51. 51. A tool for making restores of Postgres databases manageable and fault tolerant.
  52. 52. Manageable Run dbduke as a daemon with jobs
  53. 53. $ dbduke jobs  ­­­­­­­­­­­­     DBDuke  ­­­­­­­­­­­­  * restore ­ 35e1ca93­936b­4c73­8812­b1a69d708791     database: postgres     dumpfile: /data/lite­dump.dmp     started: 17:19:59 Tue Oct 11, 2016 ­0700     flags: ­­no­big­tables ­­maintenance 
  54. 54. A tool for making restores of Postgres databases manageable and fault tolerant.
  55. 55. Fault Tolerance Treat restores as a state machine and recover from failure states
  56. 56. Error handling in practice: 1. Error out 2. Log a warning 3. Retry with timeout (with or without backoff)
  57. 57. Error out db, err := sql.Open("postgres", dbUrl)  if err != nil {    log.Fatalf("Could not open postgres db @ `%s`", dbUrl)  } 
  58. 58. Log warning query := "DROP SCHEMA IF EXISTS _migration CASCADE"  _, err = db.Exec(query)  if err != nil {    log.Warnf("Query `%s` failed with err: %v", query, err)  } 
  59. 59. Retry with timeout (without backoff) func (r *Restorer) BlockTillNotInUse() {      if r.State == state.InUse {          log.Warn("State is currently InUse. Going into retry loop.")          for {              time.Sleep(time.Second * 15)              r.QuitIfTimeout()              currentState, err := state.GetCurrentState()              if err != nil {                  log.Errorf(                    "Error getting current state. Err: %v",                    err,                  )                  break              }              if currentState != state.InUse {                  r.State = currentState                  break              }          }      }  } 
  60. 60. Manageability + Fault Tolerance Go makes it easy! ™
  61. 61. Where Go won with dbduke: Error Handling (Reliability) Concurrency (Performance/Ease of Use)
  62. 62. In Conclusion In the context of tool building Go = Reliability, Performance, and Ease of Use
  63. 63. Procore is hiring! (big surprise) http://procore.com/careers
  64. 64. Thank you! Questions? aj.bahnken@procore.com / @ajvbahnken
  65. 65. Further Resources Tour of Go Effective Go (required reading) Great intoduction to using SQL within Go Why we import drivers with '_'
  66. 66. Sources Performance Graph

×