Go Systems Administration Tutorial LISA17 Conference
•
1 like•1,424 views
Golang tutorial with a focus on system administration tasks and tools
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Go Systems Administration Tutorial LISA17 Conference
Similar to Go Systems Administration Tutorial LISA17 Conference (20)
More from Chris McEniry
Recently uploaded
Recently uploaded (20)
Go Systems Administration Tutorial LISA17 Conference
- 1. October 29–November 3, 2017 | San Francisco, CA www.usenix.org/lisa17 #lisa17 Close to the Edge Systems Administration in Go Chris "mac" McEniry https://app.strigo.io/event/iEHzHx4rhedZsJyTQ token: XGVI slack: #2017-tutorial-t1-go
- 3. Goals for Today • Sysadmin Topics • File systems, Web Servers, One-liners, Containers, SSH • Go Language Features • Interfaces, Type Assertions, Function Types, Anonymous Functions, Empty Interface, Anonymous Structs, GOOS, GOARCH • Go Libraries • filepath, os, syscall, syscall, syscall, crypto/x509, crypto/tls, net/http, encoding/json, expvar, x/ crypto/ssh • Go Tools (in Go or for use on Go) • go-bindata, go-bindata-assetfs, h2i, h2c, dep, gorram, expvarmon | 3
- 4. Prerequisites • Some Go experience • Mainly need to be able to read code and follow the basic control flow 4
- 5. Schedule 9:00 - 9:10 Administrivia 9:10 - 9:20 Introduction 9:20 - 10:30 Interfaces, Files, Web Servers, TLS, HTTP/2 10:30 - 11:00 Break 11:00 - 12:30 JSON, Package Management, One Liners, Cross compilation, Metrics, Containers, SSH 5
- 6. WARNING Going to cut some corners with our code. Any errors that arise - just going to panic. This is not what you want to normally do - this is just for the sake of demonstration. DO NOT USE ANY OF THIS CODE IN PRODUCTION WITHOUT TAKING A SECOND LOOK AT IT 6 https://pixabay.com/en/animals-black-and-white-bomb-boom-985500/
- 7. Working with the Examples • `go get github.com/cmceniry/cttesa` • All exercises are meant to work with `go run` • We'll continue after any example when ~50% of the attendees say to continue (by show of hands) 7 https://app.strigo.io/
- 8. Close to the Edge...
- 9. What do you remember as your most ingenious moment? 9 https://pixabay.com/p-1872376/?no_redirect
- 10. • Built something • Completed something • Used something in a way it wasn't intended • Succeeded even though you were blocked • Learned something 10 https://upload.wikimedia.org/wikipedia/commons/thumb/c/c8/Mr_pipo_Learning.svg/1000px-Mr_pipo_Learning.svg.png
- 11. "the quality of being clever, original, and inventive." 11 https://www.google.com/search?q=ingenuity
- 12. David Blank-Edelman's Over the Edge Systems Administration Homage 12 https://c1.staticflickr.com/7/6004/6013286848_c9d1782d7f_b.jpg
- 13. • Generalize • Use a new interface • Repurpose your tools • Do the smallest or wrong thing • Transports and Tunnels Over the Edge System Administration 13 https://pixabay.com/p-1966997
- 14. Close to the Edge • Experiment • Sometimes you do it the hard way • Look under the hood • Read the code • Read the docs • Have fun 14 https://pixabay.com/p-759080
- 15. • Look at your tools. What do they tell you about themselves? • Look at what your tools are operating on. What do the tools tell you about what they operate on? • What do the tools tell you about how you do work? 15
- 18. Go Types • Most Go Types represent some form of data • Boolean, Numeric, String, Array, Slice, Struct, Pointer, Map, Channel • The Interface Type represents behavior • Bit of a placeholder • "Any value whose type has these methods can be used here." • "Any value whose type can do at least what I need of it can be used here." • Can only use the methods of that interface • This is used a lot without even knowing it 18
- 19. io.Reader • Implements just one method: Read • https://golang.org/pkg/io/#Reader • godoc io | grep -A2 -m2 '^type Reader' • Side note: error is an Interface Type. It implements `Error() string` 19 type Reader interface { Read(p []byte) (n int, err error) }
- 20. cttesa/interface • Can only call `Read([]byte) (int,error)` on `data` • But that's all we need func saveFile(filename string, data io.Reader) error { ... nr, err := data.Read(buf) ... } 20
- 21. cttesa/interface • `a` and `b` are different different types, but both satisfy `io.Reader` so can be used for `saveFile` func main() { a := bytes.NewBuffer([]byte("file1n")) err := saveFile("file1", a) ... b := strings.NewReader("file2n") err = saveFile("file2", b) ... } 21
- 22. Another io.Reader • Find another io.Reader from the library • Use saveFile to write it to disk 22
- 23. Working with file metadata... 23 https://upload.wikimedia.org/wikipedia/commons/c/c2/Logo_X-FILES.png
- 24. syscall.Stat_t • Goal: Get the inode for a file • Go concept: Type Assertion, syscall.Stat_t 24
- 25. cttesa/inode • `s.Sys()` returns a interface which must be asserted to `*syscall.Stat_t` • `*syscall.Stat_t` struct has actual the `Ino` field func getInode(path string) (uint64, error) { s, err := os.Stat(path) ... stat, ok := s.Sys().(*syscall.Stat_t) ... return stat.Ino, nil } 25
- 26. filepath.Walk • Goal: Locate files with the same Inode • Go concepts: Function Type, filepath.Walk 26
- 27. cttesa/inode-walk • The `filepath` library defines a `WalkFunc` Function Type to be used to define any custom logic for it. We can define that and provide it to the `Walk` function. type WalkFunc func(path string, info os.FileInfo, err error) error ... func Walk(root string, walkFn WalkFunc) error 27 func walkFunc(path string, info os.FileInfo, err error) error { ... fun main() { filepath.Walk(".", walkFunc)
- 28. Anonymous Function • You don't have to define the function. It can be used as an expression inline - this is called an Anonymous Function • Complete `inode-anon` to use an anonymous function. Copy portions from `inode-walk` • Place the Println to add a prefix defined in the `main` function 28 func main() { prefix := "FOUND:" filepath.Walk(".", ...) // fmt.Printf("%s %s = %dn", prefix, path, ino) }
- 29. Why Anonymous Functions? • Inline • Locate code closer to where it is used • If only used once, no reason to separate it • Allows for closures • Wrapping up of variables • Access to inside of function which wouldn't be available otherwise 29
- 30. Returning Anonymous Function • Advanced: Do-on-your-own challenge.. • Write a function which takes in a string, the prefix, and returns a `filepath.WalkFunc` • Update `inode-anon` to use this function • Behaves the same as `inode-walk` 30
- 31. Need to serve files somehow.. 31 https://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/Kickstarter_logo.svg/2000px-Kickstarter_logo.svg.png
- 32. cttesa/kickstart • Goal: Deploy a web server to handle static files • Go concepts: net/http, +Function Type, +Interface 32
- 34. 34 https://golang.org/pkg/net/http/#Dir ==> An http.Dir value (acts as http.FileSystem) can be fed to http.FileServer ==> http.FileServer returns an http.Handler which can be fed to http.Handle
- 35. Setup the static server • Fill in `kickstart` with the appropriate line so that it will serve the files in the `data` directory 35 func main() { ... fmt.Println("Listening on :8080") http.ListenAndServe(":8080", nil) }
- 36. • Goal: Make the file server stand alone • Go concepts: go generate, go-bindata-assetfs 36
- 37. Single binary • One of Go's strengths is its ability to package up as a single binary. • What if you could package up resources in that binary as well? 37
- 39. go-bindata-assetfs data/... cat bindata_assetfs.go ... func assetFS() *assetfs.AssetFS { assetInfo := func(path string) (os.FileInfo, error) { return os.Stat(path) } for k := range _bintree.Children { return &assetfs.AssetFS{Asset: Asset, AssetDir: AssetDir, AssetInfo: assetInfo, Prefix: k} } cttesa/kickstart-embedded 39
- 41. go generate • Part of build tool which runs prior to build • Can trigger manually with `go generate` • Executes what's in a generate comment • Intention is for any code generation pieces 41 //go:generate echo foo
- 42. Setup Generate and assetFS • Update `kickstart-embedded` to use `assetFS` • Update `kickstart-embedded` to generate automatically • Build/run stand alone web server 42
- 44. • Goal: Add encryption to the web server • Go concept: net/http.Server, crypto/tls, crypto/x509 44
- 45. Custom http server configuration • Been using the default http server in net/http • Can setup custom http servers - different configs, multiple at once, etc • Define the server as a value and use that instead of the default library functions 45 http.ListenAndServe(":8080", nil) s := http.Server{Addr: ":8080"} err = s.ListenAndServe()
- 46. Generate certificates • Process • Generate Private Key • Generate Certificate Information • Self-sign Certificate with Private Key • Everything else is just manipulating it so the libraries take it 46
- 47. Two Certificate Types • crypto/x509.Certificate: Used for general certificate information • crypto/tls.Certificate: Used for certificates in the context of TLS • Support for Cert/Key together • Support for name/Cert mapping • Support for Certificate chain required by TLS 47
- 49. TLS Certificates • Need to generate the certificate using crypto/x509 • Then format it into crypto/tls 49
- 50. cttesa/tls • Has a helper function : generateTLSCert • Returns the tls.Certificate since that is what is used later • Start by generating the private key • Next go into certificate information... 50 func generateTLSCert() (*tls.Certificate, error) { k, err := rsa.GenerateKey(rand.Reader, 2048)
- 51. 51 t := x509.Certificate{ SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "localhost"}, NotBefore: time.Now().Add(-30 * time.Second), NotAfter: time.Now().Add(86400 * time.Second), KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{ x509.ExtKeyUsageServerAuth, }, } cttesa/tls
- 52. • Next step is to sign the certificate (see exercise). Need an crypto/x509 function that returns a certificate format (type or bytes or...) 52 https://golang.org/pkg/crypto/x509/#CreateCertificate
- 53. • Then need to format it for tls.Certificate (PEM vs DER). Looking at the crypto/tls library, we look for a function which returns a Certificate. 53 https://golang.org/pkg/crypto/tls/#X509KeyPair
- 54. cttesa/tls • We need to format the cert and key into a pem []byte certPem := pem.EncodeToMemory(&pem.Block{ Type: "CERTIFICATE", Bytes: c, }) keyPem := pem.EncodeToMemory(&pem.Block{ Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(k), }) 54
- 55. cttesa/tls • Setup the handler as before • Setup the net/http.Server value, with a crypto/tls.Config wired up to it. • And then can start the listen and serve loop, though in this case, it's ListenAndServerTLS err = s.ListenAndServeTLS("", "") 55 http.Handle("/", http.FileServer(http.Dir("."))) s := http.Server{...}
- 57. Generating Certs and Enabling TLS • Complete `tls/main.go` by • making the appropriate call to generate the x509.Certificate • making the appropriate call to generate the tls.Certificate • making the appropriate call to configure TLS • Can skip hostname verification • Listen on localhost:8443 57
- 58. Can generate as files... • https://golang.org/src/crypto/tls/generate_cert.go 58
- 59. http2 59
- 60. • Goal: Setup for HTTP/2 • Go concept: h2i, h2c, net/http 60
- 61. HTTP/2 • Binary protocol • Required encryption • Multiplexing of requests • Header compression • Wraps (essentially) HTTP/1.1 requests into frames 61
- 62. • Golang's net/http library already has HTTP/2 built in (if you run TLS) • Several tools in Go to test it • https://github.com/fstab/h2c • https://godoc.org/golang.org/x/net/http2/h2i 62
- 63. Access web server from h2i 63 shell$ h2i -insecure localhost:8443 ... h2i> headers (as HTTP/1.1)> GET / HTTP/1.1 (as HTTP/1.1)> ...
- 64. Access web server from h2c • Go back and restart the other kickstart examples. Try the same with localhost:8080 64 shell$ h2c start & ... shell$ h2c connect localhost:8443 shell$ h2c get / ... shell$ h2c stop ...
- 66. Reading JSON • Goal: Load configuration data from a JSON structure • Go concepts: Empty Interface{}, encoding/json 66
- 67. Parsing JSON/XML/YAML • Golang's encoding libraries behave the same way • Use reflection to determine how to convert data • Or use `map[string]interface{}` to create for key/value with the value being non-specific types 67
- 68. cttesa/json-empty • The JSON structure that we're going to work with • We're going to just try to pull "bar" out. { "foo": 123, "bar": "baz", "really": true } 68
- 69. cttesa/json-empty • Initialize a map to store data. Note the inline `interface{}` -- essentially an anonymous interface definition • Now use `json.Unmarshal` to load data into c from our source. Unmarshal works on a slice of bytes, so have to convert the string to allow it. c := make(map[string]interface{}) 69 json.Unmarshal([]byte(data1), &c)
- 70. cttesa/json-empty • Now, safely get the data out barEmpty, ok := c["bar"] if !ok { panic("bar missing") } bar, ok := barEmpty.(string) if !ok { panic("bar is not a string") } fmt.Printf("bar: %sn", bar) 70
- 71. Reading JSON using a struct • The constant conversion is fairly verbose. • Can rely on the library to do the work for us. 71
- 72. cttesa/json-struct • Need to define a struct type to hold the data and give form • Then can decode the exact same way, but without the same checks type config struct { Foo int64 Bar string Really bool } 72 c := &config{} json.Unmarshal([]byte(data1), &c) fmt.Printf("bar: %sn", c.Bar)
- 73. Read XML • Perform the same exercise with XML • Complete `xml-struct`: • Fill in the fields for `config` based on the XML input • Call the right function to convert `data1` to `c` 73
- 74. Writing JSON • Goal: Output JSON • Go concepts: Anonymous Struct, Inline Initialization 74
- 75. Writing JSON • Same reflection can be used to output JSON • But you don't have to do any of the checking • Becomes dealers choice for what to output from • Existing or inline map[string]interface{} • Existing or inline (anonymous) struct 75
- 76. Writing JSON • Complete `json-output` • All blocks should output a `Name` and `Value` 76
- 77. Do the same with XML • On your own... • Do `json-output`, but with encoding/xml instead of encoding/json 77
- 78. Struct Tags • On your own... • Investigate struct tags to map other fields into the structs • E.g. a JSON field "common_name" needs to map to camelcase "CommonName" field inside of a struct 78
- 80. Pulling in Others' Code • So far, everything we used has been a command, so we didn't look at dependencies, just used them via `go get...` • Now we want to pull in an actual code dependency. • In the previous JSON examples, we converted JSON to and from • map of the empty interfaces • structs • What if we want to skip the JSON part? 80
- 81. 81 https://godoc.org/github.com/mitchellh/mapstructure Why? Some functions work with pluggable backends but have a common interface. That interface may or may not be able to use fix types and so have to resort to some more abstract pieces. It's still nice to be able to convert those.
- 82. Go Package Management • Has be very limited* • Several different package managers sprung up to fill the void • Community finally started to push to a common one • Still being sorted out, so this might change... 82 https://pixabay.com/p-311665 https://pixabay.com/p-307592
- 83. dep • Command which will handle dependency • Looks through your code, sees what is dependent (imports) • Looks at the dependencies, see what they depend on, and repeat • Takes some input as to if there are any constraints (library versions which you have said you are locked to) • Solves to pull in the best version of dependencies 83
- 84. dep Structure • `vendor` directory in project would keep all of the upstream dependencies • `Gopkg.toml` has any constraints you want to put into it • `Gopkg.lock` is the current state of the dependency solver 84
- 85. dep Commands • `dep init`: Setup `Gopkg` and `vendor` • `dep ensure`: Regularly keep dependencies in alignment • Updates to your imports • Updates to upstream version • Updates to constraints 85
- 86. Use map structure • Complete `dep/main.go`: • Fill in the appropriate import statement • Fill in the appropriate data conversion function • Use `dep` to manage the dependency • Expected out is similar to JSON exercises 86 shell$ go run main.go Property1=100
- 88. Golang Oneliners • The need for `main.main` makes it hard to do a oneliner in Go. • However, if we make some assumptions about input and output, we can pull it off... 88
- 89. 89
- 90. 90 Lots of options for inferring input/ output streams
- 91. Play with gorram 91 shell$ gorram crypto/sha1 Sum /etc/hosts shell$ echo 12345 | gorram encoding/base64 StdEncoding.EncodeToString shell$ # is there an equivalent to decode base64? shell$ gorram net/http Get https://www.usenix.org/conference/lisa17 shell$ gorram math/rand Int shell$ gorman math/rand Int # run this twice - what's up?
- 92. Notes for the future... • Think about building library functions so that you can refer to them with tools like Gorram 92
- 93. expvar 93
- 94. • Goal: Monitoring Go processes • Go concept: expvar 94
- 97. Use expvar • Instrument `kickstart` using expvar • Use a browser or command line web client to get data from expvar endpoint • Examine the metrics with `expvarmon` 97
- 98. Caveats... • This is either a really good approach or a really bad approach - you have to decide for yourself • Can use different `net/http.Server` and `expvar.Handler` to move where the metrics are exposed (exercise for yourself) 98
- 100. Cross Compilation • The Go tool chain has cross compilation built in • Can easily build on one platform for many platforms • There are edges with cgo bindings and syscall specific items • Use GOOS (pronounced goose) and GOARCH (garch) to control the target of the build 100
- 101. cttesa/cc // +build darwin 101 // +build darwin,i386 // +build windows // +build linux,amd64
- 102. Attempt Cross Compile • `cc` has three files • `main-darwin.go` • `main-linux.go` • `main-windows.go` • Attempt to cross compile with various GOOS settings and see the results 102 shell$ GOOS=... go build . shell$ file cc* ...
- 103. Make your own containers... 103 https://pixabay.com/p-1096829
- 104. Containers • Goal: Build a limited container by hand • Go concepts: os/exec, syscall 104
- 105. Containers: The Short Short Version • Processes with: • Isolation (namespaces) - Visibility • Resource constraints (cgroups) • Privileges (capabilities) 105
- 106. e.g. chroot • The first partial namespace: chroot • Would give you a different perspective to the root file system • Extend this out to the full mnt ("ns") namespace, you can control completely different sets of mounts; not just one as a child of the other. 106
- 107. Other Namespace • UTS - UNIX Timesharing System (aka hostname/domainname) • Net • PID • IPC • User • Cgroup 107
- 109. cttesa/container • Containers are built on processes; new processes handled by os/ exec.Cmd • Can connect process inputs and outputs (just like pipes and redirection) • And start the command: cmd := exec.Command("/bin/bash") 109 cmd.Stdin = os.Stdin cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr err := cmd.Run()
- 112. A couple of isolations... • Finish `container/main.go` by setting the appropriate field on `cmd`. • Isolate hostname: Change the hostname inside and out and show no cross impact • Isolate network: Run it and see the difference on `ip addr` 112
- 114. • Goal: Setting up ssh in code • Go concepts: golang.org/x/crypto/ssh 114
- 115. Using the ssh library • Three parts to using it • Get any authentication credentials • Setup the configuration and connect to the server • Run session which execute the commands we use 115
- 118. cttesa/ssh • `getKey` gets the auth key needed for ssh key authentication func getKey() (ssh.Signer) { keyFile := "/home/got/.ssh/id_rsa" pemKey, err := ioutil.ReadFile(keyFile) ... signer, err := ssh.ParsePrivateKey(pemKey) ... return signer } 118
- 119. cttesa/ssh • `connect` uses the auth key to connect to the localhost func connect(host string, signer ssh.Signer) *ssh.Client { config := &ssh.ClientConfig{ User: "got", Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)}, HostKeyCallback: ssh.InsecureIgnoreHostKey(), } client, err := ssh.Dial("tcp", "127.0.0.1:22", config) ... return client 119
- 121. cttesa/ssh • Start by opening a new session. This is for a single command to run • As usual, we will want to clean up when we're done • Connect our process to the remote command by pulling out a StdinPipe session, err := client.NewSession() 121 defer session.Close() sin, err := session.StdinPipe()
- 122. cttesa/ssh • Start a command inside of the session • Perform any needed actions • Wait till the command finishes err = session.Start(...) 122 session.Wait()
- 123. "Copy" a file over ssh • Simulate the action of "echo input | ssh ..." • Finish `ssh/main.go` to simulate the action of `echo input | ssh ...` to "copy" a file over ssh • Fill in the command to execute on the remote side • Pipe data over the ssh session (`io.Copy` will help) 123
- 125. Some laughter Some thinking Goals for today 125 https://commons.wikimedia.org/wiki/Emoji#/media/File:Twemoji2_1f914.svghttps://commons.wikimedia.org/wiki/File:Twemoji2_1f602.svg
- 126. Questions Inspired by each Exercise • Interface: How much do we rely on concrete aspects instead of behaviors? • Files: How can we provide a smarter response to `find` results? • Kickstart: Can we package *everything* that we need up into a nice package? What are the trade offs? • TLS: Is it really that scary to be secure? • http2: How much is already built into what we're using? • JSON: How can we make data transformations easy? 126
- 127. Questions Inspired by each Exercise • dep: What if other people wanted to use this? • gorram: How do we structure our code so it's reusable, not just by other libraries, but also by other tools? • expvar: Can (should?) we use our application ports for signaling and monitoring as well? What are the tradeoffs? • GOOS/GOARCH: How portable is our code really? • Containers: What does this tell us about the underlying system? • SSH: Just because we're working code, are there other methods (e.g. shell-isms) that we can bring into our code? 127
- 128. October 29–November 3, 2017 | San Francisco, CA www.usenix.org/lisa17 #lisa17 Remember to fill in your tutorial evaluation! Thank You! T1 - Close to the Edge Systems Administration Chris "mac" McEniry