Getting Started with Go

  • 7,693 views
Uploaded on

This presentation was given as a Workshop at OSCON 2014. …

This presentation was given as a Workshop at OSCON 2014.

New to Go? This tutorial will give developers an introduction and practical
experience in building applications with the Go language. Gopher Steve Francia,
Author of [Hugo](http://hugo.spf13.com),
[Cobra](http://github.com/spf13/cobra), and many other popular Go packages
breaks it down step by step as you build your own full featured Go application.

Starting with an introduction to the Go language. He then reviews the fantastic
go tools available. With our environment ready we will learn by doing. The
remainder of the time will be dedicated to building a working go web and cli
application. Through our application development experience we will introduce
key features, libraries and best practices of using Go.

This tutorial is designed with developers in mind. Prior experience with any of the
following languages: ruby, perl, java, c#, javascript, php, node.js, or python
is preferred. We will be using the MongoDB database as a backend for our
application.

We will be using/learning a variety of libraries including:

* bytes and strings
* templates
* net/http
* io, fmt, errors
* cobra
* mgo
* Gin
* Go.Rice
* Cobra
* Viper

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,693
On Slideshare
0
From Embeds
0
Number of Embeds
25

Actions

Shares
Downloads
236
Comments
0
Likes
25

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Building   your  first   Go  App 1 Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
  • 2. @spf13 •Author of Hugo, Cobra, Viper & More •Chief Developer Advocate for MongoDB •Gopher 2
  • 3. My  Go  Story •I write a medium size blog •Frustrated with Wordpress performance & maintenance •Existing Static Site Generators had complicated installations & were very slow •Began writing a S.S.G. in Go called Hugo 1.5 years ago 4
  • 4. hugo.spf13.com
  • 5. Hugo •Hugo is 2nd fastest growing Static Site Generator (40-50 * a week on github) •Hugo is one of the fastest Static Site Generators. 1000x faster than Jekyll •Easiest Installation of any SSG. Download and run •2nd most contributors of any Go project (Docker 1st)6
  • 6. Plan •Introduce Go Language •Introduce Tools & Libraries •Build our application •Tell the world how awesome we are 8
  • 7. Ground  Rules •Workshops are hard, everyone works at a different pace •We will move on when about 50% are ready •Slides are online, feel free to work ahead or catch up 9
  • 8. http:// spf13.com/ presentation/ first-go-app/ 10
  • 9. Installing  Go •You should already have Go installed •If you don’t, do it NOW •Installation Guide 11
  • 10. Installing  MongoDB •You should already have MongoDB installed •If you don’t, do it NOW •Installation Guide 12
  • 11. Git  &  Mercurial •Lastly we need Git & Mercurial •http://git-scm.com/downloads •http://mercurial.selenic.com/wiki/Download 13
  • 12. Introduction     to  Go 14
  • 13. Why  Another  Language? • Software is slow • Sofware is hard to write • Software doesn’t scale well 15
  • 14. Go  is  Fast • Go execution speed is close to C • Go compile time rivals dynamic interpretation 16
  • 15. Go  is  Friendly • Feels like a dynamic language in many ways • Very small core language, easy to remember all of it • Single binary installation, no dependencies • Extensive Tooling & StdLib 17
  • 16. Go  is  Concurrent • Concurrency is part of the language • Any function can become a goroutine • Goroutines run concurrently, communicate through channels • Select waits for communication on any of a set of channels 18
  • 17. Go’s  Inspiration •C: statement and expression syntax •Pascal: declaration syntax •Modula 2, Oberon 2: packages •CSP, Occam, Newsqueak, Limbo, Alef: concurrency •BCPL: the semicolon rule •Smalltalk: methods •Newsqueak: <-, := •APL: iota 19 … AND lessons good and bad from all those plus: C++, C#, Java, JavaScript, LISP, Python, Scala, ...
  • 18. — txxxxd “Most  of  the  appeal   for  me  is  not  the   features  that  Go  has,   but  rather  the   features  that  have   been  intentionally  left   out.” 20
  • 19. — Rob Pike “Why  would  you  have   a  language  that  is   not  theoretically   exciting?  Because  it’s   very  useful.” 21
  • 20. package main ! import "fmt" ! func main() { fmt.Println(“Hello, 世界") } Hello  World 22
  • 21. Package •Not classes or objects •No subpackages •Fundamental building block in Go •Visibility is package-level, not type-level •Programs run “main” package 23 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 22. import •Provides access to a package •Package consists of types, functions, etc… •No circular dependencies 24 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 23. “fmt" •Package path is just 
 a string •Standard library sits at the root (no path) •Core language is very small •Most functionality is in Stdlibs 25 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 24. func •Declaring a function •Same syntax for methods, anonymous functions & closures •main.main() is the function run when the program is executed 26 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 25. Main •No void •No return value •No function args 
 (command line is in os package) 27 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 26. { } •Braces not spaces •Feels like C ( & most languages ) •Newline must be after brace 
 (semicolons are inserted during compiliation) 28 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 27. fmt •Accessing the package 
 imported above •Everything visible in the package will be accessible through the name (or alias if provided) 29 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 28.  Println • Println, not println • Println, not print_ln •Capital for export •Variadic function using reflection 30 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 29. “Hello,  世界” •UTF-8 input source •Strings are immutable •Strings are UTF-8 encoded •String is a built-in type 31 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  • 30. Rob  Pike  did  it  better http://confreaks.com/videos/3419- gophercon2014-opening-day-keynote 32
  • 31. Tons  more… •types •constants •methods •interfaces •pointers •control flow •conditionals •tools •packages •concurrency 33
  • 32. Effective  Go 34
  • 33. Lets  write   some  code   ! Introduce  the     go  tools 35
  • 34. package main ! import "fmt" ! func main() { fmt.Println("Hello,World") } Hello  World 36 hello.go
  • 35. ❯ go run hello.go
 
 Hello,World Go  Run 37
  • 36. ❯ go build hello.go ! ❯ls -lh
 1.7M Jun 17 22:44 hello
 72B Jun 17 22:43 hello.go ! ❯ ./hello
 Hello, World Go  Build 38
  • 37. Go  Fmt •Automatically formats Go source code •Ends stylistic debates •Integration with editors (vim, emacs, others) •Can also refactor code 
 (see http://spf13.com/post/go-fmt ) 39
  • 38. go  Test   •Enables easy testing of application •Integrates with the testing package •Supports benchmark, functional & unit style testing •Combine with ‘looper’ to have realtime feedback 40
  • 39. package main ! import "testing" ! func TestOne(t *testing.T) { one := false if !one { t.Errorf(“Test Failed”) } } Hello  Test 41 hello_test.go
  • 40. ❯ go test ./... --- FAIL: TestOne (0.00 seconds) hello_test.go:8: Test failed FAIL FAIL_/Code/firstApp0.012s ! ! ❯ go test ./... ok _/Code/firstApp0.012s Go  Test 42
  • 41. Go   Planet 43The planet logo is based on the Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
  • 42. Why  Planet? •CLI application •Web application •Good introduction •Right sized •Database (MongoDB) •Concurrency 45
  • 43. Steps 0. Env Setup 1. Commands 2. Configuration 3. Working with Feeds 4. DB as Storage 5-6. Web Server 7-8. DB -> Templates 9. & 10. Add Polish 46
  • 44. Step  0   
 Setting  up  our   environment 47
  • 45. Go  PAth   ! $GOPATH 48
  • 46. Go  path •The GOPATH environment variable specifies the location of your workspace (and dependencies) •It must not be the same path as your Go installation 49
  • 47. ❯ mkdir $HOME/go
 ! ❯ export 
 GOPATH=$HOME/go Setting  up 50
  • 48. export GOPATH=$USER/go export GOROOT=`go env GOROOT` PATH=$PATH:$GOPATH/bin Linux  &  Mac 51 Add this to your ~/.bashrc (or equivalent)
  • 49. Windows •Control panel > System 52
  • 50. Go  PAth •/home/user/gocode/ ($GOPATH) • src/ (put your source code here) • bin/ (binaries installed here) • pkg/ (installed package objects) 53
  • 51. Creating  our   project 54
  • 52. Project  Dir •Make a directory inside of $GOPATH/src •I like mine to be in 
 $GOPATH/src/github.com/spf13/PROJECTNAME •It can not be a symlink •We will call this our working directory or $wd 55
  • 53. package main ! import "fmt" ! func main() { fmt.Println(“My Project") } main.go 56
  • 54. ❯ go run main.go
 
 My Project Run  it 57
  • 55. 58 I’m on the #gopath with @spf13 at #OSCON
 http://j.mp/onGoPath Tweet  Break
  • 56. Step  1   
 Creating  the   Command(s) 59
  • 57. Defining  
 our  first   command 60
  • 58. Structs •Short for structure •Objectish … blend of data & methods, but no inheritance •Really cheap. Struct{} is free (0 bytes). 61
  • 59. Functions •Can have multiple return values •No overloading •No optional parameters (but variadic) 62
  • 60. First  class  Functions •Function literals are anonymous functions in Go •Can assign to a variable or field •Can use as function parameters or return values •Can be created as a goroutine 63
  • 61. pointers •Function calls copy arguments into the function •Pointers reference a location in memory where a value is stored rather than the value itself •In Go a pointer is represented using the * •The & operator gives us the address of a variable •Go automatically dereferences pointers when using “.” 64
  • 62. cobra •A CLI Commander •Easy management of commands & flags •Used by bitbucket, openshift & lots more 65 http://fav.me/d5gkby7
  • 63. go get -u 
 github.com/spf13/cobra Getting  Cobra 66
  • 64. ❯ mkdir $wd/commands Commands  Dir 67
  • 65. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/cobra" ) planet.go  (1) 68
  • 66. Path  =  Url go get uses it to install 69
  • 67. var RootCmd = &cobra.Command{ Use: "dagobah", Short: `Dagobah is an awesome planet style RSS aggregator`, Long: `Dagobah provides planet style RSS aggregation. It is inspired by python planet. It has a simple YAML configuration and provides it's own webserver.`, Run: func(cmd *cobra.Command, args []string) { fmt.Println("Dagobah runs") }, } planet.go  (2) 70
  • 68. executing   cobra’s   command 71
  • 69. Variable  Assignment •‘:=‘ is the short variable declaration operator
 Declares, creates & assigns simultaneously while infering type •Assignment via ‘=‘ requires prior declaration •var foo int = 10 is the same as foo := 10 72
  • 70. Error  Handling •Errors are not exceptional. Should be part of the language •Typically one of the return values •No exceptions in Go fd, err := os.Open("test.go")
 if err != nil {
 log.Fatal(err)
 } 73
  • 71. (  Don’t)  Panic •Should rarely be used if ever •Performs a full stack trace dump for end users •Use only in the most dire circumstances (like package can’t find FileSystem) •Can use ‘recover’, but only inside deferred functions 74
  • 72. func Execute() { err := RootCmd.Execute() if err != nil { fmt.Println(err) os.Exit(-1) } } planet.go  (3) 75
  • 73. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package main ! import "github.com/spf13/firstGoApp-Planet/commands" ! func main() { commands.Execute() } main.go 76 Make sure to use your path
  • 74. ❯ go run main.go
 Dagobah runs ! ❯ go run main.go help Running  it 77
  • 75. Getting  Help •https://godoc.org/github.com/spf13/cobra •https://github.com/spf13/firstGoApp-Planet/ tree/step1 78
  • 76. Creating #CobraCommands with @spf13 at #OSCON http://j.mp/cobracmd Tweet  Break
  • 77. Step  2   
 Easy  Configuration 80
  • 78. Handing   Config  files 81
  • 79. Package  Identifiers •Export with Capital name •No automatic getters & setters •var field … func Field() … func SetField() 
 are appropriate names 82
  • 80. init() •Each source file can have one (or multiple) •Niladic function to setup state 
 (function without return value) •Called after all variable declarations in the package 83
  • 81. Scoping •Very granular scoping. •Variables declared inside { } only survive in that cotrol structure (for, if, etc) •The := trap
 if := is used to declare x and x already exists in an outer scope a new variable x will be created (shadowing) 84
  • 82. package main ! import ( "errors" "fmt" ) ! var Foo int ! func Zero() (int, error) { return 0, errors.New("yo") } ! ! func main() { ! Foo = 10 ! fmt.Println("outer:", Foo) // 10 ! if Foo, err := Zero(); err != nil { fmt.Println("inner:", Foo) // 0 } ! fmt.Println("outer again:", Foo) // still 10 } :=  Trap 85http://play.golang.org/p/i4L9Ao1P65
  • 83. Viper •A configuration manager •Easy loading of config files •Registry for application settings •Designed to work well with (or without) Cobra 86
  • 84. go get -u 
 github.com/spf13/viper Getting  Viper 87
  • 85. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/cobra" "github.com/spf13/viper" ) planet.go  (1) 88
  • 86. var CfgFile string ! func init() { cobra.OnInitialize(initConfig)
 RootCmd.PersistentFlags().StringVar(&CfgFile, "config", "", "config file (default is $HOME/dagobah/config.yaml)") } ! func initConfig() { if CfgFile != "" { viper.SetConfigFile(CfgFile) } viper.SetConfigName("config") viper.AddConfigPath("/etc/dagobah/") viper.AddConfigPath("$HOME/.dagobah/") viper.ReadInConfig() } planet.go  (2) 89
  • 87. Creating  our   config  file 90
  • 88. appname: "Dagobah" feeds: - "http://spf13.com/index.xml" - "http://dave.cheney.net/feed" - "http://www.goinggo.net/feeds/posts/default" - "http://blog.labix.org/feed" - "http://blog.golang.org/feed.atom" ! $HOME/.dagobah/config.yaml 91
  • 89. trying  it  out 92
  • 90. var RootCmd = &cobra.Command{ Use: "...", Short: `...`, Long: `...`, Run: rootRun, } ! func rootRun(cmd *cobra.Command, args []string) { fmt.Println(viper.Get("feeds")) fmt.Println(viper.GetString("appname")) } planet.go  (3) 93
  • 91. ❯ go run main.go
 [http://spf13.com/index.xml http:// dave.cheney.net/feed http:// www.goinggo.net/feeds/posts/default http://blog.labix.org/feed http:// blog.golang.org/feed.atom]
 
 Dagobah Running  it 94
  • 92. I got bit by the #goviper at #OSCON 
 http://j.mp/go-viper Tweet  Break
  • 93. https:// github.com/spf13/ firstGoApp- Planet/tree/ step2 96
  • 94. Step  3   
 Working  with  Feeds 97
  • 95. Finding   Libraries 98
  • 96. Go-Search.org 99
  • 97. Go  wiki 100
  • 98. go-pkg-rss •Fetch Rss and Atom feeds from the internet •Supports RSS .92, 1.0, 2.0 & Atom 1.0 •Respects TTL, SkipDays, CacheTimeout, etc in the feeds 101
  • 99. go get -u 
 github.com/jteeuwen/go- pkg-rss Getting  Go-PKG-RSS 102
  • 100. boilerplate  
 stuff 103
  • 101. import ( "time" ... ) ! func addCommands() { RootCmd.AddCommand(fetchCmd) } ! func Execute() { addCommands() ... } planet.go 104
  • 102. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" "time" ! rss "github.com/jteeuwen/go-pkg-rss" "github.com/spf13/cobra" "github.com/spf13/viper" ) Fetch.Go  (1) 105
  • 103. Creating  our   own  type 106
  • 104. Named  Types •Can be any known type (struct, string, int, slice, a new type you’ve declared, etc) •Can have methods declared on it •Not an alias. Explicit type. 107
  • 105. Slices  &  arrays •Array is fixed in length •Slice can be viewed as a dynamic array (can grow) •Slice is a view of an array •Slice is the core data structure in Go for lists of data 108
  • 106. type Config struct { Feeds []string Port int } Fetch.Go  (2) 109
  • 107. Defining  the   fetch   command 110
  • 108. New(    ) •new(T) Does 2 things at once: •Allocates new zeroed value of type T •Returns a pointer to the new value ( *T) •Used for all non-reference types 111
  • 109. Make(  ) •Creates slices, maps, and channels only •These types are special “reference” types •Makes a initialized (not zeroed) value of type T •Does not return a pointer 112
  • 110. Composite  Literals •An expression that creates a new value each time it is evaluated •We’ve already used these quite a bit •Can also be create values of arrays, slices, and maps •Pseudo Constructor •eg File{fd: fd, name: name} 113
  • 111. Constructors •Not built into Go •Use Composite Literals when possible •If initialization is needed use a factory •Convention is to use New___() •… or New() when only one type is exported in the package 114
  • 112. var fetchCmd = &cobra.Command{ Use: "fetch", Short: "Fetch feeds", Long: `Dagobah will fetch all feeds listed in the config file.`, Run: fetchRun
 } ! func init() { fetchCmd.Flags().Int("rsstimeout", 5, "Timeout (in min) for RSS retrival") viper.BindPFlag("rsstimeout", fetchCmd.Flags().Lookup("rsstimeout")) } Fetch.go  (3) 115
  • 113. Fetching  in  a   goroutine 116
  • 114. GoRoutines •A function executing concurrently with other goroutines in the same address space •Lightweight •Not threads, coroutines, or processes 117
  • 115. Channels •Channels are lightweight •Synchronize goroutines by communication rather than locking shared memory •Channel <- Sender —- Writes block when channel is full •<- Channel —- Reads blocks waiting for something to be read •Great talk on channels given at GopherCon
 http://confreaks.com/videos/3422-gophercon2014-a-channel-compendium 118
  • 116. Marshaling  into •We can’t return a T (no generics) •We can reflect on any value of any type •Marshal accepts the address of the value so it can manipulate the value directly 119
  • 117. func fetchRun(cmd *cobra.Command, args []string) { 
 Fetcher() ! sigChan := make(chan os.Signal, 1) signal.Notify(sigChan, os.Interrupt) <-sigChan } Fetch.go  (4) 120
  • 118. func Fetcher() { var config Config if err := viper.Marshal(&config); err != nil { fmt.Println(err) } ! for _, feed := range config.Feeds { go PollFeed(feed.Url) } } Fetch.go  (5) 121
  • 119. Building  our   fetcher 122
  • 120. Learning  about  go-pkg-rss •How do we learn how to use the package? 123
  • 121. Learning  about  go-pkg-rss •github.com/jteeuwen/go-pkg-rss •Readme is pretty lacking •godoc.org/github.com/jteeuwen/go-pkg-rss 124
  • 122. godoc.org/github.com/jteeuwen/go-pkg-rss 125
  • 123. Go  Search 126
  • 124. For •For, Foreach & While are all spelled ‘for’ in Go for init; condition; post { } 
 // Like a C for for condition { } 
 // Like a C while for { } 
 // Like a C for(;;) 127
  • 125. func PollFeed(uri string) { timeout := viper.GetInt("RSSTimeout") if timeout < 1 { timeout = 1 } feed := rss.New(timeout, true, chanHandler, itemHandler) ! for { if err := feed.Fetch(uri, nil); err != nil { fmt.Fprintf(os.Stderr, "[e] %s: %s", uri, err) return } ! fmt.Printf("Sleeping for %d seconds on %sn", feed.SecondsTillUpdate(), uri) time.Sleep(time.Duration(feed.SecondsTillUpdate() * 1e9)) } } Fetch.GO  (6) 128
  • 126. func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %sn", len(newchannels), feed.Url) } ! func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %sn", len(newitems), feed.Url) } Fetch.go  (7) 129
  • 127. ❯ go run dagobah.go fetch --rsstimeout=1
 
 15 new item(s) in http://spf13.com/index.xml
 1 new channel(s) in http://spf13.com/index.xml
 Sleeping for 300 seconds on http://spf13.com/index.xml
 25 new item(s) in http://www.goinggo.net/feeds/posts/default
 1 new channel(s) in http://www.goinggo.net/feeds/posts/default
 Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/ default
 10 new item(s) in http://dave.cheney.net/feed
 1 new channel(s) in http://dave.cheney.net/feed
 Sleeping for 299 seconds on http://dave.cheney.net/feed
 Running  it 130
  • 128. Tweet  Break I wrote my first goroutine at #OSCON with @spf13
  • 129. https:// github.com/spf13/ firstGoApp- Planet/tree/ step3 132
  • 130. Step  4   
 Storing  in  
 the  Database 133
  • 131. Pretty •github.com/kr/pretty •pretty.Println(X,Y,Z) 134
  • 132. MongoDB •Open Source •Document Database •Document == Struct or Document == Map •Works seamlessly with Go (and other languages) 135
  • 133. MongoDB •Fast & Scalable •Used by Disney, IBM, Metlife, eBay, Forbes, Craigslist, Cisco, Stripe 136
  • 134. Mgo •Developed by the open source community •Heavily used throughout Go community •"mgo is the best database driver I've ever used."
 — Patrick Crosby, Founder of StatHat •"mgo and Go are a pair made in heaven.
 — Brian Ketelsen, Author of GopherTimes.com137
  • 135. go get -u gopkg.in/mgo.v2 Getting  mgo 138
  • 136. boilerplate
 stuff 139
  • 137. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/viper" "gopkg.in/mgo.v2" ) ! var mongodbSession *mgo.Session ! func init() { RootCmd.PersistentFlags().String("mongodb_uri", "localhost", "host where mongoDB is") viper.BindPFlag("mongodb_uri", RootCmd.PersistentFlags().Lookup(“mongodb_uri")) CreateUniqueIndexes() } MongoDB.go  (1) 140
  • 138. Setting  up  a   DB  session 141
  • 139. func DBSession() *mgo.Session { if mongodbSession == nil { uri := os.Getenv("MONGODB_URI") if uri == "" { uri = viper.GetString("mongodb_uri") ! if uri == "" { log.Fatalln("No connection uri for MongoDB provided") } } ! var err error mongodbSession, err = mgo.Dial(uri) if mongodbSession == nil || err != nil { log.Fatalf("Can't connect to mongo, go error %vn", err) } ! mongodbSession.SetSafe(&mgo.Safe{}) } return mongodbSession } MongoDB.go  (2) 142
  • 140. setting  up   some   shortcuts 143
  • 141. ! func DB() *mgo.Database { return DBSession().DB(viper.GetString("dbname")) } ! func Items() *mgo.Collection { return DB().C("items") } ! func Channels() *mgo.Collection { return DB().C("channels") } MongoDB.go  (3) 144
  • 142. Setting  up   indexes 145
  • 143. func CreateUniqueIndexes() { idx := mgo.Index{ Key: []string{"key"}, Unique: true, DropDups: true, Background: true, Sparse: true, } ! if err := Items().EnsureIndex(idx); err != nil { fmt.Println(err) } ! if err := Channels().EnsureIndex(idx); err != nil { fmt.Println(err) } } MongoDB.go  (4) 146
  • 144. Preparing   the  data 147
  • 145. Data  Prep •Feeds are dirty •Go doesn’t allow “patching” of structs (not how structs work) •MongoDB works with native types & named types 148
  • 146. Tags  (Annotations) •String literal following a field declaration •Provides additional information during reflection •Used by JSON, BSON, YAML, etc 149
  • 147. for  x,y  :=  range •Provides a way to iterate over an array, slice, string, map, or channel. •Like Foreach or Each in other languages •x is key, y is value… can ignore by setting to _ 150
  • 148. Append •Append is really awesome, feels like push, but does a lot more. •Append will grow a slice (both length & capacity) •… and copy the underlying array if needed 151
  • 149. type Itm struct { Date time.Time Key string ChannelKey string Title string FullContent string Links []*rss.Link Description string Author rss.Author Categories []*rss.Category Comments string Enclosures []*rss.Enclosure Guid *string `bson:",omitempty"` Source *rss.Source PubDate string Id string `bson:",omitempty"` Generator *rss.Generator Contributors []string Content *rss.Content Extensions map[string]map[string][]rss.Extension } fetch.go 152
  • 150. func itmify(o *rss.Item, ch *rss.Channel) Itm { var x Itm x.Title = o.Title x.Links = o.Links x.ChannelKey = ch.Key() x.Description = o.Description x.Author = o.Author x.Categories = o.Categories x.Comments = o.Comments x.Enclosures = o.Enclosures x.Guid = o.Guid x.PubDate = o.PubDate x.Id = o.Id x.Key = o.Key() x.Generator = o.Generator x.Contributors = o.Contributors x.Content = o.Content x.Extensions = o.Extensions x.Date, _ = o.ParsedPubDate() ! if o.Content != nil && o.Content.Text != "" { x.FullContent = o.Content.Text } else { x.FullContent = o.Description } ! return x } fetch.go 153
  • 151. type Chnl struct { Key string Title string Links []rss.Link Description string Language string Copyright string ManagingEditor string WebMaster string PubDate string LastBuildDate string Docs string Categories []*rss.Category Generator rss.Generator TTL int Rating string SkipHours []int SkipDays []int Image rss.Image ItemKeys []string Cloud rss.Cloud TextInput rss.Input Extensions map[string]map[string] []rss.Extension Id string Rights string Author rss.Author SubTitle rss.SubTitle } fetch.go 154
  • 152. func chnlify(o *rss.Channel) Chnl { var x Chnl x.Key = o.Key() x.Title = o.Title x.Links = o.Links x.Description = o.Description x.Language = o.Language x.Copyright = o.Copyright x.ManagingEditor = o.ManagingEditor x.WebMaster = o.WebMaster x.PubDate = o.PubDate x.LastBuildDate = o.LastBuildDate x.Docs = o.Docs x.Categories = o.Categories x.Generator = o.Generator x.TTL = o.TTL x.Rating = o.Rating x.SkipHours = o.SkipHours x.SkipDays = o.SkipDays x.Image = o.Image x.Cloud = o.Cloud x.TextInput = o.TextInput x.Extensions = o.Extensions x.Id = o.Id x.Rights = o.Rights x.Author = o.Author x.SubTitle = o.SubTitle ! ... ! return x } fetch.go 155
  • 153. ... ! ! var keys []string for _, y := range o.Items { keys = append(keys, y.Key()) } x.ItemKeys = keys ! ... fetch.go 156
  • 154. Storing   Feeds  &   Items 157
  • 155. The  Blank  Identifier  ‘_’ •Used to ignore return values from a function •Used with imports as an alias • Allows import of unused packages so init functions can be executed 158
  • 156. func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %sn", len(newchannels), feed.Url) for _, ch := range newchannels { chnl := chnlify(ch) if err := Channels().Insert(chnl); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } } fetch.go 159
  • 157. func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %sn", len(newitems), feed.Url) for _, item := range newitems { itm := itmify(item, ch) if err := Items().Insert(itm); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } } fetch.go 160
  • 158. Running   MongoDB 161
  • 159. ❯ mongod
 
 MongoDB starting : pid=51054 port=27017 dbpath=/data/db Running  MongoDB 162
  • 160. ❯ mongo
 
 
 
 Mongo  Shell 163
  • 161. ❯ go run dagobah.go fetch --rsstimeout=1
 
 15 new item(s) in http://spf13.com/index.xml
 1 new channel(s) in http://spf13.com/index.xml
 Sleeping for 300 seconds on http://spf13.com/index.xml
 25 new item(s) in http://www.goinggo.net/feeds/posts/default
 1 new channel(s) in http://www.goinggo.net/feeds/posts/default
 Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/ default
 10 new item(s) in http://dave.cheney.net/feed
 1 new channel(s) in http://dave.cheney.net/feed
 Sleeping for 299 seconds on http://dave.cheney.net/feed
 Running  planet 164
  • 162. > use dagobah > db.items.findOne() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" Looking  at  the  Data 165
  • 163. _ID •MongoDB’s primary key •Default is to use an “object_id” •Object_id guaranteed to be unique across your entire cluster •MongoDB will create it for you on insert 166
  • 164. Tweet  Break Feeding mgo at #OSCON with @spf13 http://j.mp/mongomgo
  • 165. https:// github.com/spf13/ firstGoApp- Planet/tree/ step4 168
  • 166. Step  5   
 Building  a  
 web  Server 169
  • 167. Lots  of  choices •Beego
 High-performance web framework •Gin
 Martini like with better performance •Goji
 A minimalistic web framework •Gorilla 
 A web toolkit •httprouter
 A high performance router •Mango 
 modular web-app framework like Rack •Martini 
 modular web apps & services •net/http
 Standard library web package •pat
 Sinatra style, by the author of Sinatra. •Revel
 A high-productivity web framework •tigertonic
 framework for JSON web services •traffic
 Sinatra inspired web framework for Go •web.go
 A simple framework to write webapps in Go. 170
  • 168. Gin •Compatible with net/http •Uses Httprouter (really fast) •Familiar/Friendly Interface •Works well with JSON •Had to pick something 171
  • 169. go get -u
 github.com/gin-gonic/gin Getting  Gin 172
  • 170. boilerplate  
 stuff 173
  • 171. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" ) Server.Go  (1) 174
  • 172. Defining  the   Server   command 175
  • 173. var serverCmd = &cobra.Command{ Use: "server", Short: "Server for feeds", Long: `Dagobah will serve all feeds listed in the config file.`, Run: serverRun, } ! func init() { serverCmd.Flags().Int("port", 1138, "Port to run Dagobah server on") viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) } server.go  (2) 176
  • 174. func addCommands() { RootCmd.AddCommand(fetchCmd) RootCmd.AddCommand(serverCmd) } planet.go 177
  • 175. Defining  our   first  route 178
  • 176. func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go  (3) 179
  • 177. ❯ go run dagobah.go server Running  it 180
  • 178. Check  it  out http://localhost:1138 181
  • 179. Tweet  Break I wrote a webserver in go using Gin 
 with @spf13 at #OSCON http://j.mp/gin-go
  • 180. https:// github.com/spf13/ firstGoApp- Planet/tree/ step5 183
  • 181. Step  6   
 Building  a  
 Dynamic  Server 184
  • 182. Binary  Problem? •Go ships as a single binary •Need to load files from within the binary •Don’t want to embed by hand 185
  • 183. go.rice •Nicely allows local loads in dev, and embedded loads when executing binary •Stable, but not very mature yet •Best option I know of 186
  • 184. go get -u 
 github.com/GeertJohan/ go.rice Getting  Go.Rice 187
  • 185. Serving   Static  Files 188
  • 186. Get  your   static  &   templates  on c.spf13.com/OSCON/step6-static.zip 189
  • 187. import ( "fmt" "html/template" "log" "net/http" ! "github.com/GeertJohan/go.rice" "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" ) server.go 190
  • 188. func serverRun(cmd *cobra.Command, args []string) { ...
 r.GET("/static/*filepath", staticServe) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go 191
  • 189. func staticServe(c *gin.Context) { static, err := rice.FindBox("static") if err != nil { log.Fatal(err) } ! original := c.Request.URL.Path c.Request.URL.Path = c.Params.ByName("filepath") fmt.Println(c.Params.ByName("filepath")) http.FileServer(static.HTTPBox()).ServeHTTP(c.Writer, c.Request) c.Request.URL.Path = original } server.go 192
  • 190. Loading  &   Serving     Templates 193
  • 191. func loadTemplates(list ...string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } return templates } server.go 194
  • 192. func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() templates := loadTemplates("full.html") r.SetHTMLTemplate(templates) ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! r.GET("/", homeRoute) r.GET("/static/*filepath", staticServe)
 ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go 195
  • 193. func homeRoute(c *gin.Context) { obj := gin.H{"title": "Go Rules"} c.HTML(200, “full.html", obj) } server.go 196
  • 194. ❯ go run dagobah.go server Running  it 197
  • 195. Check  it  out http://localhost:1138 198
  • 196. Tweet  Break No more mr rice guy 
 with @spf13 at #OSCON http://j.mp/go-rice
  • 197. https:// github.com/spf13/ firstGoApp- Planet/tree/ step6 200
  • 198. Step  7   
 Connecting  the  Data   to  The  templates 201
  • 199. Go  Templates •Really nice to work with •Very simple, yet very powerful •Safe •Go Template Primer 202
  • 200. Understanding  
 the  data 203
  • 201. > db.items.find().sort({ "date" : -1 }).limit(1).pretty() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" } Looking  at  the  Schema 204
  • 202. Listing  POsts 205
  • 203. passing  data     into  the  template •Go’s templates accept a variety of data types •Gin’s default is ‘H’ : map[string]interface{} •Feels natural •All (exported) methods bound to values in the map are accessible inside the template 206
  • 204. func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts} c.HTML(200, "full.html", obj) } server.go 207
  • 205. ... <section id="latest-list" class="ui divided inbox selection list active tab" data-tab="recent"> {{ range $item := .posts }} {{ if $item.WorthShowing }} <a class="item" href=“#{{$item.Key | urlquery }}"> <div class="description">{{ $item.Title }}</div> <div class="right floated ui label small">{{ $item.Date.Format "Jan 2, 2006" }}</div> </a> {{ end }} {{ end }} </section> ... Full.html 208
  • 206. Adding  Helpers  
 to  itm 209
  • 207. func (i Itm) FirstLink() (link rss.Link) { if len(i.Links) == 0 || i.Links[0] == nil { return } return *i.Links[0] } ! func (i Itm) WorthShowing() bool { if len(i.FullContent) > 100 { return true } return false } fetch.go 210
  • 208. displaying  POsts 211
  • 209. ... <main> {{ with .message}}! <h1 class="ui header large"> {{.}} </h1>! {{ end }}! ! {{ range $item := .posts }}! {{ if $item.WorthShowing }}! <article data-key="{{$item.Key}}">! <header>! <a name="{{$item.Key}}">! <h1 class="ui header">{{$item.Title}}</h1>! <section class="meta-tags">! <a class="ui label large blue author" href="/channel/{{$item.ChannelKey}}">{{$item.Author.Name}}</a>! <span class="large ui label date">{{ $item.Date.Format "Jan 2, 2006" }}</span>! </section>! </header>! <section class="main-content">! {{$item.FullContent | html }}! </section>! <footer>! {{with $item.FirstLink}}! <a class="ui basic button" href="{{.Href}}">Source</a>! {{end}}! <div class="ui divider"></div>! </footer>! </article>! {{ end }}! {{ else }}! No Articles! {{ end }}! </main> ... full.html 212
  • 210. Adding  Template   Functions 213
  • 211. func loadTemplates(list ...string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } ! templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } ! funcMap := template.FuncMap{ "html": ProperHtml, "title": func(a string) string { return strings.Title(a) }, } ! templates.Funcs(funcMap) ! return templates } server.go 214
  • 212. func ProperHtml(text string) template.HTML { if strings.Contains(text, "content:encoded>") || strings.Contains(text, "content/:encoded>") { text = html.UnescapeString(text) } return template.HTML(html.UnescapeString(template.HTMLEscapeString(text))) } server.go 215
  • 213. Looking  Good http://localhost:1138 216
  • 214. Tweet  Break Conquering Go Templates 
 with @spf13 at #OSCON
  • 215. https:// github.com/spf13/ firstGoApp- Planet/tree/ step7 218
  • 216. Step  8   
 More  Routes 219
  • 217. Adding  Helpers 220
  • 218. func (c Chnl) HomePage() string { if len(c.Links) == 0 { return "" } ! url, err := url.Parse(c.Links[0].Href) if err != nil { log.Println(err) } return url.Scheme + "://" + url.Host } fetch.go 221
  • 219. ! func four04(c *gin.Context, message string) { c.HTML(404, "full.html", gin.H{"message": message, "title": message}) } server.go 222
  • 220. func AllChannels() []Chnl { var channels []Chnl r := Channels().Find(bson.M{}).Sort("-lastbuilddate") r.All(&channels) return channels } MongoDB.go 223
  • 221. Listing  channels 224
  • 222. func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", 
 "posts": posts, "channels": AllChannels()} c.HTML(200, "full.html", obj) } server.go 225
  • 223. ... <section id="channel-list" 
 class="ui large inverted vertical menu tab" data-tab=“channels"> <a class="item js-navigate" href="/" >! <i class="external home icon"></i>! <span class="ui name" href=""> All </span>! </a>! {{ range $channel := .channels }}! <div class="item js-navigate-channel" data-href="/channel/{{$channel.Key}}" 
 data-key="{{$channel.Key}}" style="cursor:pointer;">! <a href="{{$channel.HomePage}}" class="ui float right"><i class="external url sign icon"></i></a>! <span class="ui name" href=""> {{$channel.Title}} </span>! <span class="ui label">{{ len $channel.Links }}</span>! </div>! {{end}}! </section> ... full.html 226
  • 224. displaying   channels 227
  • 225. func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/channel/*key", channelRoute) r.Run(":" + port) } server.go 228
  • 226. func channelRoute(c *gin.Context) { key := c.Params.ByName("key") if len(key) < 2 { four04(c, "Channel Not Found") return } ! key = key[1:] ! fmt.Println(key) ! var posts []Itm results := Items().Find(bson.M{"channelkey": key}).Sort("-date").Limit(20) results.All(&posts) ! if len(posts) == 0 { four04(c, "No Articles") return } ... server.go  -  pt  1 229
  • 227. ...
 var currentChannel Chnl err := Channels().Find(bson.M{"key": key}).One(&currentChannel) if err != nil { if string(err.Error()) == "not found" { four04(c, "Channel not found") return } else { fmt.Println(err) } } ! obj := gin.H{"title": currentChannel.Title, "header": currentChannel.Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) } server.go  -  pt  2 230
  • 228. displaying  
 A  post 231
  • 229. func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/post/*key", channelRoute) r.Run(":" + port) } server.go 232
  • 230. func postRoute(c *gin.Context) { key := c.Params.ByName("key") ! if len(key) < 2 { four04(c, "Invalid Post") return } ! key = key[1:] ! var ps []Itm r := Items().Find(bson.M{"key": key}).Sort("-date").Limit(1) r.All(&ps) ! if len(ps) == 0 { four04(c, "Post not found") return } ... server.go  -  Pt  1 233
  • 231. ...
 ! var posts []Itm results := Items().Find(bson.M{"date": bson.M{"$lte": ps[0].Date}}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": ps[0].Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) } server.go  -  Pt  2 234
  • 232. Looking  Good http://localhost:1138 235
  • 233. Tweet  Break Making my own planet in #go with @spf13
 at #OSCON http://j.mp/go-planet
  • 234. https:// github.com/spf13/ firstGoApp- Planet/tree/ step8 237
  • 235. Step  9   
 Advanced  Routes 238
  • 236. Congrats •You’ve made it really far •Now the training wheels are off •Let’s try to add pagination & search 240
  • 237. Hints •You will need a new index •We’re talking full text here •You will also need a new route 241
  • 238. ... r.GET("/", homeRoute) r.GET("/post/*key", postRoute) r.GET("/search/*query", searchRoute) r.GET("/static/*filepath", staticServe) r.GET("/channel/*key", channelRoute) ... server.go 242
  • 239. Step  10   ! Add  Polish 243
  • 240. What’s  next  ? •Add ‘truncate’ command to remove old posts and channels •Change root command to run fetcher & server •Add infinite scroll capabilities 244
  • 241. What’s  next  ? •Make it so users can provide their own static files & templates •Documentation •Blog posts 245
  • 242. In  Conclusion 246 Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
  • 243. Thank  yOU 247
  • 244. What  have  we  done  ? •Written our first lines of Go •Written our first Go package •Written our own web server •Written our first Go application •Learned a lot & had fun doing it 248
  • 245. @spf13 •Author of Hugo, Cobra, Viper & More •Chief Developer Advocate for MongoDB •Gopher 249