A Tale of Go REPL
motemen
Go Conference 2015 Summer @ Tokyo

About Me
- @motemen
- Web engineer
- Chief engineer at Hatena, Kyoto

Mackerel mackerel.io

Agenda
- Introduction of ! motemen/gore
- Gore’s architecture
- Implementation details

Gore /gɔː/

Case: Learning New Languages
- Playing with language features
- Trying libraries

My Case — Perl
- perl ~/sketch.pl
- Edit and Run

My Case — Ruby
- irb
- pry
- A “REPL”
(Read-Eval-Print-Loop)

My Case — Go?
- tmp.go
- “packge main” “func main()”
- Removing unused variables by hand
- Question: Can gophers haz a REPL?

Go "REPL" impls (AFAIK)
- ! sbinet/igo — built upon ! sbinet/go-eval, with liner
- ! emicklei/rango — importing, statements, listing variables
- ! vito/go-repl — importing, statements, useful commands
- ! d4l3k/go-pry — attaches to running program, uses reflection
- (play.golang.org)

Needed a REPL that:
- Evaluates statements
- Imports any module
- Pretty prints values automatically
- Completes code

The Difficulties
- How can we achieve Go semantics?
- No APIs provided
- Re-implement Go compiler? 😩

Solution: “go run”

Solution: “go run”
- An “eval” to Go
- Perfect implementation & Always up-to-date
- Dead simple
- In: a program Out: the result 🎉

Gore: A REPL using “go run”
go get github.com/motemen/gore

Caveats 🙏
- Gore runs all code input for each run
- Code with side effects will be repeated
- eg. Printing output / Sleeping

Stories Inside Gore

Overview: not a “REPL” exactly
- REPL: Read-Eval-Print-Loop
- No “Eval” and “Print” phase
- As we do them by “go run”
- Much like: Read-Generate-Run-Loop

Read-Gen-Run-Loop
1. Read user input to evaluate
2. Generate a .go file that prints result
3. “go run” it
4. Back to 1.

Step: Read
- Codes go under AST (abstract syntax tree) form
- Syntax checking
- Easier to manipulate
- Input String → AST → .go → “go run”

Read: Input
- ! peterh/liner
- Line editing
- History
- Supports Windows

Read: Input to AST
- package go/parser
- Go has rich support for parsing/typing its code
- Easy to generate & manipulate

Read: Input to AST
- Parse input as an expression
- If failed: a statement
- Otherwise: continue input (multi-line)

Read: Expression
- Easy
- Dedicated API
- parser.ParseExpr(in)

Read: Statements
- $
func (s *Session) evalStmt(in string) error {
src := fmt.Sprintf("package P; func F() { %s }", in)
f, err := parser.ParseFile(s.Fset, "stmt.go", src, 0)
if err != nil {
return err
}
enclosingFunc := f.Scope.Lookup("F").Decl.
(*ast.FuncDecl)
stmts := enclosingFunc.Body.List
}

Generate: The Delicious Part
- Append the input AST nodes to our main()
- Add the code to print the value

Printing Values
- The generated program prints values (not gore)
- “New Values” should be printed
- Expression: Its resulting value
- Statement: Values assigned

Printing Values: Expression
- User: foo.Bar(a+1)
- Gore: __gore_p(foo.Bar(a+1))

Printing Values: Statements
- ast.AssignStmt stands for assign/define
- User: a, b := foo()
- Gore: a, b := foo(); __gore_p(a, b)

Printing Values: __gore_p()
- “Pretty prints” values
- Defined along with the main()
- Depends on installed packages
- ! k0kubun/pp, ! davecgh/go-spew or plain old %#v

The Initial .go File (pp)
package main
import "github.com/k0kubun/pp"
func __gore_p(xx ...interface{}) {
for _, x := range xx {
pp.Println(x)
}
}
func main() {
// User input goes here
}

The Initial .go File (go-spew)
package main
import "github.com/davecgh/go-spew/spew"
func __gore_p(xx ...interface{}) {
for _, x := range xx {
spew.Printf("%#vn", x)
}
}
func main() {
// User input goes here
}

The Initial .go File (%#v)
package main
import "fmt"
func __gore_p(xx ...interface{}) {
for _, x := range xx {
fmt.Printf("%#vn", x)
}
}
func main() {
// User input goes here
}

Generate: Append to main()
- Just append those generated statements
s.mainBody.List = append(s.mainBody.List, stmts…)
- Now we’ve got the working code!
- Really? No! %

Go compiler complaints (you know)
- “x declared but not used”
- “p imported but not used”
- “no new variables of left side of :=“
- & Should remove these to a successful go run

Quick Fixing Erroneous Program
- “x declared but not used”
- “p imported but not used”
- “no new variables on left side of :=“
Use it!!
Anonymize it!!
Make it `=‘ !!

“declared but not used”
package P
func main() {
s := "hello"
}
package P
func main() {
s := "hello"
_ = s
}

”imported but not used”
package P
import "fmt"
func main() {
}
package P
import _ "fmt"
func main() {
}

“no new variables on left side of :=“
package P
func main() {
var a int
a := 1
}
package P
func main() {
var a int
a = 1
}

! motemen/go-quickfix to do the work
- Type check source code using go/types.Check()
- Catch errors and modify AST
- Packed with a bin
- goquickfix -w main.go

Gore specific quickfix
- __gore_p() for non-value expressions

Running
- Output the AST to file — go/printer
- Then go run
- If failed to run, revert the last input

Recap: Read-Gen-Quickfix-Run-Loop
1. Read and parse the input to obtain AST
2. Add printing function call __gore_p()
3. Append the code to main()
4. Quickfix it so that it compiles well
5. go run

PRs welcome! 🍻
- ! motemen/gore
- ! motemen/go-quickfix

htn.to/intern2015

A Tale Gore
motemen
Go Conference 2015 Summer @ Tokyo

Appendix:
More Features

Code Completion
- liner has support for completion
- Great tool for editors: ! nsf/gocode
- Server/client model
- And an interface to it: motemen/gore/gocode
- If gocode binary can be located, use it

Commands
- :print
- :write <file>
- :import <pkg>
- :doc <expr>

:import <pkg>
- Imports arbitrary packages
- Just append import statement to the file
- Unused imports are handled by quickfix

:doc <expr>
- Invokes go doc for given expression
- Package: :doc json
- Function: :doc json.Marshal
- Method: :doc json.NewEncoder().Encode