Illustrate different scenarios of using Metaprogramming Go with actual example; using py for quite PoC. Opening talk for PyCon 2014 China.

1. Metaprogramming Go
Weng Wei, Nov 2014

3. About Me
• Weng Wei, Chinese in Singapore
• Full-stack developer
• .net -> python -> go
• Passionate about code & life

4. Metaprogramming (MP)
• “writing of computer programs with the ability to
treat programs as their data...program could be
designed to read, generate, analyse and/or
transform other programs”
• allows programmers to minimize the number of
lines of code to express a solution => better
produtivity

5. MP is everywhere
• Compiler is the most common MP tool, it compile
high level code to lower one
• MP for manipulating other code is also common, for
example: C Macro

6. C Macro
#define list_for_each(pos, head)
for (pos = (head)->next; pos != (head);
pos = pos->next)

7. Metaprogramming Go？
• No macro
• No generic
• Reflection in Go is awful
• “ gopher has been hard working these years… to
solve problems don’t exist in other languages……” -
@laiyonghao

8. How hard working gopher
solve generic with py?

10. • StringSet& IntSet has exact code structure, only
types are different
• Classic usage of generic

13. Simple & Effective

14. auto generate
• MakeFile？
• Real-time file monitor: github.com/gorakhargosh/
watchdog

15. How about “ORM”?

16. We need DSL:
thrift

20. Same struct definition
• CRUD
• Full-text search
• User Permission
• RPC

21. Lots of boilerplate code
for obj passing, value
assign, display etc

26. • Generate them all!
• Generated code won’t have typo

27. Implementation

28. Struct Parsing
• ptsd: github.com/wickman/ptsd
• thrift lexer/parser using ply
• ply: www.dabeaz.com/ply/
• An implementation of lex and yacc parsing tools
for Python

29. code-gen template
• www.cheetahtemplate.org

39. YAML is very handy

40. Define workflow

43. • Similar to ORM: change DSL from thrift to YAML
• YAML
• More flexible
• Non-coder friendly

44. API
• Needed in different places：
• web
• mobile web
• mobile client
• Business are the same, but may need different
wrapping

46. Web Form

47. JSON HTTP

48. Mobile API

49. We should be lazy about
writing boilerplate
wrapping code

51. So, we have 300k80line of code,
mostly generated

52. • Above are “demo codes”
• Actual code are much more complicated
• ORM：github.com/sipin/xuanwu
• Just for referencing of idea; don’t use it
• No documentation
• No maintain
• We’ve restarted

53. Customisation？
• Generate once, and change
• Generate whenever template changes：
• Provide callback interface
• Method overwriting

56. • May use MP for generic, ORM, workflow, API
wrapping, and more
• Consider MP as code-refactoring
• Write the dirty code first
• Abstract similar code repeated to template
• Choose an DSL
• Python is simple & resourceful; handy for
metaprogramming prototype/PoC

57. Challenge of MP
• Choosing the right DSL
• Provide both:
• high level of abstraction
• low level customization
• www.joelonsoftware.com/articles/
LeakyAbstractions.html

58. Code generation
• Parse (DSL) & Bind (code-template) are easy
• just like HTML template
• Generated codes are simple, thus easy to debug
• Compiler checking for free

59. Reflection for LISP way？
• No compilation
• Error checking at runtime
• Implementation needs more challenger
• Codes are more elegant
• Yup, that could be better; so, how?

60. –Ken Thompson
“When in doubt, use brute force.”

61. Actually
• Python is not so fast
• After experiencing git’s extreme speed, you can’t tolerate hg’s “not so fast”
• It’s easier to use & analyse the same language in MP
• golang.org/pkg/go/parser/
• go generate in go 1.4
• docs.google.com/document/d/1V03LUfjSADDooDMhe-
_K59EgpTEm3V8uvQRuNMAEnjg
• Metaprogramming Go with GO is the way
• with py only for quick prototype/POC

62. Q & A