Network server in go #gocon 2013-11-14

2,173 views

Published on

Develop binary based network server in Go
introduction of []byte, bytes, encoding/binary, bufio etc

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,173
On SlideShare
0
From Embeds
0
Number of Embeds
144
Actions
Shares
0
Downloads
33
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Network server in go #gocon 2013-11-14

  1. 1. NetWork & IO I/O and binary programming 2013/11/14 #GoCon
  2. 2. ● ● ● ● ● ● id: Jxck github: Jxck twitter: jxck_ about: http://jxck.io blog: http://d.hatena.ne.jp/jxck Love: music Jack
  3. 3. Go研 https://github.com/goken/goken
  4. 4. Now working on ... ● http2 server/client ○ ○ ○ ○ github.com/Jxck/http2 github.com/Jxck/hpack github.com/Jxck/logger github.com/Jxck/color ● Caution ○ ○ ○ ○ not enough error handling not enough test not enough api/feature under heavy working :p
  5. 5. network server I/O and binary programming
  6. 6. NetWork Server in Go ● ● ● ● ● ● ● net []byte bytes encoding/binary io io/ioutil bufio
  7. 7. No Error Handling in Sample Codes func main() { listener, err := net.Listen("tcp", "127.0.0.1:3000") // err if err != nil { log.Fatal(err) } for { conn, err := listener.Accept() // err if err != nil { log.Fatal(err) } defer func() { log.Println("close connection") conn.Close() }() _, err = conn.Write([]byte("hellon")) // send hello if err != nil { log.Fatal(err) } } } Hard To Read on Slide
  8. 8. import “net” tcp server in go
  9. 9. TCP Server package main import "net" func main() { listener, _ := net.Listen("tcp", ":3000") for { conn, _ := listener.Accept() conn.Write([]byte("hellon")) conn.Close() } }
  10. 10. TCP Server With Handler func main() { listener, _ := net.Listen("tcp", ":3000") for { conn, _ := listener.Accept() handleConn(conn) } } func handleConn(conn net.Conn) { conn.Write([]byte("hellon")) }
  11. 11. TCP Server with Goroutine func main() { listener, _ := net.Listen("tcp", ":3000") for { conn, _ := listener.Accept() go handleConn(conn) } } func handleConn(conn net.Conn) { conn.Write([]byte("hellon")) }
  12. 12. []byte handle octet stream in go
  13. 13. binary format (example) R Length(14) R Reserved(8) Type(8) Flags(8) Stream Identifier(31) Setting Identifier(24) Value(32) []byte{0, 8, 4, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 16, 0}
  14. 14. low level byte slice handling func main() { b := []byte{} b = append(b, []byte{0, b = append(b, 4) b = append(b, 0) // stream id 1 b = append(b, []byte{0, // settings id 1 b = append(b, []byte{0, // value 4096 b = append(b, []byte{0, fmt.Println(b) // [0 8 4 0 0 0 0 1 0 0 } 8}...) 0, 0, 1}...) 0, 0, 1}...) 0, 16, 0}...) 0 1 0 0 16 0] // length 8 // type 4 // flags 0
  15. 15. SWrap (slice wrap) import “github.com/Jxck/swrap” type SWrap ● ● ● ● ● ● ● ● ● ● ● ● func func func func func func func func func func func func New(a []byte) SWrap (sw *SWrap) Add(a byte) (sw *SWrap) Bytes() []byte (sw *SWrap) Compare(b []byte) bool (sw *SWrap) Delete(i int) (sw *SWrap) Len() int (sw *SWrap) Merge(a []byte) (sw *SWrap) Pop() byte (sw *SWrap) Push(b byte) (sw *SWrap) Replace(i int, b byte) (sw *SWrap) Shift(b byte) (sw *SWrap) UnShift() byte
  16. 16. import “io” handle IO in go
  17. 17. important: io.Reader, io.Writer type Reader interface { Read(p []byte) (n int, err error) } type Writer interface { Write(p []byte) (n int, err error) } type ReadWriter interface { Reader Writer }
  18. 18. net.Conn implements io.Writer func handleConn(conn net.Conn) { conn.Write([]byte("hellon")) } func handleConn(conn io.Writer) { conn.Write( []byte{104, 101, 108, 108, 111, 10} ) }
  19. 19. import “bytes” utility for byte slice and Buffer
  20. 20. bytes.Buffer: io.ReadWriter for []byte func handleConn(conn io.Writer) { conn.Write([]byte{104, 101, 108, 108, 111, 10}) } func main() { b := []byte{} buf := bytes.NewBuffer(b) // io.ReadeWriter handleConn(buf) actual := buf.Bytes() bytes.Equal( actual, []byte{104, 101, 108, 108, 111, 10}, ) // true }
  21. 21. import “encoding/binary” read/write fixed size value
  22. 22. we did... func main() { b := []byte{} b = append(b, []byte{0, b = append(b, 4) b = append(b, 0) // stream id 1 b = append(b, []byte{0, // settings id 1 b = append(b, []byte{0, // value 4096 b = append(b, []byte{0, fmt.Println(b) // [0 8 4 0 0 0 0 1 0 0 } 8}...) 0, 0, 1}...) 0, 0, 1}...) 0, 16, 0}...) 0 1 0 0 16 0] // length 8 // type 4 // flags 0
  23. 23. Fixed Size R Length(14) R Type(8) Flags(8) Stream Identifier(31) Reserved(8) Setting Identifier(24) Value(32) var var var var Length uint16 = 8 Type, Flags uint8 = 4, 0 StreamId, SettingsId uint32 = 1, 1 Value uint32 = 4096
  24. 24. encoding/binary.Write() buf var var var var := bytes.NewBuffer([]byte{}) Length uint16 = 8 Type, Flags uint8 = 4, 0 StreamId, SettingsId uint32 = 1, 1 Value uint32 = 4096 binary.Write(buf, binary.BigEndian, Length) binary.Write(buf, binary.BigEndian, Type) // ... binary.Write(buf, binary.BigEndian, Value) fmt.Println(buf.Bytes()) // [0 8 4 0 0 0 0 1 0 0 0 1 0 0 16 0]
  25. 25. encoding/binary.Write() buf := bytes.NewBuffer([]byte{}) frame := Frame{ Length: Type: Flags: StreamId: SettingsId: Value: } 8, 4, 0, 1, 1, 4096, type Frame struct { Length uint16 Type uint8 Flags uint8 StreamId uint32 SettingsId uint32 Value uint32 } // func Write(w io.Writer, order ByteOrder, data interface{}) error binary.Write(buf, binary.BigEndian, frame) fmt.Println(buf.Bytes())
  26. 26. import “net” tcp client in go
  27. 27. TCP Client func main() { conn, _ := net.Dial("tcp", ":3000") b := make([]byte, 100) n, _ := conn.Read(b) fmt.Print(string(b[:n])) // hello }
  28. 28. import “bufio” buffered io
  29. 29. bufio := []byte(4096) + io // bufio.Reader type Reader struct {} // bufio.Writer type Writer struct {} func NewReader(rd io.Reader) *Reader { // convert io.Reader to *bufio.Reader } func NewWriter(wr io.Writer) *Writer { // convert io.Writer to *bufio.Writer }
  30. 30. TCP Client with bufio func main() { conn, _ := net.Dial("tcp", ":3000") br := bufio.NewReader(conn) line, _ := br.ReadString('n') fmt.Print(line) // hello }
  31. 31. TCP Client with encoding/binary type Frame struct { Length uint16 Type uint8 Flags uint8 StreamId uint32 SettingsId uint32 Value uint32 } func main() { conn, _ := net.Dial("tcp", ":3000") frame := &Frame{} binary.Read(conn, binary.BigEndian, frame) fmt.Print(frame) // &{8 4 0 1 1 4096} }
  32. 32. HTTP/2.0?
  33. 33. HTTP/2.0 with Go ● tcp/tls connection ○ net ● multiplexed stream ○ goroutine / channel ● binary frame ○ static type / encoding/binary ● crypto ○ crypto ● build ○ go build (cross compile) ● test ○ go test (and testing)
  34. 34. HTTP2.0 Study (#http2study) ● http2.0 勉強会 #2 ○ http://bit.ly/158zE4C ● http2.0 hackathon ○ 12月 or 1月
  35. 35. Q & A anyone ?
  36. 36. END thanks :)

×