Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Networking and Go: An Epic Journey


Published on

One woman's quest to understand how to leverage the Go std library to build epic networking services.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Networking and Go: An Epic Journey

  1. 1. Networking and Go: an epic journey
  2. 2. software engineer @DigitalOcean networking team loves nature and cats @snehainguva
  3. 3. My journey with Go 2016: building DOCC, abstraction layer on top of k8s 2017: working on hypervisor-level daemons to configure monitoring with Prometheus 2018: working on DHCP-server implementation in Go 2018/2019: Experimenting with building network primitives outside of work
  4. 4. Why use Go to build networking services? And how?
  5. 5. The Plan ● Why use go? ★ ● Networking Review ● Layer 4+ Services ● Layer 2+ Services ● Conclusion
  6. 6. Go for Microservices Goroutines: lightweight processes Excellent concurrency support with sync package Communication primitive known as channels Low learning-curve
  7. 7. Go and Networking net package: portable interface for network I/O, Unix sockets, etc. net/http package: provides HTTP client/server implementations syscall package: provides access to low-level system primitives os package: provides platform-independent interface to OS system functionality
  8. 8. The Plan ● Why use go? ● Networking Review ★ ● Layer 4+ Services ● Layer 2+ Services ● Conclusion
  9. 9. Networking Basics: OSI Model
  10. 10. Networking Basics: A Segment, Packet, and Frame Ports -------------------------- IP ------------------ MAC-- network transport data link
  11. 11. Networking Basics: Sockets internal endpoint to send or receive data in a network Stream Socket: Data sent reliably and in-order. Used for TCP connections. Datagram Socket: Used for connectionless data transmission. Raw Socket: Packets not sent with any transport-layer formatting. Often used for low-level data transmission.
  12. 12. Networking Basics: Protocols HTTP: an application layer (7) protocol TCP: a transport layer (4) protocol providing ordered delivery of bytes UDP: a transport layer (4) protocol providing connectionless data transmission IP: a network layer (3) protocol ARP: an IPv4 protocol used to map IP to hardware addresses NDP: an IPv6, a network layer (3) protocol used to map IP to hardware addresses
  13. 13. The Plan ● Why use go? ● Networking Review ● Layer 4+ Services ★ ● Layer 2+ Services ● Conclusion
  14. 14. Layer 4+ Networking Services Layer 7 load balancer: Application-layer load balancer Can look at URL for routing purposes Layer 4 load balancer: Accept TCP connections from frontend and open TCP connections to backends Similar to IPVS - layer 4 lb built into the Linux networking stack Port scanner: Similar to nmap utility Attempts to open TCP connections to check what is opened and closed
  15. 15. Layer 7 Load Balancer // HTTP handler and server. http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { // Randomly select from list of backends. n := rand.Intn(len(backends)) r.URL.Host = backends[n] r.URL.Scheme = "https" req, err := http.NewRequest(r.Method, r.URL.String(), r.Body) if err != nil { // TODO(sneha): fix how this returns later. http.Error(w, "cannot process request", http.StatusBadGateway) return } ... ←-http handler is listening for requests ←- create new http request to backends
  16. 16. Layer 4 Proxy func handleConn(clientConn net.Conn) { n := rand.Intn(len(backends)) backendConn, err := net.Dial("tcp", backends[n]) if err != nil { log.Printf("error opening backend conn %s: %v", backends[n], err) return } var g run.Group { g.Add(func() error { return copy(clientConn, backendConn) }, func(error) { clientConn.Close() // TODO(sneha): handle errors here backendConn.Close() }) } { g.Add(func() error { return copy(backendConn, clientConn) }, func(error) { backendConn.Close() clientConn.Close() // TODO(sneha): handle errors here }) } ... ←------read from TCP streaming socket opened client-side ←------ open TCP streaming socket to one of the backends ←--- use runGroups to run routines to copy data bidirectionally
  17. 17. Port Scanner func main() { hostname := "localhost" conSema := make(chan struct{}, 10) var wg sync.WaitGroup for i := 1; i < 65535; i++ { wg.Add(1) go func(port int) { conSema <- struct{}{} addr := fmt.Sprintf("%s:%d", hostname, port) conn, err := net.Dial("tcp", addr) if err != nil { fmt.Printf("port %d closed: %vn", port, err) } else { fmt.Printf("port %d openn", port) conn.Close() } <-conSema wg.Done() }(i) } wg.Wait() } use channel to limit worker pool -------------> ←-- use waitgroup to block execution of program use net.Dial to open streaming socket ------------------> each port tested in new goroutine ------------------>
  18. 18. The Plan ● Why use go? ● Networking Review ● Layer 4+ Services ● Layer 2+ Services ★ ● Conclusion
  19. 19. Layer 2+ Services NDP/ARP proxy: ● NDP (neighbor discovery protocol) is used to map IPv6 to hardware addresses ● ARP (address resolution protocol) is used to map IPv4 to hardware addresses DHCP server: ● dynamic host configuration protocol is used by routers to allocate IP addresses to network interfaces ● DHCPv6 uses NDP and DHCPv4 uses ARP
  20. 20. ARP Package (uses raw sockets) type Client struct {...} // Dial creates a new Client using the specified network interface. func Dial(ifi *net.Interface) (*Client, error) { // Open raw socket to send and receive ARP packets using ethernet frameswe build ourselves. p, err := raw.ListenPacket(ifi, protocolARP, nil) ←-------------------- open raw socket to listen for ARP packets if err != nil { return nil, err } return New(ifi, p) } func (c *Client) Request(ip net.IP) error { if c.ip == nil { return errNoIPv4Addr } arp, err := NewPacket(OperationRequest, c.ifi.HardwareAddr, c.ip, ethernet.Broadcast, ip) if err != nil { return err } return c.WriteTo(arp, ethernet.Broadcast) }
  21. 21. Raw Package // listenPacket creates a net.PacketConn which can be used to send and receive data at the device driver level. func listenPacket(ifi *net.Interface, proto uint16, _ Config) (*packetConn, error) { ←---open a connection based on ifi *os.File var err error // Try to find an available BPF device for i := 0; i <= 10; i++ { bpfPath := fmt.Sprintf("/dev/bpf%d", i) f, err = os.OpenFile(bpfPath, os.O_RDWR, 0666) ←------- use os open a raw socket to BPF device if err == nil { // Found a usable device break } // Device is busy, try the next one if perr, ok := err.(*os.PathError); ok { if perr.Err.(syscall.Errno) == syscall.EBUSY { ←-------- check if the device is busy using syscall continue } } ...
  22. 22. NDP Package (uses datagram sockets) // Dial returns a Conn and the chosen IPv6 address of the interface. func Dial(ifi *net.Interface, addr Addr) (*Conn, net.IP, error) { addrs, err := ifi.Addrs() if err != nil { return nil, nil, err } ipAddr, err := chooseAddr(addrs, ifi.Name, addr) if err != nil { return nil, nil, err } ic, err := icmp.ListenPacket("ip6:ipv6-icmp", ipAddr.String()) ←------- listen for ICMP packets if err != nil { return nil, nil, err } pc := ic.IPv6PacketConn() ...
  23. 23. ICMP Package func ListenPacket(network, address string) (*PacketConn, error) { var family, proto int switch network { case "udp4": family, proto = syscall.AF_INET, iana.ProtocolICMP case "udp6": family, proto = syscall.AF_INET6, iana.ProtocolIPv6ICMP default: ... } var cerr error var c net.PacketConn switch family { case syscall.AF_INET, syscall.AF_INET6: s, err := syscall.Socket(family, syscall.SOCK_DGRAM, proto) ←syscall to listen from datagram socket if err != nil { return nil, os.NewSyscallError("socket", err) ←---- use os to check for syscall error } ...
  24. 24. The Plan ● Why use go? ● Networking Review ● Layer 4+ Services ● Layer 2+ Services ● Conclusion ★
  25. 25. Conclusion net package for transport layer (4) and higher use syscall and os packages to go lower if needed go has excellent concurrency primitives (goroutines, channels, sync package)
  26. 26. A special thanks Matt Layher (@mdlayher) Julius Volz (@juliusvolz) Networking Pillar at DigitalOcean
  27. 27. Links Port scanner: Layer 7 Load balancer: Layer 4 Load balancer: Raw Package: ARP Package: NDP Package: