Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Next Gen Monitoring with INT

46 views

Published on

Next Gen Monitoring with INT by Ismail

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Next Gen Monitoring with INT

  1. 1. Copyright © Arista 2018. All rights reserved. Next Gen Monitoring with INT (In Band Network Telemetry) Ismail Ali (ismail@arista.com) MyNOG 2019
  2. 2. Copyright © Arista 2018. All rights reserved. Agenda • Network Monitoring Evolution • Motivation for New INT Model • What is INT and How it Works • INT Use Cases 2
  3. 3. Copyright © Arista 2018. All rights reserved. Network Monitoring • Why we need to monitor a network? • What we need to monitor for a network? • When and How we monitor a network? • What and when we can get payback for network monitoring? 3
  4. 4. Copyright © Arista 2018. All rights reserved. Today’s Network Monitoring 4 Expensive and inefficient No fine granularity visibility NO visibility = no control
  5. 5. Copyright © Arista 2018. All rights reserved. Network Monitoring Evolution -- Telemetry • Telemetry: is an automated communications process by which measurements and other data are collected at remote or inaccessible points and transmitted to receiving equipment for monitoring • Need more finer granularity network monitoring data • Mostly vendor-specific chipset/system based • Involved some open projects/standards 5
  6. 6. Copyright © Arista 2018. All rights reserved. Network Monitoring Evolution -- INT • INT: Inband Network Telemetry (INT) is a framework that is designed to monitor, collect, and report (Telemetry) flows and network states (Network), by the data plane, without requiring intervention or work by the control plane (Inband), • Through data plane, metadata based, more vendor system depending • Network-wide, instead of original platform centralized 6
  7. 7. Copyright © Arista 2018. All rights reserved. Motivations for Next-Gen Monitoring • Realtime network monitoring - Line rate monitoring • Canonical data mode - OpenConf/YANG - Easy to multi-vendor deployment • Assurance for services/quality - SLA and proof - Per packet metadata • New services deployment - NFV - micro-service • Next-Gen Operation - Fast fault detecting - Real time path tracing - Fast responding 7
  8. 8. Copyright © Arista 2018. All rights reserved. INT First Spec Draft in 2015 8 https://p4.org/assets/INT-current-spec.pdf
  9. 9. Copyright © Arista 2018. All rights reserved. And the latest is… 9 https://github.com/p4lang/p4-applications/blob/master/docs/INT_v1_0.pdf
  10. 10. Copyright © Arista 2018. All rights reserved. INT : In-band Network Telemetry • Mechanism for collecting network state in the data plane - As close to real-time as possible - At current and future line rates - With a framework that can adapt over time • Examples of network state - Switch ID, Ingress Port ID, Egress Port ID - Egress Link Utilization - Hop Latency - Egress Queue Occupancy - Egress Queue Congestion Status - …. 10 Naga Katta, Mukesh Hira, Changhoon Kim, Anirudh Sivaraman, and Jennifer Rexford. HULA: Scalable Load Balancing Using Programmable Data Planes. In SOSR 2016.
  11. 11. Copyright © Arista 2018. All rights reserved. How INT works • Network element inserting it’s state (referred to as INT Metadata) inline into packets in the data path encapsulated within an INT Header. • Each network element in the packet path that supports INT, inserts its state onto the packet. • At the tail end these are stripped off and sent to collectors where the metadata can be analyzed to provide much deeper information regarding the network element states at the time of packet transit. 11
  12. 12. Copyright © Arista 2018. All rights reserved. INT Typical Deployment 12
  13. 13. Copyright © Arista 2018. All rights reserved. INT Packet • INT information/data is carried inline in data plane frames - Possible for every packet in the network • Two components for the INT information - INT Header: A packet header that carries INT information ≫ Identifies the INT frame and also carries information for transit nodes. One of the primary fields in the header is the ‘INT-vector’ which is typically a bit-map of data-types that each transit node collects and inserts in the frame. Note that draft-kumar, for example, uses template ids to map to a particular set of metadata as opposed to explicitly specifying the metadata set via a bit vector. ≫ Note that the draft-kumar and draft-kumar-v2 use the term ‘IFA’ (Inband Flow Analyzer) to refer to ‘INT’. For example the INT Header is referred to as the IFA Header in the drafts. - INT Metadata: Information that an INT Source inserts into the INT Header ≫ can be viewed as the collection of data-plane state that is stamped by a node in the frame. Typically this can be viewed as a variable-length array of node metadata, where each element of the array represents the metadata for a particular node. The Metadata may follow immediately after the header with each transit node inserting its metadata at the head of the array immediately after the header. ≫ Note that certain implementations mention that INT Metadata (or a part of it) may also be carried in the tail of the frame, but this is typically done to get around some hardware limitations. 13
  14. 14. Copyright © Arista 2018. All rights reserved. INT Header • Two types of INT header - Type 1: hop-by-hop type ≫ Intermediate devices must process this type of INT header - Type 2: destination type ≫ Intermediate devices must ignore this type of head and must be only consumed by INT Sink - Yet another type: ≫ When both INT header types are present, the hop-by-hop type must be precede the destination type header 14
  15. 15. Copyright © Arista 2018. All rights reserved. INT Header Over ANY Encapsulation • Basically, and INT Header can be inserted as an option or payload of any encapsulation type - INT over VXLAN (as VXLAN payload, per GPE extension) - INT over Geneve (as Geneve option) - INT over NSH (as NSH payload) - INT over TCP (as payload) - INT over UDP (as payload) - INT over GRE (as a shim between GRE header and encapsulated payload) • All devices along the way need to agree with it 15
  16. 16. Copyright © Arista 2018. All rights reserved. INT Hop-by-Hop Metadata Header Format 16
  17. 17. Copyright © Arista 2018. All rights reserved. INT Header: Potential Locations for Different Encapsulation 17
  18. 18. Copyright © Arista 2018. All rights reserved. An INT Header and Metadata Example for a Simple Topology 18 • Host1 sends a TCP packet to host2. • The ToR switch of host1 (Switch1) acts as the INT source. • Switch1 adds INT headers and its own metadata in the packet. • Switch2 prepends its metadata. • Finally, the ToR switch of host2 (Switch3) acts as the INT sink and removes INT headers before forwarding the packet to host2.
  19. 19. Copyright © Arista 2018. All rights reserved. Metadata List for Present and Future (to be added) • Switch Level - Switch id - Control plane state version number • Ingress - Ingress port identifier - Ingress timestamp - Ingress port RX pkt count - Ingress port RX byte count - Ingress port RX drop count - Ingress port RX utilization 19 • Egress - Egress port identifier - Egress timestamp - Egress port TX pkt count - Egress port TX byte count - Egress port TX drop count - Egress port TX utilization • Buffer Information - Queue id - Instantaneous queue length - Average queue length - Queue drop count • Miscellaneous - Checksum Complement
  20. 20. Copyright © Arista 2018. All rights reserved. INT Flow Event – Watchlist & Event Detection 20 INT Endpoint (source) Flow watchlist payload header Switch1 INT Metadata payload header Switch2 INT Metadata payload header Switch1 INT Metadata INT Endpoint (sink) Event detection payload header Switch2 INT Metadata Switch3 Local report report header Switch1 INT Metadata INT Endpoint (source) Flow watchlist Host1 Host2Switch1 Switch2 Switch3 Monitor Collector
  21. 21. Copyright © Arista 2018. All rights reserved. Some INT Report Types • Local flow reports — Generated from flow events. Sent from the source or sink for host-to-host data flows matching the watchlist • Drop reports — Generated from drop events. Sent for certain supported drops. Every INT-enabled switch sends these reports to the monitor- collector • Queue Congestion reports — Generated from queue-related events. Sent for packets exceeding the queue depth or latency. Every INT-enabled switch sends these reports to the monitor-collector • INT reports — Sent by the sink. When INT-encapsulated data packets are received on the sink fabric port, two reports are generated by the sink: - Local report for traffic arriving on fabric port - INT report for data received from the source 21
  22. 22. Copyright © Arista 2018. All rights reserved. INT Report Example – Drop Report 22 payload header payload header INT Endpoint Watchlist: Event detection report header Switch1 Drop information Host1 Host2Switch1 Switch2 Switch3 Monitor Collector report header Switch2 Drop information report header Switch3 Drop information payload header payload header INT Endpoint Watchlist: Event detection INT Endpoint Watchlist: Event detection
  23. 23. Copyright © Arista 2018. All rights reserved.23 A simple INT use case: Measuring and reporting end-to-end latency between virtual switches
  24. 24. Copyright © Arista 2018. All rights reserved. How packet level telemetry helps • Inflated latencies and congestion analysis • Network topology and packet traversals • Timeliness and flexibility for exceptions • Doorway to machine learning 24
  25. 25. Copyright © Arista 2018. All rights reserved. Questions INT tried to Address 25 http://www.opencompute.org/assets/Uploads/INT-In-Band-Network-Telemetry-A-Powerful- Analytics-Framework-for-your-Data-Center-OCP-Final3.pdf
  26. 26. Copyright © Arista 2018. All rights reserved. Network Path and Forwarding Rule 26
  27. 27. Copyright © Arista 2018. All rights reserved. Network Latency 27
  28. 28. Copyright © Arista 2018. All rights reserved. Congestion Cause 28
  29. 29. Copyright © Arista 2018. All rights reserved. How INT Works 29 Remote monitoring engine
  30. 30. Copyright © Arista 2018. All rights reserved. Extending INT telemetry beyond physical switches 30 Trident 3 Jericho 2
  31. 31. Copyright © Arista 2018. All rights reserved. Three INT Models • Out of Band Probes - Similar to ping/traceroute in the sense that these are admin/initiated ≫ except that unlike ping/traceroute, these are handled in data plane. - The transit nodes add INT metadata in data plane. - Typically these probes are initiated and terminated on the Host side and the network switches/nodes perform the ‘transit’ function • Inline INT Model - INT information is carried inband/inline in existing data flows. - By including the INT information (header + metadata) at the Initiating/Encapsulating node and by removing the INT headers and metadata at the Terminating/Decapsulating node. - Each transit node adds its own INT metadata ≫ Initiating and Terminating nodes may also perform the ‘transit’ role • INT over mirror - The INT Initiating node applies a selection process to select candidate frames for applying INT based on interface, ACL etc. - The candidate frames are then subject to sampling at a configured rate to arrive at the final decision to apply further INT processing. - The INT Initiating/Encapsulating node then mirrors or generates a copy of the selected frames. - The original packet goes through unchanged, whereas the copy is augmented with INT data. - The Initiating node inserts an INT header in the copy and each transit hop adds its metadata carrying INT information. - The terminating node collects the INT information, drops the copy and forwards the summarized data to collector(s). 31
  32. 32. Copyright © Arista 2018. All rights reserved. INT Deployment model • INT Initiation at TOR - The TOR switches perform the INT Initiation and Termination functions. - The servers are INT Agnostic and the Spine switches perform INT transit • INT Initiation at Server/NIC - INT Initiation/Termination happens at the NIC - TOR and Spine switches are purely transit from INT perspective. 32
  33. 33. Copyright © Arista 2018. All rights reserved. References • [draft-kumar-ifa-00]: Mechanism to sample and mirror data plane packets and carry INT info in the mirrored packets which would be dropped at the terminating node. This mechanism is referred to as IFA 1.0 in the draft. This is superseded by draft-kumar-v2. • [draft-kumar-ippm-ifa-01]: Update to draft-kumar that specifies usage of a new ‘experimental IP protocol type’ to identify INT packets. This is the current version proposed in IETF. It is referred to as IFA 2.0 in the draft. • [draft-ietf-ippm-ioam-data-03]: Mechanism to carry telemetry info in-situ i.e along with data plane packets by inserting INT headers and metadata into packets. • [https://p4.org/assets/INT-current-spec.pdf]: specified mechanism to carry INT info in normal data plane packets or special probe packets • [draft-lapukhov-dataplane-probe-01]: Active data plane probes. These don't carry application traffic i.e separate INT packets - sort of like ping, traceroute except they would be handled in the data plane in the transiting nodes. 33
  34. 34. Copyright © Arista 2018. All rights reserved.Copyright © Arista 2018. All rights reserved. Thanks 34

×