Successfully reported this slideshow.

How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf

0

Share

Loading in …3
×
1 of 37
1 of 37

How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf

0

Share

Download to read offline

How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf

Network to Code, LLC is a network automation solution provider that helps companies transform the way their networks are deployed, managed, and consumed on a day-to-day basis by leveraging network automation, software development, and DevOps technologies and principles. They provide highly sought-after training and consulting services that integrate and deploy network automation technology solutions to improve reliability, security, efficiency, time to market, and customer satisfaction while reducing operational costs.

In this session Josh VanDeraa and David Flores from Network to Code will present how to monitor your network devices with Telegraf using both the SNMP and the gNMI input plugins. They will also present what the challenges are with ingesting the same type of data from different sources and how to remediate that by normalizing the data in Telegraf using processors.

How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf

Network to Code, LLC is a network automation solution provider that helps companies transform the way their networks are deployed, managed, and consumed on a day-to-day basis by leveraging network automation, software development, and DevOps technologies and principles. They provide highly sought-after training and consulting services that integrate and deploy network automation technology solutions to improve reliability, security, efficiency, time to market, and customer satisfaction while reducing operational costs.

In this session Josh VanDeraa and David Flores from Network to Code will present how to monitor your network devices with Telegraf using both the SNMP and the gNMI input plugins. They will also present what the challenges are with ingesting the same type of data from different sources and how to remediate that by normalizing the data in Telegraf using processors.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf

  1. 1. www.networktocode.com Network to Code Confidential How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf November 2020
  2. 2. www.networktocode.com Network to Code Confidential Who is Network to Code? /company/network-to-code/ @networktocode • Founded in mid 2014 • Network Automation Solution Provider – Next-gen consulting and integration company – Focus on automation/programmability and reducing operational inefficiencies – Services only business model • Vendor Independent – Infrastructure (Cisco, Juniper, Arista, HP, Cumulus, F5 etc.) – Open Source Tools (Ansible, Salt, Python, StackStorm, Puppet, Terraform, Telegraf, InfluxDB, Prometheus, Grafana, ElasticSearch, Kibana) – IT Operations Software – ServiceNOW, Remedy, IBM www.networktocode.com
  3. 3. www.networktocode.com Network to Code Confidential About David & Josh David ● Dublin, IE ● Sr Network Automation Consultant, Network to Code ● Automating Networks since 2013 ● Telegraf user since 2019 Josh ● Minnesota, US ● Network Automation Engineer, Network to Code ● Automating Networks since 2015 ● Telegraf user since 2018 @vanderaaj@davidban77 @jvanderaa@davidban77
  4. 4. www.networktocode.com Network to Code Confidential Agenda One of the best times to be gathering telemetry data from network devices! • Network Streaming Telemetry Stack • gNMI • Gathering of Data from Network Devices with Telegraf • Enriching and Modifying metrics • Demo • Wrap Up & Tips and Tricks
  5. 5. www.networktocode.com Network to Code Confidential Telemetry Stack Collect Enrichment Store Dashboard Network Devices
  6. 6. www.networktocode.com Network to Code Confidential gNMI - Streaming Telemetry
  7. 7. www.networktocode.com Network to Code Confidential What is gNMI • Model driven configuration and retrieval of operational data over gRPC (Remote Procedure Calls) • Protocol Buffers • Subscription model
  8. 8. www.networktocode.com Network to Code Confidential gNMI Plugin Note • Previously Cisco gNMI Telemetry (cisco_telemetry_gnmi) • Renamed to just gNMI in Telegraf 1.15 • https://github.com/influxdata/telegraf/tree/rel ease-1.16/plugins/inputs/gnmi
  9. 9. www.networktocode.com Network to Code Confidential Telegraf for Network Telemetry
  10. 10. www.networktocode.com Network to Code Confidential What is Telegraf? • Telegraf is a plugin-driven server agent for collecting & reporting metrics written in Go • Pulls metrics from system running on, 3rd party APIs • Output plugins to send to various data stores
  11. 11. www.networktocode.com Network to Code Confidential Telegraf - gNMI 11 [[inputs.gnmi]] addresses = ["device1"] username = "<USERNAME>" password = "<PASSWORD>" [[inputs.gnmi.subscription]] origin = "openconfig-interfaces" path = "/interfaces/interface" subscription_mode = "sample" sample_interval = "60s" Collect Interfaces/interface counters Base config : Device + Credential
  12. 12. www.networktocode.com Network to Code Confidential Want to monitor all of this, streaming telemetry would be terrific
  13. 13. www.networktocode.com Network to Code Confidential Several devices in your environment do not yet support Streaming Telemetry!
  14. 14. www.networktocode.com Network to Code Confidential Hybrid Collection
  15. 15. www.networktocode.com Network to Code Confidential Telegraf - SNMP 15 [[inputs.snmp]] agents = ["device1"] version = 2 community = "<SNMP Community>" interval = "60s" timeout = "10s" retries = 3 [[inputs.snmp.field]] name = "hostname" oid = "RFC1213-MIB::sysName.0" is_tag = true [[inputs.snmp.table]] name = "interface" inherit_tags = [ "hostname" ] oid = "IF-MIB::ifXTable" [[inputs.snmp.table.field]] name = "name" oid = "IF-MIB::ifDescr" is_tag = true Base config : Device, Credential, interval Convert sysName.0 as tag hostname Collect IF-MIB::ifXTable Table Convert ifDescr as Tag name
  16. 16. www.networktocode.com Network to Code Confidential Challenge with Multiple Input Plugins 16 snmp gnmi interface_counters,agent_host=router1,ifName=Ethernet7 ifOutOctets=3814i,ifInOctets=469i 160519 interface,source=192.0.2.10,name=Ethernet7 in_octets=2176i 1605192 interface,source=192.0.2.10,name=Ethernet7 out_octets=316i 1605192 Measurement Tag Set Field Set TimestampInflux Line Protocol:
  17. 17. www.networktocode.com Network to Code Confidential Telegraf Pipeline
  18. 18. www.networktocode.com Network to Code Confidential Telegraf Pipeline snmp Output Plugin 40+ availableprometheus_client gnmi Execd regex Input Plugins 190+ available Processors Plugins 25+ available influxdb rename Collect Data Enrichment Output
  19. 19. www.networktocode.com Network to Code Confidential Data Enrichment
  20. 20. www.networktocode.com Network to Code Confidential Data Normalization with Telegraf - single device 20 snmp Output Plugin - Send to InfluxDB Normalize Data influxdb_v2 gnmi execd regex rename rename rename Collect Data + Add device level Tags Enrich Data (add interface role)
  21. 21. www.networktocode.com Network to Code Confidential # Processor to rename the snmp fields [[processors.rename]] [[processors.rename.replace]] field = "ifHCInOctets" dest = "in_octets" [[processors.rename.replace]] field = "ifHCOutOctets" dest = "out_octets" [[processors.rename.replace]] tag = "agent_host" dest = "source" Telegraf - Rename Data 21 Rename the field ifHCInOctets and ifHCOutOctets to match gNMI Rename the tag agent_host to match gNMI
  22. 22. www.networktocode.com Network to Code Confidential Telegraf - Add metadata 22 [[processors.regex]] [processors.regex.tagpass] host = device1 [[processors.regex.tags]] key = "name" pattern = "^ge-0/0/0$" replacement = "management" result_key = "interfacerole" [[processors.regex.tags]] key = "name" pattern = "^ge-0/0/1$" replacement = "data" result_key = "interfacerole" Apply these rules only for measurement with a tag host=device1 If a tag name=ge-0/0/0 is present, add a tag interfacerole=management If a tag name=ge-0/0/1 is present, add a tag interfacerole=data
  23. 23. www.networktocode.com Network to Code Confidential Demo Environment
  24. 24. www.networktocode.com Network to Code Confidential Demo Environment
  25. 25. www.networktocode.com Network to Code Confidential Recap • Collect Data from devices • Normalize the data • Enrich the data • Output the data • Dashboard of the data
  26. 26. www.networktocode.com Network to Code Confidential What’s Next & Tips and Tricks
  27. 27. www.networktocode.com Network to Code Confidential What we are trying to do next? • Flux language • InfluxDB 2.0 • InfluxDB IOx • Additional plugins - Input and Processor • Starlark processor
  28. 28. www.networktocode.com Network to Code Confidential Telegraf Tips & Tricks • Automate configuration with templating - Ansible is a good choice • Run in Docker • Install SNMP MIB’s Locally • Use files output for troubleshooting • SNMP/gNMI won’t work? Look at writing your own executable with the execd plugins • Input Internal Plugin (about the Telegraf Plugin) • Consider running on device natively • Use the reachability plugins (Network Response, DNS) • Look at the Plugin Documentation (search “Telegraf Plugins”)
  29. 29. www.networktocode.com Network to Code Confidential Monitor Your Network With gNMI, SNMP, and Grafana http://blog.networktocode.com/post/monitor_your_network_with_gnmi_snmp_and_grafana/ Network Telemetry for SNMP Devices http://blog.networktocode.com/post/network_telemetry_for_snmp_devices/ Monitoring Websites with Telegraf and Prometheus https://blog.networktocode.com/post/monitoring_websites_with_telegraf_and_prometheus/ Network to Code Blog Content
  30. 30. www.networktocode.com Network to Code Confidential Thank You! /company/network-to-code/ @networktocode www.networktocode.com slack.networktocode.com
  31. 31. www.networktocode.com Network to Code Confidential Appendix Items
  32. 32. www.networktocode.com Network to Code Confidential [global_tags] device_role = "spine" [agent] hostname = "telegraf-r1" Telegraf-R1: Base Input Configuration [[inputs.snmp]] agents = ["r1"] version = 2 community = "${SNMP_COMMUNITY}" interval = "60s" timeout = "10s" retries = 3 [[inputs.snmp.table]] name = "interface" oid = "IF-MIB::ifXTable" [[inputs.snmp.table.field]] name = "name" oid = "IF-MIB::ifDescr" is_tag = true Cont.
  33. 33. www.networktocode.com Network to Code Confidential Telegraf -R1 Data Normalization and Enrichment [[processors.rename]] [[processors.rename.replace]] field = "ifHCInOctets" dest = "in_octets" [[processors.rename.replace]] field = "ifHCOutOctets" dest = "out_octets" [[processors.rename.replace]] tag = "agent_host" dest = "device" [[processors.regex]] namepass = ["interface"] [[processors.regex.tags]] key = "name" pattern = "^GigaS+0$" replacement = "management" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^GigaS+0/1$" replacement = "spine" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^GigaS+0/2$" replacement = "backbone" result_key = "intf_role" Cont.
  34. 34. www.networktocode.com Network to Code Confidential Telegraf -R1 Output InfluxDB and STDOUT [[outputs.influxdb]] urls = ["http://influxdb:8086"] [[outputs.file]] files = ["stdout"] data_format = "influx"
  35. 35. www.networktocode.com Network to Code Confidential [global_tags] device_role = "spine" [agent] hostname = "telegraf-r2" Telegraf-R2: Base Input Configuration Cont. [[inputs.gnmi]] addresses = ["r2:6030"] username = "${GNMI_USER}" password = "${GNMI_PASSWORD}" redial = "10s" [[inputs.gnmi.subscription]] name = "interface" origin = "openconfig-interfaces" path = "/interfaces/interface/state/counters" subscription_mode = "sample" sample_interval = "5s"
  36. 36. www.networktocode.com Network to Code Confidential Telegraf -R2 Data Normalization and Enrichment Cont. [[processors.rename]] [[processors.rename.replace]] tag = "source" dest = "device" [[processors.regex]] namepass = ["interface"] [[processors.regex.tags]] key = "name" pattern = "^Managementd+$" replacement = "management" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^Ethernet1$" replacement = "spine" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^Ethernet2$" replacement = "backbone" result_key = "intf_role"
  37. 37. www.networktocode.com Network to Code Confidential Telegraf -R2 Output InfluxDB and STDOUT [[outputs.influxdb]] urls = ["http://influxdb:8086"] [[outputs.file]] files = ["stdout"] data_format = "influx"

×