Uploaded bypphaal
1,880 views

Network visibility and control using industry standard sFlow telemetry

The document discusses the importance of network visibility and control using sFlow telemetry for effective monitoring and management of network performance. It highlights the principles of controllability and observability, the implementation of sFlow as an industry-standard measurement technology, and its integration into various systems for real-time analytics, particularly in mitigating DDoS attacks. The content underscores the necessity of accurate metrics and data aggregation to understand network performance issues, optimize resource allocation, and respond to network anomalies.

Embed presentation

Downloaded 88 times
Network visibility and control using industry standard sFlow telemetry Peter Phaal InMon Corp.
 March, 2016 Twitter: @sFlow Blog: blog.sflow.com San Francisco Network Visibility Meetup
Why monitor? “If you can’t measure it, you can’t improve it” Lord Kelvin
Time Capacity Demand Static provisioning $ Unused capacity $$$ Service failure $$ Unused capacity
$$ Savings Time Capacity Demand Dynamic provisioning
Feedback control Measure Control System desired output measured output
Controllability and Observability Basic concept is simple, a stable feedback control system requires: 1. ability to influence all important system states (controllable) 2. ability to monitor all important system states (observable)
It’s hard to stay on the road if you can’t see the road, or keep to the speed limit without a speedometer It’s hard to stay on the road or maintain speed if your brakes, engine or steering fail Controllability and Observability driving example Observability Controllability States location, speed, direction, ... Tule fog in California Central Valley
Effect of delay on stability Measurement delay Planning delay Time Configuration delayDisturbance Response delay EffectLoop delay DDoS launched Identify target, attacker Black hole, mark, re-route? Switch CLI commands Route propagation Traffic dropped Components of loop delay e.g. Slow reaction time causes tired / drunk / distracted driver to weave, very slow reaction time and they leave the road
What is sFlow? “In God we trust. All others bring data.” Dr. Edwards Deming
Industry standard measurement technology integrated in switches http://sflow.org/
Open source agents for hosts, hypervisors and applications Host sFlow project (http://sflow.net) is center of an ecosystem of related open source projects embedding sFlow in popular operating systems and applications Host agent extends network visibility into public / private cloud
Network (maintained by hardware in network devices) - MIB-2 ifTable: ifInOctets, ifInUcastPkts, ifInMulticastPkts, ifInBroadcastPkts, ifInDiscards, ifInErrors, ifUnkownProtos, ifOutOctets, ifOutUcastPkts, ifOutMulticastPkts, ifOutBroadcastPkts, ifOutDiscards, ifOutErrors Host (maintained by operating system kernel) - CPU: load_one, load_five, load_fifteen, proc_run, proc_total, cpu_num, cpu_speed, uptime, cpu_user, cpu_nice, cpu_system, cpu_idle, cpu_wio, cpu_intr, cpu_sintr, interupts, contexts - Memory: mem_total, mem_free, mem_shared, mem_buffers, mem_cached, swap_total, swap_free, page_in, page_out, swap_in, swap_out - Disk IO: disk_total, disk_free, part_max_used, reads, bytes_read, read_time, writes, bytes_written, write_time - Network IO: bytes_in, packets_in, errs_in, drops_in, bytes_out, packet_out, errs_out, drops_out Application (maintained by application) - HTTP: method_option_count, method_get_count, method_head_count, method_post_count, method_put_count, method_delete_count, method_trace_count, method_connect_count, method_other_count, status_1xx_count, status_2xx_count, status_3xx_count, status_4xx_count, status_5xx_count, status_other_count - Memcache: cmd_set, cmd_touch, cmd_flush, get_hits, get_misses, delete_hits, delete_misses, incr_hits, incr_misses, decr_hists, decr_misses, cas_hits, cas_misses, cas_badval, auth_cmds, auth_errors, threads, con_yields, listen_disabled_num, curr_connections, rejected_connections, total_connections, connection_structures, evictions, reclaimed, curr_items, total_items, bytes_read, bytes_written, bytes, limit_maxbytes Standard counters
Simple - standard structures - densely packed blocks of counters - extensible (tag, length, value) - RFC 1832: XDR encoded (big endian, quad-aligned, binary) - simple to encode / decode - unicast UDP transport Minimal configuration - collector address - polling interval Cloud friendly - flat, two tier architecture: many embedded agents → central “smart” collector - sFlow agents automatically start sending metrics on startup, automatically discovered - eliminates complexity of maintaining polling daemons (and associated configurations) Scaleable push protocol
• Counters tell you there is a problem, but not why. • Counters summarize performance by dropping high cardinality attributes: - IP addresses - URLs - Memcache keys • Need to be able to efficiently disaggregate counter by attributes in order to understand root cause of performance problems. • How do you get this data when there are millions of transactions per second? Counters aren’t enough Why the spike in traffic? (100Gbit link carrying 14,000,000 packets/second)
• Random sampling is lightweight • Critical path roughly cost of maintaining one counter:
 if(--skip == 0) sample(); • Sampling is easy to distribute among modules, threads, processes without any synchronization • Minimal resources required to capture attributes of sampled transactions • Easily identify top keys, connections, clients, servers, URLs etc. • Unbiased results with known accuracy Break out traffic by client, server and port (graph based on samples from100Gbit link carrying 14,000,000 packets/second) sFlow also exports random samples
Integrated data model Packet Header Source Destination TCP/UDP Socket TCP/UDP Socket MAC Address MAC Address Sampled Packet Headers + Forwarding State I/F Counters NETWORK HOST CPU Memory I/O Adapter MACs APPLICATION Sampled Transactions Transaction Counters TCP/UDP Socket Independent agents sFlow analyzer joins data for integrated view
Virtual Servers Applications Apache/PHP Tomcat/Java Memcached Virtual Network Servers Network Embedded monitoring of all switches, all servers, all applications, all the time Consistent measurements shared between multiple management tools Comprehensive data center wide visibility
Picking the right tools “This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together.” Doug McIlroy
packets decode hash sendflow cache flushsample Flow Records flow cache embedded on switchswitch NetFlow IPFIX … decode hash sendflow cache flush Flow Records packets send polli/f counters sample multiple switches export sFlow packets send polli/f counters sample ... centralized software flow cache switch switch JSON/REST
 NetFlow IPFIX … • Reduce ASIC cost / complexity • Fast response (data not sitting on switch) • Centralized, network-wide visibility • Increase flexibility → software defined analytics Move flow cache from ASIC to external software Scale-out alternative to SNMP polling Traffic analytics with sFlow
sFlow-RT.com analytics engine • Low latency flow analytics for real-time control applications • Disaggregates flow cache from database. Choose external database(s) for history (InfluxDB, Logstash, etc.) • Programmable analytics pipeline through open APIs
RESTful API for defining flows http://blog.sflow.com/2013/08/restflow.html curl -H "Content-Type:application/json" -X PUT —data '{"keys":"ipsource,ipdestination,tcpsourceport,tcpdestinationport", "value":"bytes", "ipfixCollectors":["10.0.0.1"]}' http://127.0.0.1:8008/flow/tcp/json curl -H "Content-Type:application/json" -X PUT --data '{"keys":"ipdestination,icmpunreachableport", "value":"frames"}' http://127.0.0.1:8008/flow/unreachableport/json • Instantly enables network wide monitoring of flows • All switches, all ports, including hosts and virtual switches • Contrast with task of re-configuring Flexible Netflow/IPFIX caches on every switch in multi-vendor network. How many simultaneous flow definitions are allowed? What key / value combinations are allowed? curl -H "Content-Type:application/json" -X PUT --data '{"value":"frames"}' http://127.0.0.1:8008/flow/frames/json
InMon sFlow-RT active timeout active timeout NetFlow Open vSwitch SolarWinds Real-Time NetFlow Analyzer • sFlow does not use flow cache, so realtime charts more accurately reflect traffic trend • NetFlow spikes caused by flow cache active-timeout for long running connections Rapid detection of large flows Flow cache active timeout delays large flow detection, limits value of signal for real-time control applications
Counters and packet samples http://blog.sflow.com/2013/02/measurement-delay-counters-vs-packet.html • Packet samples give a fast signal that operates at scale • Counters are maintained in hardware and provide precise traffic totals. • Counters capture rare events, like packet discards, that can severely impact performance. • Counters report important link state information, like link speed, LAG group membership etc.
Metrics and Events Metrics Events Sources SNMP, sFlow, collectd Traps, syslog, IPFIX, NetFlow Crossover Event count Threshold event Collectors InfluxDB, OpenTSDB, Graphite, rrdtool Logstash, flowtools, Splunk Application Performance Security and alerts
Data models and transports sFlow SNMP NetFlow version 5 OpenConfig Telemetry IPFIX syslog Model standard measurements published by sFlow.org, Dataplane focus: based on IEEE, IETF, APIs (MIB-2, LAG-MIB, libvirt, JMX, …) standard MIBs defined by IETF standard tcp / udp / icmp flow record defined by Cisco Telemetry defined as part of YANG configuration models by OpenConfig.org
 Control plane focus: BGP, MPLS, VLAN, etc. Encoding XDR
 (RFC 4506) ASN1 (IETF) NetFlow (Cisco) protobufs, JSON, NetConf IPFIX (IETF) Syslog (RFC 5424) Transport UDP UDP UDP UDP, HTTP SCTP, TCP, UDP UDP Mode Push Pull Push Push Push Push Easy to combine multiple data sources if you disaggregate tool chain 
 e.g. separate agents from collectors, feed data from all sources into InfluxDB / Logstash
sflowtool https://github.com/sflow/sflowtool sflowtool replicate ascii Perl, Python, awk, grep pcap wireshark, tcpdump Netflow Open source command line tool
Network visibility for DevOps tools • Streaming filtering and summarization reduces data volume and increases scaleability of backend tools • Streaming flow analytics to generate application metrics
Feedback control of cloud infrastructure “You can’t control what you can’t measure” Tom DeMarco
Cloud depends on network • Server costs (both capex and power consumption) far exceed networking costs in the data center. • Network congestion caused server to wait, resulting in poor utilization of cloud infrastructure. • Optimize network to increase data center efficiency http://perspectives.mvdirona.com/2010/09/overall-data-center-costs/ “Typically the resource that is most scarce is the network.” Amin Vahdat, Google, ONS2015 Keynote http://blog.sflow.com/2015/06/optimizing-software-defined-data-center.html
http://blog.sflow.com/2013/02/sdn-and-large-flows.html Elephant flows are the small number of long lived large flows responsible for majority of bytes on network Large “Elephant” flows
ECMP visibility and control
Default sFlow settings Port Speed Large Flow Sampling Rate Polling Interval 1 Gbit/s >= 100 Mbit/s 1-in-1,000 30 seconds 10 Gbit/s >= 1 Gbit/s 1-in-10,000 30 seconds 25 Gbit/s >= 2.5 Gbit/s 1-in-25,000 30 seconds 40 Gbit/s >= 4 Gbit/s 1-in-40,000 30 seconds 50 Gbit/s >= 5 Gbit/s 1-in-50,000 30 seconds 100 Gbit/s >= 10 Gbit/s 1-in-100,000 30 seconds Supports real-time detection of large “Elephant” flows where large flow is defined as flow consuming >10% of link bandwidth for >1 second http://blog.sflow.com/2013/06/large-flow-detection.html
Elephant flow collisions http://blog.sflow.com/2015/03/ecmp-visibility-with-cumulus-linux.html
Elephant flow collisions http://blog.sflow.com/2015/03/ecmp-visibility-with-cumulus-linux.html
ECMP monitoring challenge • large number of links, 12 x 10G links • all links need to be monitored continuously, 180G total bandwidth • real-time detection of congested links • real-time detection of Elephant flows http://blog.sflow.com/2015/03/ecmp-visibility-with-cumulus-linux.html
Fabric level performance metrics • Fabric View application runs on sFlow-RT • Downloadable from sFlow-RT.com, includes captured data set from 4 node 10G ECMP fabric • Elephant collisions on spine links occur frequently • Collisions halve throughput • Collisions cause packet discards http://blog.sflow.com/2015/10/fabric-view.html
http://www.slideshare.net/martin_casado/elephants-and-mice-elephant-detection-in-the-vswitch-with-hardware-handling Large “Elephant” flows delay small “Mice” flows Separating Elephants and Mice into different queues significantly improves latency and response time Large flow marking
http://blog.sflow.com/2014/06/restful-control-of-cumulus-linux-acls.html Large flow marking
ONS2015 SDN Showcase
 CORD: Open-source spine-leaf fabric
http://blog.sflow.com/2015/06/leaf-and-spine-traffic-engineering.html CORD: Open-source spine-leaf fabric Rack Rack Rack Rack Spine http://blog.sflow.com/2015/08/cord-open-source-spine-leaf-fabric.html
SDN Central Demo Friday SDN Optimization of Hybrid Packet / Optical Data Center Fabric
http://blog.sflow.com/2014/09/sdn-control-of-hybrid-packet-optical.html Cumulus, Calient and InMon PoC. Large flows detected in real-time and diverted to direct ToR to ToR optical circuit SDN Optimization of Hybrid Packet / Optical Data Center Fabric
Software Defined Internet Router (SIR)
https://eos.arista.com/spotifys-sdn-internet-router/ IXP Routes Installed Routes not installed LINX 18672 95711 DECIX 12518 108164 EQIX-ASH 27687 75146 https://eos.arista.com/arista-eos-bgp-selective-route-download/
White box Internet router PoC
Internet Router PoC http://blog.sflow.com/2015/07/white-box-internet-router-poc.html Peer 1 Peer 2 Peer N DDoS Mitigation REST API sFlow-RT controller real-time analytics OpenFlow, REST, BGP, … Large flow marking Routing Offload BGP … BGP Router Switch http://blog.sflow.com/2015/10/active-route-manager.html
Open Networking Summit SDN Idol winning solution Real-time SDN Analytics for DDoS mitigation
DDoS Mitigation Market Opportunity DDoS Attack Megatrends [Reference 1] • High bandwidth, volumetric infrastructure layer (Layer 3 & 4) attacks increased approximately 30 percent • DDoS attack volume also increased month-to-month in 2013, with 10 out of 12 months showing higher attack volume compared to 2012 • Average DDoS attack sizes continued to increase – many over 100 Gbps, the largest peaking at 179 Gbps DDoS Mitigation Market Growth • $870M market by 2017, 18.2% CAGR – Source: IDC:Worldwide DDoS Prevention Products and Services 2013-2017 Forecast • $1049M market by 2017, 25% CAGR – Source: Infonetics: Global DDoS Prevention Appliances 2012-2017 Forecast Reference 1: Top DDoS Attack Trends http://www.itbriefcase.net/top-ddos-attack-trends-for-2013
DDoS Mitigation Use Case (1) ISP 1 ISP 2 ISP N • ISP/IX is uniquely positioned to protect customers from DDoS flood attacks • New revenue from DDoS mitigation service + differentiates ISP/IX service Attacker User Prevent attack from overwhelming customer access link Filter attack traffic in real-time Customer network DDoS target host Attack on single host can take out entire customer data center. Customer cannot mitigate flood attack without upstream help ISP / IX ISP/IX Market Segment
Customer portal DDoS Mitigation Service Web UI + RESTful programmatic API • real-time TopN analytics • programmable filtering of traffic • set thresholds + automatic blocking Real-time sFlow visibility, Hybrid OpenFlow Control capability of Brocade switches/routers REST API InMonsFlow-RT REST API OpenFlowController DDoS Mitigation Application Customer Network Internet 1. Flood attack overloads customer port 2. Attack maps to large flows [Ref. 2]. sFlow-RT detects attack (maps to large flows) and characterizes attack (srcip, dstip, protocol, ports, etc.) 3. mitigation application takes signature, applies customer policy, selects optimal control and push OpenFlow rule(s) to switch(es) 5. OpenFlow rule(s) applied to switch forwarding path to drop / mark traffic and protect link HTTPS HTTPS 4. Controller pushes OpenFlow rule(s) to switch(es) OpenFlow 1.3 Match Fields line rate filtering using Brocade switches Reference 2: IETF I2RS Working Group Draft - https://ietf.org/doc/draft-krishnan-i2rs-large-flow-use-case/
Demonstration http://blog.sflow.com/2014/03/ons2014-sdn-idol-finalist-demonstrations.html
DDoS Mitigation
Cloudflare DDoS mitigation http://blog.sflow.com/2016/02/cloudflare-ddos-mitigation-pipeline.html • sFlow used to identify attack signature • BPF created to match signature • Detailed signatures minimize collateral damage
Big Switch Network Webinar Big Tap sFlow: Enabling Pervasive Flow-level Visibility
Pervasive monitoring http://blog.sflow.com/2015/04/big-tap-sflow-enabling-pervasive-flow.html
Targeted capture http://blog.sflow.com/2015/04/big-tap-sflow-enabling-pervasive-flow.html
Comments • sFlow instrumentation is widely available in switches http://sflow.org/products/network.php • Host sFlow (sFlow.net) agent extends visibility into servers (works with libpcap, iptables, Open vSwitch to efficiently sample packets in host data plane) • Common data model ensures strong interoperability across sFlow data sources • Streaming counter and packet telemetry across network, compute and application tiers makes data center observable • Observability makes it possible to apply feedback controls
Extra slides
Host sFlow monitoring of Linux datapath Technology Reference Adapter, bridge, macvlan, ipvlan Berkeley Packet Filter (BPF) sampling function http://blog.sflow.com/ 2016/02/linux-bridge- macvlan-ipvlan-adapters.html Open vSwitch Kernel datapath has sFlow support http://openvswitch.org/ support/config- cookbooks/sflow/ Linux Firewall iptables statistic module random function with ulog http://blog.sflow.com/ 2010/12/ulog.html Top of Rack Switch ASIC provides wirespeed monitoring of attached servers http://blog.sflow.com/ 2010/04/hybrid-server- monitoring.html Efficient monitoring of high traffic production workloads
Open vSwitch Fall Conference New OVS instrumentation features aimed at real-time monitoring of virtual networks
Overlay / Underlay Visibility http://blog.sflow.com/2015/11/network-virtualization-visibility-demo.html
Open vSwitch Fall Conference OVN service injection demonstration
Service injection http://blog.sflow.com/2015/11/ovn-service-injection-demonstration.html
White Paper Actionable Intelligence in the SDN Ecosystem: Optimizing Network Traffic through FRSA
WAN Optimization http://blog.sflow.com/2015/06/wan-optimization-using-real-time.html
Dell NFV Summit 2015 Demystifying NFV Infrastructure Hotspots End-to-end Monitoring, Analytics & SDN Control
Demystifying NFV Infrastructure Hotspots http://blog.sflow.com/2016/01/demystifying-nfv-infrastructure-hotspots.html
Hybrid OpenFlow ECMP testbed
Hybrid OpenFlow ECMP testbed http://blog.sflow.com/2015/01/hybrid-openflow-ecmp-testbed.html http://mininet.org/ • Simulated ECMP network for developing visibility and control applications • sFlow support in Open vSwitch • OpenFlow for control
The sFlow Standard: Scalable, Unified Monitoring of Networks, Systems and Applications 2012 Velocity Conference
Tagged.com case study http://blog.sflow.com/2013/04/velocity-conference-talk.html

More Related Content

PDF
SDN Fundamentals - short presentation
byAzhar Khuwaja
 
PDF
Cisco Live! :: Cisco ASR 9000 Architecture :: BRKARC-2003 | Las Vegas 2017
byBruno Teixeira
 
PDF
A comparison of segment routing data-plane encodings
byGunter Van de Velde
 
PDF
Chapter 8 .vlan.pdf
bymanojkumar595505
 
PPTX
IOT Protocols
byNagesh Rao
 
PPT
Linux Networking Commands
bytmavroidis
 
PDF
How BGP Works
byThousandEyes
 
PPTX
Dhcp
byChinmoy Jena
 
SDN Fundamentals - short presentation
byAzhar Khuwaja
 
Cisco Live! :: Cisco ASR 9000 Architecture :: BRKARC-2003 | Las Vegas 2017
byBruno Teixeira
 
A comparison of segment routing data-plane encodings
byGunter Van de Velde
 
Chapter 8 .vlan.pdf
bymanojkumar595505
 
IOT Protocols
byNagesh Rao
 
Linux Networking Commands
bytmavroidis
 
How BGP Works
byThousandEyes
 
Dhcp
byChinmoy Jena
 

What's hot

PPTX
SDN 101: Software Defined Networking Course - Sameh Zaghloul/IBM - 2014
bySAMeh Zaghloul
 
PPT
JunOS Jumstart 01 - JunOS Overview.ppt
byLucTieuPhung1
 
PPTX
Data center network architectures v1.3
byJeong, Wookjae
 
PDF
“Thread - A New Wireless Networking Protocol for Internet of Things” - Ankith...
byEIT Digital Alumni
 
PPTX
Wireshark
byantivirusspam
 
PPT
Deploying Carrier Ethernet features on ASR 9000
byVinod Kumar Balasubramanyam
 
PPTX
Ns3
byRehmat Ullah
 
PPTX
Technical Overview of Cisco Catalyst 9200 Series Switches
byRobb Boyd
 
PPTX
Open network operating system (onos)
byAmeer Sameer
 
PDF
Linux Networking Explained
byThomas Graf
 
PPTX
Ipv4 & ipv6
byurooj ehsan
 
PPT
Network architecture
byOnline
 
PPTX
SD-WAN PROTOCOLS
bybilal anjum
 
PPTX
VTP
byHaidar-Mohammed
 
PDF
SD WAN Overview | What is SD WAN | Benefits of SD WAN
byAshutosh Kaushik
 
PPT
Internet control message protocol
byasimnawaz54
 
PPTX
Wireshark Basic Presentation
byMD. SHORIFUL ISLAM
 
PPTX
Address resolution protocol (ARP)
byNetProtocol Xpert
 
PPT
Static Routing
byKishore Kumar
 
PPTX
Dhcp
byTapan Khilar
 
SDN 101: Software Defined Networking Course - Sameh Zaghloul/IBM - 2014
bySAMeh Zaghloul
 
JunOS Jumstart 01 - JunOS Overview.ppt
byLucTieuPhung1
 
Data center network architectures v1.3
byJeong, Wookjae
 
“Thread - A New Wireless Networking Protocol for Internet of Things” - Ankith...
byEIT Digital Alumni
 
Wireshark
byantivirusspam
 
Deploying Carrier Ethernet features on ASR 9000
byVinod Kumar Balasubramanyam
 
Ns3
byRehmat Ullah
 
Technical Overview of Cisco Catalyst 9200 Series Switches
byRobb Boyd
 
Open network operating system (onos)
byAmeer Sameer
 
Linux Networking Explained
byThomas Graf
 
Ipv4 & ipv6
byurooj ehsan
 
Network architecture
byOnline
 
SD-WAN PROTOCOLS
bybilal anjum
 
VTP
byHaidar-Mohammed
 
SD WAN Overview | What is SD WAN | Benefits of SD WAN
byAshutosh Kaushik
 
Internet control message protocol
byasimnawaz54
 
Wireshark Basic Presentation
byMD. SHORIFUL ISLAM
 
Address resolution protocol (ARP)
byNetProtocol Xpert
 
Static Routing
byKishore Kumar
 
Dhcp
byTapan Khilar
 

Viewers also liked

PDF
Cloud Network Virtualization with Juniper Contrail
bybuildacloud
 
PDF
大規模環境のOpenStack アップグレードの考え方と実施のコツ
byTomoya Hashimoto
 
PDF
NFV evolution towards 5G
byMarie-Paule Odini
 
PDF
【AWS初心者向けWebinar】AWSから始める動画配信
byAmazon Web Services Japan
 
PPSX
Contrail Deep-dive - Cloud Network Services at Scale
byMarketingArrowECS_CZ
 
PDF
ビッグデータ処理データベースの全体像と使い分け
byRecruit Technologies
 
PDF
NFV Tutorial
byRashid Mijumbi
 
PDF
NFV and OpenStack
byMarie-Paule Odini
 
PDF
Digdagによる大規模データ処理の自動化とエラー処理
bySadayuki Furuhashi
 
PPTX
NFV : Virtual Network Function Architecture
bysidneel
 
PPTX
5 g network & technology
byFrikha Nour
 
PDF
Treasure Data Cloud Data Platform
byinside-BigData.com
 
PDF
Using Agilio SmartNICs for OpenStack Networking Acceleration
byNetronome
 
PPTX
Monitor OpenStack Environments from the bottom up and front to back
byIcinga
 
PDF
Design Principles for 5G
byOpen Networking Summit
 
PDF
Nfv orchestration open stack summit may2015 aricent
byAricent
 
PDF
170827 jtf garafana
byOSSラボ株式会社
 
Cloud Network Virtualization with Juniper Contrail
bybuildacloud
 
大規模環境のOpenStack アップグレードの考え方と実施のコツ
byTomoya Hashimoto
 
NFV evolution towards 5G
byMarie-Paule Odini
 
【AWS初心者向けWebinar】AWSから始める動画配信
byAmazon Web Services Japan
 
Contrail Deep-dive - Cloud Network Services at Scale
byMarketingArrowECS_CZ
 
ビッグデータ処理データベースの全体像と使い分け
byRecruit Technologies
 
NFV Tutorial
byRashid Mijumbi
 
NFV and OpenStack
byMarie-Paule Odini
 
Digdagによる大規模データ処理の自動化とエラー処理
bySadayuki Furuhashi
 
NFV : Virtual Network Function Architecture
bysidneel
 
5 g network & technology
byFrikha Nour
 
Treasure Data Cloud Data Platform
byinside-BigData.com
 
Using Agilio SmartNICs for OpenStack Networking Acceleration
byNetronome
 
Monitor OpenStack Environments from the bottom up and front to back
byIcinga
 
Design Principles for 5G
byOpen Networking Summit
 
Nfv orchestration open stack summit may2015 aricent
byAricent
 
170827 jtf garafana
byOSSラボ株式会社
 

Similar to Network visibility and control using industry standard sFlow telemetry

PPT
OpenFlow tutorial
byopenflow
 
PDF
Kentik Network@Scale (Dan Ellis)
bygvillain
 
PPT
network-management Web base.ppt
byAssadLeo1
 
PPTX
22IT508 - Full Stack Development with Node and React.pptx
byPandiya Rajan
 
PDF
SDN in the Management Plane: OpenConfig and Streaming Telemetry
byAnees Shaikh
 
PPT
1. Network monitoring and measurement-2.ppt
byFarid Er
 
PPTX
lect4_SDNbasic_openflow.pptx
byJesicaDcruz1
 
PDF
Multi Layer Monitoring V1
byLahav Savir
 
PDF
IT Monitoring in the Era of Containers | Luca Deri Founder & Project Lead | ntop
byInfluxData
 
PDF
Banv
bynetvis
 
KEY
The sFlow Standard: Scalable, Unified Monitoring of Networks, Systems and App...
bynetvis
 
PDF
Performance Aware SDN, LSPE talk
bynetvis
 
PPTX
FlowER Erlang Openflow Controller
byHolger Winkelmann
 
PDF
Security defined routing_cybergamut_v1_1
byJoel W. King
 
PDF
Go with the Flow-v2
byZobair Khan
 
PDF
Go with the Flow
byBangladesh Network Operators Group
 
PPT
Intelligent Network Services through Active Flow Manipulation
byTal Lavian Ph.D.
 
PDF
Network Statistics for OpenFlow
byMiro Cupak
 
PPTX
SDN Demystified, by Dean Pemberton [APNIC 38]
byAPNIC
 
PPTX
OpenStack and OpenFlow Demos
byBrent Salisbury
 
OpenFlow tutorial
byopenflow
 
Kentik Network@Scale (Dan Ellis)
bygvillain
 
network-management Web base.ppt
byAssadLeo1
 
22IT508 - Full Stack Development with Node and React.pptx
byPandiya Rajan
 
SDN in the Management Plane: OpenConfig and Streaming Telemetry
byAnees Shaikh
 
1. Network monitoring and measurement-2.ppt
byFarid Er
 
lect4_SDNbasic_openflow.pptx
byJesicaDcruz1
 
Multi Layer Monitoring V1
byLahav Savir
 
IT Monitoring in the Era of Containers | Luca Deri Founder & Project Lead | ntop
byInfluxData
 
Banv
bynetvis
 
The sFlow Standard: Scalable, Unified Monitoring of Networks, Systems and App...
bynetvis
 
Performance Aware SDN, LSPE talk
bynetvis
 
FlowER Erlang Openflow Controller
byHolger Winkelmann
 
Security defined routing_cybergamut_v1_1
byJoel W. King
 
Go with the Flow-v2
byZobair Khan
 
Go with the Flow
byBangladesh Network Operators Group
 
Intelligent Network Services through Active Flow Manipulation
byTal Lavian Ph.D.
 
Network Statistics for OpenFlow
byMiro Cupak
 
SDN Demystified, by Dean Pemberton [APNIC 38]
byAPNIC
 
OpenStack and OpenFlow Demos
byBrent Salisbury
 

Recently uploaded

PPTX
software define network authentication .pptx
byAhmedGouda93
 
PDF
10 PyData Piraeus Meetup: Retrieval and Matching at Scale: from embeddings to...
byPyData Piraeus
 
PPTX
[DSC Europe 25] Dusan Jovicic - AI Story: From on-prem to cloud and back agai...
byDataScienceConferenc1
 
PPTX
Exploring Quantum Computing Simulation Tools and Fundamentals Algorithms.pptx
byfarhanafayk
 
PDF
BCS501 Module-05 Software Engineering and Project Management.pdf
byjaadu6918
 
PPTX
[DSC Europe 25] Vid Stimac - Policy Parsimony: Between Oversimplifying and Ov...
byDataScienceConferenc1
 
PDF
[DSC Europe 25] Nikola Rajovic - Hardware Technologies Under the Hood: RISC-V...
byDataScienceConferenc1
 
PPTX
Python Functions and Modules Detailed Notes.pptx
byJayantAbhinav
 
PPTX
essential in computer science structure and other
byhncseven7
 
PPTX
Betsi- Zero-Trust Security Securing the Modern Enterprise.pptx
bymershaabdisa
 
PPTX
Presentation 1-PUM Singapore 2015- 2D.pptx
byWilliamChong71
 
PDF
[DSC Europe 25] Marija Vlajkovic & Andrea Radonjanin - Integration of AI tool...
byDataScienceConferenc1
 
PPTX
Copy of First ENGR 100 Presentation.pptx
bygenshinds971
 
PPTX
[DSC Europe 25] Jim Sterne - Adopting Generative AI Capabilities Into the Ent...
byDataScienceConferenc1
 
PPTX
Customer Segmentation: Seeing the Trees and the Forest Simultaneously - V2
bySK Palu
 
PPTX
ELECTRICITY SECTOR IN PHILIPHINE .pptx
byahmadnabil090406
 
PDF
Natal SM 2025 - Latihan Liturgi Kelas 0 - 3 SD.pdf
byWenny63
 
PPTX
Orientation-on-ARAL-Check-in-Assessment.pptx
byJenTeruel1
 
PPTX
Safe_Spaces_Act_Modern_Presentation.pptx
byairesmijares2003
 
PPTX
GRADE 8 MATH - Q3 - LESSON 1 -LINEAR EQUATION IN ONE VARIABLE.pptx
byEdlynMalangPanti
 
software define network authentication .pptx
byAhmedGouda93
 
10 PyData Piraeus Meetup: Retrieval and Matching at Scale: from embeddings to...
byPyData Piraeus
 
[DSC Europe 25] Dusan Jovicic - AI Story: From on-prem to cloud and back agai...
byDataScienceConferenc1
 
Exploring Quantum Computing Simulation Tools and Fundamentals Algorithms.pptx
byfarhanafayk
 
BCS501 Module-05 Software Engineering and Project Management.pdf
byjaadu6918
 
[DSC Europe 25] Vid Stimac - Policy Parsimony: Between Oversimplifying and Ov...
byDataScienceConferenc1
 
[DSC Europe 25] Nikola Rajovic - Hardware Technologies Under the Hood: RISC-V...
byDataScienceConferenc1
 
Python Functions and Modules Detailed Notes.pptx
byJayantAbhinav
 
essential in computer science structure and other
byhncseven7
 
Betsi- Zero-Trust Security Securing the Modern Enterprise.pptx
bymershaabdisa
 
Presentation 1-PUM Singapore 2015- 2D.pptx
byWilliamChong71
 
[DSC Europe 25] Marija Vlajkovic & Andrea Radonjanin - Integration of AI tool...
byDataScienceConferenc1
 
Copy of First ENGR 100 Presentation.pptx
bygenshinds971
 
[DSC Europe 25] Jim Sterne - Adopting Generative AI Capabilities Into the Ent...
byDataScienceConferenc1
 
Customer Segmentation: Seeing the Trees and the Forest Simultaneously - V2
bySK Palu
 
ELECTRICITY SECTOR IN PHILIPHINE .pptx
byahmadnabil090406
 
Natal SM 2025 - Latihan Liturgi Kelas 0 - 3 SD.pdf
byWenny63
 
Orientation-on-ARAL-Check-in-Assessment.pptx
byJenTeruel1
 
Safe_Spaces_Act_Modern_Presentation.pptx
byairesmijares2003
 
GRADE 8 MATH - Q3 - LESSON 1 -LINEAR EQUATION IN ONE VARIABLE.pptx
byEdlynMalangPanti
 
In this document
Powered by AI

Introduction to network visibility and control using sFlow telemetry. Presented by Peter Phaal from InMon Corp.

Quote by Lord Kelvin emphasizing the necessity of measurement for improvement.

Focus on capacity, demand, and savings through static vs dynamic provisioning in networks.

Fundamentals of feedback control systems requiring controllability and observability.

Using a driving analogy to explain observability and controllability in systems.

Effects of various types of delays on stability and control in network systems.

Definition of sFlow, its integration in switches and the ecosystem surrounding its usage.

Detailed standard metrics and the data-sampling architecture of sFlow in environments.

Counters provide partial insights into network issues; the emphasis on sampling for deeper analysis.

How sFlow integrates various data sources across different layers of network and applications.

The importance of using the right tools that work together effectively for monitoring.

Comparison of sFlow with other flow management tools like NetFlow for enhanced visibility.

Capabilities of sFlow-RT for low latency flow analytics and advantages over NetFlow.

Discussion of metrics, events, and the need for real-time monitoring in applications.

Comparison of various telemetry standards and encouraging efficient data aggregation.

Measuring and controlling cloud infrastructure's performance, including elephant flows.

Challenges faced in monitoring ECMP and detecting large flows affecting network performance.

Monitoring fabric-level performance, including collisions and the separation of flow types.

Discussing spine-leaf architecture and its relevance for SDN optimizations in data centers.

Strategies and opportunities for DDoS mitigation, including market trends and technologies.

Highlighting various tools and technologies for effective monitoring and control within networks.

Network visibility and control using industry standard sFlow telemetry

  • 1.
    Network visibility andcontrol using industry standard sFlow telemetry Peter Phaal InMon Corp.
 March, 2016 Twitter: @sFlow Blog: blog.sflow.com San Francisco Network Visibility Meetup
  • 2.
    Why monitor? “If youcan’t measure it, you can’t improve it” Lord Kelvin
  • 3.
    Time Capacity Demand Static provisioning $ Unusedcapacity $$$ Service failure $$ Unused capacity
  • 4.
  • 5.
  • 6.
    Controllability and Observability Basicconcept is simple, a stable feedback control system requires: 1. ability to influence all important system states (controllable) 2. ability to monitor all important system states (observable)
  • 7.
    It’s hard tostay on the road if you can’t see the road, or keep to the speed limit without a speedometer It’s hard to stay on the road or maintain speed if your brakes, engine or steering fail Controllability and Observability driving example Observability Controllability States location, speed, direction, ... Tule fog in California Central Valley
  • 8.
    Effect of delayon stability Measurement delay Planning delay Time Configuration delayDisturbance Response delay EffectLoop delay DDoS launched Identify target, attacker Black hole, mark, re-route? Switch CLI commands Route propagation Traffic dropped Components of loop delay e.g. Slow reaction time causes tired / drunk / distracted driver to weave, very slow reaction time and they leave the road
  • 9.
    What is sFlow? “InGod we trust. All others bring data.” Dr. Edwards Deming
  • 10.
    Industry standard measurementtechnology integrated in switches http://sflow.org/
  • 11.
    Open source agentsfor hosts, hypervisors and applications Host sFlow project (http://sflow.net) is center of an ecosystem of related open source projects embedding sFlow in popular operating systems and applications Host agent extends network visibility into public / private cloud
  • 12.
    Network (maintained byhardware in network devices) - MIB-2 ifTable: ifInOctets, ifInUcastPkts, ifInMulticastPkts, ifInBroadcastPkts, ifInDiscards, ifInErrors, ifUnkownProtos, ifOutOctets, ifOutUcastPkts, ifOutMulticastPkts, ifOutBroadcastPkts, ifOutDiscards, ifOutErrors Host (maintained by operating system kernel) - CPU: load_one, load_five, load_fifteen, proc_run, proc_total, cpu_num, cpu_speed, uptime, cpu_user, cpu_nice, cpu_system, cpu_idle, cpu_wio, cpu_intr, cpu_sintr, interupts, contexts - Memory: mem_total, mem_free, mem_shared, mem_buffers, mem_cached, swap_total, swap_free, page_in, page_out, swap_in, swap_out - Disk IO: disk_total, disk_free, part_max_used, reads, bytes_read, read_time, writes, bytes_written, write_time - Network IO: bytes_in, packets_in, errs_in, drops_in, bytes_out, packet_out, errs_out, drops_out Application (maintained by application) - HTTP: method_option_count, method_get_count, method_head_count, method_post_count, method_put_count, method_delete_count, method_trace_count, method_connect_count, method_other_count, status_1xx_count, status_2xx_count, status_3xx_count, status_4xx_count, status_5xx_count, status_other_count - Memcache: cmd_set, cmd_touch, cmd_flush, get_hits, get_misses, delete_hits, delete_misses, incr_hits, incr_misses, decr_hists, decr_misses, cas_hits, cas_misses, cas_badval, auth_cmds, auth_errors, threads, con_yields, listen_disabled_num, curr_connections, rejected_connections, total_connections, connection_structures, evictions, reclaimed, curr_items, total_items, bytes_read, bytes_written, bytes, limit_maxbytes Standard counters
  • 13.
    Simple - standard structures- densely packed blocks of counters - extensible (tag, length, value) - RFC 1832: XDR encoded (big endian, quad-aligned, binary) - simple to encode / decode - unicast UDP transport Minimal configuration - collector address - polling interval Cloud friendly - flat, two tier architecture: many embedded agents → central “smart” collector - sFlow agents automatically start sending metrics on startup, automatically discovered - eliminates complexity of maintaining polling daemons (and associated configurations) Scaleable push protocol
  • 14.
    • Counters tellyou there is a problem, but not why. • Counters summarize performance by dropping high cardinality attributes: - IP addresses - URLs - Memcache keys • Need to be able to efficiently disaggregate counter by attributes in order to understand root cause of performance problems. • How do you get this data when there are millions of transactions per second? Counters aren’t enough Why the spike in traffic? (100Gbit link carrying 14,000,000 packets/second)
  • 15.
    • Random samplingis lightweight • Critical path roughly cost of maintaining one counter:
 if(--skip == 0) sample(); • Sampling is easy to distribute among modules, threads, processes without any synchronization • Minimal resources required to capture attributes of sampled transactions • Easily identify top keys, connections, clients, servers, URLs etc. • Unbiased results with known accuracy Break out traffic by client, server and port (graph based on samples from100Gbit link carrying 14,000,000 packets/second) sFlow also exports random samples
  • 16.
    Integrated data model PacketHeader Source Destination TCP/UDP Socket TCP/UDP Socket MAC Address MAC Address Sampled Packet Headers + Forwarding State I/F Counters NETWORK HOST CPU Memory I/O Adapter MACs APPLICATION Sampled Transactions Transaction Counters TCP/UDP Socket Independent agents sFlow analyzer joins data for integrated view
  • 17.
    Virtual Servers Applications Apache/PHP Tomcat/Java Memcached Virtual Network Servers Network Embeddedmonitoring of all switches, all servers, all applications, all the time Consistent measurements shared between multiple management tools Comprehensive data center wide visibility
  • 18.
    Picking the righttools “This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together.” Doug McIlroy
  • 19.
    packets decode hash sendflowcache flushsample Flow Records flow cache embedded on switchswitch NetFlow IPFIX … decode hash sendflow cache flush Flow Records packets send polli/f counters sample multiple switches export sFlow packets send polli/f counters sample ... centralized software flow cache switch switch JSON/REST
 NetFlow IPFIX … • Reduce ASIC cost / complexity • Fast response (data not sitting on switch) • Centralized, network-wide visibility • Increase flexibility → software defined analytics Move flow cache from ASIC to external software Scale-out alternative to SNMP polling Traffic analytics with sFlow
  • 20.
    sFlow-RT.com analytics engine •Low latency flow analytics for real-time control applications • Disaggregates flow cache from database. Choose external database(s) for history (InfluxDB, Logstash, etc.) • Programmable analytics pipeline through open APIs
  • 21.
    RESTful API fordefining flows http://blog.sflow.com/2013/08/restflow.html curl -H "Content-Type:application/json" -X PUT —data '{"keys":"ipsource,ipdestination,tcpsourceport,tcpdestinationport", "value":"bytes", "ipfixCollectors":["10.0.0.1"]}' http://127.0.0.1:8008/flow/tcp/json curl -H "Content-Type:application/json" -X PUT --data '{"keys":"ipdestination,icmpunreachableport", "value":"frames"}' http://127.0.0.1:8008/flow/unreachableport/json • Instantly enables network wide monitoring of flows • All switches, all ports, including hosts and virtual switches • Contrast with task of re-configuring Flexible Netflow/IPFIX caches on every switch in multi-vendor network. How many simultaneous flow definitions are allowed? What key / value combinations are allowed? curl -H "Content-Type:application/json" -X PUT --data '{"value":"frames"}' http://127.0.0.1:8008/flow/frames/json
  • 22.
    InMon sFlow-RT active timeoutactive timeout NetFlow Open vSwitch SolarWinds Real-Time NetFlow Analyzer • sFlow does not use flow cache, so realtime charts more accurately reflect traffic trend • NetFlow spikes caused by flow cache active-timeout for long running connections Rapid detection of large flows Flow cache active timeout delays large flow detection, limits value of signal for real-time control applications
  • 23.
    Counters and packetsamples http://blog.sflow.com/2013/02/measurement-delay-counters-vs-packet.html • Packet samples give a fast signal that operates at scale • Counters are maintained in hardware and provide precise traffic totals. • Counters capture rare events, like packet discards, that can severely impact performance. • Counters report important link state information, like link speed, LAG group membership etc.
  • 24.
    Metrics and Events MetricsEvents Sources SNMP, sFlow, collectd Traps, syslog, IPFIX, NetFlow Crossover Event count Threshold event Collectors InfluxDB, OpenTSDB, Graphite, rrdtool Logstash, flowtools, Splunk Application Performance Security and alerts
  • 25.
    Data models andtransports sFlow SNMP NetFlow version 5 OpenConfig Telemetry IPFIX syslog Model standard measurements published by sFlow.org, Dataplane focus: based on IEEE, IETF, APIs (MIB-2, LAG-MIB, libvirt, JMX, …) standard MIBs defined by IETF standard tcp / udp / icmp flow record defined by Cisco Telemetry defined as part of YANG configuration models by OpenConfig.org
 Control plane focus: BGP, MPLS, VLAN, etc. Encoding XDR
 (RFC 4506) ASN1 (IETF) NetFlow (Cisco) protobufs, JSON, NetConf IPFIX (IETF) Syslog (RFC 5424) Transport UDP UDP UDP UDP, HTTP SCTP, TCP, UDP UDP Mode Push Pull Push Push Push Push Easy to combine multiple data sources if you disaggregate tool chain 
 e.g. separate agents from collectors, feed data from all sources into InfluxDB / Logstash
  • 26.
    sflowtool https://github.com/sflow/sflowtool sflowtool replicate ascii Perl, Python, awk,grep pcap wireshark, tcpdump Netflow Open source command line tool
  • 27.
    Network visibility forDevOps tools • Streaming filtering and summarization reduces data volume and increases scaleability of backend tools • Streaming flow analytics to generate application metrics
  • 28.
    Feedback control ofcloud infrastructure “You can’t control what you can’t measure” Tom DeMarco
  • 29.
    Cloud depends onnetwork • Server costs (both capex and power consumption) far exceed networking costs in the data center. • Network congestion caused server to wait, resulting in poor utilization of cloud infrastructure. • Optimize network to increase data center efficiency http://perspectives.mvdirona.com/2010/09/overall-data-center-costs/ “Typically the resource that is most scarce is the network.” Amin Vahdat, Google, ONS2015 Keynote http://blog.sflow.com/2015/06/optimizing-software-defined-data-center.html
  • 30.
    http://blog.sflow.com/2013/02/sdn-and-large-flows.html Elephant flows arethe small number of long lived large flows responsible for majority of bytes on network Large “Elephant” flows
  • 31.
  • 32.
    Default sFlow settings PortSpeed Large Flow Sampling Rate Polling Interval 1 Gbit/s >= 100 Mbit/s 1-in-1,000 30 seconds 10 Gbit/s >= 1 Gbit/s 1-in-10,000 30 seconds 25 Gbit/s >= 2.5 Gbit/s 1-in-25,000 30 seconds 40 Gbit/s >= 4 Gbit/s 1-in-40,000 30 seconds 50 Gbit/s >= 5 Gbit/s 1-in-50,000 30 seconds 100 Gbit/s >= 10 Gbit/s 1-in-100,000 30 seconds Supports real-time detection of large “Elephant” flows where large flow is defined as flow consuming >10% of link bandwidth for >1 second http://blog.sflow.com/2013/06/large-flow-detection.html
  • 33.
  • 34.
  • 35.
    ECMP monitoring challenge •large number of links, 12 x 10G links • all links need to be monitored continuously, 180G total bandwidth • real-time detection of congested links • real-time detection of Elephant flows http://blog.sflow.com/2015/03/ecmp-visibility-with-cumulus-linux.html
  • 36.
    Fabric level performancemetrics • Fabric View application runs on sFlow-RT • Downloadable from sFlow-RT.com, includes captured data set from 4 node 10G ECMP fabric • Elephant collisions on spine links occur frequently • Collisions halve throughput • Collisions cause packet discards http://blog.sflow.com/2015/10/fabric-view.html
  • 37.
    http://www.slideshare.net/martin_casado/elephants-and-mice-elephant-detection-in-the-vswitch-with-hardware-handling Large “Elephant” flows delaysmall “Mice” flows Separating Elephants and Mice into different queues significantly improves latency and response time Large flow marking
  • 38.
  • 39.
    ONS2015 SDN Showcase
 CORD:Open-source spine-leaf fabric
  • 40.
    http://blog.sflow.com/2015/06/leaf-and-spine-traffic-engineering.html CORD: Open-source spine-leaffabric Rack Rack Rack Rack Spine http://blog.sflow.com/2015/08/cord-open-source-spine-leaf-fabric.html
  • 41.
    SDN Central DemoFriday SDN Optimization of Hybrid Packet / Optical Data Center Fabric
  • 42.
    http://blog.sflow.com/2014/09/sdn-control-of-hybrid-packet-optical.html Cumulus, Calient and InMonPoC. Large flows detected in real-time and diverted to direct ToR to ToR optical circuit SDN Optimization of Hybrid Packet / Optical Data Center Fabric
  • 43.
  • 44.
    https://eos.arista.com/spotifys-sdn-internet-router/ IXP Routes InstalledRoutes not installed LINX 18672 95711 DECIX 12518 108164 EQIX-ASH 27687 75146 https://eos.arista.com/arista-eos-bgp-selective-route-download/
  • 45.
  • 46.
    Internet Router PoC http://blog.sflow.com/2015/07/white-box-internet-router-poc.html Peer1 Peer 2 Peer N DDoS Mitigation REST API sFlow-RT controller real-time analytics OpenFlow, REST, BGP, … Large flow marking Routing Offload BGP … BGP Router Switch http://blog.sflow.com/2015/10/active-route-manager.html
  • 47.
    Open Networking SummitSDN Idol winning solution Real-time SDN Analytics for DDoS mitigation
  • 48.
    DDoS Mitigation MarketOpportunity DDoS Attack Megatrends [Reference 1] • High bandwidth, volumetric infrastructure layer (Layer 3 & 4) attacks increased approximately 30 percent • DDoS attack volume also increased month-to-month in 2013, with 10 out of 12 months showing higher attack volume compared to 2012 • Average DDoS attack sizes continued to increase – many over 100 Gbps, the largest peaking at 179 Gbps DDoS Mitigation Market Growth • $870M market by 2017, 18.2% CAGR – Source: IDC:Worldwide DDoS Prevention Products and Services 2013-2017 Forecast • $1049M market by 2017, 25% CAGR – Source: Infonetics: Global DDoS Prevention Appliances 2012-2017 Forecast Reference 1: Top DDoS Attack Trends http://www.itbriefcase.net/top-ddos-attack-trends-for-2013
  • 49.
    DDoS Mitigation UseCase (1) ISP 1 ISP 2 ISP N • ISP/IX is uniquely positioned to protect customers from DDoS flood attacks • New revenue from DDoS mitigation service + differentiates ISP/IX service Attacker User Prevent attack from overwhelming customer access link Filter attack traffic in real-time Customer network DDoS target host Attack on single host can take out entire customer data center. Customer cannot mitigate flood attack without upstream help ISP / IX ISP/IX Market Segment
  • 50.
    Customer portal DDoS Mitigation Service WebUI + RESTful programmatic API • real-time TopN analytics • programmable filtering of traffic • set thresholds + automatic blocking Real-time sFlow visibility, Hybrid OpenFlow Control capability of Brocade switches/routers REST API InMonsFlow-RT REST API OpenFlowController DDoS Mitigation Application Customer Network Internet 1. Flood attack overloads customer port 2. Attack maps to large flows [Ref. 2]. sFlow-RT detects attack (maps to large flows) and characterizes attack (srcip, dstip, protocol, ports, etc.) 3. mitigation application takes signature, applies customer policy, selects optimal control and push OpenFlow rule(s) to switch(es) 5. OpenFlow rule(s) applied to switch forwarding path to drop / mark traffic and protect link HTTPS HTTPS 4. Controller pushes OpenFlow rule(s) to switch(es) OpenFlow 1.3 Match Fields line rate filtering using Brocade switches Reference 2: IETF I2RS Working Group Draft - https://ietf.org/doc/draft-krishnan-i2rs-large-flow-use-case/
  • 51.
  • 52.
  • 53.
    Cloudflare DDoS mitigation http://blog.sflow.com/2016/02/cloudflare-ddos-mitigation-pipeline.html •sFlow used to identify attack signature • BPF created to match signature • Detailed signatures minimize collateral damage
  • 54.
    Big Switch NetworkWebinar Big Tap sFlow: Enabling Pervasive Flow-level Visibility
  • 55.
  • 56.
  • 57.
    Comments • sFlow instrumentationis widely available in switches http://sflow.org/products/network.php • Host sFlow (sFlow.net) agent extends visibility into servers (works with libpcap, iptables, Open vSwitch to efficiently sample packets in host data plane) • Common data model ensures strong interoperability across sFlow data sources • Streaming counter and packet telemetry across network, compute and application tiers makes data center observable • Observability makes it possible to apply feedback controls
  • 58.
  • 59.
    Host sFlow monitoringof Linux datapath Technology Reference Adapter, bridge, macvlan, ipvlan Berkeley Packet Filter (BPF) sampling function http://blog.sflow.com/ 2016/02/linux-bridge- macvlan-ipvlan-adapters.html Open vSwitch Kernel datapath has sFlow support http://openvswitch.org/ support/config- cookbooks/sflow/ Linux Firewall iptables statistic module random function with ulog http://blog.sflow.com/ 2010/12/ulog.html Top of Rack Switch ASIC provides wirespeed monitoring of attached servers http://blog.sflow.com/ 2010/04/hybrid-server- monitoring.html Efficient monitoring of high traffic production workloads
  • 60.
    Open vSwitch FallConference New OVS instrumentation features aimed at real-time monitoring of virtual networks
  • 61.
    Overlay / UnderlayVisibility http://blog.sflow.com/2015/11/network-virtualization-visibility-demo.html
  • 62.
    Open vSwitch FallConference OVN service injection demonstration
  • 63.
  • 64.
    White Paper Actionable Intelligencein the SDN Ecosystem: Optimizing Network Traffic through FRSA
  • 65.
  • 66.
    Dell NFV Summit2015 Demystifying NFV Infrastructure Hotspots End-to-end Monitoring, Analytics & SDN Control
  • 67.
    Demystifying NFV InfrastructureHotspots http://blog.sflow.com/2016/01/demystifying-nfv-infrastructure-hotspots.html
  • 68.
  • 69.
    Hybrid OpenFlow ECMPtestbed http://blog.sflow.com/2015/01/hybrid-openflow-ecmp-testbed.html http://mininet.org/ • Simulated ECMP network for developing visibility and control applications • sFlow support in Open vSwitch • OpenFlow for control
  • 70.
    The sFlow Standard:Scalable, Unified Monitoring of Networks, Systems and Applications 2012 Velocity Conference
  • 71.