SlideShare a Scribd company logo
1 of 6
Download to read offline
Introduction to Flow Analysis


                                      Vincent Berk
                                      February 3rd , 2011



           VINCENT BERK
Copyright © 2011 Process Query Systems, LLC
Overview
                                              Computers communicate over the network,
                                              in streams of thousands of packets.
                                              Actions, such as sending email, result in
                                              streams of related packets, called “flows”.

                                              Most routers, firewalls, and switches can
                                              report summaries of all their flows.
                                              This process of reporting on flows is
                                              called “exporting” of flows.

                                              Flows are exported to a “collector”, which
                                              may aggregate, plot, or store the flows.
                                              A collector is a separate program running
                                              on a network server.


           VINCENT BERK
Copyright © 2011 Process Query Systems, LLC
Exporter
Exporters are routers, firewalls, or switches
capable of forwarding flow summaries.
Most top- and middle-tier networking hardware
is capable of exporting flow summaries.

If your hardware is not capable of exporting
flows, a software exporter can be used.
This is a program that runs on a computer which
must be attached to a SPAN/TAP/Mirror port on a
switch or router, and does the flow exporting.

Exported flows are only traffic summaries,
they do not contain any traffic content.
For instance: a flow reports the connections to
an email server, but not the content of the emails.

               VINCENT BERK
    Copyright © 2011 Process Query Systems, LLC
Collector
                                                   A collector is a server with software that
                                                   can accept and interpret flow exports.
                                                   Exporters send their flow summaries to
                                                   collectors for storage and analysis.

Most collectors summarize and aggregate the
flows before storage, discarding the records.
Although coarse, this approach is fastest.
The cost is the loss of forensic accuracy.

                                                    Some collectors store all flow records,
                                                    alowing full recall, and precise filtering.
                                                    The value of a flow product depends
                                                    mostly on the implementation of the
                                                    collector.

                VINCENT BERK
     Copyright © 2011 Process Query Systems, LLC
Flow Formats
Flow exports come in many formats. Some
Manufacturers are compatible, others not.
Adding flow capability to your network will
increase the traffic load by 1% to 5%.


                                                  Some flow formats are sampled. This means
                                                  that only some flows are reported on.
                                                  Forensic accuracy is lost to gain some speed.
                                                  sFlow ® uses this strategy.

Most flow formats report every flow, allowing
full flow recall, if the collector supports this.
NetFlow, Cflow, and Jflow use this strategy.
If your hardware only supports flow sampling,
you can always use a software exporter instead.

               VINCENT BERK
    Copyright © 2011 Process Query Systems, LLC
For Additional Information:
                        http://www.proquesys.com
                           info@proquesys.com
                               603.727.4477
ProQueSys FlowTraq                   A full fidelity flow collector. Supports: IPv6, NetFlow
                                     v1/5/7/9, sFlow v2/4/5, automated alerting, scheduled
                                     reporting, user dashboards, GUI and CLI interfaces.

ProQueSys Flow Exporter FREE downloadable software flow exporter. Supports:
                        IPv6, exporting in NetFlow v5 and v9, VLAN, IFindex
                        specification, exporters to 16 destinations at once.


             VINCENT BERK
  Copyright © 2011 Process Query Systems, LLC

More Related Content

Similar to Introduction to flow analysis

Monitoring&Logging - Stanislav Kolenkin
Monitoring&Logging - Stanislav Kolenkin  Monitoring&Logging - Stanislav Kolenkin
Monitoring&Logging - Stanislav Kolenkin Kuberton
 
Openflow overview
Openflow overviewOpenflow overview
Openflow overviewopenflowhub
 
Looking at SDN with DDS Glasses
Looking at SDN with DDS GlassesLooking at SDN with DDS Glasses
Looking at SDN with DDS GlassesAngelo Corsaro
 
Cloud Automation Manager
Cloud Automation ManagerCloud Automation Manager
Cloud Automation ManagerNithin Babu
 
Network Security and Visibility through NetFlow
Network Security and Visibility through NetFlowNetwork Security and Visibility through NetFlow
Network Security and Visibility through NetFlowLancope, Inc.
 
Music city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lakeMusic city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lakeTimothy Spann
 
PortoTechHub - Hail Hydrate! From Stream to Lake with Apache Pulsar and Friends
PortoTechHub  - Hail Hydrate! From Stream to Lake with Apache Pulsar and FriendsPortoTechHub  - Hail Hydrate! From Stream to Lake with Apache Pulsar and Friends
PortoTechHub - Hail Hydrate! From Stream to Lake with Apache Pulsar and FriendsTimothy Spann
 
Mahout low-overhead datacenter traffic management using end-host-based ...
Mahout   low-overhead  datacenter  traffic management  using end-host-based  ...Mahout   low-overhead  datacenter  traffic management  using end-host-based  ...
Mahout low-overhead datacenter traffic management using end-host-based ...João Gabriel Lima
 
FlowER Erlang Openflow Controller
FlowER Erlang Openflow ControllerFlowER Erlang Openflow Controller
FlowER Erlang Openflow ControllerHolger Winkelmann
 
Analytics driven SDN and commodity switches
Analytics driven SDN and commodity switchesAnalytics driven SDN and commodity switches
Analytics driven SDN and commodity switchesnetvis
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at SpeedC4Media
 
StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011
StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011
StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011darach
 
Software defined network and Virtualization
Software defined network and VirtualizationSoftware defined network and Virtualization
Software defined network and Virtualizationidrajeev
 
Apache flume by Swapnil Dubey
Apache flume by Swapnil DubeyApache flume by Swapnil Dubey
Apache flume by Swapnil DubeySwapnil Dubey
 
software defined network, openflow protocol and its controllers
software defined network, openflow protocol and its controllerssoftware defined network, openflow protocol and its controllers
software defined network, openflow protocol and its controllersIsaku Yamahata
 
Is 12 Factor App Right About Logging
Is 12 Factor App Right About LoggingIs 12 Factor App Right About Logging
Is 12 Factor App Right About LoggingPhil Wilkins
 

Similar to Introduction to flow analysis (20)

Monitoring&Logging - Stanislav Kolenkin
Monitoring&Logging - Stanislav Kolenkin  Monitoring&Logging - Stanislav Kolenkin
Monitoring&Logging - Stanislav Kolenkin
 
Openflow overview
Openflow overviewOpenflow overview
Openflow overview
 
Looking at SDN with DDS Glasses
Looking at SDN with DDS GlassesLooking at SDN with DDS Glasses
Looking at SDN with DDS Glasses
 
Cloud Automation Manager
Cloud Automation ManagerCloud Automation Manager
Cloud Automation Manager
 
Network Security and Visibility through NetFlow
Network Security and Visibility through NetFlowNetwork Security and Visibility through NetFlow
Network Security and Visibility through NetFlow
 
Music city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lakeMusic city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lake
 
PortoTechHub - Hail Hydrate! From Stream to Lake with Apache Pulsar and Friends
PortoTechHub  - Hail Hydrate! From Stream to Lake with Apache Pulsar and FriendsPortoTechHub  - Hail Hydrate! From Stream to Lake with Apache Pulsar and Friends
PortoTechHub - Hail Hydrate! From Stream to Lake with Apache Pulsar and Friends
 
Inside Flume
Inside FlumeInside Flume
Inside Flume
 
Mahout low-overhead datacenter traffic management using end-host-based ...
Mahout   low-overhead  datacenter  traffic management  using end-host-based  ...Mahout   low-overhead  datacenter  traffic management  using end-host-based  ...
Mahout low-overhead datacenter traffic management using end-host-based ...
 
FlowER Erlang Openflow Controller
FlowER Erlang Openflow ControllerFlowER Erlang Openflow Controller
FlowER Erlang Openflow Controller
 
Analytics driven SDN and commodity switches
Analytics driven SDN and commodity switchesAnalytics driven SDN and commodity switches
Analytics driven SDN and commodity switches
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
 
Tapping Into the Health of Your Network
Tapping Into the Health of Your NetworkTapping Into the Health of Your Network
Tapping Into the Health of Your Network
 
StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011
StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011
StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011
 
NFA - Middle East Workshop
NFA - Middle East WorkshopNFA - Middle East Workshop
NFA - Middle East Workshop
 
Software defined network and Virtualization
Software defined network and VirtualizationSoftware defined network and Virtualization
Software defined network and Virtualization
 
Apache flume by Swapnil Dubey
Apache flume by Swapnil DubeyApache flume by Swapnil Dubey
Apache flume by Swapnil Dubey
 
software defined network, openflow protocol and its controllers
software defined network, openflow protocol and its controllerssoftware defined network, openflow protocol and its controllers
software defined network, openflow protocol and its controllers
 
OVS-LinuxCon 2013.pdf
OVS-LinuxCon 2013.pdfOVS-LinuxCon 2013.pdf
OVS-LinuxCon 2013.pdf
 
Is 12 Factor App Right About Logging
Is 12 Factor App Right About LoggingIs 12 Factor App Right About Logging
Is 12 Factor App Right About Logging
 

Introduction to flow analysis

  • 1. Introduction to Flow Analysis Vincent Berk February 3rd , 2011 VINCENT BERK Copyright © 2011 Process Query Systems, LLC
  • 2. Overview Computers communicate over the network, in streams of thousands of packets. Actions, such as sending email, result in streams of related packets, called “flows”. Most routers, firewalls, and switches can report summaries of all their flows. This process of reporting on flows is called “exporting” of flows. Flows are exported to a “collector”, which may aggregate, plot, or store the flows. A collector is a separate program running on a network server. VINCENT BERK Copyright © 2011 Process Query Systems, LLC
  • 3. Exporter Exporters are routers, firewalls, or switches capable of forwarding flow summaries. Most top- and middle-tier networking hardware is capable of exporting flow summaries. If your hardware is not capable of exporting flows, a software exporter can be used. This is a program that runs on a computer which must be attached to a SPAN/TAP/Mirror port on a switch or router, and does the flow exporting. Exported flows are only traffic summaries, they do not contain any traffic content. For instance: a flow reports the connections to an email server, but not the content of the emails. VINCENT BERK Copyright © 2011 Process Query Systems, LLC
  • 4. Collector A collector is a server with software that can accept and interpret flow exports. Exporters send their flow summaries to collectors for storage and analysis. Most collectors summarize and aggregate the flows before storage, discarding the records. Although coarse, this approach is fastest. The cost is the loss of forensic accuracy. Some collectors store all flow records, alowing full recall, and precise filtering. The value of a flow product depends mostly on the implementation of the collector. VINCENT BERK Copyright © 2011 Process Query Systems, LLC
  • 5. Flow Formats Flow exports come in many formats. Some Manufacturers are compatible, others not. Adding flow capability to your network will increase the traffic load by 1% to 5%. Some flow formats are sampled. This means that only some flows are reported on. Forensic accuracy is lost to gain some speed. sFlow ® uses this strategy. Most flow formats report every flow, allowing full flow recall, if the collector supports this. NetFlow, Cflow, and Jflow use this strategy. If your hardware only supports flow sampling, you can always use a software exporter instead. VINCENT BERK Copyright © 2011 Process Query Systems, LLC
  • 6. For Additional Information: http://www.proquesys.com info@proquesys.com 603.727.4477 ProQueSys FlowTraq A full fidelity flow collector. Supports: IPv6, NetFlow v1/5/7/9, sFlow v2/4/5, automated alerting, scheduled reporting, user dashboards, GUI and CLI interfaces. ProQueSys Flow Exporter FREE downloadable software flow exporter. Supports: IPv6, exporting in NetFlow v5 and v9, VLAN, IFindex specification, exporters to 16 destinations at once. VINCENT BERK Copyright © 2011 Process Query Systems, LLC