Network processing by pid

Energia Open Source Network Monitoring by process id Nuno Martins nuno.martins@caixamagica.pt

Energia Open Source Monitoring  Understand programs actual behaviour when running  Analyse resource usage  Create a usage profile, to evaluate performance and correctness  Can be done actively (polling the monitor) or passively  Through dynamic or static instrumentation  Analysis of the data can be done online (during capture) or offline (post mortem)27.02.2012 Network Monitoring by Process id 2

Energia Open Source Network Monitoring  Network monitoring is done passively by capturing packets  Generally it is done using PCap (Packet Capture) Library with LSF (Linux Socket Filtering)  Analysis of communication protocols  Analysis of the interactions between distributed entities  Error detection, performance rating, troubleshooting, etc.  The reduction in obtained data is done through filters  This way the overhead is reduced (because it doesnt have to copy so much data)  Only capture the relevant packets for analysis27.02.2012 Network Monitoring by Process id 3

Energia Open Source Actual Network Monitoring User TCPDump User APP User APP User APP ... LibPCAP Kernel Network TCP/IP stack PACKET stack Before sending or receiving packets Hardware NIC27.02.2012 Network Monitoring by Process id 4

Energia Open Source Actual Monitoring Mechanism  A filter is a set of rules  These rules can be based on ports, addresses, protocols, etc.  Or on a specific set of bytes  Rules are combined with logical operands  Changing the filter with libpcap  Necessary to drain the socket  Possible to lose packets  When filters are more complex they cant be applied on kernel space  All packets are captured and the filter is applied in userspace  Without specifying a filter the monitor will capture all packets27.02.2012 Network Monitoring by Process id 5

Energia Open Source Challenges of network monitoring by process id  Monitoring the network activity of a process in user space is limited to some cases and can overload the system  Changing the filter by using libpcap has non-negligible latency Compile and Draining and pcap filter optimise attach the new filter 0 Time Latency when attaching a new filter27.02.2012 Network Monitoring by Process id 6

Energia Open Source Network Monitoring by Process ID  New Approach towards the reduction of data for analysis  More specific analysis  Performance and overhead issues  When we only want a subset of the packets flowing on the network card  Can simplify the use of bpf filters27.02.2012 Network Monitoring by Process id 7

Energia Open Source Why we need this  Analysis of closed protocols  No access to source code  Not easy to understand / follow some network protocols  Debug protocols when creating new ones  Determining if a program is leaking information  In production machines  Cannot stop servers for debug  Troubleshooting specific threads27.02.2012 Network Monitoring by Process id 8

Energia Open Source Kernel Space Changes  Two Main Parts  Main kernel code  Created a hook to be attached by the filter function  Changed the filter_function at the end to call the hook  Module (MRoP – “Monitorização de Rede orientada ao Processo” [Pt])  KProbes handler functions  Repository (a RB-Tree)  Filter function  User space interface (through DebugFS)27.02.2012 Network Monitoring by Process id 9

Energia Open Source Kernel Components  The Kernel module developed has 4 components  Syscalls hooked handlers Repository (of socket information)  ports and addresses Filter function DebugFS interface  Communication to/from user land  Statistics and control27.02.2012 Network Monitoring by Process id 10

Energia Open Source New Filter Mechanism (by process id) Root user App DebugFS Monitor App tcpdump User appUser space PID, stats, etc Sending Kernel or receiving Packet Kernel packets stack filter module27.02.2012 Network Monitoring by Process id 11

Energia Open Source Generic Process TCPDump Monitor Pcap Library Control API AF_PACKET KProbes Packets Instrumented Syscalls handlers Hook Repository Packet Filter NIC Function Driver NIC27.02.2012 Network Monitoring by Process id 12

Energia Open Source KProbes  Dynamic Instrumentation Mechanism on kernel space  Different types of instrumentation based on what want to be achieved  Kprobe, Jprobe, KretProbe  int 3 instruction (trap)  Does not need Debug information  Uses kallsyms  To locate symbols  Overhead of 0.6 microseconds  per probe hit  Its a mechanism not a tool27.02.2012 Network Monitoring by Process id 13

Energia Open Source Syscalls hooked handlers  Connect, accept, bind, recvfrom, sendto and sock_close function  KProbes is a kernel mechanism for instrumentation KRetProbes are probes that get the return value of the functions KRetProbes use a trampolin to catch the return value  Inside the handlers the computation must be very quick27.02.2012 Network Monitoring by Process id 14

Energia Open Source Syscalls hooked handlers (II)  Inside handlers  Need to get the socket information  The sockets information is on the parameters of the syscall or on socket descriptor  The registers have the value of the socket descriptor  The socket information is written to the repository  The are two handlers, one on the entry of the function and the other on the return  This way on the return handler using the return value we are sure if the call was successful, if not the information on the repository regarding that socket is removed27.02.2012 Network Monitoring by Process id 15

Energia Open Source Filter Function  Packets that will be accepted by the bpf program filter are evaluated by the module filter function  The filter function only evaluates TCP and UDP packets  Search the repository for the packet port If it finds verifies the protocol and the address27.02.2012 Network Monitoring by Process id 16

Energia Open Source Repository  This repository is for socket information so that the filter can know if a specific packet belongs to the target process  Implemented using a Red and Black Tree  Mainly performance (must be searched once per packet)27.02.2012 Network Monitoring by Process id 17

Energia Open Source Repository II27.02.2012 Network Monitoring by Process id 18

Energia Open Source DebugFS interface to user land  Created a directory on the DebugFS Files for controlling the filtering mechanism Search the process structure and add information to the repository Clear the repository Identify which process to monitor Files for statistics/logging purposes How many packets passed/dropped by the filter How many times the handlers functions were called How many elements have the repository27.02.2012 Network Monitoring by Process id 19

Energia Open Source Evaluation  Functional evaluation  Created small and specific programs to verify socket information on both sides (user space and kernel space)  Transfered data using http, ftp and iperf protocols  Data transferred with http and ftp was monitored and saved to a file  It was visualized on Wireshark  Application layer data was compared with the data sent using md5 and sha1 checksums27.02.2012 Network Monitoring by Process id 20

Energia Open Source Evaluation II  Performance Evaluation  Evaluated the overhead using a 1GB transfer  While transferring data it was monitored and saved to a file  Overhead introduced:  Dynamic instrumentation  Managing the repository27.02.2012 Network Monitoring by Process id 21

Energia Open Source Performance Evaluation  Two machines connected directly on a 100 Mbit/s link  Transferring 1GB data through that link using:  ftp, http protocols and iperf tool  Capturing only one flow of data  Capturing two flows of data (one being the relevant one)  Capturing only one flow using MRoP VS  Capturing two flows  Measured times:  Without monitoring (transfer only)  Using standard monitoring  Using MRoP (tcpdump + developed kernel module)27.02.2012 Network Monitoring by Process id 22

Energia Open Source Performance Evaluation II27.02.2012 Network Monitoring by Process id 23

Energia Open Source Performance Evaluation III 3.5%27.02.2012 Network Monitoring by Process id 24

Energia Open Source Network Monitoring by Process ID  Created a new kernel module to extend functionality of network monitoring of a Process  Only captures the relevant packets for analysis  Low overhead  Maintains compatibility with old bpf filters  Shows better results when the analysis is focused on a subset of the network packets27.02.2012 Network Monitoring by Process id 25

Energia Open Source More Integrated (Work in Progress)  This approach has some integration issues  It is necessary to use the debugfs to introduce process ids  Not integrated with libpcap  Filters still dont have a pid mnemonic  The core functionality will be kept27.02.2012 Network Monitoring by Process id 26

Energia Open Source Generic Process TCPDump Monitor PCap Library Control API AF_PACKET KProbes Packets Instrumented Syscalls handlers Hook Repository Packet Filter NIC Function Driver NIC27.02.2012 Network Monitoring by Process id 27

Energia Open Source Generic Process TCPDump Monitor Pcap Library Control API (as debug) AF_PACKET KProbes Packets Control API Instrumented Syscalls handlers Hook Packet Filter NIC Repository Function Driver NIC27.02.2012 Network Monitoring by Process id 28

Energia Open Source Final Considerations  These changes benefit the new monitoring system based on process id  The mechanism of applying a new filter function may also benefit other kernel developers  Wireshark developers desire a mechanism to filter packets based on application id (they have it in their wish list)  Maybe it can be used to detect malware (since the instrumentation is done below the userspace)27.02.2012 Network Monitoring by Process id 29

http://corptv.caixamagica.pt	193
http://pinguinsmagicos.blogs.sapo.pt	134
http://alphamatrix.info	4
http://blogs.sapo.pt	3
http://alphamatrix.org	3
http://spondulix9.ignisi.com	2
http://www.alphamatrix.org	1
http://www.linkedin.com	1

Network processing by pid

by CMNunoGMartins

Accessibility

Categories

Upload Details

Usage Rights

8 Embeds 341

Statistics