Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[MeetUp][1st] 오픈소스를 활용한 xflow 수집-시각화

332 views

Published on

https://www.facebook.com/groups/InfraEngineer

Published in: Technology
  • Be the first to comment

[MeetUp][1st] 오픈소스를 활용한 xflow 수집-시각화

  1. 1. 출처 : https://www.researchgate.net/figure/NetFlow-datagram-format_fig2_220110281
  2. 2. 참고&출처 : https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html
  3. 3. Traffic Sampled Netflow : Sampling 1 out of 4 Packets NetFlow Collector Traffic NetFlow Random Sampled Netflow : Sampling 1 out of 4 Packets
  4. 4. 참고 : https://sflow.org/developers/diagrams/sFlowV5FlowData.pdf
  5. 5. Traffic Sampling 1 out of 4 Packets sFlow Collector
  6. 6. 출처 : https://blog.sflow.com/2009/05/scalability-and-accuracy-of-packet.html
  7. 7. Link Speed Large Flow Sampling Rate Polling Interval 10 Mbit/s >= 1 Mbit/s 1-in-10 20 seconds 100 Mbit/s >= 10 Mbit/s 1-in-100 20 seconds 1 Gbit/s >= 100 Mbit/s 1-in-1,000 20 seconds 10 Gbit/s >= 1 Gbit/s 1-in-10,000 20 seconds 40 Gbit/s >= 4 Gbit/s 1-in-40,000 20 seconds 100 Gbit/s >= 10 Gbit/s 1-in-100,000 20 seconds 출처 : https://blog.sflow.com/2013/06/large-flow-detection.html
  8. 8. Netflow sFlow Sampling Type Flow Sampling Packet Sampling CPU / Memory Usage High Low Interface Counters Not supported Fully Supported IP/ICMP/UDP/TCP Fully Supported Fully Supported Ethernet/802.3 Not supported Fully Supported Packet Headers Specific Fields Only Fully Supported IPX, Apple Talk Not supported Fully Supported Input/Output Interfaces Fully Supported Fully Supported Input/Output VLAN Some Vendors Fully Supported Source & Destination subnet/prefix Fully Supported Fully Supported
  9. 9. 출처 : https://www.slideshare.net/pphaal/network-visibility-and-control-using-industry-standard-sflow-telemetry
  10. 10. Grafana Elastic Search 7Horizon 25 Elastic Search 6Horizon 24 Drift Drift Opennms-helm
  11. 11. Grafana Elastic Search 7 Horizon Flow Parser Flow enricher Flow Writer Flow API ☞ The location the NetFlow package is coming from ☞ The address of the exporter ☞ Node ID Flow Package(JAVA) Flow Package Flow Package (enricher) Flow Collector
  12. 12. • yum -y install java maven unzip • wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.3.2-x86_64.rpm • yum install elasticsearch-7.3.2-x86_64.rpm
  13. 13. • cat << EOF >> /etc/security/limits.conf elasticsearch - nofile 65535 elasticsearch - nproc 4096 EOF • cat << EOF >> /etc/sysctl.conf vm.max_map_count = 262144 EOF • vi /usr/lib/systemd/system/elasticsearch.service [Service] LimitMEMLOCK=infinity 출처 : https://www.elastic.co/guide/en/elasticsearch/reference/master/system-config.html
  14. 14. • curl -XGET 'localhost:9200/_cluster/health?pretty' { "cluster_name" : "elasticsearch", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 501, "active_shards" : 501, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 497, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 50.20040080160321 }
  15. 15. • wget https://github.com/OpenNMS/elasticsearch-drift-plugin/archive/es-7.3.x.zip • unzip es-7.3.x.zip • cd elasticsearch-drift-plugin-es-7.3.x • vi pom.xml <groupId>org.opennms.elasticsearch</groupId> <artifactId>elasticsearch-drift-plugin</artifactId> <version>7.3.2-SNAPSHOT</version> … <properties> <elasticsearch.version>7.3.2</elasticsearch.version> • mvn clean package • /usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/elasticsearch-drift- plugin-es-7.3.x/target/releases/elasticsearch-drift-plugin-7.3.2-SNAPSHOT.zip
  16. 16. • /usr/share/elasticsearch/bin/elasticsearch-plugin list opennms-drift • curl 'localhost:9200/_cat/plugins?v&s=component&h=name,component,version,description’ name component version description localshot opennms-drift 7.3.2-SNAPSHOT The Drift plugin exposes additional aggregations for analysis of Netflow data.
  17. 17. • yum -y install https://yum.opennms.org/repofiles/opennms-repo-stable-rhel7.noarch.rpm • rpm --import https://yum.opennms.org/OPENNMS-GPG-KEY • yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo- latest.noarch.rpm • yum install opennms yum-utils java-11-openjdk java-11-openjdk-devel postgresql10 postgresql10-server
  18. 18. • /usr/pgsql-10/bin/postgresql-10-setup initdb • systemctl start postgresql-10 #Opennms database 생성 및 계정 생성 • su - postgres • createuser -P opennms • createdb -O opennms opennms #Postgres super user 계정 password 변경 • psql -c "ALTER USER postgres WITH PASSWORD 'YOUR-POSTGRES-PASSWORD';" • Exit • vi /var/lib/pgsql/10/data/pg_hba.conf host all all 127.0.0.1/32 md5 <= md5로 수정 host all all ::1/128 md5 <= md5로 수정
  19. 19. • vi ${OPENNMS_HOME}/etc/opennms-datasources.xml <jdbc-data-source name="opennms" database-name="opennms" class-name="org.postgresql.Driver" url="jdbc:postgresql://localhost:5432/opennms" user-name="** YOUR-OPENNMS-USERNAME **" password="** YOUR-OPENNMS-PASSWORD **" /> <jdbc-data-source name="opennms-admin" database-name="template1" class-name="org.postgresql.Driver" url="jdbc:postgresql://localhost:5432/template1" user-name="postgres" password="** YOUR-POSTGRES-PASSWORD **" />
  20. 20. • vi ${OPENNMS_HOME}/etc/telemetryd-configuration.xml <!-- Netflow v5 listener & adapters --> <listener name="Netflow-5-UDP-8877" class-name="org.opennms.netmgt.telemetry.listeners.UdpListener" enabled="false"> …… <parameter key="port" value="8877"/> <!-- Netflow v9 listener & adapters --> <listener name="Netflow-9-UDP-4729" class-name="org.opennms.netmgt.telemetry.listeners.UdpListener" enabled="false"> <parameter key="port" value="4729"/> ….. <!-- SFlow listener & adapters --> <listener name="SFlow-UDP-6343" class-name="org.opennms.netmgt.telemetry.listeners.UdpListener" enabled="true"> <parameter key="port" value="6343"/> • ${OPENNMS_HOME}/bin/send-event.pl -p 'daemonName Telemetryd' uei.opennms.org/internal/reloadDaemonConfig
  21. 21. • ssh -p 8101 admin@localhost admin@opennms> config:edit org.opennms.features.flows.persistence.elastic admin@opennms> config:property-set elasticUrl http://elasticsearch-server-ip:9200 admin@opennms> config:update ctrl+d exit • less ${OPENNMS_HOME}/etc/org.opennms.features.flows.persistence.elastic.cfg elasticUrl=http:// elasticsearch-server-ip :9200 elasticIndexStrategy=daily • systemctl restart opennms
  22. 22. • ${OPENNMS_HOME}/bin/runjava –s • ${OPENNMS_HOME}/bin/install –dis • systemctl start opennms # http://<ip-or-fqdn-of-your-server>:8980/opennms ( ID/PW : admin/admin )
  23. 23. • yum -y install fontconfig freetype* urw-fonts • wget https://dl.grafana.com/oss/release/grafana-6.4.3-1.x86_64.rpm • sudo yum localinstall grafana-6.4.3-1.x86_64.rpm #opennms-helm plugin 설치 • grafana-cli plugins install opennms-helm-app • systemctl start grafana-server.service #https://Grafana-server-ip:3000 접속 ( ID/PW : admin/admin )
  24. 24. #opennms-helm enable #Datasource opennms flows / elasticsearch setting
  25. 25. 출처 : https://blog.sflow.com/2013/08/restflow.html

×