SlideShare a Scribd company logo
1 of 13
Download to read offline
Intrusion Detection with
Segmented Federated Learning
for Large-Scale Multiple LANs
Yuwei Sun, Hideya Ochiai, Hiroshi Esaki
The University of Tokyo
Agenda
• Motivation: Isolated anomaly detection in a network
• Proposal: Segmented federated learning for dynamically
collaborative learning within diverse nodes
• Evaluation: How it worked for anomaly detection with the
enhanced federated learning in stochastic network environments
• Conclusion
Remote Server
Training parameters of models
Server
in
UTokyo
Participant A Participant B Participant C Participant D
Motivation
Problems:
• Training data is too diverse for a traditional federated learning
• Different data sizes of participants, features from the ones with a smaller dataset
might be erased by the larger ones
• When participants keep increasing, the waiting time for updating the global
model goes to overflow the limit time of a round
Server
in
UTokyo
Participant A Participant B Participant C Participant D
Server
in
UTokyo
Proposal: Segmented Federated Learning
Features:
• Multiple global models
• Limited participants’ updating
• Regular performance evaluation
for a structure adjustment
Methods: Segmented federated learning (1/4)
Global parameters updating
Perfomance evaluation
Methods: Segmented federated learning (2/4)
• 𝑝t-1: former global parameters
• 𝑝i: parameters from participants who
conduct training
• 𝑞i: the other global models’ parameters
• 𝑑i: distance between each participant’s
accuracy and the average accuracy
Global parameters updating Performance evaluation
Methods: Segmented federated learning (3/4)
• Two convolution layers, each of which is followed
by a maxpooling layer and two fully-connected
layers
• Feature maps of local dataset as the input, and
the result from an expert-knowledge based
labeling as the output
• Learning rate: 0.00001 Batch size: 50 Epoch: 1
Design of a model protocol for parameters sharing
Methods: Segmented federated learning (4/4)
Broadcast data in a LAN and any communication
directly sent to the monitor device
Event
Generator
Host 1 Host m
….
Data
Collector
Capture
network traffic
Visulization and
classification with DCNN
nmap execution
e.g., ARP scan, TCP/UDP port scan, …
Local Area Network
Malici
ous
user
Two months’ network traffic data of 20 participants from
the LAN-Security Monitoring Project
How it worked for anomaly detection in stochastic network environments
Evaluation
Knowledge-based labeling
• Malicious SMB: Detection of any SYN445 to the monitor device
• TCP SYN Flooding: TCP SYN from the same IP with a frequency of more than
three times
• Malicious UDP unicast: Detection of any UDP unicast to the monitor device
(except the communications of NTP with a source port of 123 and DNS with
a source port of 53)
Validation accuracy (Malicious SMB)
rounds
Segmentation of participants’ local neural networks
Se
rve
r
in
UT
ok
yo
Se
rve
r
in
UT
ok
yo
Se
rve
r
in
UT
ok
yo
Se
rve
r
in
UT
ok
yo
Se
rve
r
in
UT
ok
yo
(Malicious SMB)
Conclusion
• Isolated anomaly detection in a network
• SFL is proposed to solve the problem of various adaptivity of participants to
the single global model in a FL scheme
• Regular performance evaluation is conducted automatically for transforming
the structure of the system
• Insights on intelligent networking and anomaly detection using distributed
neural networks, for anomaly information sharing among various networks

More Related Content

Similar to Segmented Federated Learning

Barsamian alexander-identifying-network-users
Barsamian alexander-identifying-network-usersBarsamian alexander-identifying-network-users
Barsamian alexander-identifying-network-usersProQSys
 
Ds objects and models
Ds objects and modelsDs objects and models
Ds objects and modelsMayank Jain
 
network mining and representation learning
network mining and representation learningnetwork mining and representation learning
network mining and representation learningsun peiyuan
 
Agents and P2P Networks
Agents and P2P NetworksAgents and P2P Networks
Agents and P2P NetworksJames Salter
 
Data Communication & Computer Networks
Data Communication & Computer NetworksData Communication & Computer Networks
Data Communication & Computer NetworksSreedhar Chowdam
 
Free NetFlow Analyzer training - Getting the initial settings right
Free NetFlow Analyzer training - Getting the initial settings rightFree NetFlow Analyzer training - Getting the initial settings right
Free NetFlow Analyzer training - Getting the initial settings rightManageEngine, Zoho Corporation
 
Radio Frequency Identification Technology In IOT
Radio Frequency Identification Technology In IOTRadio Frequency Identification Technology In IOT
Radio Frequency Identification Technology In IOTNikhil Patankar
 
SDN and Named Data Networking Security
SDN and Named Data Networking SecuritySDN and Named Data Networking Security
SDN and Named Data Networking Securitywolverinetyagi
 
From ensembles to computer networks
From ensembles to computer networksFrom ensembles to computer networks
From ensembles to computer networksCSIRO
 
DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...
DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...
DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...Deltares
 
Programmable Network Devices
Programmable Network DevicesProgrammable Network Devices
Programmable Network DevicesTal Lavian Ph.D.
 
Practical Considerations for Deploying a Java Active Networking Platform
Practical Considerations for Deploying a Java Active Networking PlatformPractical Considerations for Deploying a Java Active Networking Platform
Practical Considerations for Deploying a Java Active Networking PlatformTal Lavian Ph.D.
 
Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...
Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...
Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...YuweiSun5
 

Similar to Segmented Federated Learning (20)

Fuzzy Rough Set Feature Selection to Enhance Phishing Attack Detection
Fuzzy Rough Set Feature Selection to Enhance Phishing Attack Detection Fuzzy Rough Set Feature Selection to Enhance Phishing Attack Detection
Fuzzy Rough Set Feature Selection to Enhance Phishing Attack Detection
 
Barsamian alexander-identifying-network-users
Barsamian alexander-identifying-network-usersBarsamian alexander-identifying-network-users
Barsamian alexander-identifying-network-users
 
Ds objects and models
Ds objects and modelsDs objects and models
Ds objects and models
 
network mining and representation learning
network mining and representation learningnetwork mining and representation learning
network mining and representation learning
 
Agents and P2P Networks
Agents and P2P NetworksAgents and P2P Networks
Agents and P2P Networks
 
Vapt pci dss methodology ppt v1.0
Vapt pci dss methodology ppt v1.0Vapt pci dss methodology ppt v1.0
Vapt pci dss methodology ppt v1.0
 
OC_Offline_Africa
OC_Offline_AfricaOC_Offline_Africa
OC_Offline_Africa
 
Data Communication & Computer Networks
Data Communication & Computer NetworksData Communication & Computer Networks
Data Communication & Computer Networks
 
Free NetFlow Analyzer training - Getting the initial settings right
Free NetFlow Analyzer training - Getting the initial settings rightFree NetFlow Analyzer training - Getting the initial settings right
Free NetFlow Analyzer training - Getting the initial settings right
 
Решения WANDL и NorthStar для операторов
Решения WANDL и NorthStar для операторовРешения WANDL и NorthStar для операторов
Решения WANDL и NorthStar для операторов
 
CN Unit-1 PPT.pptx
CN Unit-1 PPT.pptxCN Unit-1 PPT.pptx
CN Unit-1 PPT.pptx
 
Mini Project- Implementation & Evaluation Of Wireless La Ns
Mini Project- Implementation & Evaluation Of Wireless La NsMini Project- Implementation & Evaluation Of Wireless La Ns
Mini Project- Implementation & Evaluation Of Wireless La Ns
 
Radio Frequency Identification Technology In IOT
Radio Frequency Identification Technology In IOTRadio Frequency Identification Technology In IOT
Radio Frequency Identification Technology In IOT
 
SDN and Named Data Networking Security
SDN and Named Data Networking SecuritySDN and Named Data Networking Security
SDN and Named Data Networking Security
 
From ensembles to computer networks
From ensembles to computer networksFrom ensembles to computer networks
From ensembles to computer networks
 
DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...
DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...
DSD-INT 2014 - OpenMI Symposium - Federated modelling of Critical Infrastruct...
 
Programmable Network Devices
Programmable Network DevicesProgrammable Network Devices
Programmable Network Devices
 
Practical Considerations for Deploying a Java Active Networking Platform
Practical Considerations for Deploying a Java Active Networking PlatformPractical Considerations for Deploying a Java Active Networking Platform
Practical Considerations for Deploying a Java Active Networking Platform
 
Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...
Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...
Blockchain-Based Federated Learning Against End-Point Adversarial Data Corrup...
 
Network cost services
Network cost servicesNetwork cost services
Network cost services
 

Recently uploaded

A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 

Recently uploaded (20)

A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 

Segmented Federated Learning

  • 1. Intrusion Detection with Segmented Federated Learning for Large-Scale Multiple LANs Yuwei Sun, Hideya Ochiai, Hiroshi Esaki The University of Tokyo
  • 2. Agenda • Motivation: Isolated anomaly detection in a network • Proposal: Segmented federated learning for dynamically collaborative learning within diverse nodes • Evaluation: How it worked for anomaly detection with the enhanced federated learning in stochastic network environments • Conclusion
  • 3. Remote Server Training parameters of models Server in UTokyo Participant A Participant B Participant C Participant D Motivation Problems: • Training data is too diverse for a traditional federated learning • Different data sizes of participants, features from the ones with a smaller dataset might be erased by the larger ones • When participants keep increasing, the waiting time for updating the global model goes to overflow the limit time of a round
  • 4. Server in UTokyo Participant A Participant B Participant C Participant D Server in UTokyo Proposal: Segmented Federated Learning
  • 5. Features: • Multiple global models • Limited participants’ updating • Regular performance evaluation for a structure adjustment Methods: Segmented federated learning (1/4)
  • 6. Global parameters updating Perfomance evaluation Methods: Segmented federated learning (2/4)
  • 7. • 𝑝t-1: former global parameters • 𝑝i: parameters from participants who conduct training • 𝑞i: the other global models’ parameters • 𝑑i: distance between each participant’s accuracy and the average accuracy Global parameters updating Performance evaluation Methods: Segmented federated learning (3/4)
  • 8. • Two convolution layers, each of which is followed by a maxpooling layer and two fully-connected layers • Feature maps of local dataset as the input, and the result from an expert-knowledge based labeling as the output • Learning rate: 0.00001 Batch size: 50 Epoch: 1 Design of a model protocol for parameters sharing Methods: Segmented federated learning (4/4)
  • 9. Broadcast data in a LAN and any communication directly sent to the monitor device Event Generator Host 1 Host m …. Data Collector Capture network traffic Visulization and classification with DCNN nmap execution e.g., ARP scan, TCP/UDP port scan, … Local Area Network Malici ous user Two months’ network traffic data of 20 participants from the LAN-Security Monitoring Project How it worked for anomaly detection in stochastic network environments Evaluation
  • 10. Knowledge-based labeling • Malicious SMB: Detection of any SYN445 to the monitor device • TCP SYN Flooding: TCP SYN from the same IP with a frequency of more than three times • Malicious UDP unicast: Detection of any UDP unicast to the monitor device (except the communications of NTP with a source port of 123 and DNS with a source port of 53)
  • 12. Segmentation of participants’ local neural networks Se rve r in UT ok yo Se rve r in UT ok yo Se rve r in UT ok yo Se rve r in UT ok yo Se rve r in UT ok yo (Malicious SMB)
  • 13. Conclusion • Isolated anomaly detection in a network • SFL is proposed to solve the problem of various adaptivity of participants to the single global model in a FL scheme • Regular performance evaluation is conducted automatically for transforming the structure of the system • Insights on intelligent networking and anomaly detection using distributed neural networks, for anomaly information sharing among various networks