SlideShare a Scribd company logo
Scalable 5-Tuple Packet
Classification in Overlay Network
Muhammad Arif
September 13, 2016
Agenda
• Introduction
• Background
• Problem Statement
• Platforms Overview
• Development
• 5-Tuple LCAF & Mapping System
• Rule Aggregation
• Flow Classification
• Result & Analysis
• Conclusion and Future Works
Introduction
3
Traditional Networking
Destination-based
Forwarding
No global view, multiple
control plane
Introduction
4
• Problems: Number of rules increases and rule size
increases
• processing latency increases
• storage space requirement also increases
OpenFlow
Up to 38 packet header
fields
(SDN) Has global view,
centralized control plane
Problem Statement
• Trade-off between flexibility vs performance: 5-tuple
5
• Objectives:
• Propose rule aggregation algorithm to minimize
number of entries stored  reduce the storage space
requirement
• Propose flow classification method to minimize the
lookup time  reduce processing latency
Flexibility Performance
SDN TN5-Tuple
Related Works
• Packet Classification: Well-known topic for research
• Decision-tree based:
• Well-known approach, such as in HiCut and HyperCut
• Cons: Tree size grows significantly due to finer TE
• TCAM-based:
• Fast lookup process
• Cons: high implementation cost
• Tuple Space Search (TSS):
• Very flexible
• Cons: high processing latency for high number of rules
• Main Idea: MC-SBC
• Modified TSS
• Send rules and packets to specific smaller lookup tables
6
Platform Overview
7
Open Overlay Router (OOR)
• Overlay network is one of the well-known implementation
of SDN
• OOR: Open-source platform to create programmable
overlay network
8
Vector Packet Processing (VPP)
• High performance packet processing platform and runs
entirely in the userspace
• Process multiple packets in a time by building superframe
• Packets are processed through Packet Processing Graph
9
VPP Classifier
• Using mask-and-match model
• Two main parts: Tables and Sessions
10
Developments
11
5-Tuple LCAF Implementation
2-Tuple LCAF (Current)
5-Tuple LCAF
Based on RFC Draft:
LISP support for Multi-Tuple EIDs
12
5-Tuple Mapping System
• Goals:
• Faster Lookup Times
• Reduce amount of Control Message
• Three Layers of Information Bases:
• NIB: Whole TE Rules
• RIB: Subset of NIB
• FIB: Ready-to-use Info
13
Rule Aggregation and Flow Classification
• Developed based on VPP Classification method
• Basic Ideas:
• Splitting rules into each header fields
• Limiting wildcard positions during aggregation
• Using Multiple Tables for classification
15
Rule Aggregation
• VPP classifier does not accept variable-sized header
(wildcards) mask
• Goals:
• Minimize the number of entries
• Give flexibility to use wildcards (predicted)
• Predicted vs Free Wildcard Positions (4 bits rules)
Predicted Free Wildcard Positions
1111 1111 0111 1010 0010
0111 1110 1100 0101 0100
0011 1101 1001 0110 1000
0001 1011 0011 0001
16
Rule Aggregation (Cont.)
• Inserted rules (with wildcards) are broken down into
multiple entries 𝐸𝑗
𝑖
to remove the wildcards
• The entries are converted into binary and split into each
packet header fields 𝑒 𝑘,𝑗
𝑖
and inserted to 𝑅𝑖 matrix
• Matrix 𝑅𝑖 is inserted into the the rule aggregation algorithm
and results in aggregated rules matrix 𝑅𝑖
′
17
Rule Aggregation (Cont.)
18
Rule Aggregation (Cont.)
Fields Value Binary* Size
SRC Addr 172.205.10.1 {0,0,0,1} 32 bits
DST Addr 245.206.13.2 {0,0,1,0} 32 bits
SRC Port 80 {0,0,0,0} 16 bits
DST Port 20-24 {0,1,0,0}
{0,1,0,1}
{0,1,1,0}
{0,1,1,1}
{1,0,0,0}
16 bits
Protocol TCP (6) {0,1,1,0} 8 bits
20
Flow Classification
• Goal: reduce the lookup times
• Similar to Tuple Space Search method, but based on
specific bit position
• Process: Offline and Online Stages
• Pre-filtering: select one or more bit positions to split the
rules into smaller multiple lookup tables as even as
possible
21
Flow Classification (Cont.)
• Bit positions selected for the pre-filtering  Effective Bit
Position (EBP)
• To determine EBP:
• Diversity Index: to find most even distribution of zeros
and ones
• Independence Index (for >1 EBP): to find bit positions
that draws good distinction between entries
22
Flow Classification (Cont.)
• Best case scenario: High 𝐻 𝑞 and 𝐼𝑛𝑑 𝑞
• EBP has to be updated periodically, especially if there’s
many rules addition
• In the beginning, when there’s no rules (day 0): select EBP
at random
23
Flow Classification (Cont.)
Entries SRC Addr DST Addr SRC Port DST Port Protocol Action
𝐸1 00011 0001* ***** 101** 00110 action1
𝐸2 00011 00100 ***** 101** 00110 action1
𝐸3 00101 00010 ***** 10110 10001 action2
𝐸4 00110 00010 ***** 10110 10001 action2
𝐸5 00111 0000* ***** 101** ***** action3
Offline Stage
24
Flow Classification (Cont.)
Entries SRC Addr DST Addr SRC Port DST Port Protocol Action
𝐸1 00011 0001* ***** 101** 00110 action1
𝐸2 00011 00100 ***** 101** 00110 action1
𝐸3 00101 00010 ***** 10110 10001 action2
𝐸4 00110 00010 ***** 10110 10001 action2
𝐸5 00111 0000* ***** 101** ***** action3
Online Stage
25
Implementation in VPP
Offline Stage
26
Implementation in VPP (Cont.)
Online Stage
27
Results & Analysis
28
Simulation and Testing Setup
• Testbed using VMs to implement OOR and VPP
• Synthetic rules generator: Classbench
• Packet generation: hping3
29
Name Function
VM1 VPP and Classbench
VM2 & VM3 xTR and hping3
VM4 MS and hping3
Result: Rule Aggregation
• Number of stored entries can be reduced
• Minimum achievable aggregation: 41.4%
• Higher number of original rules lead to more aggregation
30
Original Aggregated %
1607 1419 94.5
8640 6573 76.0
31571 16357 51.8
74527 31321 42.02
84815 35367 41.7
Result: Flow Classification
• Lookup times can be reduced
• After 76 entries, the performance of proposed method
surpassed the basic method
• Maximum achievable reduction: 29.6%
31
Conclusions
32
Conclusions
• SDN provides flexibility and programmability, but it comes
with scalability problem
• Implementation of proposed 5-tuple rule aggregation and
flow classification are able to reduce:
• Number of entries stored  less storage space
requirement
• Lookup times  lower processing latency
• In the testing phase, we found:
• 58.6% savings in storage space requirement
• 29.6% reduction in processing latency
33
Future Works
• Extend the proposed rule aggregation
• To check for overlapping rules
• To find faster way for calculation
• Extend the flow classification method
• Calculate and Update EBP in real time
• Other method to determine EBP
• To check the effectiveness of the proposed methods in the
real and big network
34
Thank You!
35

More Related Content

What's hot

Stateful Distributed Stream Processing
Stateful Distributed Stream ProcessingStateful Distributed Stream Processing
Stateful Distributed Stream Processing
Gyula Fóra
 
Deep Dive into Apache Apex App Development
Deep Dive into Apache Apex App DevelopmentDeep Dive into Apache Apex App Development
Deep Dive into Apache Apex App Development
Apache Apex
 
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
 IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
Apache Apex
 
Ingesting Data from Kafka to JDBC with Transformation and Enrichment
Ingesting Data from Kafka to JDBC with Transformation and EnrichmentIngesting Data from Kafka to JDBC with Transformation and Enrichment
Ingesting Data from Kafka to JDBC with Transformation and Enrichment
Apache Apex
 
Apache flink
Apache flinkApache flink
Apache flink
Ahmed Nader
 
From Batch to Streaming with Apache Apex Dataworks Summit 2017
From Batch to Streaming with Apache Apex Dataworks Summit 2017From Batch to Streaming with Apache Apex Dataworks Summit 2017
From Batch to Streaming with Apache Apex Dataworks Summit 2017
Apache Apex
 
Introduction to Apache Apex
Introduction to Apache ApexIntroduction to Apache Apex
Introduction to Apache Apex
Apache Apex
 
Windowing in apex
Windowing in apexWindowing in apex
Windowing in apex
Yogi Devendra Vyavahare
 
Flink Apachecon Presentation
Flink Apachecon PresentationFlink Apachecon Presentation
Flink Apachecon Presentation
Gyula Fóra
 
Building your first aplication using Apache Apex
Building your first aplication using Apache ApexBuilding your first aplication using Apache Apex
Building your first aplication using Apache Apex
Yogi Devendra Vyavahare
 
OpenDaylight Openflow & OVSDB use cases ODL summit 2016
OpenDaylight Openflow & OVSDB use cases ODL summit 2016OpenDaylight Openflow & OVSDB use cases ODL summit 2016
OpenDaylight Openflow & OVSDB use cases ODL summit 2016
abhijit2511
 
Apache Apex as YARN Application
Apache Apex as YARN ApplicationApache Apex as YARN Application
Apache Apex as YARN Application
Chinmay Kolhatkar
 
HPCC Systems 6.0.0 Highlights
HPCC Systems 6.0.0 HighlightsHPCC Systems 6.0.0 Highlights
HPCC Systems 6.0.0 Highlights
HPCC Systems
 
Albert Bifet – Apache Samoa: Mining Big Data Streams with Apache Flink
Albert Bifet – Apache Samoa: Mining Big Data Streams with Apache FlinkAlbert Bifet – Apache Samoa: Mining Big Data Streams with Apache Flink
Albert Bifet – Apache Samoa: Mining Big Data Streams with Apache Flink
Flink Forward
 
Third Party DNS Operator to Registries/Registrars Protocol
Third Party DNS Operator to Registries/Registrars ProtocolThird Party DNS Operator to Registries/Registrars Protocol
Third Party DNS Operator to Registries/Registrars Protocol
APNIC
 
Fabian Hueske – Cascading on Flink
Fabian Hueske – Cascading on FlinkFabian Hueske – Cascading on Flink
Fabian Hueske – Cascading on Flink
Flink Forward
 
Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink
Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache FlinkSuneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink
Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink
Flink Forward
 
Ingestion file copy using apex
Ingestion   file copy using apexIngestion   file copy using apex
Ingestion file copy using apex
Apache Apex
 
Log ingestion kafka -- impala using apex
Log ingestion   kafka -- impala using apexLog ingestion   kafka -- impala using apex
Log ingestion kafka -- impala using apex
Apache Apex
 
Building Your First Apache Apex (Next Gen Big Data/Hadoop) Application
Building Your First Apache Apex (Next Gen Big Data/Hadoop) ApplicationBuilding Your First Apache Apex (Next Gen Big Data/Hadoop) Application
Building Your First Apache Apex (Next Gen Big Data/Hadoop) Application
Apache Apex
 

What's hot (20)

Stateful Distributed Stream Processing
Stateful Distributed Stream ProcessingStateful Distributed Stream Processing
Stateful Distributed Stream Processing
 
Deep Dive into Apache Apex App Development
Deep Dive into Apache Apex App DevelopmentDeep Dive into Apache Apex App Development
Deep Dive into Apache Apex App Development
 
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
 IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
 
Ingesting Data from Kafka to JDBC with Transformation and Enrichment
Ingesting Data from Kafka to JDBC with Transformation and EnrichmentIngesting Data from Kafka to JDBC with Transformation and Enrichment
Ingesting Data from Kafka to JDBC with Transformation and Enrichment
 
Apache flink
Apache flinkApache flink
Apache flink
 
From Batch to Streaming with Apache Apex Dataworks Summit 2017
From Batch to Streaming with Apache Apex Dataworks Summit 2017From Batch to Streaming with Apache Apex Dataworks Summit 2017
From Batch to Streaming with Apache Apex Dataworks Summit 2017
 
Introduction to Apache Apex
Introduction to Apache ApexIntroduction to Apache Apex
Introduction to Apache Apex
 
Windowing in apex
Windowing in apexWindowing in apex
Windowing in apex
 
Flink Apachecon Presentation
Flink Apachecon PresentationFlink Apachecon Presentation
Flink Apachecon Presentation
 
Building your first aplication using Apache Apex
Building your first aplication using Apache ApexBuilding your first aplication using Apache Apex
Building your first aplication using Apache Apex
 
OpenDaylight Openflow & OVSDB use cases ODL summit 2016
OpenDaylight Openflow & OVSDB use cases ODL summit 2016OpenDaylight Openflow & OVSDB use cases ODL summit 2016
OpenDaylight Openflow & OVSDB use cases ODL summit 2016
 
Apache Apex as YARN Application
Apache Apex as YARN ApplicationApache Apex as YARN Application
Apache Apex as YARN Application
 
HPCC Systems 6.0.0 Highlights
HPCC Systems 6.0.0 HighlightsHPCC Systems 6.0.0 Highlights
HPCC Systems 6.0.0 Highlights
 
Albert Bifet – Apache Samoa: Mining Big Data Streams with Apache Flink
Albert Bifet – Apache Samoa: Mining Big Data Streams with Apache FlinkAlbert Bifet – Apache Samoa: Mining Big Data Streams with Apache Flink
Albert Bifet – Apache Samoa: Mining Big Data Streams with Apache Flink
 
Third Party DNS Operator to Registries/Registrars Protocol
Third Party DNS Operator to Registries/Registrars ProtocolThird Party DNS Operator to Registries/Registrars Protocol
Third Party DNS Operator to Registries/Registrars Protocol
 
Fabian Hueske – Cascading on Flink
Fabian Hueske – Cascading on FlinkFabian Hueske – Cascading on Flink
Fabian Hueske – Cascading on Flink
 
Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink
Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache FlinkSuneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink
Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink
 
Ingestion file copy using apex
Ingestion   file copy using apexIngestion   file copy using apex
Ingestion file copy using apex
 
Log ingestion kafka -- impala using apex
Log ingestion   kafka -- impala using apexLog ingestion   kafka -- impala using apex
Log ingestion kafka -- impala using apex
 
Building Your First Apache Apex (Next Gen Big Data/Hadoop) Application
Building Your First Apache Apex (Next Gen Big Data/Hadoop) ApplicationBuilding Your First Apache Apex (Next Gen Big Data/Hadoop) Application
Building Your First Apache Apex (Next Gen Big Data/Hadoop) Application
 

Viewers also liked

Le web
Le webLe web
Le web
rami1979
 
Lapset ja nuoret_netissa_webinaari
Lapset ja nuoret_netissa_webinaariLapset ja nuoret_netissa_webinaari
Lapset ja nuoret_netissa_webinaariMikko Horila
 
Yemek
YemekYemek
Yemek
gazi999
 
Cocina china 1.www.lo kotorrents.com
Cocina china 1.www.lo kotorrents.comCocina china 1.www.lo kotorrents.com
Cocina china 1.www.lo kotorrents.comTerseum
 
Tutkiva oppiminen, Tre 8.11.2011, II Kerta, Digikuvaaminen, Picasa ja kuvaga...
Tutkiva oppiminen, Tre 8.11.2011, II Kerta,  Digikuvaaminen, Picasa ja kuvaga...Tutkiva oppiminen, Tre 8.11.2011, II Kerta,  Digikuvaaminen, Picasa ja kuvaga...
Tutkiva oppiminen, Tre 8.11.2011, II Kerta, Digikuvaaminen, Picasa ja kuvaga...Mikko Horila
 
La energía estudiante 12
La energía  estudiante 12La energía  estudiante 12
La energía estudiante 12
mayi12
 
Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...
Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...
Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...
Centers for Medicare & Medicaid Services (CMS)
 
Spiceworks Unplugged UK 1 December 2011
Spiceworks Unplugged UK 1 December 2011Spiceworks Unplugged UK 1 December 2011
Spiceworks Unplugged UK 1 December 2011
Auskosh
 
1111 On Wall Street Article
1111   On Wall Street Article1111   On Wall Street Article
1111 On Wall Street ArticlePeter Furth
 
Town and Gown Working Together: Four Women Make It Happen
Town and Gown Working Together: Four Women Make It HappenTown and Gown Working Together: Four Women Make It Happen
Town and Gown Working Together: Four Women Make It Happen
Kristen Smith
 
Peskin chap02
Peskin chap02Peskin chap02
Peskin chap02
Javier Rencoret
 
Webinar: State Innovation Models Round Two - How to Apply on Grants.gov
Webinar: State Innovation Models Round Two - How to Apply on Grants.govWebinar: State Innovation Models Round Two - How to Apply on Grants.gov
Webinar: State Innovation Models Round Two - How to Apply on Grants.gov
Centers for Medicare & Medicaid Services (CMS)
 
Vmware any-cloud
Vmware any-cloudVmware any-cloud
Vmware any-cloud
Andrey Tokarchuk
 

Viewers also liked (20)

Tipos de publicidad
Tipos de publicidadTipos de publicidad
Tipos de publicidad
 
Le web
Le webLe web
Le web
 
Lapset ja nuoret_netissa_webinaari
Lapset ja nuoret_netissa_webinaariLapset ja nuoret_netissa_webinaari
Lapset ja nuoret_netissa_webinaari
 
Yemek
YemekYemek
Yemek
 
Calendarioespanol
CalendarioespanolCalendarioespanol
Calendarioespanol
 
Credit risk management3
Credit risk management3Credit risk management3
Credit risk management3
 
Cocina china 1.www.lo kotorrents.com
Cocina china 1.www.lo kotorrents.comCocina china 1.www.lo kotorrents.com
Cocina china 1.www.lo kotorrents.com
 
Seman 3 ciclo parte b ii
Seman 3 ciclo parte b iiSeman 3 ciclo parte b ii
Seman 3 ciclo parte b ii
 
Presentation
Presentation Presentation
Presentation
 
S ta r data
S ta r dataS ta r data
S ta r data
 
Tutkiva oppiminen, Tre 8.11.2011, II Kerta, Digikuvaaminen, Picasa ja kuvaga...
Tutkiva oppiminen, Tre 8.11.2011, II Kerta,  Digikuvaaminen, Picasa ja kuvaga...Tutkiva oppiminen, Tre 8.11.2011, II Kerta,  Digikuvaaminen, Picasa ja kuvaga...
Tutkiva oppiminen, Tre 8.11.2011, II Kerta, Digikuvaaminen, Picasa ja kuvaga...
 
La energía estudiante 12
La energía  estudiante 12La energía  estudiante 12
La energía estudiante 12
 
Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...
Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...
Webinar: State Innovation Models Initiative Round Two - Model Design Proposal...
 
Spiceworks Unplugged UK 1 December 2011
Spiceworks Unplugged UK 1 December 2011Spiceworks Unplugged UK 1 December 2011
Spiceworks Unplugged UK 1 December 2011
 
1111 On Wall Street Article
1111   On Wall Street Article1111   On Wall Street Article
1111 On Wall Street Article
 
Town and Gown Working Together: Four Women Make It Happen
Town and Gown Working Together: Four Women Make It HappenTown and Gown Working Together: Four Women Make It Happen
Town and Gown Working Together: Four Women Make It Happen
 
Peskin chap02
Peskin chap02Peskin chap02
Peskin chap02
 
Webinar: State Innovation Models Round Two - How to Apply on Grants.gov
Webinar: State Innovation Models Round Two - How to Apply on Grants.govWebinar: State Innovation Models Round Two - How to Apply on Grants.gov
Webinar: State Innovation Models Round Two - How to Apply on Grants.gov
 
Libya
LibyaLibya
Libya
 
Vmware any-cloud
Vmware any-cloudVmware any-cloud
Vmware any-cloud
 

Similar to Preso-v0.1

PelotonDB - A self-driving database for hybrid workloads
PelotonDB - A self-driving database for hybrid workloadsPelotonDB - A self-driving database for hybrid workloads
PelotonDB - A self-driving database for hybrid workloads
宇 傅
 
Online index rebuild automation
Online index rebuild automationOnline index rebuild automation
Online index rebuild automation
Carlos Sierra
 
Parallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC SystemsParallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC Systems
HPCC Systems
 
DevoFlow - Scaling Flow Management for High-Performance Networks
DevoFlow - Scaling Flow Management for High-Performance NetworksDevoFlow - Scaling Flow Management for High-Performance Networks
DevoFlow - Scaling Flow Management for High-Performance Networks
Jason TC HOU (侯宗成)
 
Benchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA ConstraintsBenchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA Constraints
Nicolas Poggi
 
12 processor structure and function
12 processor structure and function12 processor structure and function
12 processor structure and function
dilip kumar
 
POLARDB for MySQL - Parallel Query
POLARDB for MySQL - Parallel QueryPOLARDB for MySQL - Parallel Query
POLARDB for MySQL - Parallel Query
oysteing
 
Hhm 3470 mq v8 and more recent new things for z os
Hhm 3470 mq v8 and more recent new things for z osHhm 3470 mq v8 and more recent new things for z os
Hhm 3470 mq v8 and more recent new things for z os
Pete Siddall
 
Reduced instruction set computers
Reduced instruction set computersReduced instruction set computers
Reduced instruction set computers
Syed Zaid Irshad
 
IT209 Cpu Structure Report
IT209 Cpu Structure ReportIT209 Cpu Structure Report
IT209 Cpu Structure ReportBis Aquino
 
Tutorial: Network State Awareness Troubleshooting
Tutorial: Network State Awareness TroubleshootingTutorial: Network State Awareness Troubleshooting
Tutorial: Network State Awareness Troubleshooting
APNIC
 
Network_Layer_and_Internet_Protocols_IPv.pptx
Network_Layer_and_Internet_Protocols_IPv.pptxNetwork_Layer_and_Internet_Protocols_IPv.pptx
Network_Layer_and_Internet_Protocols_IPv.pptx
20CE112YASHPATEL
 
Metrics Are Not Enough: Monitoring Apache Kafka and Streaming Applications
Metrics Are Not Enough: Monitoring Apache Kafka and Streaming ApplicationsMetrics Are Not Enough: Monitoring Apache Kafka and Streaming Applications
Metrics Are Not Enough: Monitoring Apache Kafka and Streaming Applications
confluent
 
Improved Search With Lucene 4.0 - NOVA Lucene/Solr Meetup
Improved Search With Lucene 4.0 - NOVA Lucene/Solr MeetupImproved Search With Lucene 4.0 - NOVA Lucene/Solr Meetup
Improved Search With Lucene 4.0 - NOVA Lucene/Solr Meetuprcmuir
 
Best Practices – Extreme Performance with Data Warehousing on Oracle Databa...
Best Practices –  Extreme Performance with Data Warehousing  on Oracle Databa...Best Practices –  Extreme Performance with Data Warehousing  on Oracle Databa...
Best Practices – Extreme Performance with Data Warehousing on Oracle Databa...
Edgar Alejandro Villegas
 
400-101 CCIE Routing and Switching IT Certification
400-101 CCIE Routing and Switching IT Certification400-101 CCIE Routing and Switching IT Certification
400-101 CCIE Routing and Switching IT Certification
wrouthae
 
Data monstersrealtimeetl new
Data monstersrealtimeetl newData monstersrealtimeetl new
Data monstersrealtimeetl new
GreenM
 
Chapter 14 software testing techniques
Chapter 14 software testing techniquesChapter 14 software testing techniques
Chapter 14 software testing techniques
SHREEHARI WADAWADAGI
 

Similar to Preso-v0.1 (20)

PelotonDB - A self-driving database for hybrid workloads
PelotonDB - A self-driving database for hybrid workloadsPelotonDB - A self-driving database for hybrid workloads
PelotonDB - A self-driving database for hybrid workloads
 
Online index rebuild automation
Online index rebuild automationOnline index rebuild automation
Online index rebuild automation
 
Parallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC SystemsParallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC Systems
 
DevoFlow - Scaling Flow Management for High-Performance Networks
DevoFlow - Scaling Flow Management for High-Performance NetworksDevoFlow - Scaling Flow Management for High-Performance Networks
DevoFlow - Scaling Flow Management for High-Performance Networks
 
Benchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA ConstraintsBenchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA Constraints
 
12 processor structure and function
12 processor structure and function12 processor structure and function
12 processor structure and function
 
POLARDB for MySQL - Parallel Query
POLARDB for MySQL - Parallel QueryPOLARDB for MySQL - Parallel Query
POLARDB for MySQL - Parallel Query
 
Hhm 3470 mq v8 and more recent new things for z os
Hhm 3470 mq v8 and more recent new things for z osHhm 3470 mq v8 and more recent new things for z os
Hhm 3470 mq v8 and more recent new things for z os
 
Reduced instruction set computers
Reduced instruction set computersReduced instruction set computers
Reduced instruction set computers
 
IT209 Cpu Structure Report
IT209 Cpu Structure ReportIT209 Cpu Structure Report
IT209 Cpu Structure Report
 
Tutorial: Network State Awareness Troubleshooting
Tutorial: Network State Awareness TroubleshootingTutorial: Network State Awareness Troubleshooting
Tutorial: Network State Awareness Troubleshooting
 
Network_Layer_and_Internet_Protocols_IPv.pptx
Network_Layer_and_Internet_Protocols_IPv.pptxNetwork_Layer_and_Internet_Protocols_IPv.pptx
Network_Layer_and_Internet_Protocols_IPv.pptx
 
Metrics Are Not Enough: Monitoring Apache Kafka and Streaming Applications
Metrics Are Not Enough: Monitoring Apache Kafka and Streaming ApplicationsMetrics Are Not Enough: Monitoring Apache Kafka and Streaming Applications
Metrics Are Not Enough: Monitoring Apache Kafka and Streaming Applications
 
Improved Search With Lucene 4.0 - NOVA Lucene/Solr Meetup
Improved Search With Lucene 4.0 - NOVA Lucene/Solr MeetupImproved Search With Lucene 4.0 - NOVA Lucene/Solr Meetup
Improved Search With Lucene 4.0 - NOVA Lucene/Solr Meetup
 
Best Practices – Extreme Performance with Data Warehousing on Oracle Databa...
Best Practices –  Extreme Performance with Data Warehousing  on Oracle Databa...Best Practices –  Extreme Performance with Data Warehousing  on Oracle Databa...
Best Practices – Extreme Performance with Data Warehousing on Oracle Databa...
 
400-101 CCIE Routing and Switching IT Certification
400-101 CCIE Routing and Switching IT Certification400-101 CCIE Routing and Switching IT Certification
400-101 CCIE Routing and Switching IT Certification
 
Data monstersrealtimeetl new
Data monstersrealtimeetl newData monstersrealtimeetl new
Data monstersrealtimeetl new
 
Callgraph analysis
Callgraph analysisCallgraph analysis
Callgraph analysis
 
PraveenBOUT++
PraveenBOUT++PraveenBOUT++
PraveenBOUT++
 
Chapter 14 software testing techniques
Chapter 14 software testing techniquesChapter 14 software testing techniques
Chapter 14 software testing techniques
 

Preso-v0.1

  • 1. Scalable 5-Tuple Packet Classification in Overlay Network Muhammad Arif September 13, 2016
  • 2. Agenda • Introduction • Background • Problem Statement • Platforms Overview • Development • 5-Tuple LCAF & Mapping System • Rule Aggregation • Flow Classification • Result & Analysis • Conclusion and Future Works
  • 4. Traditional Networking Destination-based Forwarding No global view, multiple control plane Introduction 4 • Problems: Number of rules increases and rule size increases • processing latency increases • storage space requirement also increases OpenFlow Up to 38 packet header fields (SDN) Has global view, centralized control plane
  • 5. Problem Statement • Trade-off between flexibility vs performance: 5-tuple 5 • Objectives: • Propose rule aggregation algorithm to minimize number of entries stored  reduce the storage space requirement • Propose flow classification method to minimize the lookup time  reduce processing latency Flexibility Performance SDN TN5-Tuple
  • 6. Related Works • Packet Classification: Well-known topic for research • Decision-tree based: • Well-known approach, such as in HiCut and HyperCut • Cons: Tree size grows significantly due to finer TE • TCAM-based: • Fast lookup process • Cons: high implementation cost • Tuple Space Search (TSS): • Very flexible • Cons: high processing latency for high number of rules • Main Idea: MC-SBC • Modified TSS • Send rules and packets to specific smaller lookup tables 6
  • 8. Open Overlay Router (OOR) • Overlay network is one of the well-known implementation of SDN • OOR: Open-source platform to create programmable overlay network 8
  • 9. Vector Packet Processing (VPP) • High performance packet processing platform and runs entirely in the userspace • Process multiple packets in a time by building superframe • Packets are processed through Packet Processing Graph 9
  • 10. VPP Classifier • Using mask-and-match model • Two main parts: Tables and Sessions 10
  • 12. 5-Tuple LCAF Implementation 2-Tuple LCAF (Current) 5-Tuple LCAF Based on RFC Draft: LISP support for Multi-Tuple EIDs 12
  • 13. 5-Tuple Mapping System • Goals: • Faster Lookup Times • Reduce amount of Control Message • Three Layers of Information Bases: • NIB: Whole TE Rules • RIB: Subset of NIB • FIB: Ready-to-use Info 13
  • 14. Rule Aggregation and Flow Classification • Developed based on VPP Classification method • Basic Ideas: • Splitting rules into each header fields • Limiting wildcard positions during aggregation • Using Multiple Tables for classification 15
  • 15. Rule Aggregation • VPP classifier does not accept variable-sized header (wildcards) mask • Goals: • Minimize the number of entries • Give flexibility to use wildcards (predicted) • Predicted vs Free Wildcard Positions (4 bits rules) Predicted Free Wildcard Positions 1111 1111 0111 1010 0010 0111 1110 1100 0101 0100 0011 1101 1001 0110 1000 0001 1011 0011 0001 16
  • 16. Rule Aggregation (Cont.) • Inserted rules (with wildcards) are broken down into multiple entries 𝐸𝑗 𝑖 to remove the wildcards • The entries are converted into binary and split into each packet header fields 𝑒 𝑘,𝑗 𝑖 and inserted to 𝑅𝑖 matrix • Matrix 𝑅𝑖 is inserted into the the rule aggregation algorithm and results in aggregated rules matrix 𝑅𝑖 ′ 17
  • 18. Rule Aggregation (Cont.) Fields Value Binary* Size SRC Addr 172.205.10.1 {0,0,0,1} 32 bits DST Addr 245.206.13.2 {0,0,1,0} 32 bits SRC Port 80 {0,0,0,0} 16 bits DST Port 20-24 {0,1,0,0} {0,1,0,1} {0,1,1,0} {0,1,1,1} {1,0,0,0} 16 bits Protocol TCP (6) {0,1,1,0} 8 bits 20
  • 19. Flow Classification • Goal: reduce the lookup times • Similar to Tuple Space Search method, but based on specific bit position • Process: Offline and Online Stages • Pre-filtering: select one or more bit positions to split the rules into smaller multiple lookup tables as even as possible 21
  • 20. Flow Classification (Cont.) • Bit positions selected for the pre-filtering  Effective Bit Position (EBP) • To determine EBP: • Diversity Index: to find most even distribution of zeros and ones • Independence Index (for >1 EBP): to find bit positions that draws good distinction between entries 22
  • 21. Flow Classification (Cont.) • Best case scenario: High 𝐻 𝑞 and 𝐼𝑛𝑑 𝑞 • EBP has to be updated periodically, especially if there’s many rules addition • In the beginning, when there’s no rules (day 0): select EBP at random 23
  • 22. Flow Classification (Cont.) Entries SRC Addr DST Addr SRC Port DST Port Protocol Action 𝐸1 00011 0001* ***** 101** 00110 action1 𝐸2 00011 00100 ***** 101** 00110 action1 𝐸3 00101 00010 ***** 10110 10001 action2 𝐸4 00110 00010 ***** 10110 10001 action2 𝐸5 00111 0000* ***** 101** ***** action3 Offline Stage 24
  • 23. Flow Classification (Cont.) Entries SRC Addr DST Addr SRC Port DST Port Protocol Action 𝐸1 00011 0001* ***** 101** 00110 action1 𝐸2 00011 00100 ***** 101** 00110 action1 𝐸3 00101 00010 ***** 10110 10001 action2 𝐸4 00110 00010 ***** 10110 10001 action2 𝐸5 00111 0000* ***** 101** ***** action3 Online Stage 25
  • 25. Implementation in VPP (Cont.) Online Stage 27
  • 27. Simulation and Testing Setup • Testbed using VMs to implement OOR and VPP • Synthetic rules generator: Classbench • Packet generation: hping3 29 Name Function VM1 VPP and Classbench VM2 & VM3 xTR and hping3 VM4 MS and hping3
  • 28. Result: Rule Aggregation • Number of stored entries can be reduced • Minimum achievable aggregation: 41.4% • Higher number of original rules lead to more aggregation 30 Original Aggregated % 1607 1419 94.5 8640 6573 76.0 31571 16357 51.8 74527 31321 42.02 84815 35367 41.7
  • 29. Result: Flow Classification • Lookup times can be reduced • After 76 entries, the performance of proposed method surpassed the basic method • Maximum achievable reduction: 29.6% 31
  • 31. Conclusions • SDN provides flexibility and programmability, but it comes with scalability problem • Implementation of proposed 5-tuple rule aggregation and flow classification are able to reduce: • Number of entries stored  less storage space requirement • Lookup times  lower processing latency • In the testing phase, we found: • 58.6% savings in storage space requirement • 29.6% reduction in processing latency 33
  • 32. Future Works • Extend the proposed rule aggregation • To check for overlapping rules • To find faster way for calculation • Extend the flow classification method • Calculate and Update EBP in real time • Other method to determine EBP • To check the effectiveness of the proposed methods in the real and big network 34

Editor's Notes

  1. Hello Good Morning Thanks for coming Today I will present about my thesis work with title
  2. Kasi panah
  3. Kasi gambar 5-tuple
  4. TCAM: Specialized memory for fast lookup MC-SBC: multidimensional-cutting via selective bit-concatenation
  5. Salah iP
  6. Sikat semua
  7. Sikat semua