1. A user mode implementation of
filtering rule management
plane on virtualized networking
environment
Network Security Research Institute, National Institute of Information and
Communications Technology
Ruo Ando
情報通信研究機構 ネットワークセキュリティ研究所
安藤類央
CSEC66 2014-07-04
第66回CSEC・第10回SPT合同研究発表会
セッションC-5(13:25 - 14:40) http://starbed.nict.go.jp/

2. Abstract: Towards alternative access control model
仮想ネットワーク環境での新しいアクセス制御モデル
[A] The emergence of network virtualization and related technologies such as SDN and Cloud computing make us
face the new challenge of new alternative access control model.
(1) ネットワーク仮想化（抽象化）とその周辺技術（Software Defined Network, クラウドコンピューティング）の発達
により、アクセス制御技術にも革新と修正が求められている。
(2) アクセス制御技術の近年の傾向としてはカーネル空間での実装と一元化 (Centrazlization) が問題領域とネット
ワークの大規模に伴い様々な問題が生じている。
[B] Particularly, besides flexibility, fine-grained traffic engineering functionality for coping with scalability and
diversified networks is required for the deployments of SDN and Cloud Computing.
[C] Our architecture leverages NoSQL data store for handling a large scale of filtering rules. By adopting NoSQL,
we can achieve scalability, availability and tolerance to network partition. Besides, separating management plane
and control plane, we can achieve responsiveness and strong consistency at the same time.
(3) 提案システムには大規模なフィルタリングルールの処理のためにNoSQLデータストアを用いて、スケーラビリ
ティを確保し、従来のSDNのアーキテクチャを修正し、management planeをcontrol planeから明示的に分離し、
ネットワークのレスポンスを向上させた。
[D] In experiment, we have prototyped a lightweight management plane for IP filtering. Access filtering rules
including target IP address, prefix and gateway is represented as radix tree. It is shown that proposed method can
achieve reasonable utilization in filtering IP packets
(4) 評価実験では、radix treeを用いたIPフィルタリングを実装し、10～100以内のフィルタリングルールセットでは許
容可能な負荷率で稼動する（CPU負荷率では3%以内）ことを示した。

3. Related works: Long-term trend
Centralized access
control model
2000 2005 2010 2014
Virtualization
Technology
(hypervisor)
Cloud computing
Domain Specific /
Declarative Language
Open Flow
DSL for IDS and access
control
New Security
Concerns
DSL for SDN
Rules and conflict
management
sHype
Pyretic
Xen
Chimera
ForNox
RuleBricks
Remote Attestation
Attack on multi-tenant
アクセス制御の
一元化
ネットワーク
抽象化
アクセス制御の
表現方法の問題
Open vSwitch
vMAC
sandbox xenprobe
subvirt
仮想化、ハイパーバイザー内でのAC一元化
スケーラブル、合成可能な
観測システムの構築BGP attack
ネットワークのスライシング
仮想化技術を用いた
動的解析
ConfAid
BotMiner
解析の大規模化
大規模システムでの設定・
Access Ctrlの検査
Flowvisor
HyperDex Ensenble

4. Avenues of Attack
Sensitive data
Enterprise Network
Missing
Security Patches
Misconfigured
Database
Advanced Attacks
Sensitive
Data Leaks
Escalating
User Privileges
Default
Passwords
Weak
Passwords
Unauthorized
Database
Weak
PRNG
CDP:Functional & Operational Firewall Pattern - AWS-CloudDesignPattern
Nemesis: preventing authentication & access control vulnerabilities in web
applications, SSYM'09 Proceedings of the 18th conference on USENIX security
symposium
Detecting BGP configuration faults with static analysis, NSDI'05 Proceedings of the
2nd conference on Symposium on Networked Systems Design & Implementation
A security enforcement kernel for OpenFlow networks, HotSDN '12 Proceedings of the
first workshop on Hot topics in software defined networks
Misconfigured
Filtering

5. proposed system （設計方針）：building management
plane for scalability and manageability
DataStoreの
適用
Scalabilityを
確保する
一元管理：
設定管理
ミスをなくす
Data planeとControl planeは
同一ホスト（インスタンス）内で
分離しない。
フィルタリング・設定情報の
管理用にManagement planeを
設置して管理。
全てのcomponentをuser modeで実装することでdeploymentのコストを下げ、迅速にする。

6. Alternative access control model for virtualized
computing environment 仮想ネットワーク
• Virtualized Networking environment = SDN + Cloud computing.
• Instance is virtualized
• Network is also virtualized
hypervisor
VM VM
hypervisor
VM VM
SDN controller Network operating system
Routing Traffic
Engineering
Load
Balancing
Open Flow Switch Open Flow Switch
Data Plane
Control Plane
各ノードは仮想化
され、マルチテナント
として同一物理NIC
上に配置される。
ネットワークも仮想化
され、フィルタリングと
マッチング処理が分離
一元化される。
Open vSwitch. B. Pfaff, J. Pettit, K. Amidon, M. Casado, T. Koponen, an
d S. Shenker. Extending Networking into the Virtualization Layer. In
HotNets, 2009
Operational /
Functional FW
Amazon EC2
Design Pattern

7. What is SDN and network virtualization ?
Myth: “SDN is network virtualization”
x86 / ARM
Virtualization
Layer
Windows Linux
Open Flow
Virtualization
Or Slicing
NOX NOX
CPU, Hardisk, PIC, IO
X86 instruction set
Xen, QEMU, etc
Windows Linux
Hardware Resources
Abstraction layer
Virtualization Layer
slice slice
Bandwidth, CPU, FIB
OpenFlow
FlowVisor
Controller Contoller
Definition of a slice
• Slice is a set of flows (called flowspace) running on a topology of switches.
https://www.clear.rice.edu/comp529/.../tutorial_4.pdf

8. 設計上の問題：仮想化環境とスケーラビリティ
Centralized access
control model
2000 2005 2010 2014
Virtualization
Technology
(hypervisor)
Cloud computing
Domain Specific /
Declarative Language
Open Flow
DSL for IDS and access
control
New Security
Concerns
DSL for SDN
Rules and conflict
management
sHype
Pyretic
Xen
Chimera
ForNox
RuleBricks
Remote Attestation
Attack on multi-tenant
アクセス制御の
一元化
ネットワーク
抽象化
アクセス制御の
表現方法の問題
Open vSwitch
vMAC
sandbox xenprobe
subvirt
仮想化、ハイパーバイザー内でのAC一元化
スケーラブル、合成可能な
観測システムの構築BGP attack
ネットワークのスライシング
仮想化技術を用いた
動的解析
ConfAid
BotMiner
解析の大規模化
大規模システムでの設定・
Access Ctrlの検査
Flowvisor
HyperDex Ensenble

9. “when virtual is harder than real”
drawbacks of virtualized network
Tal Garfinkel , Mendel Rosenblum, When virtual is harder than real: Security challenges in
virtual machine based computing environments, HotOS 2005
Scalability. Growth in physical machines is ultimately limited by setup time and bounded
by organization‘s capital equipment budget. In contrast creating a new VM is as easy as
copying file. Users will frequently have several or even dozens of special purpose VMs .
Thus，total number of VMs in an organization can grow at an explosive rate．Rarely all
administrative tasks completely automated.
Diversity. Many IT organizations tackle security problems by enforcing homogenity. all
machines must run the most current patched software. This creates a range of problems
as one must try and maintain patches or other protection for a wide range of OS and deal
with the risk posed by having many unpatched machines on the network.
Access Control should be centralized !
CloudPolice: Taking access control out of the network Lucian Popa, Minlan Yu, Steven Y. Ko, Ion Stoica, Sylvia
Ratnasamy 9th ACM Workshop on Hot Topics in Networks (HotNets-IX). Monterey, CA, October 2010.
Jonathan M McCune, Stefan Berger, Trent Jaeger, Reiner Sailer: Shamon -- A System for Distributed Mandatory
Access Control. 22nd Annual Computer Security Applications Conference (ACSAC), Miami Beach, Florida, December
2006

10. Design requirement: fine grained traffic
functioning for scalability, diversity and flexibility.
[1] Scalability and diversity: Garfinkel pointed that creating a new virtual instance is far easier
than physical environment. the rapid and unpredictable growth can exacerbate management
tasks and in worse case the impact of catastorophic events can be multiplied where all instances
should be patched. Enforcing homogenity is difficult in the situation that users can have their own
special purpose VM easily without expensive cost, like copying files.
[2] Flexibility: In SDN, networks are diversified, programmable and elastic. For a long period, from
active networks to advanced network technologies like cloud and SDN, one of the general goals
of net working research has been arrived at a network which is flexible.
[3] Fine-grained traffic functioning: commercial corporations，private Enterprises and universities
emplos datacenters to run variety of applications and cloud based services. Their study reveals
that existing traffic engineering perform 15%to 20% worse than the optimal solution.
MicroTE: fine grained traffic engineering for data centers, CoNEXT '11 Proceedings of the
Seventh COnference on emerging Networking EXperiments and Technologies
Lucian Popa, Ion Stoica, Sylvia Ratnasamy: Rule-based Forwarding(RBF): Improving Internet’s
flexibility and security. HotNets 2009

11. Tradeoffs between manageability and performance
"Logically centralized?: state distribution trade-offs in software defined networks", Dan Levin, Andreas
Wundsam, Brandon Heller, Nikhil Handigol and Anja Feldmann, HotSDN '12 Proceedings of the first workshop
on Hot topics in software defined networks
Controller component choices:
[1] Strongly consistent – controller components always operate on the same
world view. Imposes delay and overhead.
[2] Eventually consistent – controller components incorporate information as
it becomes available but may make decisions on different world views.
http://www.richardclegg.org/node/21
C A
P
NoSQLRDBMS
Consistency Availability
Tolerance to network
partition
CAP Theorem (Eric Brewer 2000)
Enforced Consistency Eventual Consistency パケットフィルタなどの処理では
Strongly Consistentであることが
求められる（望ましい）
NoSQLを用いることで
A (availability)
P (Tolerance to network partition)
S (Scalability)
が達成される。

12. Basic SDN architecture and proposed system
Node (VM)
Node (VM)
Node (VM)
Flow
Table
ControllerSecure
Channel
Node (VM)
Node (VM)
Node (VM)
Filtering
rule
Table
Data store
match
match
判定処理(match)はノード側で行い、パケットは転送
しない。→ これにより、フィルタリングの一元管理を
達成しつつ、Control/Management Planeの負荷を下
げる。
Ingress packets
Ingress packets
Data plane Control plane
Control and Data plane Management plane
VCRIB: Virtualized rule management in the cloud Masoud Moshref, Minlan Yu, Abhishek Sharma, Ramesh Govindan the
4th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud). Boston, MA, June 2012.
Basic SDN
提案手法

13. Adopting datastore (NoSQL) on management plane
auto_ptr<mongo::DBClientCursor> cursor =
client.query(ns, mongo::BSONObj());
while(cursor->more()) {
mongo::BSONObj p = cursor->next();
mongo::OID oid = p["_id"].OID();
string dest = p["dest"].str();
int mask = p["mask"].numberInt();
string gateway = p["gateway"].str();
const char *p0 = dest.c_str();
const char *p1 = gateway.c_str();
add_rtentry(p0, mask, p1);
int res;
res = find_route(dstAddress);
if(res==0)
printf("route find n");
/* flush entry /*
rm_rtentry(p0, mask);
{"_id":
"$oid":"53370eaeb1f58908a9837910"
"dest":"10.0.0.0","mask":
8,"gateway":"192.168.0.2"}
Radix treeのエントリを動的に読み出すことで
リモート(management plane)から任意の
タイミングでフィルタリングルールを変更できる
ようにする。
Filtering ruleはBSON (JSON)で記述
a radix tree (also patricia trie or radix trie or
compact prefix tree) is a space-optimized trie
data structure where each node with only one
child is merged with its parent.
14 entry.addr = ntohl(addr dst.s addr);
15 entry.prefix len = 32;
17 radix tree<rtentry, in addr>::iterator it;
18
19 it = rttable.longest match(entry);
20 if (it == rttable.end()) f
21 std::cout << ‘‘no route to ‘‘ << dst << std::endl;
22 return 1;

14. Experimental result on Amazon VPC
We compiled our system on ubuntu12 LTS with
Linux kernel 3.2.0. proposed system is hosted
on Intel Xeon E5645 with 2.4 GHZ clock.
ルール数が１０～１００以内であれば、CPU負
荷率（３％以内）、コンテキストスイッチ（５０００
回以内）とも許容範囲内で稼動。
設定した環境内では、1000～10000のルール
を用いるケースは稀なため、提案手法は妥当。
vNIC1 vNIC2
Bridge
IP capture
1
2
3
MongoDB
5
8
7
8
Radix Module
6
0
Management plane Control plane
Python module
パケットキャプチャは
ユーザモードでの
pcapを利用。ルールは
Pythonを用いて設定

15. Further works
Centralized access
control model
2000 2005 2010 2014
Virtualization
Technology
(hypervisor)
Cloud computing
Domain Specific /
Declarative Language
Open Flow
DSL for IDS and access
control
New Security
Concerns
DSL for SDN
Rules and conflict
management
sHype
Pyretic
Xen
Chimera
ForNox
RuleBricks
Remote Attestation
Attack on multi-tenant
アクセス制御の
一元化
ネットワーク
抽象化
アクセス制御の
表現方法の問題
Open vSwitch
vMAC
sandbox xenprobe
subvirt
仮想化、ハイパーバイザー内でのAC一元化
スケーラブル、合成可能な
観測システムの構築BGP attack
ネットワークのスライシング
仮想化技術を用いた
動的解析
ConfAid
BotMiner
解析の大規模化
大規模システムでの設定・
Access Ctrlの検査
Flowvisor
HyperDex Ensenble

16. Conclusions
[A] The emergence of network virtualization and related technologies such as SDN and Cloud computing make us
face the new challenge of new alternative access control model.
(1) ネットワーク仮想化（抽象化）とその周辺技術（Software Defined Network, クラウドコンピューティング）の発達によ
り、アクセス制御技術にも革新と修正が求められている。
(2) アクセス制御技術の近年の傾向としてはカーネル空間での実装と一元化 (Centrazlization) が問題領域とネット
ワークの大規模に伴い様々な問題が生じている。
[B] Particularly, besides flexibility, fine-grained traffic engineering functionality for coping with scalability and
diversified networks is required for the deployments of SDN and Cloud Computing.
[C] Our architecture leverages NoSQL data store for handling a large scale of filtering rules. By adopting NoSQL, we
can achieve scalability, availability and tolerance to network partition. Besides, separating management plane and
control plane, we can achieve responsiveness and strong consistency at the same time.
(3) 提案システムには大規模なフィルタリングルールの処理のためにNoSQLデータストアを用いて、スケーラビリティを
確保し、従来のSDNのアーキテクチャを修正し、management planeをcontrol planeから明示的に分離し、ネットワーク
のレスポンスを向上させた。
[D] In experiment, we have prototyped a lightweight management plane for IP filtering. Access filtering rules
including target IP address, prefix and gateway is represented as radix tree. It is shown that proposed method can
achieve reasonable utilization in filtering IP packets
(4) 評価実験では、radix treeを用いたIPフィルタリングを実装し、10～100以内のフィルタリングルールセットでは許容
可能な負荷率で稼動する（CPU負荷率では3%以内）ことを示した。
www.slideshare.net/RuoAndo
https://github.com/RuoAndo/cBridge