• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Practical Wireless Mesh Networks and Their Applications
 

Practical Wireless Mesh Networks and Their Applications

on

  • 3,328 views

This is my PhD defense talk, given on Nov 11, 2009 at Johns Hopkins University.

This is my PhD defense talk, given on Nov 11, 2009 at Johns Hopkins University.

Statistics

Views

Total Views
3,328
Views on SlideShare
3,323
Embed Views
5

Actions

Likes
1
Downloads
143
Comments
0

2 Embeds 5

http://www.slideshare.net 4
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • explain how it works

Practical Wireless Mesh Networks and Their Applications Practical Wireless Mesh Networks and Their Applications Presentation Transcript

  • Practical Wireless Mesh Networks and Their Applications Raluca Musăloiu-E. Johns Hopkins University November 11, 2009 Joint work with: Yair Amir, Claudiu Danilov, Michael Hilsdale, Michael Kaplan, Nilo Rivera
  • Deploying a wireless network...
  • ... using access points. Follows client-server paradigm. For more coverage, install more access points. Each access point is connected to the Internet.
  • That’s a lot of wires for a wireless network!
  • Mesh networks are a paradigm shift. Two classes of participants: Clients, which are mobile. Nodes, which are relatively stationary. Only a few nodes are connected to the Internet (Internet gateways). They are multi-hop networks.
  • Lots of research on wireless networks relies on simulations. The Mistaken Axioms of Wireless-Network Research: (Kotz et al., Darmouth College, Tech Report, 2003) “The world is flat.” “A radio's transmission area is circular.” “All radios have equal range.” “If I can hear you, you can hear me (symmetry).” “If I can hear you at all, I can hear you perfectly.” “Signal strength is a simple function of distance.”
  • We try to bridge the gap between theory and practice and build a real mesh system.
  • Effort has already been made to make mesh networks a reality. However... Some of them are experimental testbeds. Use expensive hardware for mesh nodes. Have limited support (or none) for mobility.
  • For example, In academia: In industry: MIT Roofnet Metricom Ricochet Microsoft MCL Nokia Rooftop UCSB MeshNet Firetide Rice Taps Meraki Rutgers ORBIT Lab Tropos Networks Stony Brook iMesh Community Networks: Champaign-Urbana Community Wireless Network (CUWiN) NYCwireless Freyfunk (Germany)
  • What we are looking for is Seamless access for mesh users. Fast handoff while roaming. Rapid deployment. Robustness. Low cost. Security. Applications.
  • This thesis introduces The architecture of the first high-throughput wireless mesh network with fast handoff using off-the-shelf routers. The first robust Push-To-Talk service for wireless mesh networks.
  • Introducing SMesh
  • The entire mesh network is seen by the client as an omniscient access point! We use 802.11 IBSS mode (ad-hoc). MobiSys 2006 WoWMoM 2007 WiMesh 2008
  • SMesh communication infrastructure uses the Spines Messaging System. Spines was created by Yair Amir and Claudiu Danilov (DSN03, NOSSDAV05, TOM06). Spines daemons create overlay networks on the fly. We run a Spines daemon on each mesh node. Spines provides unicast, anycast, multicast communication.
  • Mesh topology formation Direct links are created Client C between nearby nodes via an auto-discovery mechanism. Client B Virtual links are created between Internet gateways (they form a fully connected graph, communicating over an overlay multicast group). Client A
  • Mesh topology is hybrid: we use both wireless and wired links. The routing metric gives preference to wired links.
  • SMesh provides a seamless interface to the mobile clients. Standard DHCP protocol. Client always gets the same IP address (private IP in 10.0.0.0/8 address space; based on the MAC address). Client routes all packets through a Virtual Default Gateway.
  • NAT We associate two overlay multicast groups for each client.
  • Control Group 26 30 14 NAT We associate two overlay multicast groups for each client.
  • Control Group 26 Data 30 Group 14 NAT We associate two overlay multicast groups for each client.
  • Fast intra-domain handoff. The handoff is controlled from the mesh infrastructure! We use multiple access points during the handoff, to avoid losing packets.
  • Fast inter-domain handoff. SMesh runs in a private address space (NAT is performed at each Internet gateway). Connection oriented protocols expect packets to come from the same source. Solution: Route each stream through the Internet gateway used during connection establishment.
  • We deployed SMesh in our campus. 18-nodes testbed, covering three buildings in our campus (NEB/Shaffer, Maryland, Barton). Linksys WRT54G 802.11b/g routers, BCM947XX radios, omni-directional antennas, 16 MB RAM, 4 MB flash memory, 200 Mhz CPU. Available as open-source software (smesh.org).
  • How did we do what?
  • SMesh Routing Architecture
  • 1 2 Node 5 routing rules 3 Source Destination Ne Node 1 client 1 6 … … 4 Node 2 client 1 6, 7 5 … … 6 7 Client 1 Fig. 1. The routes to a mobile client (multipath routin To route, we need to use source based multicast trees. Multipath routing is not supported by current operating systems. an overlay network to increase the reliability of to-end path. End-System-Multicast [14] and Spine
  • For flexibility in routing we could use user-space routing, but...
  • With user-space routing all the packets are moved to application level. Spines Spines Spines User space Kernel space Spines
  • We need flexibility in routing... but without losing performance. We developed a new architecture that maintains the control in user space, while the data is routed at the kernel level.
  • Node’s 5 kernel routing tables: 1 2 Node 5 routing rules 3 Source Destination Next-Hop(s) Node 1 client 1 6 … … 4 Node 2 client 1 6, 7 5 … … Node 3 6 Node 2 7 Node 1 Client 1 Fig. 1. The routes to a mobile client (multipath routing). Fig. 2 an overlay network to increase the reliability of the end- space to user space in orde to-end path. End-System-Multicast [14] and Spines [3] also routing decision is made, th Each route through an application router to support overlay multicast spaceone for sent on th node maintains multiple kernel routing tables, where it is each without infrastructure support. node in the network. boundary must be crossed Other work has looked into operating system support for We describe next a me wireless ad-hoc routing protocols. Chakeres and Belding dundant multipath routing showed in [9] an in-kernel design and implementation of the is simple: each node maint ad-hoc AODV protocol using Netfilter modules, and showed one for each node in the
  • One of node’s 5 routing tables: 1 2 Node 5 routing rules 3 Source Destination Node 2 Next-Hop(s) Node 1 client 1 6 … … 4 5 Node 2 Destination client 1 6, 7 Next-hops … … 6 client 1 6, 7 client 2 3 7 ... ... Client 1 Fig. 1. The routes to a mobile client (multipath routing). Fig. 2 an overlay network to increase the reliability of the end- space to user space in orde Eachto-end path. End-System-Multicast [14] and Spines [3] also route may have multiple next-hops. routing decision is made, th route through an application router to support overlay multicast space where it is sent on th without infrastructure support. boundary must be crossed Other work has looked into operating system support for We describe next a me wireless ad-hoc routing protocols. Chakeres and Belding dundant multipath routing showed in [9] an in-kernel design and implementation of the is simple: each node maint ad-hoc AODV protocol using Netfilter modules, and showed one for each node in the
  • Fig. 2. Architecture.
  • Some interesting implementation details...
  • 1. Encode entry node in the packet’s IP header. 0 7 8 15 16 23 24 31 Version IHL TOS Total length Identification (IPID) Flags Fragment offset TTL Protocol Header checksum Source IP Destination IP Options and padding
  • 1. Encode entry node in the packet’s IP header. 0 7 8 15 16 23 24 31 Version IHL TOS Total length Identification (IPID) Identification (IPID) Flags Fragment offset TTL Protocol Header checksum Source IP Destination IP Options and padding
  • 1. Encode entry node in the packet’s IP header. 0 7 8 15 16 23 24 31 Version IHL TOS TOS Total length Identification (IPID) Identification (IPID) Flags Fragment offset TTL Protocol Header checksum Source IP Destination IP Options and padding
  • 2. Use policy routing and define multiple routing tables. # iptables -A PREROUTING -t mangle -m u32 --u32 "2&0xFFFF=35" -j MARK --set-mark 35 # ip rule add fwmark 35 table 35
  • 3. Build a kernel module to deliver packets to multiple next hops. Use CONFIG_IP_ROUTE_MULTIPATH kernel option. # ip route add 10.233.59.169/32 table 35 nexthop via 10.0.11.32 dev eth1 nexthop via 10.0.11.33 dev eth1 # iptables -A POSTROUTING -t mangle -j MULTIHOP
  • A packet path in the kernel.. entry point: set IPID set TOS all routers: set fwmark fwmark MULTIHOP module Routing NF_IP_PRE_ROUTING NF_IP_FORWARD NF_IP_POST_ROUTING Decision Routing Decision NF_IP_LOCAL_IN NF_IP_LOCAL_OUT
  • ... and a packet path in the mesh. Node 5
  • ... and a packet path in the mesh. Node 5
  • ... and a packet path in the mesh. IPID: 5 TOS set Node 5
  • ... and a packet path in the mesh. IPID: 5 TOS set Node 5
  • ... and a packet path in the mesh. IPID: 5 TOS set Node 5 IPID: 5 TOS set
  • ... and a packet path in the mesh. IPID: 5 TOS set IPID: 5 Node 5 TOS set
  • ... and a packet path in the mesh. Node 5
  • Evaluation
  • We used three configurations. 1. One router 2. 5 nodes wireless “line” setup 3. 17 nodes wireless testbed
  • Rate 24 Mbps Transmission power 50 mW Retransmission limit 7 VoIP stream 64 Kbps
  • 1. Single router Packets / s (sent + received) Packets / s (sent + received) 100 800 1600 3200 4000 5000 6000 7000 100 800 1600 3200 4000 5000 6000 7000 100 1.0 Overlay routing Overlay routing 80 0.8 60 0.6 CPU load Loss rate Native SMesh kernel routing 40 0.4 SMesh 20 0.2 Native kernel routing 0.0 0 1 4 8 16 32 40 50 55 60 70 1 4 8 16 32 40 50 55 60 70 Number of VoIP streams (each direction) Number of VoIP streams (each direction) We route up to 50 duplex VoIP streams, before the CPU is saturated. User space overlay routing 4 streams 512 Kbps SMesh 50 streams 6.4 Mbps Native kernel routing 60 streams 7.6 Mbps
  • 2. “Line” topology of 5 nodes TCP Throughput (Mbps) 10.1 With one wireless 5.1 hop, we get 10 Mbps. 3.3 2.7 2.1 2.1 2.1 2.0 1.8 1.8 1 hop 2 hops 3 hops 4 hops 5 hops Overlay routing SMesh
  • 3. Testbed of 17 nodes User-space overlay routing SMesh kernel routing Access Access Point ID Point ID 10 5 hops 28 10 5 hops 28 4 hops 26 4 hops 26 3 hops 25 3 hops 25 8 24 8 24 2 hops 2 hops Throughput (Mbps) Throughput (Mbps) 36 36 33 33 1 hop 32 1 hop 32 6 6 31 31 4 4 2 2 0 0 0 80 160 240 320 0 40 80 120 160 Time (s) Time (s) Results are close to the “line” topology (8.5 Mbps for one hop).
  • Application: Push-To-Talk System for First Responders
  • What is PTT? Half-duplex communication system with multiple participants. While one person speaks, the others listen.
  • Push-To-Talk systems require an arbitration mechanism (“floor control”) that determines the order in which participants speak. In cellular networks, Push-To-Talk systems are centralized.
  • We need a robust, distributed Push-To-Talk system that works even when Mesh nodes crash. The network is partitioned or it merges.
  • Our Push-To-Talk system allows regular phone users to remotely join a PTT session established inside the mesh!
  • How does the system work?
  • Mobile Client with VoIP Software SIP RTP PTT Controller Distributed RTP Proxy Mobile, Fault-Tolerant SIP 3PCC DTMF Voice Floor Monitor Management Mobile Client Mesh Node State sip_call_id, PTT Session sip_cseq, rtp_port, Manager ptt_group, ptt_state Client PTT PTT PTT Data Control CMonitor Controller Group Group Group Group Routing Daemon (Discovery, Topology Management, Group Management) Wireless Mesh Network
  • Interface with Mobile Client We use the standard Session Initiation Protocol (SIP) to interact with users. The entire mesh is seen by the user as a single, distributed third party call controller (3pcc).
  • A user interacts with the system using a VoIP application. sip : ptt @ 192.168.1.10 How to connect: (that’s a virtual SIP server) How to join a group: type # 12 # How to request to speak: type 5 How is notified when he receive a “beep-beep” audio signal has permission to speak:
  • We use multicast groups to manage the client and the PTT sessions. These are overlay multicast groups.
  • Client management Session management
  • Client management Control Group Session management
  • Client management Control Group Session management
  • Client management Control Group Session management Controller Group
  • Client management Control Group Session management Controller Group Monitoring Group
  • Client management Control Group Session management Controller Group Monitoring Group Data Group
  • How are users’ requests arbitrated?
  • Sending Mesh Controller client node
  • Sending Mesh Controller client node Sends “Request to speak” to the access point.
  • Sending Mesh Controller client node Floo r Re Sends “Request to speak” Con ques t trol to the access point. ler g roup
  • Sending Mesh Controller client node Floo r Re Sends “Request to speak” Con ques t trol to the access point. ler g roup k e st Ac Requ st u nica
  • Sending Mesh Controller client node Floo r Re Sends “Request to speak” Con ques t trol to the access point. ler g roup k e st Ac Requ st u nica k to Spea sion up Pe rmis ol gro C ontr Receives “Permission lient Mesh C granted” from the node access point. This can be a different node!
  • Sending Mesh Controller client node Floo r Re Sends “Request to speak” Con ques t trol to the access point. ler g roup k e st Ac Requ st u nica k to Spea sion up Pe rmis ol gro C ontr Receives “Permission lient Mesh C granted” from the node access point. PTS Ac k unic This can be a different node! ast
  • The controller is rotated periodically, according to participants’ locations in the network. We want to maintain the controller node in the “center of gravity” of the nodes handling PTT clients on a certain group.
  • Controller
  • Mesh Controller node When a “better” controller is available stop handling and queueing requests.
  • Mesh Controller node When a “better” controller is available stop handling and queueing requests. INVI TE ( Grou p, Q unic ueue ast ) Join Controller group Join Monitoring group Start handling requests
  • Mesh Controller node When a “better” controller is available stop handling and queueing requests. INVI TE ( Grou p, Q unic ueue ast ) Join Controller group Join Monitoring group Start handling ) requests (G roup k TE Ac INVI st u nica Controller change succeeded Leave Controller group Leave Monitoring group (if necessary)
  • The system needs to withstand node crashes, network partitions and merges.
  • Network
  • Network Controller
  • Sending Network Controller node
  • Sending Network Controller node Sending client
  • PING_CMON Sending Network Controller Monitoring group node On timeout: Controller is lost Join Controller group (if lowest IP). Start handling requests. Sending client
  • PING_CMON PTS_PING (client) Sending Network Controller Monitoring group Controller group node On timeout: Controller is lost On timeout: Sending node is lost Join Controller group (if lowest IP). Handle the next client in the Start handling requests. queue. Sending client
  • PING_CMON PTS_PING (client) Sending Network Controller Monitoring group Controller group node On timeout: Controller is lost On timeout: Sending node is lost Join Controller group (if lowest IP). Handle the next client in the Voice Start handling requests. queue. Sending client
  • In summary, We instantiate a controller for each PTT group that arbitrates the requests on that group. The controller is monitored and rotated if a more suitable node is available. We achieve high availability with a monitoring mechanism that uses overlay multicasts groups.
  • System Evaluation
  • Nodes 14 Rate 18 Mbps Transmission power 50 mW Retransmission limit 7 VoIP stream 64 Kbps Speak duration 20 sec
  • We evaluated the system when 1. The network is stable (normal operation). 2. The number of users in a PTT group increases. 3. The number of groups increases. 4. Large-scale scenario (40 clients, 10 groups, network partitions and merges).
  • 1. Normal operation Throughput 80 Kbps Sender node 2 Sender node 12 Sender node 14 14 nodes, 60 Kbps 40 Kbps 4 users, each speaks for 20 seconds. 20 Kbps Overhead 0 Kbps 0 10 20 30 40 50 60 70 Time (s) Sender node 2 Sender node 12 21.7 21.8 21.9 22.0 22.1 22.2 22.3 22.4 Packet Arrival time (s)
  • 2. Scalability with the #clients Average latency Average loss rate 1.2 % 120 ms 114 1 % 100 ms 0.92 0.8 % 80 ms No PTT support 60 ms 0.6 % No PTT support 40 ms 0.4 % 27 Single radio 28 Single radio 23 26 0.23 18 16 21 0.18 0.19 25 0.2 % 20 ms 17 21 19 16 Dual radio 0.08 0.09 0.1 0.15 0.05 0.13 Dual radio 0 ms 0 % 2 4 8 14 28 42 2 4 8 14 28 42 Number of clients Number of clients Using dual radios, it scales to at least 42 users in a single PTT session, in our testbed.
  • 3. Scalability with the #groups Average latency Average loss rate 2500 ms 20 % Dual radio Single radio 18 2227 2000 ms Single radio 15 % 14 Dual radio Dual radio+ 1541 1500 ms packing 1270 10 % Single radio+ Dual radio+ packing packing 1000 ms 6 6 Single radio+ 5 % 500 ms 360 packing 2 2 244 1 178 183 1 0 0 74 53 0 0 0 24 26 28 24 26 30 37 0 ms 0 % 1 2 4 6 8 10 12 14 16 18 20 1 2 4 6 8 10 12 14 16 18 20 Number of groups Number of groups Using dual radios and packing, it scales up to 18 groups, with 4 users in each PTT group.
  • 4. Large-scale scenario (10 PTT groups, 4 clients on each group) Data & overhead traffic: 5000 Data Traffic (kbps) 4000 C D F PTT control traffic Routing control traffic 3000 A B E 2000 1000 G 0 0 50 100 150 200 250 300 350 400 450 Time (s) Overhead traffic only: 100 C D F PTT control traffic Traffic (kbps) 80 A B E Routing control traffic 60 40 G 20 0 0 50 100 150 200 250 300 350 400 450 Time (s) (A) clients join (D) the network partitions (B) clients request to speak (E) stable network after partition (C) regular operation (F) the network merges (G) clients stop speaking
  • In conclusion, we presented The architecture of the first high-throughput wireless mesh network with fast handoff using off-the-shelf routers. The first robust Push-To-Talk service for wireless mesh networks.
  • Some final words...
  • Turning research ideas into practical systems is not an easy task, but it’s definitely a lot of fun.
  • Wireless Mesh Networks is a promising technology. Killer applications are yet to come. They may require infrastructure support.
  • The SMesh system is available as open-source at http://smesh.org Try it for yourself and let us know what you think! :-)