1. Comparing Simulation Tools and
Experimental Testbeds for Wireless Mesh
Networks
Written by:
Kefeng Tan, Daniel Wu, An (Jack) Chan, Prasant Mohapatra
Presented by:
Ioanna Tsalouchidou

2. Presentation Outline:
• Simulations Vs. Experimental Testbeds, discrepancies and more
• Methodology
• Point to Point measurements
• PHY-layer: Beyond inaccurate channel modeling
• MAC-layer: Flow level unfairness due to interference
• IP-layer: Route stability
• Conclusions

3. Experimental Testbeds
• Real world scenarios
• Affected by the testing environment
• Conditions highly random and uncontrollable
• High costs
• Hardware limitations

4. Simulations
• Network scenarios easily constructed and modified
• Data can be easily collected
• Modeling of large scale network topologies
• Do not require special hardware and labor resources
• Low cost comparing to testbeds
• Not as trustworthy as testbed
experiments
• PHY layer inadequate modeling

5. Methodology
• Simulations
o NS-2
o QualNet
• Experimental Testbeds:
o single antenna embedded system (Soekris)
o Laptops with modern Wi-Fi antennas
o QuRiNet

6. Network scenarios
• Metrics
oGoodput
o Delay, Delay jitter
o Probability distribution
• Parameters
o PHY layer: path loss and antenna diversity
o MAC layer: traffic load, bit rate, number of hops, flow fairness,
interference
o Routing layer: route length, persistence, stability and diversity
of routes

7. Point to Point measurement
• Send packets from one laptop to an other
• TCP and UDP protocols
• Different bitrates, distances and traffic loads
o Indoor and outdoor
o Transmission distance of 40 and 80 meters
o Packet delivery 5 and 10 Mbps
o Bitrate: auto and 54Mbps

8. Point to Point measurement
• Simulations do not affect much • UDP goodputs are higher
by the environment indoor than outdoor
• In experiments laptops have • Qualnet always has the
better performance indoor than highest results
outdoor

9. PHY: Beyond Inaccurate channel modeling
• Antenna Diversity
o Missing from simulators
o Difference of signal when antennas spaced one wavelength
apart
• Configuring PHY Channel
o Path loss determines how fast a signal fades
 NS-2 path loss global parameter
 QualNet does not have this setting
o Can be configured to fit real world scenarios

10. PHY: Beyond Inaccurate channel modeling
• QualNet constant curves
• NS-2 goodput changes as path loss increases
• Laptop-outdoor outperforms laptop-indoor
• Soekris doesn't show consistent results

11. MAC: Impact of transmission data rate
• At low rates all results increase linearly
• NS-2 results match laptop-outdoor
• The drop of laptop indoor is not captured

12. MAC: Impact of multihop
• Goodput and delay vary with the number of hops
• For small number of hops NS-2, QualNet and testbed agree

13. Routing: Route Stability
• Network settings
oThe dominating factor for routing stability is the channel quality
and stability.
• Route prevalence and persistence
o Prevalence: occurrence of a route over that of the dominant
route
o Persistence: the duration(sec) that a route lasts before it
changes
• Spatial prevalence and spatial distribution
o Spatial prevalence: number of cumulative occurrences of one
node over the number of all records

14. Routing: Route Stability
• (a) Dominating route occurs much more often in QuRiNet
• (c) NS-2: the most spatial diversity with approximately 17% of routes
covering 90% of nodes,then QuRiNet (20%) and finally QualNet (60%
spatially concentrated)
• Routes in simulations are less stable and persistent

15. Conclusions
● NS-2 and QualNet simulations.
● Testbets: Soekris, laptops, QuRiNet .
● PHY, MAC and IP layers.
● Antenna diversity, path loss, multihop, transmission rate, routing stability.
● Discrepancies due to inaccurate channel modeling, antenna diversity.
● Simulators: path loss is well modeled but routes less stable and persistent.