Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

  • Be the first to comment


  1. 1. Network Simulation and Emulation in a Testbed Era <ul><ul><li>Tom Henderson </li></ul></ul><ul><ul><li>University of Washington </li></ul></ul><ul><ul><li>The Boeing Company </li></ul></ul><ul><ul><li>UW EE Colloquium </li></ul></ul><ul><ul><li>February 24, 2009 </li></ul></ul>
  2. 2. Boeing Research & Technology organization Information Assurance Digital Communications & Network Technology Information Management and Transformation Applied Mathematics Architecture and Networked Systems Technology Contract Research and Development Programs <ul><li>Development of high assurance multi-level security technologies </li></ul><ul><ul><li>EAL7 common criteria evaluation experience </li></ul></ul><ul><ul><li>Formal methods in support of EAL 7 evaluation </li></ul></ul><ul><li>High assurance guard solutions for edge, core, & tactical environments </li></ul><ul><li>Security modeling & formal method analysis of end to end systems </li></ul><ul><li>Policy-based secure computing </li></ul><ul><ul><li>Systems of Systems IA approach </li></ul></ul><ul><li>Intrusion detection correlation technologies </li></ul><ul><li>Secure workflow and SOA technologies </li></ul><ul><li>Information operations </li></ul><ul><li>System security engineering </li></ul><ul><li>Advanced trusted system implementations </li></ul><ul><li>Text analysis for information assurance </li></ul><ul><li>Proprietary program support in IA technologies </li></ul><ul><li>Secure RFID infrastructure </li></ul><ul><li>Anti-tampering technologies </li></ul>Networked Systems Technology <ul><li>Phased array antennas </li></ul><ul><li>Free air laser communications </li></ul><ul><li>High capacity RF communications </li></ul><ul><li>Bandwidth efficient modulation </li></ul><ul><li>On board wireless communications </li></ul><ul><li>Network processors </li></ul><ul><li>AFRL RAPID </li></ul><ul><li>Integrated communications systems </li></ul><ul><li>MANET protocols </li></ul><ul><li>Reliable routing </li></ul><ul><li>Network management for tactical environments </li></ul><ul><li>Auto-configuration </li></ul><ul><li>Network gateways </li></ul><ul><li>End to end QoS </li></ul><ul><li>Network security </li></ul><ul><li>UAV network router </li></ul><ul><li>Data & Text Mining </li></ul><ul><li>Human Language technologies </li></ul><ul><li>Decision support information </li></ul><ul><li>Information visualization </li></ul><ul><li>Machine translation </li></ul><ul><li>Knowledge representation </li></ul><ul><li>Data integration </li></ul><ul><li>Simulation visualization </li></ul><ul><li>3D visualization </li></ul><ul><li>Model-based collaboration </li></ul><ul><li>Social networks analysis </li></ul><ul><li>Intelligent graphics & docs. </li></ul><ul><li>Collab. human robot systems </li></ul><ul><li>Autonomous systems </li></ul><ul><ul><li>Architectures </li></ul></ul><ul><ul><li>World modeling </li></ul></ul><ul><ul><li>Multi-vehicle behaviors </li></ul></ul><ul><li>Reasoning </li></ul><ul><li>Adaptive planning </li></ul><ul><li>Learning </li></ul><ul><li>Distributed multi-agents </li></ul><ul><li>Global information sharing </li></ul><ul><li>RFID info. mgmt. services </li></ul><ul><li>Adaptive intelligent info. integration </li></ul><ul><li>Semantic modeling, ontologies, adaptive workflow modeling </li></ul><ul><li>Quality of Service info. algorithms </li></ul><ul><li>Wireless mobile dynamic info. mgmt. </li></ul><ul><li>Sensor networks </li></ul><ul><li>Data & information fusion </li></ul><ul><li>Info. assessment aids </li></ul><ul><li>Trends analysis/intent inferencing </li></ul><ul><li>Situation assessment </li></ul><ul><li>Resource allocation techniques </li></ul><ul><li>Deconfliction algorithms </li></ul><ul><li>Course of action analysis/generation </li></ul><ul><li>Variation analysis & reduction </li></ul><ul><li>Reliability & time dependent data analysis </li></ul><ul><li>Analysis of large datasets </li></ul><ul><li>Financial modeling & decisions </li></ul><ul><li>Survey & test design </li></ul><ul><li>Statistical training </li></ul><ul><li>Quality assurance & Lean+ </li></ul><ul><li>Design of Experiments </li></ul><ul><li>Computer-aided geometric design </li></ul><ul><li>Numerical optimization </li></ul><ul><li>Design space exploration </li></ul><ul><li>Multidisciplinary Design Optimization </li></ul><ul><li>Geometric modeling </li></ul><ul><li>Multivariate data fitting </li></ul><ul><li>Optimal control </li></ul><ul><li>Engineering modeling & analysis </li></ul><ul><li>Computational mechanics </li></ul><ul><li>Sensor Fabric design & operation </li></ul><ul><li>System dynamics & control modeling </li></ul><ul><li>Data structures for complex product families </li></ul><ul><li>Kalman filter applications </li></ul><ul><li>Operations research </li></ul><ul><li>Systems of Systems modeling & analysis </li></ul><ul><li>Discrete/network/stochastic/ Systems of Systems optimization </li></ul><ul><li>Data fusion </li></ul><ul><li>Discrete-event simulation </li></ul><ul><li>Transportation, manufacturing, business, communication & military system modeling </li></ul><ul><li>UML architecting </li></ul><ul><li>Arch. analysis </li></ul><ul><li>CONOPS development </li></ul><ul><li>System requirements </li></ul><ul><li>NR-KPP compliance </li></ul><ul><li>SoSE </li></ul><ul><li>Systems engineering </li></ul><ul><li>NCO demo. </li></ul><ul><li>LVC technologies </li></ul><ul><li>LVC tools </li></ul><ul><li>DIS router </li></ul><ul><li>Data link translation </li></ul><ul><li>Gateways </li></ul><ul><li>Networked manufacturing </li></ul><ul><li>Location services </li></ul><ul><li>Planning / visibility tools </li></ul><ul><li>Functional integration </li></ul><ul><li>High performance computing </li></ul><ul><li>GRID computing </li></ul><ul><li>Resource management </li></ul><ul><li>Functional integration </li></ul><ul><li>CINS lab </li></ul><ul><li>System/network mgmt. </li></ul><ul><li>QoS </li></ul><ul><li>Mobile agents </li></ul><ul><li>Situation awareness </li></ul><ul><li>Reconfigurable computing </li></ul><ul><li>S/W architecture </li></ul><ul><li>S/W processes </li></ul><ul><li>Arch. evaluation (ATAM) </li></ul><ul><li>Performance assessment (PAPM) </li></ul><ul><li>QoS </li></ul><ul><li>Real-time system development </li></ul><ul><li>Tactical info. mgmt. with an emphasis in C2/tactical collaboration </li></ul><ul><li>Quality of Service network utilization </li></ul><ul><li>Software enabled control for UAV automation </li></ul><ul><li>Dynamic network management </li></ul><ul><li>Affordable validation & verification of flight critical software & systems </li></ul><ul><li>Collaborative anticipatory systems </li></ul><ul><li>Cyberwarfare / Cybercraft </li></ul><ul><li>Effects-based operations </li></ul><ul><li>System & vehicle health management </li></ul><ul><li>Diagnostics </li></ul><ul><li>InfoCenter & information management </li></ul><ul><li>Database technology & implementation </li></ul><ul><li>Distributed sensor system technology </li></ul>Home organization
  3. 3. What is a network simulator? <ul><li>A software tool and set of models used to evaluate computer network behavior </li></ul><ul><li>A discrete-event network simulator describes networks in terms of a sequence of discrete events </li></ul><ul><ul><li>The simulation kernel temporally orders and dispatches events to models </li></ul></ul><ul><ul><li>Models generate output or more events </li></ul></ul><ul><ul><li>Time advances in discrete steps </li></ul></ul>
  4. 4. The basic model Application Application Protocol stack Node NetDevice NetDevice Application Application Protocol stack Node NetDevice Channel Packet Simul
  5. 5. The basic model // Define your topology Node n0 = new Node; Node n1 = new Node; AddInternetStack (n0, n1); Channel c0 = new Channel;. Connect (n0, n1, c0); Application a0 = new TrafficGenerator; // Configure things a0.SetDataRate (1Mb/s); a0.Start (10.0 seconds); // Define outputs WriteTraceFile (“outfile”); // Run the simulator Simulator::Run();
  6. 6. Network simulation remains popular <ul><li>Many commercial products </li></ul><ul><ul><li>e.g. OPNET, QualNet, Shunra VE, Matlab Simulink, OMNet++, NCTUns, SSFNet, Extend </li></ul></ul><ul><li>A few free, open source projects </li></ul><ul><ul><li>ns-2 is the most widely used such tool </li></ul></ul><ul><ul><li>roughly 10,000 downloads/month </li></ul></ul><ul><li>Widely used in research </li></ul>Source: Search of ACM Digital Library papers citing simulation, 2001-04
  7. 7. Skepticism in the research community <ul><li>“For years, the community had to rely on simulators, which now seem a little dated, and it’s not clear who was convinced to adopt anything new based on ns2 simulations;” </li></ul><ul><li>Nick McKeown, VINI public review, ACM Sigcomm 2006 </li></ul>
  8. 8. What has changed? <ul><li>Simulators have historically been used for experiments in place of real machines </li></ul><ul><ul><li>A tradeoff of realism for scale </li></ul></ul><ul><li>But now... </li></ul><ul><li>Cost of test hardware has plummeted </li></ul><ul><li>Virtualization technologies proliferate </li></ul>
  9. 9. Emulab (University of Utah) <ul><li>Source: </li></ul>
  10. 10. Emulab <ul><li>Original Emulab allowed experiments of roughly ~100 nodes </li></ul><ul><ul><li>FreeBSD Dummynet and VLANs to interconnect the nodes </li></ul></ul><ul><li>Emulab now offers virtualization </li></ul><ul><ul><li>Experiments of up to ~2000 (virtual) nodes </li></ul></ul><ul><ul><li>Requires new approaches for experimental configuration </li></ul></ul><ul><li>Also, some 802.11 wireless nodes </li></ul><ul><li>Source: M. Hibler et al., “Large-scale Virtualization in the Emulab Network Testbed,” Proc. 2008 Usenix Technical Conference. </li></ul>
  11. 11. PlanetLab: Global-scale Internet testbed <ul><li>Source: </li></ul>PlanetLab “slices”
  12. 12. PL-VINI and Trellis <ul><li>PL-VINI adds virtual networking capabilities (via network overlay) to PlanetLab </li></ul><ul><ul><li>A. Bavier et al, “In VINI Veritas: Realistic and Controlled Network Experimentation,” Proc. ACM Sigcomm, 2006 </li></ul></ul><ul><li>Trellis is a kernel-based implementation of the same concept </li></ul><ul><ul><li>S. Bhatia et al, “Trellis: A Platform for Building Flexible, Fast Virtual Networks on Commodity Hardware,” ROADS Workshop, Dec. 2008 </li></ul></ul>
  13. 13. ORBIT (Rutgers WINLAB) <ul><li>Source: </li></ul><ul><li>802.11a/b/g radios </li></ul><ul><li>Bluetooth </li></ul><ul><li>GNU/USRP radios </li></ul><ul><li>Noise generators </li></ul>
  14. 14. Simulation “out of the loop” <ul><li>Source: NSF GENI project: </li></ul>
  15. 15. Skepticism in the research community <ul><li>“For years, the community had to rely on simulators, which now seem a little dated, and it’s not clear who was convinced to adopt anything new based on ns2 simulations;” </li></ul><ul><li>Nick McKeown, VINI public review, ACM Sigcomm 2006 </li></ul>
  16. 16. Overheard* on e2e-interest mailing list <ul><li>“ ...Tragedy of the Commons...” </li></ul><ul><li>“ ...around 50% of the papers appeared to be... bogus...” </li></ul><ul><li>“ Who has ever validated NS2 code?” </li></ul><ul><li>“ To be honest, I'm still not sure whether I will use a simulation in a paper.” </li></ul><ul><li>“ ...I will have a hard time accepting or advocating the use of NS-2 or any other simulation tool” </li></ul><ul><li>... </li></ul><ul><li>* September 2005 archives of the e2e-interest mailing list </li></ul>
  17. 17. Questions <ul><li>Do we need to simulate networks anymore? </li></ul><ul><li>What kind of network simulator is needed? </li></ul><ul><li>Why should you believe network simulation results? </li></ul>
  18. 18. Why simulate? <ul><li>Wireless </li></ul><ul><ul><li>Reproducibility </li></ul></ul><ul><ul><li>Fidelity (especially, real-time constraints) </li></ul></ul><ul><ul><li>Radios may not exist or be available </li></ul></ul><ul><ul><li>Field tests in realistic conditions cost $$ </li></ul></ul><ul><li>Scalability </li></ul><ul><ul><li>10,000+ nodes? </li></ul></ul><ul><ul><li>For smaller configurations, execution time </li></ul></ul>Simulation is often an appropriate tool for the job
  19. 19. Challenges for ns (and simulators)‏ <ul><li>Align with how research is now conducted </li></ul><ul><li>Improve credibility </li></ul>Can a new simulator help with these problems?
  20. 20. What is ns-3? <ul><li>An open source project building a new network simulator to replace ns-2 </li></ul>
  21. 21. ns-3 project goal <ul><li>Develop a preferred, open simulation environment for networking research </li></ul><ul><ul><li>1) a tool aligned with the simulation needs of modern networking research </li></ul></ul><ul><ul><li>2) an open-source project that encourages community contribution, peer review, and validation of the software </li></ul></ul>
  22. 22. Relationship to ns-2 <ul><li>ns-3 is a new simulator, without backward compatibility </li></ul><ul><li>Similarities to ns-2: </li></ul><ul><li>C++ software core </li></ul><ul><li>GNU GPLv2 licensing </li></ul><ul><li>ported ns-2 models: random variables, error models, OLSR, Calendar Queue scheduler, (more planned) </li></ul><ul><li>Differences: </li></ul><ul><li>Python scripting (or C++ programs) replaces OTcl </li></ul><ul><li>most of the core rewritten </li></ul><ul><li>also based on the yans and GTNetS simulators </li></ul><ul><li>new animators, configuration tools, etc. are in work </li></ul>
  23. 23. Funding support <ul><li>ns-3 project is supported by: </li></ul><ul><li>NSF CISE CRI awards (2006-10) </li></ul><ul><ul><li>PIs: Tom Henderson and Sumit Roy (University of Washington), George Riley (Georgia Institute of Technology) </li></ul></ul><ul><ul><li>NSF CNS-0551686 (University of Washington) </li></ul></ul><ul><li>INRIA Planete and the French government </li></ul><ul><li>With additional support from: </li></ul><ul><li>2008 Google Summer of Code </li></ul><ul><li>University of Washington </li></ul><ul><li>Georgia Institute of Technology </li></ul>‘
  24. 24. Challenges for ns (and simulators)‏ <ul><li>Align with how research is now conducted </li></ul><ul><li>Improve credibility </li></ul>
  25. 25. Test and Evaluation Options <ul><li>When mathematical analysis alone won’t do </li></ul>Increasing realism Increasing complexity Pure simulation Simulation cradles Virtual/Physical testbeds Field experiments Live networks Test and evaluation options Can we develop tools to span this space?
  26. 26. Some ns-3 design goals <ul><li>Realism </li></ul><ul><li>Software reuse </li></ul><ul><li>Emulation </li></ul><ul><li>Composition </li></ul>
  27. 27. 1) Attention to realism <ul><li>Research should often involve a mix of simulations and testbed or live experiments </li></ul><ul><li>If the simulator cannot be made to closely model a real system: </li></ul><ul><ul><li>hard to compare results or validate the model </li></ul></ul><ul><ul><li>hard to reuse software between the two domains </li></ul></ul>When in doubt, do what implementations do
  28. 28. Attention to realism (example) <ul><li>An ns-3 Node is a husk of a computer to which applications, stacks, and NICs are added </li></ul>Application Application Application “ DTN”
  29. 29. Realism (cont.) <ul><li>Align ns-3 to be more faithful representation of real computers </li></ul><ul><ul><li>sockets API </li></ul></ul><ul><ul><li>packets are “packed” binary representations ready for direct serialization to/from network devices </li></ul></ul><ul><ul><li>alignment with Linux architecture </li></ul></ul><ul><ul><li>multiple interfaces handled properly </li></ul></ul>
  30. 30. ns-3 follows a Linux architecture <ul><li>Linux corollaries </li></ul><ul><ul><li>each NetDevice is modelled like struct net_device </li></ul></ul><ul><ul><li>each layer-3 protocol has a struct in_device component for each NetDevice </li></ul></ul><ul><ul><li>this interface (transmit side) is aligned with Linux dev_queue_xmit() </li></ul></ul><ul><ul><li>on receive side, demux is modelled after Linux protocol handlers </li></ul></ul>class NetDevice class Ipv4Protocol
  31. 31. 2) software integration <ul><li>Do not reimplement models and tools for which open-source implementations abound </li></ul><ul><li>ns-3 conforms to standard input/output formats so that other tools can be reused. </li></ul><ul><ul><li>e.g., pcap trace output, ns-2 mobility scripts </li></ul></ul><ul><li>ns-3 is adding support for running implementation code </li></ul><ul><ul><li>Network Simulation Cradle integration has met with success: Linux TCP code </li></ul></ul><ul><ul><li>ns-3 “process” API </li></ul></ul>
  32. 32. ns-3 simulation cradle <ul><li>Port by Florian Westphal of Sam Jansen’s Ph.D. work </li></ul>Figure reference: S. Jansen, Performance, validation and testing with the Network Simulation Cradle. MASCOTS 2006.
  33. 33. ns-3 simulation cradle <ul><li>For ns-3: </li></ul><ul><li>Linux 2.6.18 </li></ul><ul><li>Linux 2.6.26 </li></ul><ul><li>Linux 2.6.28 </li></ul><ul><li>Others: </li></ul><ul><li>FreeBSD 5 </li></ul><ul><li>lwip 1.3 </li></ul><ul><li>OpenBSD 3 </li></ul><ul><li>Other simulators: </li></ul><ul><li>ns-2 </li></ul><ul><li>OmNET++ </li></ul>Figure reference: S. Jansen, Performance, validation and testing with the Network Simulation Cradle. MASCOTS 2006.
  34. 34. ns-3 “processes” and Posix API <ul><li>Support for a synchronous Posix socket API </li></ul><ul><li>each Posix type and function is redefined in the simulator </li></ul><ul><li>processes get their own private stack </li></ul><ul><ul><li>somewhat like a lightweight virtual machine </li></ul></ul><ul><li>Benefits: </li></ul><ul><ul><li>makes porting real world application code much easier </li></ul></ul><ul><ul><li>makes writing applications easier because the BSD socket API is faithfully followed </li></ul></ul><ul><li>see the “mathieu/ns-3-simu” code repository </li></ul>
  35. 35. 3) emulation support <ul><li>Support moving between simulation and testbeds or live systems </li></ul><ul><li>A real-time scheduler, and support for two modes of emulation </li></ul>
  36. 36. ns-3 emulation modes ns-3 1) ns-3 interconnects real or virtual machines real machine virtual machine virtual machine real machine ns-3 Testbed real machine ns-3 2) testbeds interconnect ns-3 stacks Various hybrids of the above are possible
  37. 37. Example: ORBIT and ns-3 <ul><li>Support for use of Rutgers WINLAB ORBIT radio grid </li></ul>
  38. 38. 4) Composition ns-3 is a component of an overall workflow <ul><li>Define ns-3 with </li></ul><ul><li>interfaces to support </li></ul><ul><li>integration with other </li></ul><ul><li>tools </li></ul><ul><li>Topology generators </li></ul><ul><li>Traffic generators </li></ul><ul><li>Data analysis frameworks </li></ul><ul><li>Configuration tools </li></ul><ul><li>Animators </li></ul>
  39. 39. Common Open Research Emulator <ul><li>Scalable Network Emulator </li></ul><ul><li>Network lab “in a box” </li></ul><ul><ul><li>Efficient and scalable </li></ul></ul><ul><ul><li>Easy-to-use GUI canvas </li></ul></ul><ul><li>Kernel-level networking efficiency </li></ul><ul><ul><li>Reference passing packet sending </li></ul></ul><ul><li>Runs real binary code </li></ul><ul><ul><li>No need to modify applications </li></ul></ul><ul><li>Connects with real networks </li></ul><ul><ul><li>Hardware-in-the-loop </li></ul></ul><ul><ul><li>Distributed - runs on multiple servers </li></ul></ul><ul><ul><li>Virtual nodes process real packets </li></ul></ul><ul><li>Fork of the IMUNES project </li></ul><ul><ul><li>University of Zagreb </li></ul></ul><ul><li>Open Source </li></ul><ul><ul><li> </li></ul></ul>
  40. 40. netgraph system Tcl/Tk GUI core_wlan core_span CORE API ng_wlan NIC FreeBSD kernel virtual images (vimages) tunnels userspace Overview of CORE Components Three key components 1. virtual machines 3. graphical user interface 2. networking subsystem
  41. 41. CORE features <ul><li>IMUNES base: </li></ul><ul><li>Flexible GUI-based management of FreeBSD lightweight virtual containers </li></ul><ul><li>Virtual topology can be hooked to real devices </li></ul><ul><li>CORE extensions: </li></ul><ul><li>GUI enhancements, kernel enhancements, wireless networking, Linux support (OpenVz and NetEm), distributed emulation, and packaging. </li></ul><ul><li>Software leads: Jeff Ahrenholz and Claudiu Danilov, Boeing </li></ul><ul><li>http:// /work/core </li></ul>
  42. 42. CORE and ns-3 integration ns-3 program
  43. 43. ns-3 and research alignment <ul><li>In summary, make it easier to move from simulation to emulation to experiments </li></ul><ul><li>Realism and alignment with popular interfaces </li></ul><ul><li>Support use of real code </li></ul><ul><li>Emulation capabilities </li></ul><ul><li>Combine ns-3 with other tools </li></ul>
  44. 44. Challenges for ns (and simulators)‏ <ul><li>Align with how research is now conducted </li></ul><ul><li>Improve credibility </li></ul>
  45. 45. Background on simulation credibility <ul><li>[1] “Why We STILL Don’t Know How to Simulate Networks” </li></ul><ul><ul><li>Mostafa Ammar, Georgia Institute of Technology, Annual Simulation Symposium 2005 </li></ul></ul><ul><li>[2] “Maintaining a Critical Attitude Towards Simulation Results” </li></ul><ul><ul><li>Sally Floyd, WNS2 Workshop Keynote, October 2006 </li></ul></ul><ul><li>[3] “MANET Simulation Studies: The Incredibles” </li></ul><ul><ul><li>Kurkowski, Camp, and Colagrosso, ACM Sigmobile, MC2R, Volume 9, Issue 4, October 2005 </li></ul></ul><ul><li>[4] “An Integrated Approach to Evaluating Simulation Credibility” </li></ul><ul><ul><li>Muessig, Laack, and Wrobleski, U.S. Naval Air Warfare Center, August 2001 </li></ul></ul>
  46. 46. Criteria for Credibility <ul><li>Repeatable </li></ul><ul><li>Unbiased </li></ul><ul><li>Realistic Scenarios </li></ul><ul><li>Statistically Sound </li></ul><ul><li>Model Accuracy </li></ul><ul><li>Results Accuracy (Validation) </li></ul><ul><li>Data Accuracy </li></ul><ul><li>Usability </li></ul>from [3] from [4]
  47. 47. Repeatability <ul><li>A hallmark of the scientific method... </li></ul><ul><li>Papers should identify simulator, version, operating system, parameters, etc. </li></ul><ul><li>Better yet, make code and configuration scripts available to the community </li></ul><ul><ul><li>Yet, 0 out of 84 ACM Mobihoc MANET simulation papers (2000-2004) referenced publicly available code (from [3]) </li></ul></ul>
  48. 48. ns-3 approach <ul><li>We are adding support to systematically document how simulations were conducted </li></ul><ul><ul><li>A configuration subsystem that dumps simulation parameters to a configuration file (input and output) </li></ul></ul><ul><li>We will host code, publications, validation results, etc. </li></ul>
  49. 49. Model accuracy <ul><li>“error-free-ness” of software and models </li></ul><ul><li>ns-3 goals here: </li></ul><ul><ul><li>Support real code where possible </li></ul></ul><ul><ul><li>Open source models </li></ul></ul><ul><ul><li>Maintainers </li></ul></ul>
  50. 50. Open source models <ul><li>“ Given enough eyeballs, all bugs are shallow” </li></ul><ul><ul><li>Eric Raymond, “The Cathedral and the Bazaar” </li></ul></ul><ul><li>ns-3 needs ways to certify models, too </li></ul><ul><ul><li>capture level of community acceptance </li></ul></ul><ul><ul><li>publication lists, cross-reference </li></ul></ul><ul><ul><li>need to identify maintainers, or state the absence of a maintainer </li></ul></ul><ul><ul><li>validation techniques and results </li></ul></ul>
  51. 51. Example: ns-3 Wifi development <ul><li>Several research groups are maturing the original INRIA model: </li></ul><ul><li>Karlsruhe Institute of Technology: 802.11 PHY, 802.11e </li></ul><ul><ul><li>Equalizing PHY models including capture effects, user-definable coding rates (e.g. 5.9 GHz from 802.11p), EDCA QoS extensions of 802.11e, Nakagami/Rayleigh propagation loss model </li></ul></ul><ul><li>University of Florence: 802.11n features </li></ul><ul><ul><li>Frame Aggregation, Block ACK, HCF (EDCA and support for HCCA),TXOP, HT terminal (also with protection modes), MIMO </li></ul></ul><ul><li>Russian Academy of Sciences: 802.11s </li></ul><ul><ul><li>a complete model of IEEE802.11s D2.0 Draft Standard </li></ul></ul><ul><li>Deutsche Telekom Laboratories in Berlin: 802.11 PHY </li></ul><ul><li>Boeing: 802.11b channel models, validation </li></ul><ul><li>(and others...) </li></ul>
  52. 52. Pledge break <ul><li>ns-3 needs participation from the research community </li></ul><ul><ul><li>1) improving simulation credibility </li></ul></ul><ul><ul><li>2) contributed and supported models </li></ul></ul><ul><ul><li>3) maintainers </li></ul></ul><ul><li>Please support your open source projects! </li></ul>
  53. 53. Summary of simulation credibility <ul><li>Learn from good and bad examples of simulation research, to produce credible simulations </li></ul><ul><li>Consider open source (or publishing of models and scripts) to be integral part of your research </li></ul><ul><li>Please give back to the simulators that you use </li></ul>
  54. 54. Acknowledgments <ul><li>Thanks to: </li></ul><ul><li>the core development team and research project leads </li></ul><ul><ul><li>Raj Bhattacharjea, Gustavo Carneiro, Walid Dabbous, Craig Dowell, Joe Kopena, Mathieu Lacage (software lead), George Riley, Sumit Roy </li></ul></ul><ul><li>2008 Google Summer of Code mentors and students </li></ul><ul><li>many code authors and testers </li></ul><ul><li>the ns-2 PIs and developers for creating ns-2 and for supporting ns-3 activities </li></ul><ul><li>USC ISI for hosting project mailing lists </li></ul><ul><li>Contact: (office M354) </li></ul>