This document discusses the challenges of distributed simulation. It describes how distributed simulation links together systems from separate purposes, technologies, vendors, and platforms. It outlines the types of simulations as live, virtual, and constructive. It then details the Distributed Interactive Simulation standard and architecture, including autonomous nodes, transmission of information, and protocol data units. It concludes by discussing some persistent issues in achieving correlation between simulations.

1. Challenges of Distributed
Simulation
CETES Tech Night
January 18, 2011
Presented by Bill Millspaugh

2. Challenges of Distributed
Simulation
Presented by Bill Millspaugh
bmillspaugh@tecmasters.com
580-284-4427

3. Goal of DIS is to link:
• Systems built for separate purposes
• Technologies from different eras
• Products from separate vendors
• Platforms from various services
• Human in the loop simulators
• Live entities

4. Types of Simulations
Live - Operations with real
equipment in the field
Virtual - Systems & troops in
simulators fighting on
synthetic battlefields
Constructive - Wargames, models,
analytic tools

5. Distributed Interactive Simulation (DIS)
• Distributed • Interactive
– Physically separated – Simulated players interact
– Multiple simulators with an exercise
– Dissimilar devices – Players affect each other
• Shoot
– Multiple computers
• Kill
• Collide
• Communicate
– Interact with environment
• Terrain
• Culture
• Atmosphere
• Water & Ocean

6. DIS Supporting Infrastructure
• Interface standards
• Communication architectures
• Technical forums
• Management structures

7. Architecture Characteristics
• Autonomous simulation nodes
• Transmission of “ground truth” information
• Transmission of state change information
• Object/event simulation architecture
• Use of “dead reckoning” algorithms to
extrapolate entity states

8. Autonomous Nodes
• Events are broadcast, available to all
• Receiving nodes are responsible for calculating
the effects of an event on the objects it is
simulating
• Effects may include the generation of new
events
• Nodes can join or leave an exercise in progress
• Initiating nodes need not calculate what effects
or who is affected by their event/stimulus

9. Transmission of “Ground Truth”
• Each node transmits the absolute truth about the
state of the object(s) it represents
• Receiving nodes are sole responsible for
determining whether their objects can perceive
an event and whether they are affected by it
• Degradation of information is performed by the
receiving node in accordance with an
appropriate model of sensor characteristics
before it is presented to human crew members
or automated crews

10. Object/Event Architecture
• Information about non-changing objects in
the virtual world is assumed to be known
to all simulations and need not be
transmitted
• Dynamic objects keep each other informed
of their movements and the events that
they cause through the transmission of
Protocol Data Units (PDUs)

11. Common Environment
• Each simulation creates and maintains its
own copy of the common environment
• Terrain
• Coordinate System
• Infrastructure (roads, bridges, buildings…)
• Weather
• Dead reckoning
• Effectiveness calculations

12. Information Exchanged
• Simulation management
• Entity state information
• Fire events
• Detonation events
• Radio communication events
• Electronic emissions

13. Simulation Communications
• Information is exchanged over the network in
packets called Protocol Data Units (PDUs)
• These formatted data units are exchanged
between networked simulations to convey
messages about entities and events
• PDUs provide data concerning simulated entity
states, the types of entity interactions that take
place in a DIS exercise, and they provide data
for the management and control of a DIS
exercise.

14. PDU Families
• Entity information interaction
• Warfare
• Logistics
• Simulation management
• Distributed emission regeneration
• Radio communications
• Entity management
• Minefield
• Synthetic environment
• Simulation management with reliability
• Live entity
• Non-real time protocol

15. Information in a PDU
• Example: Entity State
– Header: Protocol version, Exercise ID, Type
PDU, Time stamp, PDU length
– Body: Description of entity, Location, Velocity,
Orientation, Dead reckoning used,
Appearance, Description of articulated parts
and capabilities of the entity

16. Entity State PDU

17. Entity State PDU (cont)

18. Example Entity Enumerations
kind dom country cat subcat spec extra description
1 1 45 5 6 0 0 China_107mm_RKT
1 1 45 10 4 0 0 OPFOR_60mmMTR_TY31
1 1 78 2 6 0 0 Recon_Vehicle_HJ62C_Armored
1 1 222 0 1 11 0 Truck_Bridge_PMP_Engineer
1 1 222 1 2 7 0 OPFOR_T72BK
1 1 222 10 13 0 0 Mortar_2S12_120mm_Towed
1 1 222 28 4 2 2 ADA_Launcher_SA10
1 1 222 28 4 3 2 Radar_ADA_FLAPLID
1 1 225 2 1 8 0 M2A3_FIST
1 1 225 2 5 2 0 LAV_25_Recon
1 1 225 2 5 3 0 LAV_AT
1 1 225 2 5 6 0 US_HMMWV_PROPHET
1 1 225 2 5 28 0 US_Stryker_FIST
1 1 225 2 5 29 0 US_Stryker_Eng
1 1 225 2 5 31 0 US_Stryker_ATGM_Vehicle

19. Example Ammunition Enumerations
kind dom country cat subcat spec description
2 1 106 1 2 0 Aspide: HE-Frag
2 1 206 2 1 4 35mm gun: AHEAD
2 1 222 1 19 2 SA-7
2 1 222 1 30 0 SA-18
2 1 222 1 31 0 SA-19
2 1 225 1 2 3 AIM-120C (AMRAAM)
2 1 225 1 15 3 FIM-92 Stinger RMP
2 1 225 1 16 1 THAAD MISSILE
2 2 222 2 2 2 30MM/APDS
2 2 222 2 2 2 30mm APDS
2 2 222 2 2 4 30MM/HE-FRAG
2 2 222 2 2 4 30mm FRAG-HE
2 2 222 2 2 4 30mm HEI-T
2 2 225 1 2 4 25mm Bushmaster II
2 9 225 1 20 4 GBU-24

20. World Coordinate System

21. Entity Coordinate System

22. Euler Angles
• Entity Location is specified as the position
of the origin of the entity coordinate
system in world coordinates
• Orientation is specified using three angles
that describe successive rotations required
to transform from the world coordinate
system to entity coordinate system.
• These angles are called Euler angles.

23. First Rotation

24. Second Rotation

25. Third Rotation

26. Coordinate Transformations
• World coordinates Latitude/Longitude
• Latitude/Longitude Universal Transverse
Mercator (UTM)
• Latitude/Longitude Military Grid Reference
System (MGRS)
• UTM Game Coordinates

27. Dead Reckoning
= DR Position
= True Position
= True/DR Coincide
Veh A
Veh B
Initial Constant True & DR
state course & diverge, Threshold A gets msg,
speed threshold exceeded, corrects his
not B sends position of B,
exceeded update starts new DR
Notes: Scheme also applies to orientation and position
of articulated parts.
Smoothing algorithms remove jerky motion.

28. DIS Standards
• IEEE Std 1278.1-1995 (R2002) –Application Protocols
• IEEE Std 1278.1a – Supplement to IEEE Std 1278.1-
1995
• IEEE Std 1278.2 1995 (R2002) – Communication
Services and Profiles
• IEEE Std 1278.3-1996 (R2002) – Exercise Management
and Feedback
• IEEE Std 1278.4-1997 (R2002) – Verification, Validation
and Accreditation
• Simulation Interoperability Standards Organization
(SISO) Ref-010-2006 – Enumeration and Bit Encoded
Values for use with Protocols for Distributed Interactive
Simulation Applications (Accompanies 1278.1 and
1278.1a)

29. Some Distributed Simulation
Applications

30. Distributed Mission Operations
Simulator/Jet + DMO + Network
F-16
AWACS
B-52
F-15 F-15
F-15
F-15E F-16 F-15E
F-15
B-1
F-15E
F-16
F-16
RJ
F-16
RQ-1 F-16
F-15 F-22
F-15
B-2
F-16
F-16
A-10 F-15E
F-16 F-16
32 AOC
F-16
AWACS
F-16
JSTARS
B-52
F-117
A-10
F-16 B-1
F-15
F-15
F-15
F-16
F-15
AWACS
F-16
OPERATIONAL FUNDED LIMITED DMT UNFUNDED
Link Geographically Separated
Live, Virtual, and Constructive Entities
In Shared Joint & Coalition
Synthetic Theater Environments

32. Functional Concept Integrating Experiment 2010 (FCIE 10)
Colorado Springs, CO (SMDC)
Ft Leonard Wood, MO
Langley AFB, VA
Ft Monroe, VA
Ft Bragg GA
Ft Gordon, GA
Ft Benning, GA
Huntsville, AL
Ft Sill, OK Hurlburt, FL
Ft Rucker, AL

33. FCIE 10 MODELS & SIMULATIONS
MBL – Fort Benning USAF - Hurlburt Fires BL - Fort Sill AMBL - Fort Rucker
Models, Tools and Systems (505th) Models, Tools and Systems
OneSAF Models, Tools and Systems Models, Tools and Systems
Fires Simulation XXI (FireSim XXI)
FireSim Interface Air Warfare Simulation(AWSIM) Advanced Tactical Combat
Reconfigurable Tactical Operations Simulator
AMDWS Theater Battle Management Model (ATCOM)
(RTOS)
TAIS (10.*) Core System (TBMCS) NVT
Network Visualization Toolkit (NVT)
CPOF Joint Deep Operations OH58D Virtual Simulator
Advanced Field Artillery Tactical Data System
ForceXXI Battle Command, Coordination System(JADOCS) GCS-GUSS
(AFATDS)
Brigade and Below (FBCB2) AFSERS TAIS (10.*)
Air Missile Defense Workstation (AMDWS)
Representation: GCCS? Aviation Mission Planning
Tactical Airspace Integration System (TAIS 10.*)
Maneuver – IBCT,SBCT GIAC Station (AMPS)
Total Battle Space Awareness (TBSA)
Host Nation Trucks ACE-IOS CPOF
Joint Deep Operations Coordination System
Representation: FireSim Interface
(JADOCS)
Fixed Wing(SOF,USAF) Representation:
SIGCEN - Fort Gordon Tactical air Control Party Close Air Support
Marine Air Aviation Brigade
System (TACP-CAS)
Models, Tools and Systems Civilian Aircraft ADA - Radar
TBMCS (clients)
FCIE Portal Server RWA
Forward Area Air defense Engagement
CPOF UAS
Operations (FAAD-EO)
Scalable Network Technologies USAF - Langley Air Defense System Integration (ADSI(14.*))
RWA
Comm Effects Server (SNT-CES) Models, Tools and Systems Command Post of Future (CPOF)
Network Visualization Tool (NVT) Call Manager CPOF Server
Call Manager Adobe Connect Server Battle Command Server (BCS)
Adobe Connect Server OpenFire Extended Air Defense Simulation (EADSIM)
OpenFire Representation: Ground Based Laser (GBL) - Boeing BCBL-Leavenworth
FireSim Interface FireSimXXI Interface Models, Tools and Systems
OneSAF USN – Ft Sill Representation: CPOF
CPOF Server
Representation: Artillery
TCN Models, Tools and Systems
FireSim Interface MPAD
ESB
Navy Ship ADA – non-radar MSBL - Ft Leonard Wood
EMGUN Maneuver Models, Tools and Systems
SMDC - Redstone TBM & CM OneSAF
Models, Tools and Systems SOF – Ft Bragg Artillery ( +120mm Mortars)
UAV(Class 1 UAV &TUAV)
CPOF
EADSIM JBC2S
Models, Tools and Systems ADA BETSS-C
JEMS
FireSim Interface Hospital Elements, HQ Icon FireSim Interface
GCCS-A
CPOF Sustainment BDE Elements Representation:
FireSimXXI Interface
Representation: Host Nation BDE Elements? MEB
Representation:
SOF Civilians Civilians
National Assets
National ISR SMDC-BL USMC - Ft Sill Maneuver
Comms & Comms Relay Models, Tools and Systems
Platforms Colorado Springs FireSim Interface
Models, Tools and Systems
MEU
FireSimXXI Interface

34. Persistent Major Issues

35. Issues
• Lack of correlation of common environment
• Terrain data correlation
– Multiple terrain database architectures (Open Flight, Objective
One SAF, Compact Terrain Data Base, Evans and
Southerland,…)
– Both visual and underlying simulation databases required
– Visual databases differ from simulation to simulation
– Trees and vegetation are added differently for visualization and
simulation
– Lack of models for calculating blast damage to buildings and
terrain
– Visualization of building damage is very primitive
– Need models to generate rubbling of buildings
– No mechanism/standard exists to communicate terrain and
infrastructure damage in one simulation to others in the network
– Micro terrain is implementation dependent

36. Issues (cont)
• Terrain database generation is labor intensive
• Weather is ad hoc
• Each simulation handles smoke and dust differently
• Visual dust and smoke effects are probably not
correlated to internal smoke and dust simulated effects
• Need automated voice recognition in high noise
environments
• Need natural language understanding
• Need artificial intelligence to control computer generated
forces
• Tactical messaging to simulation units for command and
control (tactical messages are difficult to decode and
keep changing)

37. Issues (cont)
• Need good crowd modeling
• Need open source stealth viewing devices
for virtual environment
• Need good interface control documents
with exercise control information and
enumeration databases (Good exercises
are a result of good management)

38. Skills Needed for Modeling and
Simulation in a Distributed
Environment

39. Skills Required
• Mathematics
– Calculus
– Matrix algebra
– Vector mathematics
– Geometry/trigonometry
– Statistics
• Random sampling techniques
• Random number generation

40. Skills (cont)
• Physics/mechanics/dynamics
– Light
– Optics
– Momentum
– Kinetic energy
– Projectile equations

41. Skills (cont)
• Computer science
– Object oriented design/programming
– Sorting
– Searching
– Hashing
– Data/information structures
• Lists
– Stacks, queues
– Single/doubly linked lists
• Trees
– Binary tree representation
– Traversing binary trees
– M-way trees
• Multilinked structures
• Dynamic storage allocation

42. Skills (cont)
• Networking
– General network troubleshooting
– Switch programming capabilities
– Router programming capabilities
– Understanding of IPv4
• Subnetting
• How broadcast/unicast/multicast networks work
• Gateway and routing configuration
• DNS standards

43. Questions?