• Save

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Like this? Share it with your network

Share

A Methodology to Predict the Performance of Distributed Simulation Systems

on

  • 338 views

Conference Presentation at the 2010 PADS (Principles of Advancement in Distributed Simulation)

Conference Presentation at the 2010 PADS (Principles of Advancement in Distributed Simulation)

Statistics

Views

Total Views
338
Views on SlideShare
338
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

A Methodology to Predict the Performance of Distributed Simulation Systems Presentation Transcript

  • 1. A Methodology to Predict the Performance of Distributed Simulation Systems Daniele Gianni1,2, Giuseppe Iazeolla2, and Andrea D’Ambrogio2 daniele.gianni@esa.int 1European Space Agency {iazeolla, dambro}@info.uniroma2.it 2Dept. of Computer Science University of Rome TorVergata24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 1
  • 2. Methodology Objective A local simulator is available We wish to turn it into a distributed simulator Before implementing it (i.e. at design time) we wonder: will its execution time be shorter that local simulator execution time? (In some cases the distributed version may run slower than the equivalent local version) Methodology objective: predict (at design time) distributed simulator execution time If prediction meets performance requirements, then implement it24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 2
  • 3. Presentation Overview Terminology Problem Statement Methodology Presentation • Modelling Assumptions / Execution Graph / Performance Model / Model Parameterization Case of Study • Simulated System / OMNET++ Simulator / Model Prediction / Model Validation 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 3
  • 4. Terminology  = Simulated System SL() =  Local Simulator (LS) SD() =  Distributed Simulator (DS) PM(SD()) = Performance Model to predict SD() execution time 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 4
  • 5. Problem Statement We want to simulate   presents intrinsic parallelism Will SD() be faster than SL()? 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 5
  • 6. Predictive Evaluation Process build SL(Σ) YES SL(Σ) Stop exec time OK? NO identify tentative partitioning build and evaluate prediction model PM(SD(Σ)) pred exec time NO alternative YES SD(Σ) partitioning? OK? YES NO build Stop SD(Σ)24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 6
  • 7. Modelling Assumptions Communication between federates is based on a decentralized RTI Federation time management is conservative Hosts’ CPUs and communication networks are the only machinery devices affecting the federation execution We consider SD() consisting of 2 federates (however the model can be immediately generalized to n federates) 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 7
  • 8. Model Building Identification of machinery devices and their interconnections Derivation of the Execution Graph (an extended flow chart whose blocks are associated to device time requests) Derivation of the Performance Model (Extended Queueing Network) Model parameterization based on data from the RTI implementation and from the local simulator 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 8
  • 9. Execution Graph START Start RTI Interface SN0 C0 Federate1 Federate2 Fork C1S C2S Local Local Initialization Initialization Execution (LEXI) Execution (LEXI) HLA HLA Initialization Initialization C1S C2S C1RL C2RL S1 S2 C1 C1 C2 C2 B1 Local Local Execution SM1 Execution SM1 AND AND (LEX) p1NSYNC (LEX) p2NSYNC SM4 PoTMT1 SM4 PoTMT2 HLA RTI HLA RTI p1RTI Semaphore p2RTI Semaphore Exit Service MT1 Exit Service MT2 Test Execution p1SYNC Test Execution p2SYNC (HLAR) (HLAR) p1QUIT p2QUIT C1RL C2RL B2S2 HLA HLA Execution Execution AND SC AND SC Wait C1RC2 Wait C2RC1 (HLAF-Wait) (HLAF-Wait) SM1 SM1 SM2 SM2 HLA Execution HLA Execution From Federate 2s SM4 From Federate 1s SM4 Exit B2S1 Exit (HLAF-Ex) (HLAF-Ex) SM3 SM3 LEW LEW MT1 PoTMT1 MT2 PoTMT2 p1SYNC p2SYNC Release Release Release Release Test Semaphore Test Semaphore MT1 MT2 1 – p1SYNC 1 – p2SYNC Join END B2 B2S3 END SM0 C0 LS Segments END HLA Segments24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 9
  • 10. Federate 1 Execution Graph (B1) C1 C1 B1 Local Execution SM1 AND (LEX) p1NSYNC SM4 PoTMT1 HLA RTI p1RTI Semaphore Exit Service MT1 Test Execution p1SYNC (HLAR) p1QUIT LS Segments Join HLA Segments C0 END24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 10
  • 11. Federate 1 Execution Graph (B2) C1RL p1QUIT C1RL B2S2 HLA Execution AND SC Wait C1RC2 (HLAF-Wait) SM1 SM2 HLA Execution From Federate 2s SM4 Exit B 2 S1 (HLAF-Ex) SM3 LEW MT1 PoTMT1 p1SYNC Release Release Test Semaphore MT1 1 – p1SYNC B2S3 END B2 LS Segments HLA Segments24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 11
  • 12. Performance Model SN4 RTIAmbassador services invocation P2QUIT SN1 C2 CHLA2 C2 C2 CPU2 SA2 Net21 R21 Sink C2 S2 C2RL1 C2RC1 C2 PoT12 PoT21 FederateAmbassador P2SYNC-FA C2RC1 p2SYNC-FA SN1 services exit point 1-p2SYNC-FA SC21 C1RL2 A12 1-p2SYNC-FA PoTMT2 PoTMT2 RMT2 SN2 AMT2 FederateAmbassador services entry point C2RC1 Sink SN3 SN0 SN0 Federate 2 C0 C0 Sink Source Fork Join Simulation SN4 RTIAmbassador services invocationSystem start-up SN1 p1QUIT C1 CHLA1 command C1 C1(initialization phase not modelled) CPU1 SA1 Net12 R12 Sink S1 C1RL2 C1RC2 C1 C1 PoT21 PoT12 FederateAmbassador p1SYNC-FA C1RC2 p1SYNC-FA SN1 services exit point 1-p1SYNC-FA SC12 C1RL2 A21 1-p1SYNC-FA PoTMT1 PoTMT1 RMT1 SN2 AMT1 FederateAmbassador services entry point C1RC2 Sink SN3 Federate 1 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 12
  • 13. Performance Model - SN0 Federate 2 SN0 SN0 C0 C0 Sink Source Fork Join Simulation SN4 RTIAmbassador services invocationSystem start-up SN1 p1QUIT C1 CHLA1 command C1 C1(initialization phase not modelled) CPU1 SA1 Net12 R12 Sink S1 C1RL2 C1RC2 C1 C1 PoT21 PoT12 FederateAmbassador p1SYNC-FA C1RC2 p1SYNC-FA SN1 services exit point 1-p1SYNC-FA SC12 C1RL2 A21 1-p1SYNC-FA PoTMT1 PoTMT1 RMT1 SN2 AMT1 FederateAmbassador services entry point C1RC2 Sink SN3 Federate 1 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 13
  • 14. Performance Model - Federate 1 SN1 SN4 RTIAmbassador services invocation SN1 p1QUIT C1 CHLA1 C1 C1 CPU1 SA1 Net12 R12 Sink S1 C1RL2 C1RC2C1 C1 PoT21 PoT12 FederateAmbassador p1SYNC-FA C1RC2 p1SYNC-FA SN1 services exit point 1-p1SYNC-FA SC12 C1RL2 A21 1-p1SYNC-FA PoTMT1 PoTMT1 RMT1 SN2 AMT1 FederateAmbassador services entry point C1RC2 Sink SN3 Federate 1 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 14
  • 15. Performance Model - Federate 1 SN2 SN4 RTIAmbassador services invocation SN1 p1QUIT C1 CHLA1 C1 C1 CPU1 SA1 Net12 R12 Sink S1 C1RL2 C1RC2C1 C1 PoT21 PoT12 FederateAmbassador p1SYNC-FA C1RC2 p1SYNC-FA SN1 services exit point 1-p1SYNC-FA SC12 C1RL2 A21 1-p1SYNC-FA PoTMT1 PoTMT1 RMT1 SN2 AMT1 FederateAmbassador services entry point C1RC2 Sink SN3 Federate 1 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 15
  • 16. Performance Model - Federate 1 SN3 SN4 RTIAmbassador services invocation SN1 p1QUIT C1 CHLA1 C1 C1 CPU1 SA1 Net12 R12 Sink S1 C1RL2 C1RC2C1 C1 PoT21 PoT12 FederateAmbassador p1SYNC-FA C1RC2 p1SYNC-FA SN1 services exit point 1-p1SYNC-FA SC12 C1RL2 A21 1-p1SYNC-FA PoTMT1 PoTMT1 RMT1 SN2 AMT1 FederateAmbassador services entry point C1RC2 Sink SN3 Federate 1 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 16
  • 17. Performance Model - Federate 1 SN4 SN4 RTIAmbassador services invocation SN1 p1QUIT C1 CHLA1 C1 C1 CPU1 SA1 Net12 R12 Sink S1 C1RL2 C1RC2C1 C1 PoT21 PoT12 FederateAmbassador p1SYNC-FA C1RC2 p1SYNC-FA SN1 services exit point 1-p1SYNC-FA SC12 C1RL2 A21 1-p1SYNC-FA PoTMT1 PoTMT1 RMT1 SN2 AMT1 FederateAmbassador services entry point C1RC2 Sink SN3 Federate 1 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 17
  • 18. Performance Model – Job Flow SN4 RTIAmbassador services invocation P2QUIT SN1 C2 CHLA2 C2 C2 CPU2 SA2 Net21 R21 Sink C2 S2 C2RL1 C2RC1 C2 PoT12 PoT21 FederateAmbassador P2SYNC-FA C2RC1 p2SYNC-FA SN1 services exit point 1-p2SYNC-FA SC21 C1RL2 A12 1-p2SYNC-FA PoTMT2 PoTMT2 RMT2 SN2 AMT2 FederateAmbassador services entry point C2RC1 Sink SN3 SN0 SN0 Federate 2 C0 C0 Sink Source Fork Join Simulation SN4 RTIAmbassador services invocationSystem start-up SN1 p1QUIT C1 CHLA1 command C1 C1(initialization phase not modelled) CPU1 SA1 Net12 R12 Sink S1 C1RL2 C1RC2 C1 C1 PoT21 PoT12 FederateAmbassador p1SYNC-FA C1RC2 p1SYNC-FA SN1 services exit point 1-p1SYNC-FA SC12 C1RL2 A21 1-p1SYNC-FA PoTMT1 PoTMT1 RMT1 SN2 AMT1 FederateAmbassador services entry point C1RC2 Sink SN3 Federate 1 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 18
  • 19. Model Parameterization Procedure to determine model parameters from SL() and RTI (as instance of simulation infrastructure) From EG, performance engineering standard procedures can derive: • For each class of jobs • Service times at each center (i.e.: tCPU1, tCPU2, tNet12, and tNet21) • Routing probabilities (i.e.: p1QUIT, p2QUIT, p1SYNC, and p2SYNC) 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 19
  • 20. Case of Study Simulated System (LS)24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 20
  • 21. Case of Study DS partitioning24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 21
  • 22. Case of Study Hardware and SD() Configuration Hardware • Host 1 and Host 2 connected by a MAN • RTT = 20.5 ms • Bandwidth = 94 KB/s SD() consisting of 2 federates • Federate 1: simulates S1 and S2 and is run by Host 1 • Federate 2: simulates S3 and S4 and is run by Host 2 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 22
  • 23. Case of Study Problem Domain We consider two scenarios (A and B) Scenario Alight processing times to simulate S1 through S4, specifically DCPU = 10 ms wall-clock time to for each processing Scenario B:heavy processing times to simulate S1 through S4, specifically DCPU = 500 ms wall-clock time for each processing 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 23
  • 24. Case of Study Problem Statement Is SD() faster than SL() in Scenario A? Is SD() faster than SL() in Scenario B? 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 24
  • 25. Case of Study Design Question Scenario A: DCPU = 10 ms (light processing) Scenario B: DCPU = 500 ms (heavy processing) In scenario B, will the parallelization speed-up compensate the HLA overhead? I.e. is predicted SD() execution time shorter than SL() time? Can we give a YES answer in the Predictive Evaluation Process, and then proceed to implement SD()? 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 25
  • 26. OMNet++ Performance Model Implementation 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 26
  • 27. Model prediction Simulation Execution Time Scenario System (sec) SL() 5 A SD() 17 SL() 1400 B SD() 84624th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 27
  • 28. Model Validation The execution time prediction has been validated by: • implementing the actual DS system, and • measuring and comparing  the actual execution time with  the predicted execution time 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 28
  • 29. Model ValidationScenario SD() SD() predicted execution time actual execution time (sec) (sec) A 17 20 B 846 880 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 29
  • 30. Conclusions We have introduced a predictive methodology that given • a simulation model • a distributed simulation infrastructure • a model partitioning, and • a set of distributed hostsEnables simulator designers to predict the distributed simulator execution time 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 30
  • 31. Conclusions We have shown an example application to the evaluation of the distribution of a local simulator SL()over a metropolitan area network in the cases of • light event processing times (Scenario A) • heavy event processing times (Scenario B) 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 31
  • 32. Conclusions We have validated the execution time prediction by • implementing the actual DS simulator • measuring and comparing  the actual execution time with  the predicted execution time 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 32
  • 33. AcknowledgementsFIRB Projects “Performance Evaluation of ComplexSystems”, “Software frameworks and technologiesfor the development of open-source distributedsimulation code”, (Italian Ministry of Research andCERTIA Research Center of the University of RomaTorVergata); FP7 euHeart Project (EuropeanCommission); Research Fellowship of theEuropean Space Agency.Ylenia Cannone and Luca Marcheggiani for theirimplementation and validation of the OMNet++simulator. 24th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2010), May 17 – 19, 2010, Atlanta, GA, US 33