While developing autonomous navigation and space robotics technologies, we have realized that all the currently available tools for training and testing autonomous space missions are lacking. Thus, our team has started developing the Mission Design Simulator (MDS) to enable rapid prototyping without the need to use multiple software packages, generate synthetic data for training DL models and CV algorithms, and provide a standardized framework for robotic mission design and testing. Implementing such a tool created some unique challenges. Most notably, we have chosen to design a framework that was able to run in real-time, maintain high accuracy, and be capable of generating high fidelity images that are useful for AI models and CV algorithms. These conflicting objectives led to a delicate balancing act between performance and accuracy. In this talk, our project lead Arūnas will discuss our approach to addressing this problem, including modular architecture, multiple physics calculation layers, and other techniques that we have used. We will also present the current state of our tools and potential areas for further development.