A small presentation about a recent research article titled, "Simulators as Drivers of Cutting Edge Research."

1. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
SIMULATORS AS DRIVERS OF CUTTING EDGE
RESEARCH
Muhammad Adil Raja
Roaming Researchers, Inc.
cbnd
February 7, 2016

2. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
OUTLINE I
1 MOTIVATION
2 INTRODUCTION
3 A RATIONALE FOR ADOPTING SIMULATORS
4 SIMULATORS FOR OTHER FEATS
5 HOW GOOD IS THE SIMULATED STUFF?
6 INNOVATION THROUGH SIMULATION
7 POPULAR SIMULATORS
Software Dovetailing Issues
8 APPLICATION INTEGRATION SOFTWARE
9 INTERACTIONS WITH FLIGHTGEAR
10 CONCLUSIONS
11 REFERENCES

3. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
MOTIVATION
Problems of research.
Bewildered early stage researchers.
Lack of appropriate problems.
How should we find and solve problems?
In order to conduct good quality research in various
scientific and academic disciplines, it is not only important,
rather inevitable to employ simulators.

4. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
INTRODUCTION
Why this presentation?
Why do people use simulators?

5. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
SIMULATORS AND ESCAPISM
Traditional motives for using simulators.
To avoid responsibility.
To avoid programming.
Doing a bunch of simulations could save a lot of time and
mental effort.

6. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
A RATIONALE FOR ADOPTING SIMULATORS I
Availability of advanced simulation software and
frameworks.
There are obvious advantages of them.
Traditionally, most of the engineering feats have required
specialized human experts.
An engineer spends years in learning a particular set of
skills.
The manufacturing process, whether automated or
requiring manual labour, has had to address all the
underpinnings of the successful development of the
technology.
The recent advances in computing technologies have
changed this.

7. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
A RATIONALE FOR ADOPTING SIMULATORS II
The arrival of cloud computing technologies, coupled with
advancements in computing hardware, have begun to
enable the realization of the numerous fantastic
engineering gadgets that were possible only theoretically
previously.

8. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
AN EXAMPLE FROM SOLAR ENERGY SYSTEMS
The solar energy panels were developed previously by
domain-experts.
A certain product specification which addressed the
electrical and physical properties of the panel to be
developed.
At the very best the engineer had access to a simulation
software that could be used to simulate a solar panel of
desired specifications.
Successful designing followed by fabrication.
Designing was a black-art.

9. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
A PARADIGM SHIFT
Current advances in computing technologies have
changed such trends.
Simulators can be designed and developed that allow the
target technology to be emulated to best match the target
specifications.
Power of the cloud is a driving force.
Augmenting a simulator with a machine learning (ML)
algorithm speeds up the design process.

10. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
A REFLECTION ON ML
A sub-field of artificial intelligence.
A computing machine is expected to learn to find solutions
to a user-specified problem all by itself with minimal human
intervention.
A machine is provided with a specification of the problem,
along with relevant data, and it is expected to find a
solution for the problem all by itself.

11. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
ADVANTAGES OF ML
1 The overall development life-cycle can be automated.
2 As the development life-cycle is mostly automated,
workable prototypes of desired products can be produced
with a lot more agility.
3 Despite producing counter-intuitive models, achieved
models are often a lot more efficient than what could have
been achieved by a hand-crafted counterpart.

12. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
LIMITATIONS OF ML
ML algorithms should work with simulators or emulators.
As ML algorithms make progress in finding an optimal
solution for the problem at hand, they do so by making
errors.
ML algorithms have to be allowed to make errors as they
proceed to find optimal solutions.
By making errors they tend to escape the regions of
sub-optimal solutions and move to more optimal regions of
the solution space.
It becomes prohibitive to employ them to real systems to
find solutions.
Computational requirements are also a concern!

13. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
AN EXAMPLE FROM VEHICLE ROUTING I
An ML algorithm can be used to address a vehicle routing
problem in which the algorithm trains a vehicle to
successfully traverse a path by fulfilling certain user
specified objectives.
In finding the optimal solution, a typical algorithm will
normally generate a large number of candidate solutions
randomly and improve them (or their successors)
iteratively over time or several generations.
Since the initial solutions are chosen randomly, they can be
expected to be erroneous.
The vehicle they govern will crash with a high probability.
It is only due to the guided nature of the underlying search
algorithm that solutions begin to improve after a certain
number of iterations.

14. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
AN EXAMPLE FROM VEHICLE ROUTING II
Eventually a routing pattern begins to emerge that can
lead the vehicles to the desired destinations successfully.
Now assume what would happen if such an algorithm was
run to address the vehicle routing problem using real
vehicles.
Clearly, as erroneous solutions are deliberately produced
and tested with real vehicles, quite a lot of them would end
up in catastrophic crashes.
This can make the whole enterprise of solving vehicle
routing problems unaffordable.

15. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
SIMULATORS FOR OTHER FEATS
The rationale for employing simulators can be considered
to hold for many other feats.
Consider the problem of discovering an optimal design for
a solar energy panel.
Clearly, one approach could be to design a new panel from
scratch by hand.
Such a design would naturally be a function of the whims
and caprices of the designer.
Another approach could be to employ a suitable ML
algorithm to do the job of design discovery.

16. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
HOW GOOD IS THE SIMULATED STUFF?
It all depends on how good are the simulators.
Accurate simulators shall develop accurate systems.
A vehicular simulation system should model everything as
perfectly as possible to mimic the real world.
Moreover, the environment of the vehicle should also be
modelled as perfectly as possible.

17. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
INNOVATION THROUGH SIMULATION I
Unmanned aerial vehicles.
Solar energy systems.
Social and ecosystem modelling.
Vehicle routing.

18. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
POPULAR SIMULATORS I
Flight:
FlightGear Craighead et al. (2007).
Microsoft flight simulator Williams (2006).
X-Plane Bittar et al. (2014)
Ecosystem:
Echo, Avida, Polyworld, BubbleWorld.Evo, Farmsticks and
Ecosim Gras et al. (2015).
Social:
Mason Luke et al. (2004).
Power:
UWPFLOW, AMES, THYME, TEFTS, DCOPFJ, minpower,
MatPower, OpenDSS, Dome, PSAT, MatDyn, 4DIAC,
InterPSS, PowerSystems, MatACDC, GridLAB-D,
OpenETran, OpenPMU, rapid61850, OpenlEC and
liblEC Comittee (2013).

19. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
POPULAR SIMULATORS II
Wind:
RETScreen, wind data generator (WDG) are used for
feasibility analysis.
QBlade and Vortexje are open-source simulators for turbine
design.
WindSim, ZephyTOOLS and Windie are for flow modelling.
openWind, WindFarmer, WindPRO, meteodyn WT and
WindSim are for wind farm modelling.
openWind and WindPRO are for farm visualization.
HiRLAM and GFS are for prediction Ximo (2012).
Solar:
RETScreen, PV F-Chart, SolarDesignTool, INSEL,
TRNSYS, NREL Solar Advisor Model, ESP-r 11.5,
PVSYST 4.33, SolarPro, PV DesignPro-G, PV*SOL Expert
and HOMER Lalwani et al. (2010).

20. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
SOFTWARE DOVETAILING ISSUES I
How to dovetail domain-specific simulators with various
machine learning algorithms.
It would be ideal to come up with a general application
integration framework that accepts a pair of a randomly
chosen simulator and an ML algorithm and dovetails them.
Integration should be such that the ML algorithm should be
able to invoke various objects in the simulator, perform
computations on them and adapt its states according to
the performance of those objects.
As various ML algorithms implement some sort of an
iterative mechanism, a natural requirement could be to
have a means of data transfer between the pair of
applications.

21. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
SOFTWARE DOVETAILING ISSUES II
Data transfer can be required to test the solutions
generated by the learning algorithm using the simulator.
So, a repetitive data transfer back and forth between the
two applications should be naturally sought.
There are other objectives a nice dovetailing application
should also address. For instance, it would be quite
worthwhile if the dovetailing application can exploit the
computing power of a distributed computing environment,
such as a cloud.
Such issues would have to be taken into account while
developing the dovetailing application, and should be
included in its software design issues to be addressed.
At present such dovetailing applications do not exist that
integrate a randomly chosen pair of a machine learning
algorithm and a domain-specific simulator.

22. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
APPLICATION INTEGRATION SOFTWARE
Apache Camel.
Talend ESB.
AnyPoint Studio By MuleSoft.
Matlab and Octave.

23. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
INTERACTIONS WITH FLIGHTGEAR
Implemented in C.
Wrapped in Nasal.
Comes with full-fledged Telnet and http servers.
Can be interfaced with Matlab: Matlab interface for
FlightGear.
Through Matlab it can be integrated with numerous ML
algorithms.
A wide variety of aircraft can be modelled.

24. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
CONCLUSIONS
This position paper argues about the viability of employing
simulators for cutting-edge research.
Recent advancements in computing systems are opening
endless possibilities to simulate real-world phenomena that
were traditionally only possible in human imagination and
thought experiments.
With the sophistication that is being achieved by simulation
frameworks, it is now more convenient to simulate various
types of problems.
The real world artefacts can be created afterwards, after
simulations have succeeded.
This is not only supposed to reduce production costs, but a
handful of other benefits are conceived and presented.

25. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
REFERENCES I
Bittar, A., de Oliveira, N. M. F., and de Figueiredo, H. V. (2014).
Hardware-in-the-loop simulation with x-plane of attitude
control of a suav exploring atmospheric conditions. Journal
of Intelligent & Robotic Systems, 73(1-4):271–287.
Comittee, I. P. (2013). Task force on open source software for
power systems. http://ewh.ieee.org/cmte/psace/
CAMS_taskforce/software.htm.
Craighead, J., Murphy, R., Burke, J., and Goldiez, B. (2007). A
survey of commercial & open source unmanned vehicle
simulators. In Robotics and Automation, 2007 IEEE
International Conference on, pages 852–857. IEEE.
Gras, R., Golestani, A., Hendry, A. P., and Cristescu, M. E.
(2015). Speciation without pre-defined fitness functions.
PLoS ONE, 10.

26. Motivation Introduction A Rationale for Adopting Simulators Simulators for Other Feats How Good is the Simulated Stu
REFERENCES II
Lalwani, M., Kothari, D., and Singh, M. (2010). Investigation of
solar photovoltaic simulation softwares. International journal
of applied engineering research, 1(3):585–601.
Luke, S., Cioffi-Revilla, C., Panait, L., and Sullivan, K. (2004).
Mason: A new multi-agent simulation toolkit. In Proceedings
of the 2004 swarmfest workshop, volume 8, page 44.
Williams, B. (2006). Microsoft Flight Simulator as a training aid:
a guide for pilots, instructors, and virtual aviators. Aviation
Supplies & Academics.
Ximo, C. (2012). Wind Energy Software. Ject Press.