Vs games2012


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Vs games2012

  1. 1. Development of a mechanical maintenance training simulator in OpenSimulator for F-16 aircraft engines André Pinheiro, Paulo Fernandes Daniela Pedrosa Dep. Engenharias, Escola de Ciências e Tecnologia Faculdade de Psicologia e de Ciências da Educação, UTAD–University of Trás-os-Montes e Alto Douro UC - Universidade de Coimbra, Portugal Vila Real, Portugal daniela55pedrosa@gmail.com {andrepinheiro.infor, pklides}@gmail.com Benjamim Fonseca, Hugo Paredes, Paulo Martins, Ana Maia, Gonçalo Cruz Leonel Morgado, Pro-Chancellery for Innovation & Information INESCTEC/UTAD – University of Trás-os-Montes e Management Alto Douro UTAD – University of Trás-os-Montes e Alto Douro Vila Real, Portugal Vila Real, Portugal {benjaf, hparedes, pmartins, leonelm}@utad.pt {margaridam, goncaloc}@utad.pt Lt. Jorge Rafael Portuguese Air Force – Air Base Nr. 5, Serra de Porto de Urso, Monte Real – Leiria, Portugal tenjorgerafael@gmail.comAbstract – Mechanical maintenance of F-16 engines is training, and consequently not to be available forcarried out as a team effort involving 3 to 4 skilled engine service. Also, procedure errors in training may in sometechnicians. This paper presents the development of a cases produce costly component damage. Further,mechanical maintenance simulator for their training. This several of the technical procedures need to be executedsimulator aims to enable technician training to be by a team, meaning that time allocation of differentenhanced with cooperation and context prior to the trainees, trainers, and experienced technicians needs totraining phase with actual physical engines. We describethe requirements that were identified with the Portuguese be managed, in order for a full team to be available forAir Force, the overall software architecture of the system, on-the-job training to ensue. These various resourcethe current stage of the prototype, and the outcomes of the requirements place constraints on the availability of on-first field tests with users. the-job training opportunities and emphasize the need to optimize it.Keywords: virtual worlds, OpenSimulator, virtual The development of a 3D multi-userlearning, cooperation, task coordination, aircraft engine mechanical training simulator for this scenario thusmaintenance aims to provide trainees and trainers with more opportunities to conduct training, with the goal of allowing trainees to reach on-the-job training better INTRODUCTION prepared and thus to optimize the effectiveness of the resource-intensive training occasions with physical At the Portuguese Air Force, engine engines. This is a joint effort of the Portuguese Airtechnicians go through an initial training process at the Force and the University of Trás-os-Montes e AltoCentre for Military and Technical Training Air Force Douro (UTAD).(CFMTFA), and are subsequently placed at different air From a software engineering perspective, therebases, with specific engines and requirements. At each is also a goal of lessening the resource requirements ofof these bases, they receive further training, focused on simulation development, by conducting it in a readilythe specific engines and aircraft deployed and serviced available virtual world platform (OpenSimulator) butthere. In the case of the F-16 aircraft, this takes place at implementing the control code and decision-makingAir Base Nr. 5, near Monte Real. Since technicians may logic at an external system. The rationale for thisbe re-deployed to other bases, training of technical architectural choice was to focus the softwareprocedures for maintenance of specific engines is a development effort on the simulation behaviour in acommon and frequent process. The training process has way that could be rendered independent from thean initial theory phase, based on technical documents graphical platform of user interaction [2].known as “Technical Orders” or TOs [1]. Then an on-the-job training phase ensues, with trainees acting I. BACKGROUNDdirectly on an engine, in actual maintenancecircumstances. F-16 aircraft of the Portuguese Air Force This final on-the-job training phase is resource- employ Pratt&Whitney F100-PW-220/220E engines,demanding, since it requires engines to be available for with large number of mechanical maintenance
  2. 2. procedures. The team at UTAD conducted meetings virtual worlds it supports collaboration, includingwith the training team and experts at air base Nr. 5, to awareness of the presence of other users andascertain the most relevant procedures for technicians communication, immersive interaction, and a crediblethat are initiating their training for this specific engine. 3D representation. The role of OpenSimulator in theThrough these meetings, the procedures for installation simulation community has been defended for variousof the engine inside the F-16 aircraft fuselage were scenarios, excluding pure science and target user groupsselected as a first simulation target. of simulation experts [11]. This installation involves a series of steps forproperly installing and connecting the engine, which II. PROTOTYPE ASPECTSneed to be done not only effectively but also safely.Briefly, these steps involve preparing the engine for To develop the system prototype, we expeditedtransportation towards the aircraft fuselage, transporting the 3D environment modelling using the built-in toolsit and preparing it for insertion, its insertion into the of OpenSimulator/Second Life client viewers, andaircraft fuselage, establishing the engine connections, employed QAvimator to recreate in 3D the movementsand testing the installation. The level of detail of the of the technicians. The goal in this prototype stage wassimulation of each task was also determined in not one of photorealistic visuals, but simply to becooperation with the trainers at Air Base Nr. 5. credible for testing and development. Following the The use of virtual world platforms as the overall goal of separation of concerns betweenenvironment for developing and deploying training simulation rendering and software control/decision-simulations is frequent, in fields as diverse as making, we implemented control and decision-makingemergency response [3], business management [4], as an autonomous system, available to themedical and health scenarios [5], and security forces OpenSimulator platform as a Web Service. This Web[6]. Military training scenarios have also employed Service receives reports of all interactions taking placevirtual worlds as their development platform, such as in the 3D environment, decides how the system mustthe OLIVE platform [7], and this use follows in the react and responds accordingly, with commandsstead of a long history of gaming and gaming specifying the intended behaviour of the simulation (seetechnologies in military training [8]. Given that training Fonseca et al. for more architecture details – [2]). Theis critical for the success of military operations, for decision-making was implemented in the form of antechnical operations such as aircraft maintenance as for hierarchical state machine.tactical operations and combat. In this sense, virtual In order to separate the control system from theworld environments with multi-user abilities allow specificities of OpenSimulator client-serverpersonnel to interact in a simulated face-to-face communication, we developed simple text-basedenvironment. There are several examples of serious protocols for reporting of events by the 3D environmentgames used by multiple users for training combat and to the Web Service, and for the Web Service responsetactical activities [9][10], but the field of multi-user to OpenSimulator with commands. Using thesetechnical training simulations is still in the beginning. protocols, we managed to use a single script that is As mentioned above, the use of a virtual world running concurrently in all virtual world objects. Thisplatform aims to lessen the resource requirements of script, developed in LSL (Linden Scripting Language,simulation development. The rationale is that the virtual commonly used for scripting in Second Life andworld provides a set of basic features such as content OpenSimulator worlds) acts as an interfacing module,rendering, user login and interaction, user messaging, to report events to the Web Service, and process theand object physics, among other aspects. This enabled Web Service response. The commands in the responsedevelopment to focus on the behavioural elements of may be intended for the object originating the event, orsimulation development. We have been exploiting the for other objects, in which case the script broadcasts ordevelopment of this scenario to create and further a relays the commands and parameters, depending onsimulation control architecture which attempts to have what the Web Service specified in its response.more concern independence between the simulation’s A consequence of this approach is that newbehaviour logic and its human interaction and visual objects can be deployed in the simulator without newaspects [2]. OpenSimulator-specific code having to be developed: In order to leverage this scenario for this since the script is identical for all objects, only thepurpose, we set to plan the prototype development by decision-making component needs to be aware of theusing OpenSimulator, due to its ability to enable new objects.communication with external systems without requiring We emphasize that the 3D environment doeschanges to the code of the virtual world platform. It not conduct any decision-making: it is only responsibleshares its client-server protocol with Second Life and for reporting to the web service the events triggered bytherefore we could also benefit from the full set of the technician’s avatars or other objects and then waitcoding resources and community that develop scripts for commands issued by the Web Service, in responseand client applications for both platforms. to these reports. In this sense, it is an embryonicOpenSimulator specifically is used by many different implementation of the Model-View-Controllergroups and for different purposes, and as with many architectural style [12] for virtual worlds. This holds the
  3. 3. potential to render decision-making independent from the involvement of all technicians, as it entails severalthe 3D environment: already in its current form we can tasks that cannot be performed by a single individual.replace the decision-making algorithm entirely, without One such task is the raising of the engine tohaving to change a single line of code in the 3D align it with the empty hull of the aircraft fuselage,environment. In a parallel effort, we have used the same which requires all 3 technicians: two on the left side,3D models and scenario to implement the Partial-Order- another on the right side, as shown in Figure 1. WhenPlanning algorithm as a reasoning model to replace the process checker issues the command (i.e,, this maysome of the human-controlled avatars by an intelligent be the fourth element in the team), the three technicianssoftware agent. The approach we used enables us to will operate screw driving machines in concert to liftnow combine these decision-making approaches the motor simultaneously and co-ordinately, usingwithout having to change the 3D scripts. speech as a means of synchronization, to avoid tilting Currently, the 3D script is the one aspect which the engine excessively while raising it.still binds the system to a specific virtual worldtechnology. It is reporting OpenSimulator/Second Lifeevents and its copies are meant to be runningindependently. But the script is detached from thedecision-making process. Thus, we have the ambition ofmaking this approach evolve in order to render it viablefor other virtual world platforms. Ideally, by developinga new event-reporting module and a new translationmechanism of decision-making commands into thespecific requirements of the new technology one wouldlike to be able to benefit from existing simulations withmuch lessened development efforts and resources. The current decision-making system, while notthe focus of our development concerns, is presentedherein for clarity. It works as follows: when it receives Figure 1 - Mechanics raising the engine at Air Base Nr. 5an event from OpenSimulator, it first queries a datastore with all current data on simulation state. Then, The multi-user virtual world platformbased on this information and the date of the OpenSimulator enables three or four trainees to practiceOpenSimulator-originating event, an hierarchical state the synchronization in this task in a similar way, as longmachine algorithm determines the adequate response. as voice chatting is available. In the current prototype,For example, if a technician is in a “Free Hands” state, once all preconditions are adequate (such as havingthis state can transition to the state named steadied the bearing cart and having adequate tools in“HoldingScrewdriverInHand” upon receiving a hand and fitting for the tools in place), the engine lifting“ClickedOnScrewdriver” event. Finally, the transition is can take place. Once it is initiated, we elected totranslated into virtual-world specific commands, which implement it in the following way: the simulator willare provided to the originating script for execution. take control of the arrow keys and change their function. During the lifting process, up and down arrow III. SAMPLE SIMULATOR TASKS keys will respond as if operating the direction of screw driving machines or the turning of lifting wrenches The installation process of a Pratt & Whitney (Figure 2).F100 engine in an F-16 aircraft is quite extensive andcomplex, requiring three technicians to do the variousprocedures. Plus, a specific role in the process is that ofprocess checker, which may lead to a fourth personbeing involved, should none of the three requiredtechnicians have the credentials to perform this role. All necessary procedures are specified in adocument known as the Job Guide, which refers to allthe Technical Orders containing the necessaryinformation. Installation of the engine inside the aircraftfuselage, the first procedure being implemented in theprototype, is a single process, but it is typically Figure 2 - Arrow in virtual spacesubdivided into four jobs, known as PT1, PT2, PT3, andPT4. Currently the prototype is implemented to support The three technicians need to press theirsimulation throughout the PT1 job, which is a respective keys in the correct direction at roughly theprecondition for executing jobs PT2, PT3, and PT4. But same time (using voice chatting to synchronize theirmore important, it enables testing of the cooperation actions, as in the physical world) in order to lift orsituations of the process, because this PT1 job requires lower the engine. We defined "roughly" as a 2-second interval. Currently, this is a precondition for
  4. 4. lifting/lowering to occur, but in the future we intend to r,expand the simulator behaviour to cover errorconditions and accidents caused by wrong operations operations.Figure 3 shows the avatars of trainees during thelifting of the engine in the virtual space. Figure 4 – Room arrangement during the test The first day of our stay was employed in the room preparation, and installation of the software on the Figure 3 - Mechanics lifting the engine in virtual space air base computers. The actual training sessions took to place on day 2, each lasting about 1 hour 15 minutes. IV. USER TESTING The sessions consisted of a brief introduction with a project presentation, an explanation of the motive , In order to evaluate the simulator prototype, to imulator for the team being present, and a presentation of theinform its subsequent development, a 2, 2-day stay at Air work plan for the session. All trainers read and signedBase Nr. 5 was planned and conducted, for conducting consent forms regarding data collection for this test, and stests with prospective users. An early version of the filled in the characterization questionnaire. During the questionnaireprototype had been demonstrated earlier to the air base simulation session proper, the whole dynamic wascommand, but no actual user tests. With this 2-day stay being projected onto a screen from the perspective of een,the team aimed to make the prototype available to the computer whose user (a development team member) sertechnicians that are also trainers, in orde to identify order played the role of supervisor for the engine installationtheir satisfaction as users, and their expectations simulation. The session began with an activity forregarding subsequent development – particularly in acquainting users with the platform, aimed at theareas such as technical details and pedagogical acquisition of basic skills for moving in the virtualaffordances, user interface choices, and interaction world and interacting with objects. nteractingmethods. We also wanted to ascertain w whether the This acquaintance activity had 4 steps. Thetechnological context at the air base (computer group of avatars started by meeting in a location away meethardware and network) supported the execution from the virtual hangar. At each step there was a virtualrequirements of the prototype. poster with a task for users to accomplish. The first was Since the air base is located 260 km away from to enter avatar flight mode and move near the aircraftUTAD, the test preparation was done remotely, via e e- parked outside the hangar (Figure 5). Then they had tomail exchanges. Two test sessions were planned, which est walk into the hangar, up to a table with objects on itincluded 3 trainers each, chosen on the basis of being (Figure 6). At this table they had to pick the objects,potential future users of the simulator in the context oftraining sessions. These trainers were the people in se saving them to their avatars’ inventories. Next, we inventoriescharge of training new technicians as they are assigned asked them to carry the objects to another table, near theto the air base, for operation on F-16 aircraft engines. 16 first one (Figure 7). Here they had to place the objects heyI.e., they were the air base most qualified technicians on this second table, i.e., taking them from theregarding the mechanical maintenance procedures for inventory. Finally, the acquaintance activity wasF-16 engines. completed by asking them to return to the aircraft The 2 sessions were similar. Both took place in parking location, and from there to the engine, toa side room of the main engine maintenance hangar initiate the simulation of engine installation.(Figure 4). This room had 2 group tables with chairs, a During the engine maintenance simulation,projector and a white board. 4 computers were used for trainers were asked to employ the think aloud protocoltests, 3 for the users and 1 as an OpenSimulator server. throughout the process, communicating their thoughtsThe test preparation consisted in gathering and sted and feelings. After completing the simulation process, a processassembling all necessary supporting materials final group interview took place, recorded for later(computers, networking, cameras and microphones for analysis. The interview attempted to measure users attempttest recording, software installation, and distribution of satisfaction towards the system, and collect suggestionsconsent forms for data collection, questionnaire for questionnaires and recommendations for improvement. improvementcharacterization of user profiles, and paper guide).
  5. 5. prototype, we needed a network gateway, a service that was not provided by the provided network switch. So we proceeded with testing employing a small router we had planned as an alternative. The laptops had adequate memory and processing powers, but some limitations in terms of graphic cards, which were able to run the client software (Second Life viewer) but not in the best conditions. One was replaced by a developer’s laptop so that the test could continue. The computer being used as server also revealed itself with insufficient performance, and this Figure 5 – Setting activity: first station exposed some deficiencies in data consistency between the Web Service and the 3D environment. With the simulator prototype: From analysis of interviews, respondents (trainers) mentioned some aspects for improvement and correction. For instance, avatar positioning was not always correct for all procedures. This is an important component of training, since inadequate positioning may expose a technician to unnecessary danger or inability to perform adequately. Regarding the usefulness of the simulator, respondents found it useful, an asset in support of Figure 6 - Setting activity: second station practice. But they expressed the need to take into consideration not only the actual procedures, but also the prior safety inspection checklist. For instance, “In terms of safety of the airplane stability, this is why I told you of the amount of fuel that...” (a plane without enough fuel weight will not remain stable on the hangar floor if the engine is removed); “That part is always important (...) to safeguard his safety of the rest of the team’s work”; “While typically the aircraft will have all these elements, sometimes (...) a seat may not be there (..,) which needs to be corrected...” (Respondent 2, March 2nd, 2012). Figure 7 – Setting activity: third and fourth stations C. Other aspects mentioned in the interviews: V. RESULTS From a user perspective, respondents of both groups classified the level of knowledge andA. Trainers Profile preparation required to deploy the 3D virtual world as acceptable, although depending on actual computer and By analysis of questionnaires, All trainers software capabilities.taking part in this test were male, with an average age Regarding the acquaintance activity with theof 42. They reported being regular computer users, virtual world, respondents felt it facilitated the contact,using computers more than once per day. Their main with expressions such as “Ah ... Yes, yes”, “To knowuses of computing technology are searching for the steps…”, “It is an introduction” (Respondents 1,information (100%), work completion (83%), and March 2nd, 2012). They found the time for thesocialization (83%). Regarding the use of virtual acquaintance activity to be “enough” and “appropriate”worlds, none of the trainers had used them before, nor (Respondents 1, id.).were they aware of them. Regarding the use of the actual simulator, the usefulness of having an introduction was reported, asB. Problems and limitations found was the importance of a checklist with common errors and required tasks to accomplish in the simulator.With the local hardware: Trainers considered that support by training staff and The air base assigned for this test: four laptops, colleagues will be important at an early phase, but willa network switch, and network cables. We found out eventually no longer be required. E.g., “the second timethat this equipment was not entirely suitable: to run this I think I got it…” (Respondents 1, id.).
  6. 6. As for the pedagogical content of the REFERENCESsimulator, trainers considered that it did allow trainingof necessary skills for installation of F-16 engines, but [1] Lockheed-Martin-Corp. (2009). TO 1F-16AM-2-noted that it is necessary to increase the level of detail. 70JG-10-21 - Organizational Maintenance -E.g., with statements such as literally “more details” or Engine Removal and Installation – Model F100-“Instead of placing things on the ground, to check PW-220/220E – USAF/EPAF Series – F-16/Bmaterials, you could watch them here...” (Respondents Mid-Life Update aircraft, Technical Manual Job1, ibid.). On the collaboration aspects of simulator use, Guide. Bethesda, MD, USA: Lockheed Martinthey considered that it achieved the desired goals, since Corporation.it allowed the practice of these aspects. [2] Fonseca, Benjamim; Paredes, Hugo; Rafael, Jorge; Overall, they found that that the simulator Morgado, Leonel; Martins, Paulo (2011). Acould benefit the training of the installation of F-16 Software Architecture for Collaborative Trainingengines, reporting aspects such as “the advantage of the in Virtual Worlds: F-16 airplane engineperson being able to correct mistakes” (Respondents 2, maintenance. In Adriana S. Vivacqua, Carlibid.), and “when new staff arrives, they learn, they do Gutwin and Marcos R.S. Borges (Eds.)stuff, but until a team gets automated it takes time” "Collaboration and Technology: 17th International(Respondents 2, ibid.). Conference, CRIWG 2011, Paraty, Brazil, October On the subject of virtual worlds, they 2-7, 2011. Proceedings", 102-109. Berlin:considered them an alternative to the physical training Springer.of tasks, and an appealing one, able to “create more [3] Chen, Y.F., Rebolledo-Mendez, G., Liarokapis, F.,enthusiasm” (Respondents 2, ibid.). de Freitas, S., Parker, E. (2008). The use of virtual Regarding the graphical interface, trainers world platforms for supporting an emergencypointed out that some engine parts have to be more response training exercise, Procedings of the 13threalistic, to the point of being able to identify part when International Conference on Computer Games: AI,watched “really close” (Respondents 2, ibid.). They did Animation, Mobile, Interactive Multimedia,not find it difficult to identify the objects, but Educational & Serious Games, Wolverhampton,corrections are needed, since some aspects simply do UK, 47-55.not occur in reality (e.g., “the engine cannot leave the [4] Rodrigues, Clara; Coelho, Dalila; Morgado,front, that’s impossible” – Respondents 2, ibid.). Leonel; Varajão, João; Haidimoschi, Aura; They pointed out that interaction with the Doppler, Gerhard; Koivusalo, Hillevi; Jokinen,system is adequate and sufficient, yet that there is the Paivi; Velegrakis, George; Sancin, Chiara;need to get used to the conditions that the system offers. Carmenini, Valentina (2009). Virtual Learning for Finally, some suggestions were provided the management of successful SMEs in Europe. Inregarding technical training, including on the issue of Morgado, Leonel; Zagalo, Nelson; Boa-Ventura,security awareness, and details such as ensuring use of a Ana (Eds.) "Proceedings of the SLACTIONS 2009toolbox in certain circumstances, or identification of International Conference - Life, imagination, andoccasional errors in the installation sequence of tasks. word using metaverse platforms", pp. 165-170,Interface cueing requests such as displaying an object ISBN 978-972-669-924-8. Vila Real, Portugal:description or name upon it on mouse over, were also Universidade de Trás-os-Montes e Alto Douro.recorded. [5] Boulos, M., Hetherington, L., & Wheeler, S. (2007). Second Life: an overview of the potential VI. CONCLUSIONS AND FUTURE WORK of 3-D virtual worlds in medical and health education, Health Information & Libraries The test confirmed that the selected technology Journal, 24 (4), 233-245.is feasible for use by the actual trainers involved in [6] Hudson, K., deGast-Kennedy, K. (2009).technician training at the air base. The nature of most Canadian Border Simulation at Loyalist College,omissions in terms of simulation aspects revealed that Journal of Virtual Worlds Research, 2 (1), 3-11.the procedure analysis needs to be fine-tuned, taking [7] de Freitas, S. (2008). Serious virtual worlds. Ainto account tactical know-how and systemic aspects scoping study. Bristol, UK: JISC.(such as the security component of certain avatar [8] Smith, R. (2010). The long history of gaming inpositions or the importance of using specific accessories military training. Simulation & Gaming, 41 (1), 6-such as the toolbox for better team coordination). 19. The prototype is being further developed and [9] Orvis, K.A., Moore, J.C., Belanich, J., Murphy,the architectural goals of independence between virtual J.S., Horn, D.B. (2010). Are soldiers gamers?world platform and decision-making/control logic has videogame usage among soldiers and implicationsshown its feasibility. We hope not only to complete the for the effective use of serious videogames forsimulator, but to be able to pursue further the military training. Military Psychology 22 (2), 143-development of this software engineering approach, in 157.view of faster, and less consuming simulation [10] Sotomayor, T.M. (2010). Teaching tactical combatdevelopment. casualty care using the TC3 sim gamebased
  7. 7. simulation: a study to measure training effectiveness. Studies in health technology and informatics, 154, 176–179.[11] Fishwick, P. A. (2009). An introduction to OpenSimulator and virtual environment agent- based M&S applications. In Winter Simulation Conference (WSC 09), 177-183.[12] Krasner, G.; Pope, S. (1988). A cookbook for using the model-view controller user interface paradigm in Smalltalk-80, Journal of Object- Oriented Programming, 1 (3), 26-49.