Bio-inspired, Learning and Intelligent systems for security                                      Bio-inspired, learning   ...
flight and require constant monitoring from the human                    The payload functions are represented by the miss...
o The power management system, included                      systems.         optimising the power consumption and maximis...
Surveillance/IFF                                                      UAV Mission System                                  ...
•   Mission Goal Manager                    no longer sufficient to provide the additional capabilities                   ...
provide the current world state situation awareness and       advantages and drawbacks of implementing the individual     ...
Mission                    Mission                         Objectives                Requirements                         ...
Mid-Level Command                   Terrain database       Commands from       Mission Manager                            ...
Upcoming SlideShare
Loading in …5

Ai in decision making capability on board unmanned aerial vehicle


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ai in decision making capability on board unmanned aerial vehicle

  1. 1. Bio-inspired, Learning and Intelligent systems for security Bio-inspired, learning intelligent Systems for Security Artificial intelligence methodologies applicable to support the decision-making capability on board Unmanned Aerial Vehicles Isabella Panella Thales UK, Aerospace Division, Manor Royal, Crawley, UK Abstract systems, as single methodologies cannot address The need for Unmanned Air Vehicles (UAVs) to operate independently the complexity of the problem of machine autonomously and to manage their operation with autonomy. minimal intervention from the ground control station, in The architecture specifies the relationship between the order to reduce the datalink utilization and maximize various subsystems that are seen to be key enablers to the their exploitation in beyond line of sight (BLOS) autonomy of the vehicle combined with legacy systems operations, has been long recognized within industry and and the artificial intelligence (AI) techniques that could be research institutes. Many artificial intelligence (AI) used in order to implement them. techniques try to address the challenge of moving UAV The motivation behind this work is to provide a towards full autonomy. However, no single technique has possible UAV functional architecture within which the AI been able to provide the required autonomy for methodologies can be embedded and the need for them to unmanned platforms. This paper presents a Unmanned complement each others weaknesses and interrelationship Air Systems (UAS) architecture within which the different appreciated. AI methodologies applicable to each subsystem are In this paper, unmanned air platforms refer to both the presented. air vehicle and the payloads fitted on the air vehicle in order to provide the required capabilities for the given mission. 1. Introduction The paper is organised as follows. Section 2 provides a definition of automatic versus autonomous. These terms It has long being recognized that the employment of are often confused and misused and it is appropriate to be Unmanned Air Systems (UAS) could provide significant clear about the difference between them. Section 3 advantages in military applications for ‘dirty, dull, and provides an outline of the major functional elements of the dangerous’ missions as well as for civil and commercial platform and the payload is also provided. applications, such as search and rescue, border Section 4 reports some of the key mission level management, pipeline monitoring. functionalities and capabilities that are required for the UAV performance can be improved by increasing the systems to work autonomously. Section 5 identifies the AI autonomy on-board of the platforms in order to reduce the fields and techniques that can be applicable to solve some workload placed on the operators and users. The of the identified issues and provides a mapping of those to implementation of autonomy on UAV requires the the identified UAV’s functionalities. Diagrams of the application of innovative and at times new ways of architecture are also included. Finally, section 6 provides implementing the UAV on board functionalities, the conclusions of this study. specifically the need to implement artificial intelligence methodologies within the functional architecture of 2. Automatic versus Autonomous UAVs. This paper suggests a simple and highly modular UAV The major driver for the adoption of unmanned system architecture for the on board UAV mission system platforms is to have systems that can continuously provide and identifies a relationship between the functionalities information and situation awareness without the risk of the UAV should present and the possible artificial losing human lives. The need of gathering information is intelligence methodologies that could be applied in order driven by the need to make decisions and to react to the to achieve them. situation that presents itself as soon and effectively as The aim is to present a possible solution to the possible. problem of autonomy on board of the platform and Current UAV systems are mostly examples of promote a hybrid approach to the issue of autonomous automatic systems, which depend on pre-programmed978-0-7695-3265-3/08 $25.00 © 2008 IEEE 111DOI 10.1109/BLISS.2008.14
  2. 2. flight and require constant monitoring from the human The payload functions are represented by the missionoperator (HO). Their use up to now has been described as functional system, i.e. those systems which are missionthe use of “goggles in the sky” by a human operator on the specific, and they are defined in [Ref. 1] (chp1, pp1), asground. those functions that include the set of all functions that By automatic is meant a repetitive action that does not directly relate to the mission of a given vehiclerequire external influence or control, but which repeats The payload of each platform is mission specific and ititself based on some set conditions. A well known will vary with the specific mission requirement and it willautomatic system is feedback control system, which need to be fitted on the air vehicle and its control definedautomatically adjusts the input in order to obtain the and implemented. Examples of possible sensors that coulddesired outputs be fitted on a UAV are: The major difference between automatic and • Electro-Optical including Optical and IR sensors forautonomous systems is that autonomous systems can obstacle detection and target detectionchange their behaviour in response to unanticipated • Ground Moving Target Indicator (GMTI) radar andevents, whereas automatic systems would produce the Synthetic Aperture Radar (SAR)same outputs regardless of any changes experienced by • GPS/INS navigation system.the system or its surroundings. In robotic applications, The vehicle management system (VMS), on the otherautomatic actions allow the machines to be operationally hand, is defined in [1] as the collection of functions thatautonomous, but they do not allow them to have are required for the vehicle to understand, plan, control,decisional autonomy. and monitor the air vehicle operations. They usually An autonomous action or event is defined as an action represent the safety critical functionalities required for theor event that is ‘independent in mind or judgment’, a self- safe employment of the platform; hence they include alldirected, self-governing entity, not controlled by others or the flight-critical and safety-related outside forces. The mission system functional capabilities can be Autonomous UAVs are herein defined as air vehicle classified according to the level of autonomy that each ofsystems with embedded autonomous functionalities. them would require if the HO was not included in the In this research the focus is on identifying loop.methodologies to allow the level of autonomy for UAVs Three major levels of autonomy have been identified:to increase and the implementation of a dynamic, • Low Level - The low level of autonomy can beresponsive behaviour on the platform itself to reduce the considered the reactive side of the UAV, where,operator workload and provide independence from pre- given an event, the UAV will automatically reactprogrammed flight. according to pre-defined limits and following given dynamic models. This level is characterised by the3. UAV Mission System group of functionalities that would be required in order to fly the platform remotely. These include, but The main gaps in current UAV systems are are not limited, to the flight control system, therepresented by the reliance on datalink, the ‘dumbness’ of actuator function, the engine or propulsion control,the control and payload systems, which means that they and the aircraft flight mechanics and air dataare not reactive to the environment, and the latency of acquisition.information to the ground control station. In order to • Medium Level – The medium level of autonomyunderstand where the identified technological gaps can captures the ‘reflective’ capability that should bepresent a significant drawback within the UAV systems, embedded on board of the platform in order to enablethe general UAV functionalities are captured. it to reason about its internal and external state. That A UAV mission system is seen to provide the is the ability of the platform to detect internal faultstranslation of mission objectives into quantifiable, or malfunctions, to self-regulate when subjected toscientific descriptions that provide a measure to judge the unexpected events while carrying out the mission.performance of the platform, i.e. the system in which the The functionalities of the mission systems that havemission objectives are transformed into system been identified to belong to this class are:parameters. According to the AGARD report on o The flight path command and performanceIntegrated Vehicle Management Systems [Ref. 1], the envelope protection, such as the waypointmission management functions of modern aerospace following system and the guidance andvehicle can be split into two different functional elements: navigation functions;the payload functions and the vehicle management o The health manager and fault tolerant control, infunction. order to detect and react to possible system failures and malfunctions; 112
  3. 3. o The power management system, included systems. optimising the power consumption and maximising By developing those on the platform, the air vehicle is the mission time. envisaged to achieve the required level of autonomy for• High Level – The high level represents the most the drone. The key capabilities and the associated system sophisticated layer of autonomy that it is desirable to where it can be built in pointed out are reported in Table provide on board the air vehicle. It provides the 1. A possible application of these fields is provided in the platform with decision making capabilities and with implementation of the systems reported in Table 1. the ability of interoperate with other platforms and /or systems in order to gather and analyse information, as well as the ability to reason and act upon the 7DEOH /LQN EHWZHHQ 8$9 UHTXLUHG FDSDELOLWLHV DQG conclusions it draws. The functional systems which IXQFWLRQDO VXEVVWHPV have been identified to require a higher level of autonomy are: UAV capabilities Functional subsystem o the fault detection and identification (ID), i.e. the React to changes in Mission Planner ability for the platform to detect the malfunction, environment and be capable of Mission Goal Manager reason about it and take an appropriate action re-planning Sensor Manager without any HO intervention; Navigation through complex Flight Path o the situation awareness manager, responsible to terrain, possibly at high speed maintain and update the state of the world (i.e. the Dynamic allocation of on board Power Manager representation of the environment in which the resources, i.e. fuel, power, Sensor Manager UAV is operating) and communicate detected sensors. changes to the mission manager. Interoperability with other Sensor and Integrated Signal A high level of autonomy is also required to make systems and platforms and Data Processing Managerdecisions and to provide interoperability with other Constant maintenance of Sensor Manager, Integratedsystems. Therefore, the mission goal manager and sensor situational awareness Signal and Data Processingmanager, as well as the integrated signal and data Managerprocessing managers fall into this layer. The mission goal Situational awareness Managermanager is responsible for redefining the mission Perform autonomous Mission Plannerobjectives and goals and assessing their validity while manoeuvre at the limits of the Flight path envelope protectioncooperating with other air platforms or systems. The flight envelope to maximisesensor manager needs to monitor, select, and allocate the performancesensor when the target is detected and decide both the Provide on board data and Integrated Signal and Datarequired accuracy and precision of information for the information processing Processing Managergiven task, and also the data that need to be recorded in capabilityorder to successfully support the mission execution. Data Operate outside communication Mission Goal Managermay come from the own sensor platforms mounted on the link limits Mission PlannerUAV or from other air vehicles or systems. The integrated Sense and avoid threats, Sensor Managersignal and data processing manager has the task of collision avoidance Mission Plannerprocessing the multiple source information, including the Autonomous reconfiguration of Fault Tolerant/Reconfigurableoperator commands and reasoning about their soundness systems in case of fault controland applicability within the mission context, and also to Fault Detection and ID Managerprocess them into appropriate information useful by theother subsystems. Figure 1 captures the envisaged mission 5. Artificial intelligence techniques applicablesystem architecture and the three major autonomy levels to UAVsas described above. It is important to observe that the application of Conventional control systems and deterministicartificial intelligence technique is envisaged to be used optimisation techniques provide a means to deal withmainly in the high-level autonomy layer. uncertainty within certain boundaries. However, the multitude of case scenarios faced by the UAV even before4. Identified mission level functionalities for it starts its mission requires a real time adaptive systemUAVs capable of reacting to unforeseen events to the best of its possibilities. Machines are now required to ‘think’ and In this section the key mission-level capabilities for a ‘learn’ from the environment, as it has been pointed out inUAV are discussed and linked to the above functional the previous paragraphs. 113
  4. 4. Surveillance/IFF UAV Mission System Target Detection and Tracking Mission Objectives Battle Damage Assessment/Reconnaissance/ Search and Rescue Mission )LUH &RQWURO :HDSRQ requirements 6VWHP Mission Communication Mission Design Payload Functions Integration Sensor Control/Sensor Fusion Vehicle Management Aerial Refuelling/Rearming Functions Deployment of Humanitarian Aids /RZ /HYHO $XWRQRP 0HGLXP /HYHO $XWRQRP $XWRQRP +LJK /HYHO $XWRQRP • )OLJKW &RQWURO - )OLJKW 3DWK &RPPDQGV (QYHORSH • )DXOW HWHFWLRQ DQG , • )OLJKW PHFKDQLFV 3URWHFWLRQ • 0LVVLRQ 3ODQQHU • $FWXDWRU FRQWURO - 0XOWLIXQFWLRQ ,QWHJUDWHG 1DYLJDWLRQ • 6LWXDWLRQ $ZDUHQHVV 0DQDJHU • 3URSXOVLRQ (QJLQH 6VWHP HFLVLRQ 0DNLQJ VXEVVWHPV &RQWURO - +HDOWK 0DQJHU • 0LVVLRQ *RDO 0DQDJHU - )DXOW 7ROHUDQW 5HFRQILJXUDEOH &RQWURO • 6HQVRU 0DQDJHU - 3RZHU 0DQDJHPHQW • ,QWHJUDWHG 6LJQDO DQG DWD 3URFHVVLQJ 0DQDJHU • Monitoring – identify changes in a system’s observed )LJXUH 8$9 RQ ERDUG PLVVLRQ VVWHPV state;Therefore, the introduction of artificial intelligence • Risk Analysis – identify issues within a given coursemethodology in order to implement, enhance, and improve of action and plan/provide mitigations, such asthe UAV’s autonomous functionalities seems to be a alternative routes in case of detected obstacles;natural evolution of more conventional feedback control • Data Analysis/Processing – for instance data mining,theory. identification of trends, extrapolation of data; The following branches of AI seem to be the most • Optimisation – streamlining a system or object, suchsignificant for potentially providing autonomous as the route plan or fuel consumption for givencapabilities for the air vehicle. They were adapted manoeuvre, to achieve the best performance, resourcepartially from [2], pp. 248, and they are as follows: allocation;• Interpretation – consists of data analysis coupled with • Classification – Assignment of a category to an domain knowledge forming high-level conclusions; object, for instance threat identification;• Prediction – projecting probable consequences of • Control of Systems – Governing the behaviour of given situations; complex systems. Manipulation of a system’s• Diagnostic – Determining the cause of malfunctions interaction with the world adjusting (actuating) the in complex situations based on observable symptoms. control surfaces, for instance to maintain a flight path Identify abnormalities in the observed states of the The applicability of those fields to the functional UAV’s systems and possibly suggest remedies to subsystems previously outlined is shown in the following mitigate fault; table and it is based on the author’s engineering judgment.• Design – finding a configuration of system components, which can meet the required 7DEOH $SSOLFDWLRQ RI $, ILHOGV· WR IXQFWLRQDO VXEVVWHPV performance while satisfying the constraints; AI Field Functional Subsystem• Planning/Scheduling – devise a series of actions to Interpretation • Mission Planner achieve certain goal and co-ordinate them • Situation Awareness Manager sequentially, for instance sensor pointing time or • Fault Detection and ID route planning; • Mission Goal Manager• Decision Support/Decision Making – advise the HO, • Sensor Manager both on the ground or on a fighter aircraft acting as Prediction • Mission Planner mission manager, to aid them in difficult cognitive • Situation Awareness Manager tasks; • Fault Detection and ID 114
  5. 5. • Mission Goal Manager no longer sufficient to provide the additional capabilities • Sensor Manager of the UAV, it is still crucial to the implementation ofDiagnostic • Fault Detection and ID adaptive and reconfigurable control subsystems. Also, • Fault optimisation is more and more being adopted for Tolerant/Reconfigurable subsystems requiring resource allocation, planning and Control scheduling. In fact, heuristic searches and genetic • Health Manager algorithms are found to be well suited providing a quick and optimal solution when faced with multiple variablePlanning and • Mission Planner scenarios and incomplete information.Scheduling • Mission Goal Manager Over the years, several AI techniques have been • Sensor Manager developed, each claiming to provide significantDecision • Mission Planner advantages over the others. No individual technique hasSupport/Making • Situation Awareness Manager proved to be the answer to the problem of creating • Fault Detection and ID machine autonomy. Therefore, it is necessary to blend the • Mission Goal Manager different methodologies and provide a new level of • Sensor Manager integration in order to create hybrid systems. Only byMonitoring • Mission Planner combining different methodologies and matching them to • Situation Awareness Manager the systems requirements it will be possible to move • Fault Detection and ID autonomy forward. • Mission Goal Manager In this research, the focus has been on identifying • Sensor Manager methodologies that could allow an increased capability for • Fault a dynamic, responsive behaviour and its implementation Tolerant/Reconfigurable on the platform itself so to reduce the operator workload Control and provide independence from pre-programmed flight. • Health Manager Therefore, this paper has focused on techniques, whichRisk Analysis • Mission Planner are believed to be matured enough or most appropriate for • Situation Awareness Manager the application to UAV. The AI methodologies selected to provide autonomy to • Fault Detection and ID the UAV were: • Mission Goal Manager • Artificial Neural Networks (ANN or NN) ([7], • Sensor Manager [2],[11]);Data Analysis & • Sensor Manager • Fuzzy Logic (Fuzzy) ([16],);Processing • Integrated Signal and Data • Genetic Algorithms (GA) (16], [11], [2]); Processing Manager • Reinforcement Learning (RL) ([4], [8], [13]);Optimisation • Sensor Manager • Temporal Logic (TL) ([15], [2]); • Power Manager • Knowledge Based Systems (KBS); • Situation Awareness Manager • Rule Base Systems (RBS) [14], [10], [11]);Classification • Mission Planner • Case Based Reasoning (CBR) ([9], [11], [2]); • Situation Awareness Manager • Constrain Satisfaction Problem (CSP) ([3], [11]); • Fault Detection and ID • Model Based Reasoning (MBR) ([2], [11]). • Mission Goal Manager In order to understand what techniques could be • Sensor Manager applied to the given AI fields in order to enable moreControl of Systems • Health Manager autonomous functionalities on board of the platform, the • Flight Protection selected techniques were tabulated against the identified Commands/Envelope AI fields and they were rated according to their protection applicability. The following table (Table 3) shows how • Multifunction Integrated each technique could contribute to the set tasks which Navigation System need to be performed. • Fault tolerant/Reconfigurable In order to provide interpretation and diagnostic Control capabilities to the UAV systems, it is important to introduce data fusion techniques and data mining procedures to quickly process and analyse the data. It is important to observe that each subsystem requires Information and data fusion become particularly importantmultiple AI fields in order to satisfy the required level of in the sensor management system, where the sensorautonomy and that even though control theory in itself is information are collected and processed in order to 115
  6. 6. provide the current world state situation awareness and advantages and drawbacks of implementing the individual update the belief functions on board of the UAV. systems with the suggested techniques and analyse the The following is a qualitative and judgmental analysis. issues associated with their integration. Moreover, by The analysis carried out in the previous paragraphs led implementing specific systems, different techniques may to the definition of a functional UAV mission system become more appealing compared to the one suggested. architecture with possible AI techniques associated with Therefore, more research should be carried out on those subsystems. The result is shown in Figure 2 and overall UAV functional system design and to their Figure 3. Figure 2 captures the high level decision making integration. The key difference to current systems is the functionalities, usually performed by the pilot or the HO. inclusion of artificial intelligence techniques within the Figure two represents the subsystems such as the guidance functional domain. system that require a medium level of autonomy and the most inner part of the control loop, which can be fully 7. References automatic. The architecture has been subdivided into three components each of which groups together the functional [1] AGARD Advisory Report, ‘Integrated Vehicle Management subsystems according to their required autonomy level System’, AR 343, NATO, April 1996. and to their functional flow of information. [2] Luger, G. F., ‘Artificial Intelligence: Structures and The AI techniques selected for each subsystem are Strategies for Complex Problem Solving’, Addison Wesley, drawn from Table 3. IV Edition, 2002 [3] Barták, R., ‘Constraint Processing’, IJCAI_07 Tutorial [4] Berenji, H. R., et alt.,‘Co-evolutionary Perception-based 7DEOH 0DSSLQJ RI $, 7HFKQLTXHV WR $, )LHOGV Reinforcement Learning For Sensor Allocation in Autonomous Vehicles’, IEEE, 2003 NN Fuzzy GA RL TL KBS RBS CBR CSP MBS [5] Dufrene, W. R., ‘Approach for Autonomous Control ofScheduling/ Unmanned Aerial Vehicle Using Intelligent Agents forPlanning Knowledge Creation’, IEEE, 2004 [6] Grelle, C., Ippolito, L., Loia, V., and Siano, P., ‘Agent –Decision based architecture for designing hybrid control systems’,Support/ Information Sciences, Vol. 176, pp. 1103-1130, 2006.Decision [7] Haykin, S., ‘Neural Networks: A comprehensiveMaking Foundation’, Prentice Hall, II EditionDiagnostic [8] Harmon, M. E., and Harmon, S., ‘Reinforcement Learning:Risk analysis A Tutorial’. [9] Lees, B., ‘9th UK Workshop on Case-Based Reasoning:Data Proceeding’, SGAI, December 2004Analysis/ [10] Nilsson, N. J., ‘Introduction to Machine Learning: An earlyProcessing draft of a proposed textbook’, Robotics Laboratory, Dep. OfMonitor Computer Science, Stanford University, Dec 1996Optimisation [11] Russell, S. and Norvig, P., ‘Artificial Intelligence: A Modern Approach’, Prentice Hall, 2003Interpretation [12] Shirazi, M. A., and Soroor, J., ‘An intelligent agent-basedClassification architecture for strategic information system application’. Knowledge Based Systems, 2006.Control [13] Ten Hagen, S. and Kröse, B., ‘A Short Introduction toSystem, Reinforcement Learning’, 7th Belgian-Dutch Conference onPrediction Machine Learning, pp 7-12, 1997 [14] Tunstel, E., et al., ‘Rule-based reasoning and neuralDesign networks for safe off-road robot mobility’, Expert Systems, Vol.19, No 4, September 2002 Legend: [15] Vila, L., ‘A Survey on Temporal Reasoning in Artificial Intelligence’, AI Communications, Vol. 7, No. 1, March Highly applicable 1994 Potentially applicable [16] Johnson, J. and Picton, P., ‘Concepts in Artificial Not applicable Intelligence: designing Intelligent Machines, Volume 2’, Butterworth-Heinemann editions in association with the Open University, 2001 7. Future work There are several issues that have not been addressed in this paper. Future research should investigate the 116
  7. 7. Mission Mission Objectives Requirements Fault Detection & ID Mission Goal Manager CBR & CSP GOAL Manager: • Update Goals; Flight Path Commands • Eliminate the one reached/unachievable; Mission Planner • Generate new goals CBR, CSP, RL, GA CSP, RL, GA PAYLOAD Integrated Signal and Data Processing Integrated INS/ Signal Integrated Situation GPS Processing Data Fusion Awareness Processing Manager OUTPUTS: Radio • Range KBS, GA, RL Alt117 • Range Rate KBS • Angle ANN, GA, • Potential ID, EO Size, Shape RL World KBS MBS on Air Database vehicle SAR/ GMTI High Level Control On-Board PC Sensor Manger Fault TL, RL, CBR, GA Detection & ID CBR, GA, NN MBS on Sensor )LJXUH $, WHFKQLTXHV DVVRFLDWHG WR 8$9 IXQFWLRQDO VXEVVWHPV UHTXLULQJ KLJK OHYHO RI DXWRQRP
  8. 8. Mid-Level Command Terrain database Commands from Mission Manager Multifunction Flight Path Commands integrated navigation UAV States Adaptive Mode Trajectory Generator Transition RL, CSP, TL KBS, RL, TL, ANN Flight Control/ UAV States Loads/Propulsion/Flight Mechanics Flight Low Level Flight mechanics/Propulsi Control Envelope Protection on/FCS/ Actuators RL, CSP Actuato FCS r Model UAV NN, 6- GA, DOF Fuzzy118 Model Flight Health Manager Mechanics/Propulsion/ CBS, MBS, TL Sensor FCS/Electrical/Air Data Inputs Engine Model DATALINK/INTERFACE Fault Tolerant Control Reconfiguration MBS & CBS, TL, ANN, Fuzzy UAV States MBS for MBS for MBS for FCS Engine Actuators )LJXUH $, WHFKQLTXHV DVVRFLDWHG WR 8$9 IXQFWLRQDO VXEVVWHPV UHTXLULQJ PHGLXP DQG ORZ OHYHO DXWRQRP