1. Integrated Computer-Aided Engineering 14 (2007) 141–159 141IOS PressAutonomous and cooperative roboticbehavior based on fuzzy logic and geneticprogrammingJames F. Smith III∗ and ThanhVu H. NguyenCode 5741, Naval Research Laboratory, Washington, DC, 20375-5320, USAAbstract. Advances in a fuzzy decision theory that allow automatic cooperation between unmanned aerial vehicles (UAVs)are discussed. The algorithms determine points the UAVs are to sample, ﬂight paths, and the optimal UAVs for the task andrelated changes during the mission. Human intervention is not required after the mission begins. The algorithms take intoaccount what is known before and during the mission about UAV reliability, fuel, and kinematics as well as the measurementspace’s meteorological states, terrain, air trafﬁc, threats and related uncertainties. The fuzzy decision tree for path assignmentis a signiﬁcant advance over an older fuzzy decision rule that was previously introduced. Simulations show the ability of thecontrol algorithm to allow UAVs to effectively cooperate to increase the UAV team’s likelihood of successfully measuring theatmospheric index of refraction over a large volume. A genetic program (GP) based data mining procedure is discussed forautomatically evolving fuzzy decision trees. The GP is used to automatically create the fuzzy decision tree for real-time UAVpath assignments. The GP based procedure offers several signiﬁcant advances over previously introduced GP based data miningprocedures. These advances help produce mathematically concise fuzzy decision trees that are consistent with human intuition.1. Introduction for sampling and a small number of the best UAVs to conduct the mission. Autonomous cooperative teams of robots will be The planning algorithm takes into account the inputused for many applications in the near future. Funda- properties of each UAV under consideration including:mental to this process will be mission planning prior the UAVs risk tolerance, i.e., how much risk the UAVsto the mission and algorithms for automatic control owner will allow it to experience, expert estimates ofand cooperation of the robots in real-time during the the UAV’s sensor and non-sensor system reliabilities;mission. and the amount of fuel the UAV carries. The planning In the following, algorithms based on fuzzy logic are algorithm also takes into account what is known aboutdescribed that can be used to plan missions for a coor- the atmospheric volume where measurements are to bedinated team of ﬂying robots. The robotic unmanned made, i.e., the measurement space prior to the mission.aerial vehicles (UAVs) will be rendered autonomous. This information includes weather, e.g., rain, turbu-Human intervention is not required after the mission lence, and icing conditions; physical obstructions suchbegins when the algorithms described below are used. as mountains, and high tension wires; enemy behavior; The UAVs’ goal is to cooperate to measure the atmo- air trafﬁc such as civilian or military aircraft or animalspheric index of refraction. The fuzzy mission planning life. This information is used to form a measure of risk,algorithm uses human expertise to determine the points using a fuzzy risk tree. The risk tree allows a simpleto be sampled, the points to avoid, the best ﬂight paths mathematical formulation of the mission risk. Also, the planning algorithm takes into account elec- ∗ Corresponding author. Tel.: +1 202 767 5358; Fax: +1 202 404 tromagnetic propagation [4,9] as well as human exper-7690; E-mail: james.smith@nrl.navy.mil. tise to determine points in the atmosphere to sample,ISSN 1069-2509/07/$17.00 2007 – IOS Press and the author(s). All rights reserved
2. 142 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming AUP RMP RMP SR NSR MP VMR SR NSR MP RISK-TOL VALUE FAST LOW-RISK Fig. 1. The AUP subtree.their priority and a simple mathematical formulation of fuzzy decision trees that are mathematically more con-the mission priority. cise. The trees also assume a form that is more con- A real-time algorithm expressed as a fuzzy decision sistent with human intuition making them easier to un-tree is formulated for assignment of the best UAV to derstand. By making the tree more concise and easiereach path. The algorithm that “assigns UAVs to paths” to understand it is easier to introduce new rules on the(AUP) is referred to as the AUP fuzzy decision tree and trees for the purpose of innovation. Also, concise andit is depicted in Fig. 1. intuitive results frequently facilitate validation. The AUP fuzzy decision tree is a simple elegant Classical if-then rules have been used in the pastmathematical formula determining the degree to which to guide GP evolution, notably for the purpose of re-each UAV belongs to the path in terms of risk-tolerance, verse engineering hardware designs [21,25]. The usesensor and non-sensor reliability, UAV fuel limitations, of fuzzy rules to guide the GP is a signiﬁcant advancemission priority and mission risk. over using classical if-then rules. Fuzzy logic offers a The predecessor to the AUP fuzzy decision tree was better way of dealing with the uncertainties associatedthe AUP fuzzy decision rule [20,22,24]. Both the AUP with partial knowledge.decision rule and AUP decision tree were initially con- A GP is a computer program based on the theorystructed based on human expertise. The AUP fuzzy of evolution that automatically evolves other computerdecision rule had some of the properties of the tree, programs or mathematical expressions [7]. The math-but was more limited in its decision making ability and ematical expressions evolved here are fuzzy decisionsubject to certain types of errors that the tree was de- trees.signed to avoid. The AUP decision tree is a signiﬁcant The GP based procedure is a DM technique, a kindadvance over the AUP decision rule [22,24]. of pattern recognition. The GP mines a database of The AUP fuzzy decision tree is used by both the scenarios to produce an optimal tree. An optimal so-planning and real-time control algorithm. It can make lution is one that maximizes the ﬁtness function. Theextremely fast assignments of UAVs to paths, while ﬁtness function is constructed using the database ofallowing the various input concepts to remain explicit scenarios. The GP is guided by fuzzy logic to improvein the formulation. the GP’s convergence time, to reduce size of the tree Although the AUP decision tree was originally con- and produce elegant mathematical formulations under-structed using human expertise, another method for standable by human beings. Elegance and concisenessevolving it using a genetic program (GP) [7] as a data of form, and understandability are essential to furthermining (DM) function has been created. The GP is innovation. The fuzzy decision trees that are created bycapable of recreating the original tree, but has subse- the GP based DM procedure are unlike black box algo-quently produced trees that are different but arguably rithms like neural nets where understanding parametersuperior in their performance properties. The GP is relations is generally out of the question.guided by built in fuzzy logic based on partial exper- Being able to automatically generate decision algo-tise and related uncertainty. This procedure results in rithms using GP based data mining is a signiﬁcant ad-
3. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 143vance. It is frequently difﬁcult to acquire enough if- The planning and control algorithms described be-then rules from experts to produce a full tree, whereas low could be used for many different cooperative atmo-experts can often provide opinions about the status of spheric measurement processes such as tracking chemi-a scenario. cal plumes [26], determining properties of atmospheric Other approaches to creating optimal ﬂight paths ducts [4], rain systems, etc. The application that moti-with UAV assignments might use a genetic algorithm vated this work was a need to measure the atmospheric(GA) or dynamic programming. Both approaches index of refraction for the purpose of geo-location [20,might be suitable for a pre-mission planning algorithm 24].where there is plenty of time for computationally in- Section 2 discusses the electromagnetic measure-tensive algorithms to run. They are unlikely to be suit- ment space, UAV risk, and the planning algorithm. Sec-able for real-time application, when dealing with slow tion 2 also discusses the UAV path construction algo-legacy processors. The AUP fuzzy decision tree re- rithm that determines the minimum number of UAVsquires little CPU time and can produce effective deci- required to complete the task, a fuzzy logic based ap-sions even on slow legacy processors. Also, the one- proach for assigning paths to UAVs and which UAVstime decisions made by the GA [8,27] or dynamic pro- should be assigned to the overall mission. Finally Sec-gramming approaches [1,30] would be represented as tion 2 discusses a genetic program based data miningnumbers, not mathematical expressions. It would be procedure for evolving the fuzzy decision tree for as-difﬁcult or impossible to extract explicit relationships signing UAVs to paths. Section 3 describes the con-between input quantities like reliability, risk-tolerance, trol algorithm that renders the UAVs autonomous. Sec-etc., and output quantities like the ultimate assignment tion 3 also describes the priority for helping algorithm,of a UAV to a path. a part of the control algorithm based on fuzzy logic that The planning algorithm makes determinations prior determines which UAV should support another UAV re-to the mission’s execution. Inevitably, during the mis- questing help. The three subclasses of help requests aresion, events will occur that demand changes to the pre- also discussed in this section. Section 4 discusses ex-viously determined ﬂight path. The real-time control perimental results including UAV path determination,algorithm allows the UAVs’ task to change during the UAV path assignment, determination of which UAVsmission. These changes can include alteration in ﬂight should ﬂy the mission and the result of a request for helppath, changing sample points and the need for auto- during the mission. Section 5 provides conclusions.matic cooperative behavior between UAVs. As paths Finally, Section 6 describes future research directions.change in real-time the AUP fuzzy decision tree is usedfor reassignment. A fuzzy decision rule referred to as the priority ofhelping (PH) decision rule is also provided as part of the 2. Planning, AUP tree and the evolution of logiccontrol algorithm that allows the UAVs to cooperate au-tomatically in three ways. This automatic cooperation The measurement space consists of the electromag-is based on communication; there is no ﬁxed or cen- netic propagation environment where UAVs and thetral command platform: the UAVs automatically self- IP make their measurements. This environment in-organize. The information transferred between UAVs cludes sample points and the desirable neighborhoodsis a small number of fuzzy grades of membership, so that surround them. The sample points or the desirablecommunication bandwidth requirements are very low. neighborhoods are where the UAVs will make measure- The PH fuzzy decision rule allows UAVs to collabo- ments. The method of determining the sample pointsrate to make atmospheric measurements. It also allows and desirable neighborhoods is described below.them assist each other when malfunctions are suspected The measurement space also includes taboo pointsor take over when a malfunction has occurred. and the undesirable neighborhoods that surround them. The PH fuzzy decision rule is a simple elegant mathe- The taboo points are points of turbulence and other phe-matical relationship between various fuzzy concepts es- nomena that could threaten the UAVs. The undesirablesential to collaboration. It makes explicit relationships neighborhoods surrounding them also represent variousthat would be impossible to discern using a black box degrees of risk. The method of specifying taboo pointslike a neural net [6] or an optimization procedure [1,5, and quantifying the degree of risk associated with their8,11,12,27,28,30] that delivers only a numerical output undesirable neighborhoods employs fuzzy logic and islike a GA or dynamic programming. discussed in this section.
4. 144 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming The planning algorithm allows the determination of point or reside within an undesirable neighborhood.the minimum number of UAVs needed for the mission In the case the sample point coincides with or is nearsubject to fuel constraints, risk, UAV cost, and impor- a taboo point and at least part of the sample point’stance of various points for sampling. Risk refers to desirable neighborhood falls within the taboo point’sturbulent regions or regions undesirable for other rea- undesirable neighborhood, the UAV may only samplesons, e.g., the presence of enemy observers or physi- within a desirable neighborhood that is consistent withcal obstructions. The planning algorithm automatically its risk tolerance.establishes the order in which to send the UAVs tak- A point may be labeled taboo for a variety of rea-ing into account the UAV’s value; onboard sensor pay- sons. A taboo point and the undesirable neighborhoodsload; onboard resources such as fuel, computer CPU containing the point generally represent a threat to theand memory; etc. The priority of sample points and UAV. The threat may take the form of high winds,their desirable neighborhoods are taken into account. turbulence, icing conditions, mountains, etc. The un-The planning algorithm also calculates the optimal path desirable neighborhoods around the taboo point relatearound undesirable regions routing the UAVs to or at to how spatially extensive the threat is. A methodleast near the points to be sampled. of quantifying risk and incorporating it into the path In the planning phase, the location of the electro- assignment algorithm is presented that offers concep-magnetic source (EMS) is unknown. Some positions tual improvements over an approach previously de-are more likely than others for the EMS’s location. veloped [20]. This section uses fuzzy logic to quan-When establishing likely positions for the EMS, human tify how much risk a given neighborhood poses for aexperts are consulted. The experts provide subjective UAV. This quantitative risk is then incorporated into theprobabilities of the EMS being located at a number UAV’s cost for traveling through the neighborhood asof positions. These likely EMS locations are referred described in this section. Once the cost is establishedas hypothesis positions. Ray-theoretic electromagnetic an optimization algorithm is used to determine the bestpropagation [4,20] is conducted from each hypothesis path for the UAV to reach its goal.position to each interferometer element on the interfer- When determining the optimal path for the UAVsometer platform (IP). The points on the sampling grid to follow both the planning algorithm and the controlnearest the points of each ray’s passage are the sample algorithm running on each UAV take into account taboopoints. The priority of a sample point is related to the points and the undesirable neighborhood around eachsubjective probability of the hypothesis position from taboo point. The path planning algorithm and controlwhich the associated ray emerges. Sample points aris-ing from the highest probability hypothesis positions algorithm will not allow a UAV to pass through a taboohave priority one; sample points associated with lower point. Depending on the UAV’s risk tolerance a UAVprobability hypothesis positions, priority two; etc. may pass through various neighborhoods of the taboo Each sample point is surrounded by what are referred point, subsequently experiencing various degrees ofto as desirable neighborhoods. Depending on local risk. Both the concepts of risk and risk tolerance areweather, topography, etc., the desirable neighborhoods based on human expertise and employ rules each ofare generally concentric closed balls with a degree of which carry a degree of uncertainty. This uncertaintydesirability assigned to each ball. The degree of de- is born of linguistic imprecision [24,29], the inabilitysirability characterizes the anticipated variation in the of human experts to specify a crisp assignment for risk.index of refraction. If for that region of the measure- Owing to this uncertainty it is very effective to specifyment space, the spatial variation of the index of refrac- risk and risk tolerance in terms of fuzzy logic.tion is slow, the degree of desirability may assume itsmaximum value of unity for a ball of radius measured 2.1. Risk fuzzy decision treein miles. For regions of space where the index of re-fraction’s spatial variation is greater, the degree of de- Risk is represented as a fuzzy decision tree [3,15–sirability may fall off much more rapidly, approaching 19]. The risk subtree deﬁned below and displayed inthe minimum value of zero after just a mile or two. Fig. 2 is a subtree of the larger risk tree that was actually The desirable region need not have spherical geom- used. The risk tree is used to deﬁne taboo points andetry. Rotational symmetry may be broken by a variety the undesirable neighborhoods surrounding the tabooof processes, e.g., an elevated duct, a radio hole, etc. points. The notion of a desirable neighborhood is motivated The root concepts on the risk tree use the membershipby the fact that a sample point may also be a taboo function deﬁned in Eqs (1)–(3),
5. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 145 Risk Hostile Action/ Traffic Weather Observation Birds/Insects Turbulence Other Temperature Neutral UAVs Civilian Wind Wind- Military Shear Cold Heat Allied Air Pocket Military Own Precipitation Military Icing Hail Suspended Systems & Rain Snow Contaminants Fog Sleet Air- Pollution Smoke Physical Plumes Obstructions Suspended Mountains Trees Sand Buildings High Tension Wires Fig. 2. The fuzzy risk tree and its 25 fuzzy root concepts. µγ (qtaboo , x) (1) Suspended Sand, Birds/Insects, 1, if r = 0 Other UAVs, Air Polution, Civilian, 3/4, if 0 < r 1 · ∆l √ Own Military, Allied Military, = 1/2, if √· ∆l < r 1 √2·∆l 1 /4, if 2·∆l < r √ 3·∆l Neutral Military, Cold, Heat, 0, if r > 3·∆l Icing, Rain, Fog, Sleet, Snow, Hail, r = x − qtaboo , (2) Air Pocket, Wind, Wind Shear, Hostile Action/Observation}. (4) qtaboo = position of taboo point. (3) The norm in Eq. (2) is typically taken as an Euclideanwhere the “taboo point,” q taboo is the point at which the distance. The quantity ∆l for most applications isrisk phenomenon has been observed. The root concepts generally assigned a value of one mile or more. Inused on the risk subtree are given in Eq. (4), and the extreme cases it can be much larger than a mile.subscript γ is an element of the root concept set, RC,i.e., The fuzzy membership function for the composite concept “risk” is deﬁned as γ ∈ RC = {Mountains, High Tension Wires, µrisk (qtaboo , x) = max µα (qtaboo , x) . (5) Buildings, Trees, Smoke Plumes, α∈RC
6. 146 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming n−12.2. Optimal paths and AUP fuzzy decision tree total cost(Pathi ) ≡ path cost(rj , rj+1 ). (9) j=1 The best path algorithm is actually an optimizationalgorithm that attempts to minimize a cost function Determining the optimal path for the i th UAV con-to determine the optimal trajectory for each UAV to sists of minimizing the total path cost given by Eq. (9)follow, given a priori knowledge. The cost function for such that there is enough fuel left to complete the path.the optimization algorithm takes into account various An A-star algorithm [13] is used to determine thefactors associated with the UAV’s properties, mission point that will ultimately minimize the cost function.and measurement space. Two signiﬁcant quantities that A-star is a heuristic algorithm. It was selected becausecontribute to the cost are the effective distance between it is relatively fast and can be polynomial in time [13].the initial and ﬁnal proposed positions of the UAV and Empirically it is signiﬁcantly faster than the Dijkstrathe risk associated with travel. algorithm [13]. It is very effective for cases where a For purposes of determining the optimal path, the signiﬁcant percentage of the atmospheric volume doesUAV is assumed to follow a rectilinear path consisting not change over the course of a mission.of connected lines segments, where the beginning and The A-star algorithm as implemented in the planningending points of each line segment reside on the UAV’s and control algorithms is an easily replaceable mod-sampling lattice. Let A and B be two grid points on ule. It may well be replaced by a more sophisticatedthe UAV’s sampling grid with corresponding position algorithm in the future [5,11,12,28].vectors, rA and rB , respectively. Denote the Euclidean The planning algorithm determines the path eachdistance between A and B as d (r A , rB ). Let v (rA , rB ) UAV will pursue, which points will be sampled, thebe the speed at which the UAV travels in going from minimum number of UAVs required for sampling therA to rB . If both rA and rB are sample points then the points and makes assignments of UAVs for measure-UAV travels at sampling velocity, otherwise it travels ments at particular points. UAVs are assigned as aat non-sampling velocity. The path cost is given by function of their abilities to sample high priority points path cos t (rA , rB ) = ﬁrst. The planning algorithm determines ﬂight paths by ntaboo assigning as many high priority points to a path as pos- d(rA ,rB )+β· µrisk (ti ,rB ) . (6) sible taking into account relative distances including i=1 v(rA ,rB ) sampling and non-sampling velocity, risk from taboowhere ntaboo is the number of taboo points, i.e., points, and UAV fuel limitations. Once ﬂight pathscolumns in the taboo point matrix are determined, the planning algorithm assigns the best UAV to each path using the fuzzy logic decision tree Taboo ≡ t1 , t2 , . . . , tntaboo (7) for path assignment described in this section. The planning algorithm must assign UAVs to theand ti , i = 1, 2, . . . , ntaboo are the taboo points de- ﬂight paths determined by the optimization proceduretermined to exists in the measurement space when described below in this section. This is referred to aspath cos t (rA , rB ) is calculated. The number of points the UAV path assignment problem (UPAP). The plan-in the measurement space is ﬁnite so n taboo is ﬁnite. ning algorithm makes this assignment using the follow-The quantity, β, is an expert assigned parameter. Note ing fuzzy logic based procedure. To describe the de-that path cos t (rA , rB ) is an effective time. When risk ntaboo cision tree it is necessary to develop some preliminaryis not present, i.e., β · µrisk ti , rB is zero, then concepts and notation. i=1 Each UAV will ﬂy from lattice point to lattice point,path cos t (rA , rB ) is the actual travel time. When risk i.e., grid point to grid point, let one such route be givenis present then the travel time is increased. The param- by the matrix of points,eter β helps to penalize risky paths, thus decreasing theprobability they are selected. Path = P1 , P2 , . . . , Pnpath , P1 (10) If the candidate path for the mission consists of thefollowing points on the UAV lattice given by the path where the ordering of points gives the direction ofmatrix in Eq. (8), the route, i.e., starting at P1 and ending at P1 . Let the taboo points be those given in Eq. (7). Let the Pathi = r1 , r2 , . . . , rn , (8) degree of undesirability of the neighborhood associ-then the total path cost is deﬁned to be ated with taboo points, ti , i = 1, 2, . . . , ntaboo be
7. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 147denoted µrisk ti , Pj for the route points Pj , j = onboard the UAV(i). This fuzzy concept relates to any1, 2, . . . , npath . The deﬁnition of the mission risk (MR) non-sensor system, e.g., propulsion, computers, hardis disk, deicing systems, etc. The value of UAV(i) in units of $1000.00 is denoted as V (U AV (i)). The mission risk (Taboo,Pathk ) ≡ amount of fuel that UAV(i) has at time t is denoted ntaboo npath f uel (U AV (i) , t). All the UAVs participating in a µrisk ti , Pj (11) mission are assumed to leave base at time, t = t o . i=1 j=1 Let UAV(i)’s fuzzy grade of membership in the fuzzy concept “risk tolerance” be denoted as µ risk−tol The degree to which the k th path belongs to the (U AV (i)). The quantity, µ risk−tol (U AV (i)), is arelated fuzzy concept MR is given by number between zero and one and will be simply re- µMR (Taboo,Pathk ) ≡ ferred to as UAV(i)’s risk tolerance. If the risk toler- mission risk (Taboo,Pathk ) ance is near zero then the UAV should not be sent on (12) very risky missions. If the UAV’s risk tolerance is near max mission risk Taboo,Pathj j one then it can be sent on very risky missions. It seems The “max” operation in Eq. (12) is taken over the natural to compare “risk tolerance” to “Value.” So theset of all possible UAVs that can be assigned to the comparison can be carried out on the same footing, amission. fuzzy concept of value should be deﬁned. A fuzzy concept related to “mission risk” is “low The fuzzy grade of membership of each UAV thatrisk.” The fuzzy membership function for “low risk” can be assigned to the mission in the fuzzy conceptdenoted as µLR is deﬁned as “Value” is deﬁned as εV · Value (U AV (i)) µLR (Taboo,Pathk ) ≡ µV (U AV (i)) ≡ (16) max {Value (U AV (j))} j min (1, α + 1 − µMR ) (13) The quantity ε V is an expert assigned value. It iswhere α ∈ (0, 1) is an expert deﬁned parameter. The a number between zero and one that insures that thefunction of α is to make sure that “low risk” does not most valuable UAV can still be assigned to a path. Thedominate calculations developed below. motivation for deﬁning this parameter will be clearer Within the path speciﬁed by Eq. (10), let there be after the AUP subtree is deﬁned below, since it can bethe following sample points to be measured, Sj , j = observed that if ε V is zero, then for the most valuable1, 2, . . . , nsp . Let the function prio assign priorities UAV in the team, µAUP will take the value zero. Thisto the sample points, i.e, prio( Sj ) is the priority of the would prevent it from being assigned a path which isj th sample point. The values that prio( Sj ) can take undesirable.are positive integers with one representing the highest The advantage of the concept of “risk tolerance” ispriority, two the next highest priority, etc. The mission that it gives the user an extra concept to exploit. If thepriority (MP) for the k th Pathk is deﬁned to be UAV is not of great relative value, but it still might be nsp needed for a crucial mission after the current one, it 1 mission prio (Pathk ) ≡ . (14) might be useful to give it a low risk tolerance so that it i=1 prio Si is not lost on the current mission. This may allow it to be used on a subsequent mission. The degree to which the k th path belongs to the Another fuzzy concept and related fuzzy member-related fuzzy concept MP is given by ship function that will be deﬁned is “fast.” A UAV mission prio (Pathk ) is said to be fast if it permitted to pursue a particular µMP (Pathk ) ≡ . (15) max {mission prio (Pathj )} path and the time it would take to complete the path j is small. It is not allowed to travel the path unless the The fuzzy degree of reliability experts assign to the UAV’s fuel level, reliability, risk-tolerance and missionsensors of UAV(i) is denoted as µ sr (U AV (i)). This is priority exceed certain tolerances.a real number between zero and one with one implying Let the T (U AV (i) , Path) be the amount of time itthe sensors are very reliable and zero that they are will take UAV(i) to ﬂy and make measurements alongtotally unreliable. Likewise, µ nsr (U AV (i)) is the Path. The fuzzy membership function for the conceptfuzzy degree of reliability of other non-sensor systems “fast” is deﬁned as follows:
8. 148 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming µfast (U AV (i) , P ath) ≡ Λrrtmp · (17) µV MR ≡ min [min (µrisk−tol , 1 − µV ) , AN D2 (µfast , µLR )] (19) T (U AV (i) , Path) min 1, α + 1 − max {T (U AV (j) , Path)} The use of AN D2 in Eq. (19) allows distinctions j to be made between various values of µ fast and µLR .and If AN D2 were replaced by a min in Eq. (19) then if µfast is low enough then min (µ fast , µLR ) would take Λrrtmp ≡ χ [fuel (U AV (i) , to ) + εfuel the value µfast independent of the value of µ LR this −T (U AV (i) , Path)] · would not allow ﬁne distinctions to be made. The logical connective AN D 2 is deﬁned as χ [min (µsr , µnsr ) − ε1,rel · (18) AN D2 (µA , µB ) ≡ µA · µB (20) min(1 − µrisk−tol , The fuzzy concept “RMP” combines the fuzzy con- max(1 − µMP , ε2,MP )) − ε3,rel ] cepts sensor reliability (sr) non-sensor-reliability (nsr)where ε1,rel , ε2,MP , ε3,rel , εfuel ∈ (0, 1] are expert as- and “MP.” The fuzzy membership function for “RMP,”signed parameters. The Heaviside step functions de- denoted as µRMP is deﬁned asnoted as χ in Eq. (18) takes the value one when its µRMP ≡ min (µsr , µnsr , µMP ) . (21)argument is greater than or equal to zero and is zero Table 1 provides an example of the application ofotherwise. Eq. (21) for a three UAV scenario. The factor in Eq. (18) determines whether the UAV Both the membership functions for “VMR” andwill be permitted to travel the path at all. If its amount of “RMP” can be represented as fuzzy decision trees.fuel, sensor and non-sensor reliabilities, risk-tolerance Finally, the fuzzy membership function for the fuzzyand mission priorities do not exceed certain expert de- concept “assignment of UAV(i) to the path” (AUP) isﬁned tolerances it will not be permitted to travel and as deﬁned assuch it will certainly not be “fast.” The term ε1,rel · min(1 − µrisk−tol , max(1 − µMP , µAUP ≡ AN D2 [µRMP ,ε2,MP )) in the Heaviside step function’s argument in AN D2 (µRMP , µV MR )]Eq. (18) can result in Λ rrtmp going to zero if µ risk−tolor µMP are small enough. = µ2 RMP · µV MR (22) The parameter α is selected so that µ fast in Eq. (17) The fuzzy membership function for AUP is a deci-does not go to zero for the UAV that takes the longest sion tree that combines both “VMR” and “RMP” astime to navigate Path. By preventing µ fast from going subtrees. The use of AN D 2 in Eq. (22) in two placesto zero in this case, the slowest UAV can be selected if renders µAUP more sensitive to the values of µ RMPit can complete the path and its grades of membership and µV MR than it would be if the membership func-in the other fuzzy concepts found in Eq. (17) are high tion for AUP took the value min (µ RMP , µV MR ). Ifenough. µAUP were to take the value min (µ RMP , µV MR ) then If “Risk tolerance” and “mission priority” take low a small value of µRMP such that µRMP < µV MRvalues then depending on the value of ε 1,rel , the mem- would cause µAUP to take the value of µ RMP inde-bership function for the fuzzy concept “fast” may take pendent of the value of µ V MR . The use of AN D 2 in-the value zero. The parameter ε 2,MP limits the effect stead of min allows ﬁner distinctions to be made. Theof “mission priority.” Even if the mission priority is second degree dependence of µ RMP in Eq. (22) resultsvery high, risk tolerance plays an important role. If the in a small value of µ AUP if µRMP is small, but µAUPUAV has high risk tolerance and the path, high mis- is still dependent on µ V MR . This is consistent withsion priority the UAV must have a minimum reliability expertise. If the sensor or non-sensor reliabilities orgiven by ε 3,rel . Finally, the motivation for the concept mission priority are small, µ AUP should be small. Low“fast” is that a fast UAV experiences a lower relative reliability or priority results in a faster decline in µ AUPrisk since it is in the ﬁeld less time and may be exposed than high mission risk, high UAV value, low UAV riskto risk for a shorter duration. tolerance or the fact that a reliable and risk-tolerant A fuzzy concept that combines “Value” and “mission UAV is slow.risk” is “VMR” and its membership function denoted It should be observed that the decisions made by AUPas µV MR is deﬁned as are taken over a pool of candidates. So it is reasonable
9. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 149 Table 1 Three UAV example for the composite concept RMP Three UAV Numerical Example for Use of RMP Fuzzy Membership Function UAV 1 UAV 2 UAV 3 Sensor reliability, µsr 0.8 0.7 0.8 Non-sensor reliability, µnsr 0.9 0.4 0.8 Risk Tolerance, µrisk−tol 0.3 0.4 0.5 Mission Priority, µM P 0.6 1 0.4 µRM P ≡ min (µsr , µnsr , µM P ) 0.6 0.4 0.4those UAVs in the pool that have greater sensor and of obtaining Eq. (22) is to evolve it using a geneticnon-sensor system reliabilities and greater relative mis- program (GP). A GP is a computer program based onsion priority, should have a higher fuzzy grade of mem- the theory of evolution that evolves mathematical ex-bership for assignment to the path under consideration. pressions or computer programs that can be consideredAUP’s second degree dependence on RMP may allow optimal in a sense. The GP has been used as a dataRMP to be too dominant in the calculations. In the mining function to create the decision tree in Eq. (22).genetic program based approach described below ver- The GP data mined a scenario database where eachsions of RMP were evolved where the power of RMP scenario had been labeled by an expert. Expert ruleswas between 1.5 and 1.7. These different versions of were also incorporated to guide the evolutionary pro-RMP are still under study. cess and improve convergence time. The decision tree The fuzzy concept AUP is depicted as a tree in Fig. 1. in Eq. (22) has been evolved many times. The GP ﬁndsLeaves of the tree, i.e., those vertices of degree one are the same AUP decision tree, over and over again inde-labeled by the names of the fuzzy concepts described pendent of the seed of the random number generatorabove. Vertices are labeled by the speciﬁc logical con- used to simulate a random evolutionary process. Thenective used, i.e., min or AN D 2 . A circle on an edge GP based procedure is described in greater detail in theindicates the fuzzy logic modiﬁer not. The fuzzy mod- next subsection and an earlier version in [19].iﬁer not is deﬁned as the complement of the fuzzy set,i.e., let µA be the fuzzy membership function for the 2.3. GP creation of the AUP treefuzzy concept A then membership function for not Ais given by 1 − µA . A three UAV example is provided The AUP tree given in the previous subsection arosein Table 1 to illustrate the use of the RMP subtree of from rules provides by human experts. In this subsec-Fig. 1. tion a method of evolving the AUP tree using data min- Given the fuzzy grade of membership it is necessary ing is discussed. This procedure has been successfulto defuzzify, i.e., make deﬁnite UAV-path assignments. in evolving the exactly same tree found in Fig. 1. TheSimply assigning the UAV with the highest fuzzy grade fact that two signiﬁcantly different methods of obtain-of membership for a particular path to that path can give ing the AUP algorithm give the same results provides aless than desirable results. The approach to defuzziﬁ- signiﬁcant level of conﬁdence in the AUP tree. Finally,cation taken is as follows: if the number of UAVs is the fact that the data mining approach is an optimiza-denoted as nUAV and likewise, the number of paths is tion procedure, i.e., it gives results that are optimal in adenoted by n path , where nUAV npath then consider sense should further strengthen conﬁdence in the AUPthe set of all possible permutations of the n path UAVs tree.selected from a total of n UAV UAVs. For each assign- Data mining is the efﬁcient extraction of valuablement of npath UAVs to the paths, add up the values of non-obvious information embedded in a large quantityµAUP for that assignment over the paths. This sum is of data [2]. Data mining consists of three steps: thereferred to as the assignment beneﬁt (AB). The assign- construction of a database that represents truth; the call-ment with the highest AB is the one selected. Finally, ing of the data mining function to extract the valuablea similar procedure is followed if n UAV < npath . information, e.g., a clustering algorithm, neural net, The decision tree for AUP given in Eq. (22) was con- genetic algorithm, genetic program, etc; and ﬁnally de-structed using expertise provided by human experts. termining the value of the information extracted in theIt is a signiﬁcant improvement over a previously de- second step, this generally involves visualization.veloped fuzzy decision rule for path assignment also In a previous paper, a genetic algorithm (GA) wasconstructed from expertise [24]. An alternate method used as a data mining function to determine parameters
10. 150 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programmingfor fuzzy membership functions [14]. Here, a different regression [10]. It is found in symbolic regressiondata mining function, a genetic program [7] is used. A that candidate solutions are frequently not in algebraicgenetic program is a problem independent method for simplest form and this is the major source of their excessautomatically evolving computer programs or mathe- length. When candidate solutions are too long this ismatical expressions. referred to as bloat [10]. The GP data mines fuzzy decision tree structure, i.e., By including in the terminal set a terminal and itshow vertices and edges are connected and labeled in complement, e.g., “risk-tol,” and “not-risk-tol”; “val-a fuzzy decision tree. The GP mines the information ue” and “not-valuable”; etc., it is found that bloat isfrom a database consisting of scenarios. less and convergence of the GP is accelerated. This is The GP evolves a population of decision trees. Each a recent innovation which was not used when anothertree is a candidate solution for the AUP tree. To create resource manager, i.e., the electronic attack resourcethese candidate solutions the GP requires two input manager (EARM) was evolved using GP based datasets. These are the terminal set and function set. mining (DM) [17]. Additional bloat control procedures At the end of each generation the GP ultimately rates are described below and in [23] which provides muchor determines the ﬁtness of each candidate solution pro- less detail than found here.duced during that generation. Its method of rating the The mathematical form of the complement whethercandidate solutions is to calculate their ﬁtness using a it appears in the terminal set or is preﬁxed with a “NOT”ﬁtness function. When using a GP as a data mining logical modiﬁer from the function set is one minus thefunction the GP requires a third type of input. This membership function. To make this more explicitthird input set is a scenario database where each sce- µN OT (A) = µnot−A = 1 − µA , (25)nario has been labeled by an expert with a real numberbetween zero and one. This label corresponds to the where N OT (A) refers to the application of the logi-decision that the optimal decision tree should make. cal modiﬁer N OT from the function set to the fuzzySo it is natural when calculating the ﬁtness for a given concept A from the terminal set. The notation, not-Acandidate solution for the AUP tree to compare its ﬁ- refers to the terminal which is the complement of thenal composite concept fuzzy membership value to the terminal A. The function set, denoted as F, consists ofvalue labeling that scenario. So the scenario databaseis used to construct the ﬁnal ﬁtness function for ulti- F = {AN D1, OR1, AN D2, OR2, N OT } (26)mately determining the optimal AUP decision tree as where the elements of Eq. (26) are deﬁned in Eqs (20),determined by the GP. (27)–(30). Let A and B represent fuzzy membership The terminal set, function set, and ﬁtness functions functions then elements of the function set are deﬁnednecessary for the GP to be used as a data mining func- astion to automatically create the AUP tree are describedbelow. The terminal set used to evolve the AUP tree AN D1 (A, B) = min (A, B) ; (27)consisted of the root concepts from the AUP tree andtheir complements. The terminal set, T, is given by OR1 (A, B) = max (A, B) ; (28) T = {risk-tol, value, fast, low-risk, sr, nsr, OR2 (A, B) = A + B − A · B; (29) MP, not-risk-tol, not-valuable, not-fast, and not-low-risk, not-sr, not-nsr, not-MP}. (23) N OT (A) = 1 − A. (30) Let the corresponding fuzzy membership functions The database to be data mined is a scenario databasebe denoted as kindred to the scenario database used for evolving the EARM [17]. In this instance scenarios are character- {µrisk−tol , µvalue , µfast , µlow−risk , µsr , ized by values of the fuzzy membership functions for µnsr , µMP , µnot−risk−tol , (24) the elements of the terminal set plus a number from µnot−valuable , µnot−fast , zero to one indicating the experts’ opinion about the µnot−low−risk , µnot−sr , µnot−nsr ,µnot−MP } . value of the fuzzy membership function for AUP for When mathematical expressions are constructed by that scenario.a GP that reproduce the entries in a database within GPs require a ﬁtness function [7]. As its name im-some tolerance, the process is referred to as symbolic plies the ﬁtness function measures the merit or ﬁtness
11. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 151of each candidate solution represented as a chromo- The following is a subset of the rules used to accel-some. The ﬁtness used for data mining is referred to as erate the GP’s convergence and to help produce a resultthe input-output ﬁtness. consistent with human expertise. The input-output ﬁtness for mining the scenario R1. “not-valuable” and “risk-tol” must share a log-database takes the form ical connective, denoted as C 1 , i.e., it is desired that γshare (T,not-valuable,risk-tol) = 1. fIO (i, ndb ) ≡ R2. “not-valuable” and “risk-tol” strongly inﬂuence 1 each other, so they should be connected by AND1 or ndb . (31) AND2. So it is desired that 1+2· |µgp (i, ej ) − µexpert (ej )| j=1 µcom (T,not-valuable,risk-tol,C 1 ) = 0.4.where ej is the j th element of the database; n db is R3. “fast” and “low-risk” have an afﬁnity for eachthe number of elements in the database; µ gp (ej ) is the other. They should share a logical connective, denoted asC2 , i.e., it is desired that γshare (T,fast,low-risk) = 1.output of the fuzzy decision tree created by the GP for R4. The fuzzy root concepts “fast” and “low-risk”the ith element of the population for database element strongly inﬂuence each other, so they should be con-ej ; and µexpert (ej ) is an expert’s estimate as to what the nected by AND1 or AND2. So it is desired thatfuzzy decision tree should yield as output for database µcom (T,fast,low-risk,C2 ) = 0.4.element ej . R5. There is an afﬁnity between the fuzzy root con- The AUP tree is evolved in three steps. The ﬁrst step cepts C1 (not-valuable,risk-tol) and C 2 (fast,low-risk),involves evolving the VMR subtree; the second step, they are connected by a logical connective denoted asthe RMP subtree and the ﬁnal step, the full AUP tree. C3 , i.e., it is desired that,In the second and third steps, i.e., evolving the RMPsubtree and full AUP tree from the RMP and VMR γshare (T, C1 (not-valuable,risk-tol) , . (33) C2 (fast,low-risk)) = 1subtrees, only the input-output (IO) ﬁtness in Eq. (31)is calculated, i.e., the rule-ﬁtness described below is When the EARM was evolved by GP based data min-not used. ing [17] bloat was controlled using adhoc procedures When evolving the VMR subtree a rule-ﬁtness is based on tree depth and parsimony pressure. Most ofcalculated for each candidate solution. Only when the the bloat in evolving mathematical expressions with acandidate’s rule ﬁtness is sufﬁciently high is its input- GP arises from the expressions not being in algebraicoutput ﬁtness calculated. The use of the rule-ﬁtness simplest form [10]. With that observation in mind,helps guide the GP toward a solution that will be con- computer algebra routines have been introduced thatsistent with expert rules. Also the use of the rule ﬁtness allow the GP to simplify expressions. The followingreduces the number of times the IO ﬁtness is calculated is a partial list of algebraic simpliﬁcation techniquesreducing the run time of the GP. After some prelimi- used during the evolution of the EARM and the AUP tree. The simpliﬁcation routines used when evolvingnary deﬁnitions of crisp and fuzzy relations, a set of AUP are more sophisticated than those applied to thecrisp and fuzzy rules that were used to help accelerate creation of EARM [17].the GP’s creation of the VMR subtree are given. The One routine simpliﬁes expressions of the formrules are combined to formulate the rule ﬁtness. NOT(NOT(A)) = A. This can be more complicated than Let T be a fuzzy decision tree that represents a ver- it initially appears, since the NOT logical modiﬁers cansion of the VMR subtree, that is to be evolved by a ge- be separated on the fuzzy decision tree.netic program. Let A and B be fuzzy concepts. Then Another simpliﬁcation procedure consists of elimi-let γshare (T, A, B) = 1 if A and B share a logical nating redundant terminals connected by an AND1 log-connective denoted as C and γ share (T, A, B) = 0, ical connective. An example of this is AND1(A,A) = A.otherwise. Like the case with the logical modiﬁer NOT there can Furthermore, deﬁne the fuzzy relation be a separation between the AND1s and the terminals that add complexity to the simpliﬁcation operation. µcom (T, A, B, C) = (32) The third algebraic simpliﬁcation example is like the 0.4 if C = AN D1 or AN D2 second. It involves simplifying terminals connected by 0.1 if C = OR1 or OR2 . OR1s. Like AND1, separation between terminals and 0, otherwise OR1 can increase the complexity of the operation.
12. 152 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming Other types of algebraic simpliﬁcation use DeMor- in an environment where change is conﬁned to smallgan’s theorems in combination with the above proce- regions.dures. This can signiﬁcantly reduce the length of an A UAV may requests help if it discovers a potentialexpression. elevated system like a radio hole, malfunctions or sus- Another algebraic procedure that reduces the length pected malfunctions. All of these conditions can resultof expressions includes replacement of forms like in help messages being transmitted between the UAVs.AND2(A,A) by the square of “A,” i.e., A 2 . Still an- These help messages can result in interactions betweenother length reducing simpliﬁcation includes replacing the UAVs based on transmission of the results of prior-NOT(A) with not-A, its complement from the terminal ity calculations for rendering support to the requestingset listed in Eq. (23). UAVs. There is always a question of how much algebraic Currently in the control stage, when a UAV discov-simpliﬁcation should be conducted from generation to ers an interesting physical phenomenon, is malfunc-generation as such the simpliﬁcation algorithm allows tioning, or suspects due to internal readings that it islevels of simpliﬁcation. If a low level of simpliﬁcation malfunctioning, it sends out a request for help. Eachis selected then some parts of an expression remain that UAV receiving this message calculates its priorities formight be eliminated during full simpliﬁcation. This has providing assistance to the UAV in need. This prioritytwo advantages: it leaves chromosome subcomponents calculation gives rise to a number between zero andthat may prove useful during mutation or crossover and one, inclusive, which is subsequently transmitted to theit takes less CPU time. original UAV desiring support. The requesting UAV Algebraic simpliﬁcation produces candidate solu- sends out an omni-directional message with the ID oftions in simpler form making it easier for human ob- the UAV with highest priority for contributing support.servers to understand what is being evolved. Having The high priority UAV then ﬂies into the necessarycandidate solutions that are easier to understand can neighborhood of the requesting UAV to provide help.be an important feature for improving the evolution of There are three classes of help request. The ﬁrstGPs. occurs when a UAV, the requester, determines it may have discovered an interesting physical phenomenon. This phenomenon may be an elevated duct, radio hole, rain system or some other type of system with phys-3. Control algorithm ical extent. The requester desires to determine if the phenomenon has signiﬁcant extent. It will request that Each UAV has a real-time algorithm onboard it a helping UAV or UAVs sample likely distant pointsthat allows recalculation of paths during ﬂight due to within this phenomenon.changes in environmental conditions or mission prior- The second class of help request relates to a UAV thatities. These changes typically become apparent after according to internal diagnostics may be experiencingthe planning algorithm has run during the pre-ﬂight a sensor malfunction. This UAV will requests thatstage. As in the case of the planning algorithm the con- another UAV or UAVs measure some of the points thattrol algorithm uses an A-star algorithm [13] to do the the requesting UAV measured. This will help determinebest path calculation, employs fuzzy logic and solves a if the UAV is actually malfunctioning. If the requestingconstrained optimization problem. A-star’s and fuzzy UAV is determined to be malfunctioning, then it willlogic’s CPU time requirements have been quite satis- ﬂy back to base, if it is capable. The determinationfactory for this application. of whether it is actually malfunctioning requires some The control algorithms’ recalculation of ﬂight paths consideration. Since the second UAV will probablycan be triggered by a number of events such as weather be measuring a distant point at a time different thanbroadcasts that indicate new taboo regions or changes the original requesting UAV made its measurements,of priority of sample points. For those changes that potential variation in the index of refraction over timedo not require UAVs supporting each other, the control must be taken into account.algorithm does not differ from the planning algorithm. When a UAV sends out an omni-directional requestThe control algorithm is faster by virtue that it only need for help, those UAVs receiving the message will calcu-process those parts of the measurement space where late their fuzzy priority for helping, denoted as “PH.”there have been changes relative to sample or taboo The UAV that will ultimately help the requester is thepoints. The A-star algorithm is particularly effective one with the highest fuzzy priority for helping. The
13. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 153fuzzy priority for helping takes into account a variety It is assumed that all evaluations are processed atof properties of the potential helper. The set of UAVs time, t, so time dependence is suppressed in Eqs (35)–that receive the request for help from UAV(i) at time (39) for notational convenience. A more sophisticatedt is denoted as help(i, t). If UAV(i) request help at version of the control logic that takes path risk, changestime t and UAV(j) receives the message then UAV(j) in risk, UAV reliability, UAV risk-tolerance and missedwill take into account the amount of time, denoted, sample points into account will be the subject of a futurehelp time (U AV (j)), it will take it to ﬂy from the publication.point where it received the request to the point whereit would provide support. It also takes into account theamount of fuel UAV(j) has left at the time of the re- 4. Computational experimentsquest, denoted fuel (U AV (j) , t); UAV(j)’s fuzzy con-cept of price denoted as “price”, and UAV(j)’s fuzzy The planning and control algorithms described in theconcept of “mission priority” at time, t. Let the set of previous sections have been the subject of a large num-relevant UAV properties be denoted as U AV prop and ber of experiments. This section provides a descriptionbe deﬁned as of a small subset of these experiments. They serve to illustrate how the algorithms were tested. U AV prop = UAV experiments using only one UAV demonstrate {help time,fuel,mission prio,price} (34) how the planning and control algorithm will determine the route the UAV ﬂies so that it is successful in making The fuzzy priority for helping denoted as µ P H takes measurements at sample points in space, while the UAVthe form avoids taboo points, that is points in space that could µP H (U AV (i) , U AV (j)) = damage or destroy the UAV. Experiments using two UAVs illustrate how the control algorithm allows the wδ · µδ (U AV (j)) (35) UAVs to automatically support each other to increase δ∈UAV prop the probability their joint mission is successful. The quantities wδ and µδ for δ ∈ U AV prop are Figures 2–6 use the same labeling conventions. Sam-expert deﬁned weights and fuzzy membership func- ple points are labeled by concentric circular regionstions, respectively. The fuzzy membership functions colored in different shades of gray. The lighter theare deﬁned in Eqs (36)–(39) and given below, shade of gray used to color a point, the lower the point’s grade of membership in the fuzzy concept “desirable µhelp time (U AV (i) , U AV (j)) = (36) neighborhood.” The legend provides numerical values −1 for the fuzzy grade of membership in the fuzzy concept help time (U AV (j)) + 1 “desirable neighborhoods”. If the fuzzy degree of de- max {help time (U AV (k))} sirability is high then the index of refraction is consid- k∈help(i,t) ered to be close to the index of refraction of the sam- µfuel (U AV (i) , U AV (j)) = (37) ple point at the center of the desirable neighborhood. This allows the UAV to make signiﬁcant measurements fuel (U AV (j)) while avoiding undesirable neighborhoods. max {fuel (U AV (k))} Each sample point is labeled with an ordered pair. k∈help(i,t) The ﬁrst member of the ordered pair provides the in- dex of the sample point. The second member of the µmission prio (U AV (i) , U AV (j)) = (38) ordered pair provides the point’s priority. For example, −1 if there are nsp sample points and the q th sample point mission prio (U AV (j)) + 1 is of priority p, then that point will be labeled with the max {mission prio (U AV (k))} ordered pair (q, p). k∈help(i,t) Points surrounded by star-shaped neighborhoods µprice (U AV (i) , U AV (j)) = (39) varying from dark grey to white in color are taboo −1 points. As with the sample points, neighborhoods with Value(U AV (j)) darker shades of gray have a higher grade of member- + 1 ship in the fuzzy concept “undesirable neighborhood.” max {Value(U AV (k))} k∈help(i,t) The legend provides numerical values for the fuzzy
14. 154 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming PLANNING PHASE Fig. 3. One UAV trajectory as determined by planning algorithm. CONTROL PHASE Fig. 4. One UAV trajectory as determined by real-time control algorithm.grade of membership in the fuzzy concept “undesirable star as well as being labeled by a dialog box as being anneighborhood.” UAVs with high risk tolerance may ﬂy “old taboo point.” New taboo points and their associ-through darker grey regions than those with low risk ated undesirable neighborhoods are labeled with dialogtolerance. When comparing planning and associated boxes indicating that they are “new.”control pictures, if a point ceases to be taboo, the neigh- Each UAV has three states of motion, it can be at rest,borhood where it resides is marked by a very dim gray traveling at sampling speed or non-sampling speed.
15. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 155 PLANNING PHASE Fig. 5. Trajectory of two UAVs as determined by planning algorithm. CONTROL PHASE Fig. 6. During ﬂight, updates about changes cause the real-time control algorithms on the two UAVs to change their trajectories.Within the simulation the UAV properties other than labeled with a diamond-shaped marker. They ﬂy in thespeed are its risk-tolerance, cost, fuel, sensor reliability direction of the arrows labeling the various curves inand non-sensor reliability. The values associated with Figs 2–5.each property can vary over the pool of UAVs in use. In Figs 2–6, ﬂight paths that can be represented in two UAVs start their mission at the UAV base which is dimensions have been selected for easy comprehension.
16. 156 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programmingIn the general case the ﬂight paths are three dimensional measuring sample points one and two as assigned it byspatial curves, which can be difﬁcult to visualize. the planning algorithm. Just beyond sample point two, Figure 3 provides the sample points, taboo points and UAV(1) experiences a malfunction. UAV(1)’s real-timesample path for one UAV as determined by the planning control algorithm subsequently sends out a help requestalgorithm. It is important to notice that the UAV’s path informing the only other UAV in the ﬁeld, UAV(2) ofpasses directly through each sample point, i.e., through the malfunction. UAV(2)’s control algorithm deter-the center of the concentric circular regions represent- mines a new path for UAV(2) to ﬂy so that the prioritying the fuzzy degree of desirability of neighborhoods. one points, labeled (3,1) and (4,1), that UAV(1) wasFortuitously, the taboo points and their neighborhoods not able to sample are subsequently measured. Af-are so positioned that they do not interfere with the ter UAV(2) measures sample point ﬁve, its new ﬂightUAV’s measurement process or its return to base. path allows it to measure sample points three and four. Figure 4 depicts the actual path the UAV ﬂies as UAV(2)’s control algorithm determined it was very im-determined by the UAV’s real-time control algorithm. portant that these priority one points be measured. Un-The path determined by the control algorithm differs fortunately, due to the extra fuel expended in reassign-from the one created by the planning algorithm due to ing sample points three and four to UAV(2), UAV(2)real-time changes in taboo points. After leaving the did not have enough fuel to measure sample pointsUAV base new weather data was acquired informing the seven and eight which were of priority two. UAV(2)’sUAVs that the exact position of the third sample point, real-time control algorithm determined the best possi-i.e., the one labeled (3,1) actually resides within an ble solution in the face of changing circumstances andundesirable neighborhood. Due to the high priority of limited resources.the sample point and the UAV’s risk-tolerance, the UAV It is important to note that the control algorithmsﬂies into the taboo points’ undesirable neighborhood running on UAV(1) and UAV(2) direct both UAVs toas indicated in Fig. 4. alter their return paths to the base due to the emer- In both the planning and control algorithms the UAV gence of new taboo points making the planning algo-measures sample points of two different priorities, with rithm determined ﬂight paths too dangerous. The con-the direction of the ﬂight path selected so that the higher trol algorithm uses each UAV’s fuzzy risk-tolerance topriority points are measured ﬁrst. By measuring high determine how near each UAV may approach a taboopriority points ﬁrst, the likelihood of an important mea- point.surement not being made is diminished, if the UAV can Figure 7 provides an example of the AUP decisionnot complete its mission due to a malfunction, changein weather, etc. tree’s assignment of three UAVs to three paths. The Also, due to movement of old taboo points or the highest priority locations are assigned to UAV(1) as itemergence of new taboo points which are marked has the greatest fuel capacity, i.e., 90 minutes. UAV(1)“New,” the path determined for the UAV using the con- however does not have enough fuel to handle the hightrol algorithm is signiﬁcantly different than the one cre- priority points located at positions six and seven andated by the planning algorithm. The path change rep- therefore UAV(2) is assigned these points along withresents the control algorithm’s ability to reduce UAV the second degree high priority locations.risk. Table 2 provides numerical details of the tasks de- Figure 5 depicts the sampling path determined by picted in Fig. 7. The column labels have the followingthe planning algorithm for an experiment involving two interpretation: “Location,” the UAV coordinates on theUAVs. The ﬁrst, UAV(1) follows the dashed curve; map; “Fly mode,” whether the UAV sampled from itsthe second, UAV(2), the solid curve. The UAVs were previous location to its current position. If the UAVassigned to the different paths by the fuzzy path assign- sampled then a “S” was entered. “NS” was enteredment decision tree described in Section 2. UAV(1) is if sampling did not occur. “Fuel Time” refers to howassigned to sample all the highest priority points, i.e., much fuel remained by the time the UAV reached thethe priority one points. UAV(2) samples the lower pri- associated location.ority points, i.e.; those with priority two. Due to thegreedy nature of the point-path assignment algorithm,the highest priority points are assigned for sampling 5. Conclusions and summaryﬁrst. Figure 6 depicts the actual ﬂight path the UAVs take Fuzzy logic algorithms for planning and control of aduring real-time. Initially, UAV(1) is successful in coordinated team of robots making atmospheric mea-
17. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 157 Table 2 Details of three UAV mission depicted in Fig. 7 Three UAV Mission UAV 1 MISSION UAV 2 MISSION UAV 3 MISSION Locations Fly Fuel Time Locations Fly Fuel Time Locations Fly Fuel Time Mode Remain Mode Remain Mode Remain (minutes) (minutes) (minutes) Base 90.0 Base 85.0 Base 85.0 (1,1) NS 76.5088 (6,1) NS 67.9691 (11,3) NS 64.2839 (2,1) S 61.5088 (7,2) S 55.2412 (12,3) S 51.0412 (3,1) S 54.2662 (8,2) S 47.9986 (13,3) S 39.5559 (4,1) S 42.7809 (9,2) S 39.5133 (14,3) S 31.0706 (5,1) S 28.2956 (10,2) S 22.028 Base NS 6.2574 Base NS 6.7113 Base NS 11.7854 PLAN PHASE X Plane Y Plane Fig. 7. Three UAV mission described in Table 2, an example of the AUP decision tree’s assignment.surement have been developed. The algorithms exhib- for subsequent innovation. Only having an implicitited excellent performance under extensive testing in knowledge of the relationship between concepts as fre-digital simulation. quently occurs if only an optimization approach is used The fuzzy logic algorithms provide explicit, con- is less desirable. Such implicit relationships can makecise, elegant mathematical relationships between input innovation considerably more difﬁcult.root concepts like risk-tolerance,sensor reliability, non- The fuzzy logic based algorithms require little CPUsensor system reliability, mission-risk, mission-priority time and can function in real-time even on many slowand output composite fuzzy concepts like “assign UAV legacy processors. They also require very little memoryto path.” Such mathematical relationships are valuable storage.
18. 158 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming The fuzzy logic algorithms allow a group of ﬂying consulting with experts to obtain rules. In the absencerobots to collaborate without a central or ﬁxed com- of a good set of rules it should be possible to construct amander. Decisions are made based on communica- scenario data base for mining by the GP. A partial set oftion. The team of robots automatically self-organize. rules about the anticipated shape of fuzzy decision treesThis is valuable since the loss of anyone robot will not can be used to help accelerate the GP’s convergence anddestabilize the team. Late arriving robots may join the produce simpler, more concise results understandablegroup and contribute to the team’s well-being without by human beings as in the AUP decision tree case.difﬁculty. From the practical perspective of modeling physical Self-organization arises by the UAV’s transmitting systems, the ﬁdelity of the underlying UAV models, theonly small amounts of information. The UAVs only electromagnetic propagation model, and environmentaltransmit processed information, fuzzy grades of mem- models will be increased.bership, resulting in very low bandwidth requirements. Increases in ﬁdelity of the UAV model will includeSince the bandwidth requirements are low, no expen- speciﬁcs of the UAV engine model, e.g., how doessive data compression is required. This means physical its efﬁciency change with altitude. The ray-theoreticpower and time are saved. This is a valuable feature for electromagnetic propagation model will be partially re-future inexpensive, disposable, low powered systems. placed with a ray-wave-theoretic hybrid [9]. This will A method of creating fuzzy decision logic using an allow greater ﬁdelity in modeling rough surface scat-algorithm related to the theory of evolution, a genetic tering from terrain and also various types of ductingprogram (GP) is discussed. This algorithm’s evolution phenomena. By using a hybrid model increases in CPUis guided by expert provided scenarios in a data base time and memory requirements can be kept in checkand expert rules in the form of fuzzy logic. Ultimately, as the ﬁdelity of the modeling of underlying physicsthe GP evolves a fuzzy decision tree that is optimal, increases.with respect to the expertise provided for the desired Changes in modeling ﬁdelity will also result in al-task. The GP has been successful in reproducing known terations of the cost function related to ﬂight path de-results as well as creating new distinct algorithms. termination. It will probably also prove useful, eventu- The GP’s output, the fuzzy decision logic is concise, ally to explore more sophisticated algorithms for pathelegant and understandable by humans. This is largely determination [5,11,12,28] that will replace A-star. Andue to the use of partial expertise embedded in the GP algorithm of greater sophistication may be useful forin the form of fuzzy rules and innovative bloat con- rapidly changing environments or when there are moretrol mechanisms like computer algebra. All the bloat obstructions or threats within the environment.control mechanisms contribute to shorter solutions, butthe use of fuzzy rules within the GP to guide evolutionproduces results closer to human intuition. Acknowledgements In the case where the decision logic is the same asthat found by interviewing experts, it is easy to argue This work was sponsored by the Ofﬁce of Navalthe GP’s results are understandable by humans,after all, Research. The authors would also like to acknowledgethe GP has reproduced results handed down by human Mr. Alan Schultz, Dr. Lawrence Schuette, Dr. Jeffreyexperts. In the case where the results are different from Heyer, Dr. Francis Klemm, and Dr. Gregory Cowart.those obtained from expertise, they can be related tothe expert results and the differences understood. References6. Future directions [1] M. Azizoglu, S. Subrarnaniam and A.K. Somani, Converter placement on wavelength-routed network paths, in: Con- The various techniques for controlling GP based verter placement on wavelength-routed network paths III, John Senior, ed., San Diego, SPIE Proceedings, October 1997,bloat, i.e., generating more concise results must be ex- pp. 265–275.amined to determine their assets and liabilities. Also, [2] J.P. Bigus, Data Mining with Neural Nets, New York,various techniques for accelerating the convergence of McGraw-Hill, 1996. [3] S. Blackman and R. Popoli, Design and Analysis of Modernthe GP will be explored. Tracking Systems, Chapter 11, Boston, Artech House, 1999. Additional fuzzy logic algorithms will be added to [4] L.V. Blake, Radar Range-Performance Analysis, Artechgive the UAVs greater ﬂexibility. This will require House, Boston, 1986.
19. J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming 159 [5] K.R. Buck, R. Gassner, A.B. Poore and X. Yan, Visibility, [18] J.F. Smith, Fuzzy logic resource manager: real-time adapta- Constrained Routing of Unmanned Aerial Vehicles, in: Sig- tion and self organization, in: Signal Processing, Sensor Fu- nal Processing, Sensor Fusion, and Target Recognition VIII, sion, and Target Recognition XIII, Vol. 5429, I. Kadar, ed., (Vol. 3720), I. Kadar, ed., SPIE Proceedings, Orlando, Florida, Orlando, SPIE Proceedings, April 2004, pp. 77–88. April 1999, pp. 256–266. [19] J.F. Smith, Genetic Program Based Data Mining for Fuzzy De- [6] C. Jeffries, Tracking, code recognition, and memory manage- cision Trees, in: Proceedings of the International Conference ment with high order neural networks, in: Applications and for Intelligent Data Engineering and Automated Learning, H. Science of Artiﬁcial Neural Networks, Vol. 2492, S. Rogers, Yin, ed., Exeter, Springer-Verlag, August 2004, pp. 464–470. ed., SPIE Proceedings, Orlando, Florida, April 1995, pp. 364– [20] J.F. Smith and T.H. Nguyen, Distributed autonomous systems: 373. resource management, planning, and control algorithms, in: [7] J.R. Koza, F.H. Bennett III, D. Andre and M.A. Keane, Genetic Signal Processing, Sensor Fusion, and Target Recognition Programming III: Darwinian Invention and Problem Solving, XIV, Vol. 5809, I. Kadar, ed., Orlando, SPIE Proceedings, Morgan Kaufmann Publishers, San Francisco, 1999. April 2005, pp. 65–76. [8] J. Latourell, B. Wallet and B. Copeland, Genetic Algorithm [21] J.F. Smith, III and T.H. Nguyen, Data mining based automated to Solve Constrained Routing Problem with Applications for reverse engineering and defect discovery, in: Data Mining, In- Cruise Missile Routing, in: Applications and Science of Com- trusion Detection, Information Assurance, and Data Networks putational Intelligence, Vol. 3390, S. Rogers, ed., SPIE Pro- Security 2005, B. Dasarathy, Vol. 5812, April 2005, Orlando, ceedings, Orlando, Florida, April 1998, pp. 490–500. SPIE Proceedings, 2005, pp. 232–242. [9] M. Levy, Parabolic equation methods for electromagnetic [22] J.F. Smith and T.H. Nguyen, Fuzzy Logic Based UAV Allo- wave propagation, Institution of Electrical Engineers, Herts, cation and Coordination, in: Proceedings of the Third Inter- United Kingdom, 2000. national Conference on Informatics in Control, Automation[10] S. Luke and L. Panait, Fighting Bloat with Nonparametric Par- and Robotics: Intelligent Control Systems and Optimization, simony Pressure, in: Parallel Problem Solving from Nature – J.A. Cetto, ed., Setubal, Portugal, ICINCO 2006 Proceedings, PPSN VII, 7th International Conference Proceedings, LNCS August 2006, pp. 9–18. Vol. 2439, J.J.M. Guervos, ed., Berlin, Springer-Verlag, 2002, [23] J.F. Smith III and T.H. Nguyen, Guiding Genetic Program pp. 411–421. Based Data Mining Using Fuzzy Rules, in: Proceedings of the[11] R. Mittu and J.K. Uhlmann, Routing and Advanced Display International Conference for Intelligent Data Engineering and Technologies within STOMPM, in: Robotic and Semi-Robotic Automated Learning, H. Yin, ed., Burgos, Spain, Springer- Ground Vehicle Technology Vol. 3366, G. Gerhart, ed., Or- Verlag, September 20–23, 2006. lando, Florida, SPIE Proceedings, April 1998, pp. 177–188. [24] J.F. Smith and T.H. Nguyen: Resource manager for an au-[12] H.G. Nguyen, H.R. Everett, N. Manouk and A. Verma, tonomous coordinated team of UAVs, in: Signal Processing, Autonomous mobile communication relays, in: Unmanned Sensor Fusion, and Target Recognition XV: 62350C, I. Kadar, Ground Vehicle Technology IV Vol. 4715, G.R. Gerhart, C.M. ed., Orlando, SPIE Proceedings, April 2006, 104–114. Shoemaker and D.W. Gage, eds, Orlando, Florida, SPIE Pro- [25] J.F. Smith, III and T.H. Nguyen, Genetic program based data ceedings, April 2002, pp. 50–57. mining to reverse engineer digital logic, in: Data Mining,[13] S.J. Russel and P. Norvig, Artiﬁcial Intelligence: A Modern Intrusion Detection, Information Assurance, and Data Net- Approach, (2nd ed.), Englewood Cliffs, Prentice-Hall, 2002. works Security 2006, Vol. 6241, B. Dasarathy, Orlando, SPIE[14] J.F. Smith, III and R. Rhyne, II, A Resource Manager for Proceedings, April 2006. Distributed Resources: Fuzzy Decision Trees and Genetic [26] D. Spears and D. Zarzhitsky, Multi-Robot Chemical Plume Optimization, in: Proceeding of the International Conference Tracing, in: Multi-Robot Systems: From Swarms to Intelligent on Artiﬁcial Intelligence, IC-AI’99, Vol. II, H. Arabnia, ed., Automata, Vol. III, A. Schultz, ed., New York, Springer, May Las Vegas CSREA Press, 1999, pp. 669–675. 2005, pp. 211–221.[15] J.F. Smith, Co-evolutionary Data Mining to Discover Rules for [27] S.R. Thangiah and K.E. Nygard, School bus routing using Fuzzy Resource Management, in: Proceedings of the Inter- genetic algorithms, in: Applications of Artiﬁcial Intelligence national Conference for Intelligent Data Engineering and Au- X: Knowledge-Based Systems, Vol. 1707, G. Biswas, Orlando, tomated Learning, H. Yin, ed., Manchester, Springer-Verlag, Florida, SPIE Proceedings, April 1992, pp. 387–398. August, 2002, pp. 19–24. [28] K. Thyagarajan, R. Batta and M.H. Karwan, Planning Dissim-[16] J.F. Smith, Data Mining for Fuzzy Decision Tree Structure ilar Paths for Military Units, Progress Report Department of with a Genetic Program, in: Proceedings of the International Industrial Engineering, University of Buffalo SUNY, Buffalo, Conference for Intelligent Data Engineering and Automated NY 14260, USA, 2004. Learning, H. Yin, ed., Manchester, Springer-Verlag, August [29] L.H. Tsoukalas and R.E. Uhrig, Fuzzy and Neural Approaches 2002, pp. 13–18. in Engineering, Chapter 5, New York, John Wiley and Sons,[17] J.F. Smith, Fuzzy logic resource manager: decision tree topol- 1997. ogy, combined admissible regions and the self-morphing prop- [30] S. Verma and A.K. Saxena, Polynomial Time Algorithm for erty, in: Signal Processing, Sensor Fusion, and Target Recog- Hop Constrained QoS Routing, in: Internet Quality of Service, nition XII, Vol. 5096, I. Kadar, ed., Orlando, SPIE Proceed- Vol. 5245, M. Atiquzzaman and M. Hassan, eds Orlando, ings, April 2003, pp. 104–114. Florida, SPIE Proceedings, April 2003, pp. 131–135.