Presentation of the paper published in IEEE Conference on Games 2019. ABSTRACT: One of the main problems in the design throughoptimization of car racing bots is the inherent noise in the optimization process: besides the fact that the fitness is a heuristic based on what we think are the keys to success and as such just a surrogate for the ultimate objective, winning races, fitness itself is uncertain due to the stochastic behavior of racing conditions and the rest of the (simulated) racers. The fuzzy-based genetic controller for the car racing simulator TORCS that we have defined in previous works is based on two fuzzy subcontrollers, one for deciding on the wheel steering angle and another to set the car target speed at the next simulation tick. They are both optimized by means of an Evolutionary Algorithm, which considers an already tested fitness function focused on the maximization of the average speed during the race and the minimization of the car damage. The noisy environment asks for keeping diversity high during evolution, that is why we have added a Blend Crossover (BLX-alpha) operator, which is, besides, able to exploit current results at the same time it explores. Additionally, we try to address uncertainty in selection by introducing a novel selection policy of parents based in races, where the individuals are grouped and compete against others in several races, so just the firsts ranked will remain in the population as parents. Several experiments have been conducted, testing the value of the different controllers. The results show that the combination of a dynamic BLX-alpha crossover operator plus the pole position selection policy clearly beats the rest of approaches. Moreover, in the comparison of this controller with one of the participants of the prestigious international Simulated Car Racing Championship, our autonomous driver obtains much better results than the opponent.

1. Beating uncertainty
in racing bot evolution through
enhanced exploration and pole
position selection
Mohammed Salem (University of Mascara, Algeria)
Antonio M. Mora (University of Granada, Spain)
J.J. Merelo (University of Granada, Spain)
IEEE Conference on Games 2019
20-23 August
London, UK

2. Index
Beating uncertainty
in racing bot
evolution
through
enhanced exploration
and
pole position
selection

3. The Open Racing Car Simulator.
Used in Competitions and Testbed for AI
Controllers (Driving bots)
Inputs/Outputs:
– Sensors: Car/Track distances
– Actuators: Steer, Break,
Accelerator, Gear
http://torcs.sourceforge.net/
TORCS
Racing bot

4. TORCS
Racing bot

5. Controller Architecture
Racing bot

6. Target Speed Fuzzy Subcontroller
Racing bot
Mamdani Fuzzy System with
3 Trapezoidal Membership
Functions (Low, Medium, High).

7. Target Speed Fuzzy Subcontroller
Racing bot
IF Front = High THEN TargetSpeed is 280
IF Front = Medium THEN TargetSpeed is 240
IF Front = Low AND M5 = High THEN TargetSpeed is 220
IF Front = Low AND M5 = Medium THEN TargetSpeed is 180
IF Front = Low AND M5 = Low AND M10 = High THEN TargetSpeed is 120
IF Front = Low AND M5 = Low AND M10 = Medium THEN TargetSpeed is 60
IF Front = Low AND M5 = Low AND M10 = Low THEN TargetSpeed is 30
IF Front = MAXDISTSPEED OR M5 = MAXDISTSPEED OR M10 =
MAXDISTSPEED THEN TargetSpeed = 300
Uses a Set of Fuzzy Rules:

8. Steer Control Fuzzy Subcontroller
Racing bot
Mamdani Fuzzy System with 3
Trapezoidal Membership
Functions.
Same inputs (FRONT, M5 and M10).
Set of Fuzzy Rules:
IF Front = High THEN steer = 0
IF Front = Medium AND M10 = High THEN steer = 0.25
IF Front = Medium AND M10 = Medium AND M5 = Medium THEN steer = 0.25
IF Front = Medium AND M10 = Medium AND M5 = Low THEN steer = 0.5
IF Front = Low AND M10 = High THEN steer = 0.5
IF Front = Low AND M10 = Medium AND M5 = Medium THEN steer = 0.75
IF Front = Low AND M10 = Medium AND M5 = Low THEN steer = 0.75

9. Beating uncertainty
in racing bot
evolution
through
enhanced exploration
and
pole position
selection
Index

10. Genetic Algorithm
evolution
Wikimedia

11. Genetic Algorithm
evolution
Population: set of controllers defined each one by
a vector which determines the shape of the
membership functions.
Evaluation: every
candidate controller must
be evaluated in the
environment (TORCS races).
A value must be assigned
to its performance =>
Fitness Function.

12. Genetic Algorithm
evolution
Fitness Function:
- Maximize average speed on the whole circuit while
minimizing damage.
- Focuses on objectives of a (human) driver during a
race, not just on winning or losing => human-like.

13. Genetic Algorithm
evolution
Evaluation process:
AVG(Speed)
Damage

14. Beating uncertainty
in racing bot
evolution
through
enhanced exploration
and
pole position
selection
Index

15. Challenges
uncertainty
Reliable and effective
evaluation process:
 Good individuals <=>
Competitive controllers.
 Selection of the best.
Variable results for the
same controller:
 Weather conditions.
 Track status.
 Non-deterministic behavior of rivals.

16. Solutions
uncertainty
Fitness definition and
evaluation:
 Considering the main factors
(just speed and damage).
 Several races.
 Several laps.
 No rivals.
 Track combining difficult and
easy parts.
Alpine 2 Track

17. Beating uncertainty
in racing bot
evolution
through
enhanced exploration
and
pole position
selection
Index

18. Enhanced
Exploration
Blend Crossover Operator (BLX-a):
 Generates different descendants with random genes.
 The differences (depend on a value).
 If a > 0.5 => higher exploration
 If a < 0.5 => higher exploitation
Two approaches:
 a constant
 a variable

19. Beating uncertainty
in racing bot
evolution
through
enhanced exploration
and
pole position
selection
Index

20. Pole position
selection
Or race-based selection:
 Individuals arranged in groups
of 10.
 They compete in several races of
several laps (same car).
 They obtain a F1-based score
depending on their final
position in every race.
 At the end, those 5 with the
best global score are selected
as parents.
Advantages:
 Less sensitive to uncertainty (fitness-less selection)
 Selected individuals will be almost surely the best.

21. Beating uncertainty
in racing bot
evolution
through
enhanced exploration
and
pole position
selection
Index

22. Experimental Setup
beating
GA parameters:
 Generations = 50
 Population size = 60
 Crossover rate = 0.85
 Mutation rate = 0.09
Genetic operators:
 Random initialization
 Race-based selection [every 5 generations]
Tournament selection [the rest]
 Non-uniform mutation
 BLX-a crossover
- CONSTANT => a = 0.5
- VARIABLE => a = 1-(generation/TOTAL_GENS)
Evaluation:
 Alpine 2 Track, 20 laps, Solo race, car1-tbr1 (moderate performance)

23. Selection of the best
beating
At the end of GA run:
 10 best individuals
(according to fitness).
 5 races.
 5 laps.
 Alpine 2 Track.
 Random starting grid.
 F1-based score: 1st - 25 points, 2nd - 18, 3rd - 15,
4 - 12, 5 - 10, 6 - 8, 7 - 6, 8 - 4, 9 - 2, 10 – 1.
At the end of 10 runs:
 The 10 best individuals (selected before).
 5 races/5 laps in Alpine 2.
 Random starting grid.

24. Results
beating
Facts:
 New approaches improve previous controllers.
 GFC-VA clearly outperforms all the rest.
 Excellent performance even in an unknown track (E-Track 5).
Controllers:
 GFC -> our previous best controller (AVS Fitness).
 GFC-RS -> our previous controller with race-based selection.
 GFC-FA -> new controller with race-based selection and BLX-a
with constant value.
 GFC-VA -> new controller with race-based selection and BLX-a
with variable value.

25. Results
beating
Facts:
 New approaches improve previous controllers.
 GFC-VA clearly outperforms all the rest.
 Excellent performance even in an unknown track (E-Track 5).
Controllers:
 GFC -> our previous best controller (AVS Fitness).
 GFC-RS -> our previous controller with race-based selection.
 GFC-FA -> new controller with race-based selection and BLX-a
with constant value.
 GFC-VA -> new controller with race-based selection and BLX-a
with variable value.

26. PSRI Controller
beating
New Challenger: PSRI
 Controller created by Pérez-
Liébana, Recio, Sáez and
Isasi.
 4th in Simulated Car Racing
Competition 2009.
 Finite State Machine using
fuzzy rules for transitions.
 Optimized by means of NSGA-
II.
 J48 classifier for prediction
of parts of the track.

27. Results
beating
Facts:
 Both GFC-FA and GFC-VA improve all PSRI results.
 PSRI is able to win one of the races on each track.
 GFC-VA is the best of the three (wins 4 and 3 races).
5 races / 20 laps per track. Average results.

28. We have
Future Work:
 Implement completely Race-based selection (in all the
generations).
 Optimize the output of the fuzzy rules.
 Optimize the set of fuzzy rules (Genetic Programming).
Beaten uncertainty
in racing bot evolution
through
enhanced exploration and
pole position selection
conclusions

29. amorag@ugr.es
@amoragar