Car Racing
    Competition
Daniele Loiacono, Julian Togelius
      and Pier Luca Lanzi



   CIG, 17 December 2008, Perth,...
Goal
 Learn  or design a controller for TORCS that
  races as fast as possible alone or in the
  presence of others drive...
Car Racing Competition meets
TORCS
 More  representative of real game AI
 Better interaction with human players
 Many g...
4
The Open Racing Car
Simulator
The Open Racing Car
Simulator
 TORCS is a state of the art open source simulator
  written in C++
 Main features
      ...
Competition API
 To make TORCS more easy to
  use we developed an API based
                                     TORCS
  ...
Sensors
 Rangefinders    to sense
   Opponents
   Edges of the track

 Speed
 Positionon track
 Rotation speed of wh...
Effectors
 Basically    4 effectors
   Steeringwheel [-1,+1]
   Gas pedal [0, +1]
   Brake pedal [0,+1]
   Gearbox {-...
Effectors
 Basically    4 effectors
   Steeringwheel [-1,+1]
   Gas pedal [0, +1]
   Brake pedal [0,+1]
   Gearbox {-...
The Competitors
Submissions
5   entries have been submitted to the competition:
    Matt Simmerson – Independent Researcher
  Luigi Car...
Matt Simmerson:
     NEAT Controller
• Controller implemented as an arbitrary-
  topology neural network, evolved with the...
Politecnico di Milano




    Evolving a
  Fast Controller
    for TORCS
   Using NEAT


Luigi Cardamone
 Politecnico di M...
Overview


  We applied NEAT to evolve a fast controller for the
   Car Racing Simulator
  To evaluate the candidate sol...
Sensors and effectors




                        Neuroevolution in TORCS Cardamone
                                      ...
Sensors and effectors




                        Neuroevolution in TORCS Cardamone
                                      ...
Sensors and effectors




                        Neuroevolution in TORCS Cardamone
                                      ...
Sensors and effectors




                        Neuroevolution in TORCS Cardamone
                                      ...
Sensors and effectors




                        Neuroevolution in TORCS Cardamone
                                      ...
Sensors and effectors




                        Neuroevolution in TORCS Cardamone
                                      ...
Sensors and effectors


   The following sensors were used:
    – Track sensors at -90°,-60°,-30°,+30°,+60°,+90°
    – Fr...
Controller design


   To avoid wasting time with fast but slower controller,
    we set gas pedal to 1 (the max value) w...
Evaluation of Controllers


   The fitness is computed on an entire lap as




    – Where Tout is the number of game tic...
An example of behaviors evolved




                                  Neuroevolution in TORCS Cardamone
                  ...
Aravind Gowrisankar
 Based  on NEAT
 Preliminary work




                      20
“Redjava” (Chung-Cheng Chiu)
   The driving policy is based on determining which sensor detects the
    most distant valu...
The Results
Submissions
5   entries have been submitted to the competition:
    Matt Simmerson – Independent Researcher
  Luigi Car...
Scoring setup
A server with 2 Quad-core Xeon 2.66 GHz,
 8GB RAM, running Fedora Core 6
Scoring process: first stage
 Scoring
        process involves three tracks
 (unknown to the competitors):
  C-Speedway
 ...
First Stage: C-Speedway
                                        C-Speedway

                  Redjava
                    ...
First Stage: E-Track 6
                                          E-Track 6

                     Luigi
                  R...
First Stage: Wheel 2
                                           Wheel 2

                  Redjava
                     Lu...
Comments on firs stage
 Two   clear winners:
  Redjava
  Luigi

 WCCI  Champ still competitive in all but
  Wheel 2 tra...
The final stage
 Who   is the best controller in presence of
  opponents?
 In the final stage we compared the best
  entr...
Final results
 After   25 races…




          Luigi Cardamone is the winner of the
             CIG 2008 Simulated Car R...
Summary of results
 Why   did Luigi win this competition?
 It is not the fastest but the most reliable one:
  Redjava’s...
Summary of results
 Good   things:
  Significant improvements with respect to the previous
   competition!
  One new le...
CIG-2008 Car Racing Competition
Upcoming SlideShare
Loading in …5
×

CIG-2008 Car Racing Competition

1,348 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,348
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CIG-2008 Car Racing Competition

  1. 1. Car Racing Competition Daniele Loiacono, Julian Togelius and Pier Luca Lanzi CIG, 17 December 2008, Perth, Australia
  2. 2. Goal  Learn or design a controller for TORCS that races as fast as possible alone or in the presence of others drivers  “Spiritual successor” of CEC 2007 Competition, direct successor of WCCI 2008
  3. 3. Car Racing Competition meets TORCS  More representative of real game AI  Better interaction with human players  Many good programmed controllers available  Challenges  How to make it easy accessible?  Not designed for Machine Learning!  More similar to a real-world problem
  4. 4. 4
  5. 5. The Open Racing Car Simulator
  6. 6. The Open Racing Car Simulator  TORCS is a state of the art open source simulator written in C++  Main features  Sophisticated dynamics  Provided with several cars, tracks, and controllers  Active community of users and developers  Easy to develop your own controller  OS Support  Linux: binaries and building from sources suppo  Windows: binaries and “limited” bulding from sources support  OSX: legacy binaries and no building from sources support 
  7. 7. Competition API  To make TORCS more easy to use we developed an API based TORCS on socket (UDP) Patch  Values of sensors and effectors are sent through UDP Server BOT  3 components  Torcs Patch UDP  Server Bot (C++)  Client API (C++ and Java) Client Controlller
  8. 8. Sensors  Rangefinders to sense  Opponents  Edges of the track  Speed  Positionon track  Rotation speed of wheels Opponent Sensors  RPM  Angle with track  Distance raced  Fuel and damage  ... Track Sensors
  9. 9. Effectors  Basically 4 effectors  Steeringwheel [-1,+1]  Gas pedal [0, +1]  Brake pedal [0,+1]  Gearbox {-1,0,1,2,3,4,5,6}
  10. 10. Effectors  Basically 4 effectors  Steeringwheel [-1,+1]  Gas pedal [0, +1]  Brake pedal [0,+1]  Gearbox {-1,0,1,2,3,4,5,6}
  11. 11. The Competitors
  12. 12. Submissions 5 entries have been submitted to the competition:  Matt Simmerson – Independent Researcher  Luigi Cardamone – Politecnico di Milano  Chung-Cheng Chiu “Redjava” - Academia Sinica  Diego Perez - University Carlos III, Madrid  Aravind Gowrisankar - UT Austin, USA 3 more controllers have been considered  Daniele’s heuristic C++ controller  Julian’s heuristic Java controller  WCCI 2008 champion (entered by Matt Simmerson)
  13. 13. Matt Simmerson: NEAT Controller • Controller implemented as an arbitrary- topology neural network, evolved with the NEAT4j implementation of NEAT • Inputs: speed, angle to track axis, wall rangefinders, track position, gear, wheel spin, RPM • Output: driving and gear selection
  14. 14. Politecnico di Milano Evolving a Fast Controller for TORCS Using NEAT Luigi Cardamone Politecnico di Milano Neuroevolution in TORCS Cardamone Luigi
  15. 15. Overview  We applied NEAT to evolve a fast controller for the Car Racing Simulator  To evaluate the candidate solution we measured their performance on a complete lap  The evolved neural controller was then coupled with a gear shifting policy and with a basic obstacle avoidance behavior Neuroevolution in TORCS Cardamone Luigi
  16. 16. Sensors and effectors Neuroevolution in TORCS Cardamone Luigi
  17. 17. Sensors and effectors Neuroevolution in TORCS Cardamone Luigi
  18. 18. Sensors and effectors Neuroevolution in TORCS Cardamone Luigi
  19. 19. Sensors and effectors Neuroevolution in TORCS Cardamone Luigi
  20. 20. Sensors and effectors Neuroevolution in TORCS Cardamone Luigi
  21. 21. Sensors and effectors Neuroevolution in TORCS Cardamone Luigi
  22. 22. Sensors and effectors  The following sensors were used: – Track sensors at -90°,-60°,-30°,+30°,+60°,+90° – Frontal sensor: max reading among the frontal track sensors at -10°,0°,10° – Car speed  The network controls – Steering wheel – Gas/Brake pedals Neuroevolution in TORCS Cardamone Luigi
  23. 23. Controller design  To avoid wasting time with fast but slower controller, we set gas pedal to 1 (the max value) when the car is on a straight (i.e., when the frontal sensor return 100m as reading)  In addition, the neural controller does not deal with gear shifting and the scripted policy provided is used instead  We also used a very simple scripted policy to avoid at least the opponents that are “close and in front” of the bot Neuroevolution in TORCS Cardamone Luigi
  24. 24. Evaluation of Controllers  The fitness is computed on an entire lap as – Where Tout is the number of game tics the bot was outside the track – Savg is the average speed – D is the distance raced  As soon as a Tout becomes greater than 500 game tics, the evaluation is immediately stopped Neuroevolution in TORCS Cardamone Luigi
  25. 25. An example of behaviors evolved Neuroevolution in TORCS Cardamone Luigi
  26. 26. Aravind Gowrisankar  Based on NEAT  Preliminary work 20
  27. 27. “Redjava” (Chung-Cheng Chiu)  The driving policy is based on determining which sensor detects the most distant value, and turn the steer according to its angle between driving direction.  When the car is in the track and the gear is above 3, then it act in the similar policy as SimpleDriver (but without speed limit). Otherwise, accelerate. (brake is opposite)  The policy except steering adopt similar code as SimpleDriver (also the unstuck detection).  The code is naive since my original attempt wai to apply a machine learning method but I got bad results, thus I removed that part. 21
  28. 28. The Results
  29. 29. Submissions 5 entries have been submitted to the competition:  Matt Simmerson – Independent Researcher  Luigi Cardamone – Politecnico di Milano  Redjava - ???  Diego Perez - University Carlos III, Madrid  Aravind - ??? 3 more controllers have been considered  Daniele’s heuristic C++ controller  Julian’s heuristic Java controller  WCCI 2008 champion (entered by Matt Simmerson)
  30. 30. Scoring setup A server with 2 Quad-core Xeon 2.66 GHz, 8GB RAM, running Fedora Core 6
  31. 31. Scoring process: first stage  Scoring process involves three tracks (unknown to the competitors):  C-Speedway  E-Track6  Wheel 2  Onlya controller at once is tested and performance is defined as the distance covered within 10000 game tics
  32. 32. First Stage: C-Speedway C-Speedway Redjava Luigi Diego Competitors Matt Aravind WCCI Champ Daniele Julian 0 5000 10000 15000 20000 Distance Raced
  33. 33. First Stage: E-Track 6 E-Track 6 Luigi Redjava Aravind Competitors Diego Matt WCCI Champ Daniele Julian 0 2000 4000 6000 8000 Distance Raced
  34. 34. First Stage: Wheel 2 Wheel 2 Redjava Luigi Matt Competitors Diego Aravind WCCI Champ Daniele Julian 0 2250 4500 6750 9000 Distance Raced
  35. 35. Comments on firs stage  Two clear winners:  Redjava  Luigi  WCCI Champ still competitive in all but Wheel 2 track  Matt’s new entry is outperformed by the WCCI champ in all tracks but Wheel 2 (the most difficult track)
  36. 36. The final stage  Who is the best controller in presence of opponents?  In the final stage we compared the best entries with the previous champion  For each track we run 5 races with random starting grids  The final score is computed as the median of the 5 races using the GP2 point system (10 to first, 8 to second, 6 to third AND 1 bonus point for the fastest lap)
  37. 37. Final results  After 25 races… Luigi Cardamone is the winner of the CIG 2008 Simulated Car Racing
  38. 38. Summary of results  Why did Luigi win this competition?  It is not the fastest but the most reliable one:  Redjava’s controller is extremely fast but often looses the control of the car  It often reaches the max amount of damage or just crashes into other opponents  Better shifting policy? (though both hard-coded)
  39. 39. Summary of results  Good things:  Significant improvements with respect to the previous competition!  One new learned controller performed significantly better than programmed controllers!  Hand-coding still very competitive  An important step to the actual level of the best programmed controller in TORCS… but the distance is still big  Bad things:  Still poor performance in avoiding and overtaking opponents!  Many crazy and “dirty” behaviors to go as fast as possible… is really what we are looking for

×