Your SlideShare is downloading. ×
  • Like
2009 Simulate Car Racing Championship
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

2009 Simulate Car Racing Championship

  • 4,047 views
Published

Results of the 2009 Simulated Car Racing Championship

Results of the 2009 Simulated Car Racing Championship

Published in Sports , Automotive
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
4,047
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
49
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The 2009 Simulated Car Racing Championship Daniele Loiacono, Julian Togelius, and Pier Luca Lanzi Simulated Car Racing Championship @ CIG-2009
  • 2. 2009 Simulate Car Racing Championship  The goal is to submit a learned or developed controller for TORCS, an open source racing simulator, that is able to drive fast alone and in presence of opponents  Championship involves 9 races and will be run during 3 Conferences IEEE CEC-2009, Trondheim (Norway), May 18th – May 21st ACM GECCO-2009, Montreal (Canada), July 8th – 12th IEEE CIG-2009, Milan (Italy), September 7th – 11th  Winners awarded on the basis of their scoring in each conference competition.  At the end, the team with highest overall score will be the winner the championship Simulated Car Racing Championship @ CIG-2009
  • 3. Motivations Simulated Car Racing Championship @ CIG-2009
  • 4. Motivations  Simulated Car Racing is a funny and exciting competition  It is possible to see and play with the entries of this competition  It looks really attractive to students and people outside the field Simulated Car Racing Championship @ CIG-2009
  • 5. Goals of Simulated Car Racing  Proposing a challenging competition: not designed with Computational Intelligence in mind computationally expensive noisy and “not completely” deterministic more similar to a real-world problem  Proposing a relevant game-based competition: more representative of commercial games AI real-time dealing with a lot of practical issues a lot of programmed AI available for comparison human players can interact with AI Simulated Car Racing Championship @ CIG-2009
  • 6. The Open Racing Car Simulator
  • 7. The Open Racing Car Simulator  TORCS is a state of the art open source simulator written in C++  Main features Sophisticated dynamics Provided with several cars, tracks, and controllers Active community of users and developers Easy to develop your own controller  OS Support Linux: binaries and building from sources Windows: binaries and “limited” bulding from sources support OSX: legacy binaries and no building from sources support  Simulated Car Racing Championship @ CIG-2009
  • 8. Competition API  To make TORCS more easy to use we developed an API based on socket (UDP)  Values of sensors and effectors are sent TORCS through UDP Patch  3 components Torcs Patch Server BOT Server Bot (C++) Client API (C++ and Java) UDP Client Controlller Simulated Car Racing Championship @ CIG-2009
  • 9. Sensors  Rangefinders for… …edges of the track …opponents  Speed, RPM, fuel, damage, angle with track, distance race, position on track, etc. Simulated Car Racing Championship @ CIG-2009
  • 10. Effectors  Basically 4 effectors Steering wheel [-1,+1] Gas pedal [0, +1] Brake pedal [0,+1] Gearbox {-1,0,1,2,3,4,5,6} Simulated Car Racing Championship @ CIG-2009
  • 11. Submissions
  • 12. Entries submitted  CEC-2009 Luigi Cardamone, Politecnico di Milano (champ CIG-2008) Diego Pérez & Yago Sáez, University Carlos III, Madrid Jan Quadflieg, TU Dortmund Thies Lönneker & Martin V. Butz, University of Würzburg Enrique Onieva & David A. Pelta., Consejo Superior de Investigaciones Cientificas & Universidad de Granada Chung Cheng Chiu, Academia Sinica  GECCO-2009 Jorge Muñoz, University Carlos III, Madrid Bernardi, Ciambelli, Fiocchetti, Manfucci and Pizzo, University of Perugia Wong Ka Chun, The Chinese University of Hong Kong Witold Szymaniak, Poznan University of Technology  CIG-2009 Marc Ebner and Thorsten Tiede, Eberhard Karls Universität, Tübingen Wolf-Dieter Beelitz, BHR Engineering Ltd., Pforzheim Simulated Car Racing Championship @ CIG-2009
  • 13. Types of entries  Mainly evolved/Learned Luigi Cardamone, Politecnico di Milano (champ CIG-2008) Bernardi, Ciambelli, Fiocchetti, Manfucci and Pizzo, University of Perugia Witold Szymaniak, Poznan University of Technology Marc Ebner and Thorsten Tiede, Eberhard Karls Universität, Tübingen  Programmed behaviors developed with a significant CI contribute Thies Lönneker & Martin V. Butz, University of Würzburg Jan Quadflieg, TU Dortmund Diego Pérez & Yago Sáez, University Carlos III, Madrid  Mainly programmed Enrique Onieva & David A. Pelta., Consejo Superior de Investigaciones Cientificas & Universidad de Granada Wolf-Dieter Beelitz, BHR Engineering Ltd., Pforzheim Chung Cheng Chiu, Academia Sinica Jorge Muñoz, University Carlos III, Madrid C. Guse and D. Vrajitorum, Indiana University South Ben Wong Ka Chun, The Chinese University of Hong Kong Simulated Car Racing Championship @ CIG-2009
  • 14. Championship standings Competitor CEC-2009 GECCO-2009 CIG-2009 Total Onieva&Pelta 22 32 ? 54 COBOSTAR 28.5 16.5 ? 45 Luigi (Champ2008) 20 23 ? 43 Perez&Saez 16 11 ? 27 Mr. Racer 19 0 ? 19 Redjava 14 0 ? 14 Epic 0 13 ? 13 Simplicity 0 12.5 ? 12.5 DRT 0 11 ? 11 Jorge Muñoz 0 8.5 ? 8.5 Witold 0 0 ? 0 Simulated Car Racing Championship @ CIG-2009
  • 15. First evaluation stage
  • 16. Scoring process: first stage  Scoring process involves three tracks (unknown to the competitors): Buzzard-Raceway Forza Migrants  Only a controller at once is tested and performance is defined as the distance covered within 10000 game tics Simulated Car Racing Championship @ CIG-2009
  • 17. First Stage: Buzzard-Raceway DRT Witek Witold Epic Perez&Saez Ebner&Tiede Simplicity Luigi (Champ2008) Wolf-Dieter Redjava Jorge Fuentes COBOSTAR Onieva&Pelta Mr. Racer 0 2000 4000 6000 8000 10000 Simulated Car Racing Championship @ CIG-2009
  • 18. First Stage: Forza DRT Witek Witold Epic Perez&Saez Ebner&Tiede Simplicity Luigi (Champ2008) Wolf-Dieter Redjava Jorge Fuentes COBOSTAR Onieva&Pelta Mr. Racer 0 2000 4000 6000 8000 10000 12000 14000 Simulated Car Racing Championship @ CIG-2009
  • 19. First Stage: Migrants DRT Witek Witold Epic Perez&Saez Ebner&Tiede Simplicity Luigi (Champ2008) Wolf-Dieter Redjava Jorge Fuentes COBOSTAR Onieva&Pelta Mr. Racer 0 2000 4000 6000 8000 10000 12000 Simulated Car Racing Championship @ CIG-2009
  • 20. First stage: summary Rank Competitor Forza Buzzard Migrants Total 1 COBOSTAR 6 10 10 26 2 Onieva&Pelta 8 8 8 24 3 Jorge Muñoz 5 5 6 16 4 Mr. Racer 10 4 0 14 5 Wolf-Dieter 3 6 5 14 6 Redjava 4 2 4 10 7 Luigi (Champ2008) 2 3 2 7 8 Perez&Saez 0 0 3 3 9 Ebner&Tiede 0 1 1 2 10 Simplicity 1 0 0 1 11 Witek Witold 0 0 0 0  COBOSTAR and Onieve&Pelta Clearly the fastest controllers… … but also the most reliable  In each track the performances of best controllers are very close  COBOSTAR outperfoms Berniw (one of the best programmed controller provided with TORCS) on two tracks Simulated Car Racing Championship @ CIG-2009
  • 21. Diego Pérez et al. University Carlos III, Madrid 8th Simulated Car Racing Championship @ CIG-2009
  • 22. CIG-2008 Champ Luigi Cardamone Politecnico di Milano 7th Simulated Car Racing Championship @ CIG-2009
  • 23. Redjava Chung Cheng Chiu Academia Sinica, Taipei 6th Simulated Car Racing Championship @ CIG-2009
  • 24. Wolf-Dieter Beelitz, BHR Engineering Ltd., Pforzheim 5th Simulated Car Racing Championship @ CIG-2009
  • 25. Mr Racer Jan Quadflieg 4 th TU Dortmund Simulated Car Racing Championship @ CIG-2009
  • 26. Jorge Muñoz University Carlos III, Madrid 3rd Simulated Car Racing Championship @ CIG-2009
  • 27. Enrique Onieva Consejo Superior de Investigaciones Cientificas, La Poveda-Arganda del Rey, Madrid David A. Pelta Universidad de Granada Simulated Car Racing Championship @ CIG-2009 2nd
  • 28. COBOSTAR Thies Lönneker and Martin V. Butz University of Würzburg 1st http://www.coboslab.psychologie.uni-wuerzburg.de Simulated Car Racing Championship @ CIG-2009
  • 29. Final evaluation stage
  • 30. The final stage  Who is the best controller in presence of opponents?  In the final stage we compared all the entries with the previous CIG-2008 champion  For each track we run 5 races with random starting grids  The final score is computed as the median of the 5 races using the F1 point system (10 to first, 8 to second, 6 to third 5, 4, 3 to the last)  In addition two 2-points bonus are awarded to The controller that reported less damages at the end of the race The controller that achieved the fastest lap of the race Simulated Car Racing Championship @ CIG-2009
  • 31. The Final Stage: Buzzard-Raceway Competitor Buzzard Forza Migrant Total COBOSTAR 11 11 Onieva&Pelta 9 9 Jorge Muñoz 6 6 Luigi (Champ 2008) 4.5 4.5 Wolf-Dieter 4 4 Mr. Racer 3.5 3.5 Perez & Saez 3 3 Redjava 2.5 2.5 Simulated Car Racing Championship @ CIG-2009
  • 32. The Final Stage: Forza Competitor Buzzard Forza Migrant Total Onieva&Pelta 9 12 21 COBOSTAR 11 7 18 Jorge Muñoz 6 3.5 9.5 Mr. Racer 3.5 6 9.5 Wolf-Dieter 4 5.5 9.5 Luigi (champ 2008) 4.5 4 8.5 Redjava 2.5 4 6.5 Perez & Saez 3 2.5 5.5 Simulated Car Racing Championship @ CIG-2009
  • 33. The Final Stage: Migrants Competitor Buzzard Forza Migrant Total COBOSTAR 11 7 12 30 Onieva&Pelta 9 12 8 29 Wolf-Dieter 4 5.5 6 15.5 Jorge Muñoz 6 3.5 5 14.5 Luigi (champ 2008) 4.5 4 4 12.5 Mr. Racer 3.5 6 2.5 12 Redjava 2.5 4 4 10.5 Perez & Saez 3 2.5 4 9.5 COBOSTAR is the winner of CIG-2009 Car Racing Competition! Simulated Car Racing Championship @ CIG-2009
  • 34. Championship 2009 Competitor CEC-2009 GECCO-2009 CIG-2009 Total Onieva&Pelta 22 32 29 83 COBOSTAR 28.5 16.5 30 75 Luigi (Champ2008) 20 23 12.5 55.5 Perez&Saez 16 11 9.5 36.5 Mr. Racer 19 0 12 31 Redjava 14 0 10.5 24.5 Jorge Muñoz 8.5 14.5 23 Wolf-Dieter 15.5 15.5 Epic 13 0 13 Simplicity 12.5 0 12.5 DRT 11 0 11 Witold 0 0 0 Ebner&Tiede 0 0 Onieva & Pelta winners of 2009 Simulated Car Racing Championship! Simulated Car Racing Championship @ CIG-2009
  • 35. Conclusions (1)  Really a lot of improvement from the first edition of TORCS- based competition at WCCI-2008 (June, 2008) WCCI-2008 CIG-2009 Simulated Car Racing Championship @ CIG-2009
  • 36. Conclusions (2)  Pool of competitors is getting bigger and competition is getting more and more difficult  The best controllers are at the same fast and very reliable: they are able to win even from the last position of the starting grid  Many controllers improved their opponent avoidance/overtaking behavior  The best controllers among the submitted ones are competitive with the programmed controllers in the game! Simulated Car Racing Championship @ CIG-2009
  • 37. What about CI?  Learn or evolve a controller from scratch does not seem to lead to the best performance  The winner is actually mainly programmed  However most of the controllers involves CI: Parameters optimization Learning target speed/trajectory Learning complex behaviors (?) Simulated Car Racing Championship @ CIG-2009
  • 38. What next?  We plan to organize again this competition next year keeping the championship formula  We are definitely interested in performances but we also wish to see novel approaches and CI-relevant works: Introducing a training stage to encourage learning Improving sensory inputs to be competitive with “professional” TORCS controller Evaluating controllers also on specific tasks (e.g., race start, overtaking, etc.) as a part of the competition  Simulated Car Racing requires a big effort… how to keep people interested? Awards and prizes at major conferences Dissemination of results Simulated Car Racing Championship @ CIG-2009
  • 39. Thank You! Any Question? Simulated Car Racing Championship @ CIG-2009