![The pseudo-code for evolving
neural networksInput:
int MAX_POPULATION, int MAX_GENERATION,
int number_of_game
GENE[] PacMan::EANN{
int generation=0;
int number_of_population=0;
int number_of_game=0;
int score=0;
int avg_score=0;
NEAT::GENE[] POPULATION=new GENE(MAX_POPULATION);
NEAT::Parameters params=new ECParameters();
LoadParameters(params);
RandomPopulation(POPULATION);
for(int i=0;j<MAX_GENERATION;j++){
for(int j=0;i<MAX_POPULATION;i++){
PacManSimulator(
POPULATION[j],number_of_game
);
}
if(i<MAX_GENERATION-1){
/* Fitness sharing, Selection,
crossover,and mutation */
NEAT::Generation(POPULATION, params);
}else Sort_by_fitness(POPULATION);
}
return POPULATION[0]; // best gene
}](https://image.slidesharecdn.com/random-151013135717-lva1-app6891/75/AI-44-2048.jpg)
More Related Content
![[Nov 26] introduction to AI / ML](https://cdn.slidesharecdn.com/ss_thumbnails/nov26introductiontoaiml-211127034520-thumbnail.jpg?width=640&height=640&fit=bounds)
![[SwiftStudy 2016] 2장. Swift 타입 파트 1](https://cdn.slidesharecdn.com/ss_thumbnails/swift-study-02-type-part1-160801152736-thumbnail.jpg?width=640&height=640&fit=bounds)
![[SwiftStudy 2016] 3장. 함수](https://cdn.slidesharecdn.com/ss_thumbnails/swift-study-03-function-160808124058-thumbnail.jpg?width=640&height=640&fit=bounds)
![[SwiftStudy 2016] 1장. Swift 소개](https://cdn.slidesharecdn.com/ss_thumbnails/swift-study-01-introduction-160730154405-thumbnail.jpg?width=640&height=640&fit=bounds)
게임 AI를 통해 본 인공지능 기본 개념
- 1. 게임 AI를 통해본 인공지능 기본 개념 EPP개발팀 오근현
- 2.
- 3.
- 4.
- 5. 인공지능에 대한 다양한관점 • 데이터 • 수학/통계 • 정보 검색 • 심리/인지과학 • 컴퓨터과학 • and so on
- 6. 튜링테스트 • 튜링 테스트(Turingtest): 기 계가 인간과 얼마나 비슷하게 대화할 수 있는지를 기준으로 기계에 지능이 있는지를 판별 하고자 하는 테스트 • “컴퓨터로부터의 반응을 인간 과 구별할 수 없다면 컴퓨터는 생각(사고, thinking)할 수 있는 것” - 앨런 튜닝
- 7. 오늘의 관점 Artificial Intelligent? Human-likeIntelligent
- 8. Human-Like? Definition: Software Agentsin AI An agent is a computer system that is situated in some environment, and that is capable of autonomous action in this environment in order to meet its design objectives
- 9. 대다수 인공지능 연구의목적 Problem Solving 사람을 대신하거나 도와서 어떤 문제를 해결
- 10.
- 11.
- 12.
- 13.
- 14. 문제해결을 위해 필요한것 • 경험: Explicit Memory ==> Data • 규칙: Implicit Memory ==> Hand-coded Rules • 사고방식: Implicit Memory ==> Model, Machine Learning
- 15. 기계학습 • 학습: 직간접적경험이나 훈련에 의해 지속적으로 지각하고, 인지하며, 변 화시키는 행동 변화 • 기계학습 정의 • 컴퓨터가 학습할 수 있도록 하는 알고리즘과 기술을 개발하는 분야 • 컴퓨터에게 배울 수 있는 능력, 즉 코드로 정의하지 않은 동작을 실행하 는 능력에 대한 연구 분야 • 기계 학습에서의 일반화는 훈련 이후 새롭게 들어온 데이터를 정확히 처리할 수 있는 능력 • 문제해결의 규칙/수식을 알고 기계에게 알려주는 방식은 기계학습이 아님
- 16. Three niches formachine learning • Data mining: using historical data to improve decisions • ex) medical records —> medical knowledge • Software applications we can’t program by hand • ex) autonomous driving, speech recognition • Self customizing programs • ex) Newsreader that learns user interests • (출처: lecture slides for textbook Machine Learning, T. Mitchell, MacGraw Hill, 1997)
- 17. Ms. Pac-Man 관련기계학습방법 • 인공신경망(Artificial Neural Networks) • 진화연산(Evolutionary Computation) • 진화신경망(Evolved Artificial Neural Networks) • 공통특징 • 소프트컴퓨팅 • 학습 결과 모델이 추출됨 • 사람의 수학/통계적 해석이 어려움(거의 불가능)
- 18. 인공신경망 • 신경망: 인간의중추신경계. 감각을 인식하고 반응을 하기까지 정보 처리는 뉴런들 사이의 정보 이동을 통해 진행 • 인공신경망: 인공신경망은 시냅스의 결합으로 네트워크를 형성한 인 공 뉴런(노드)이 학습을 통해 시냅스의 결합 세기를 변화시켜, 문제 해결 능력을 가지는 모델 출처: http://study.zum.com/book/13893 출처: https://ko.wikipedia.org/wiki/인공신경망
- 19. 인공신경망 연구 예시 •이미지 인식 • 글씨 인식 • 군집화(SOM: Self- Organized Map) • 딥러닝(Deep Learning) http://www.cs.cmu.edu/~tom/faces.html
- 20. When to considerANN • Input is high-dimensional discrete or real-valued • Output is discrete or real valued • Output is a vector of values • Possibly noisy data • Form of target function is unknown • Human readability of result is unimportant • ex) Image classification, Financial prediction, Speech phoneme recognition • (출처: lecture slides for textbook Machine Learning, T. Mitchell, McGraw Hill, 1997)
- 21. 진화연산 • EC: EvolutionaryComputation • 자연계의 진화에서 아이디어를 얻음(다윈주의) • 환경에 적합한 유전자를 가진 종이 다음 세대에 자신의 유전자를 전달할 확률이 높음 • 환경에 대한 반응을 유전자로 정의하고 실제 환경에 얼마 나 적합한지를 평가하여 다음 세대에 반영
- 22. 유전자 알고리즘 • 유전자:개체의 특징이 정의 (주로 배열) • 적합도(Fitness): 유전자가 환경에 적합한 점수. 선택의 기준이 됨 • 선택(Selection): 다음 세대에 유전자를 전달할 개체 후보 선택 1 1 0 1 0 0 1 1
- 23. 유전자 알고리즘(계속) • 교차/교배(Crossover):두 부모 유전자의 특성을 조합하 여 새로운 유전자를 생성 • 돌연변이(Mutation): 교배없이 유전자가 스스로 변이 출처: http://www.j-kosacs.or.kr/journal/article.php?code=11856
- 24. 진화 연산 연구예시 • Game Theory • Genetic Programming • Artificial Life • Evolving the user interface • https://blog.mozilla.org/ ux/2013/04/firefox-next- evolving-the-user- interface-using-genetic- algorithms/
- 25. 진화신경망 • ENN: EvolvedNeural Networks, Evolutionary Neural Networks, Evolving Neural Networks • 진화연산을 이용하여 신경망 학습 • 게임, 로봇 등 평가를 위해 시뮬레이션이 필요한 학습에 다양하게 이용 • HyperNEAT/NEAT • Kenneth O. Stanley, et al. “A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks,” Artificial Life journal vol. 15, no. 2, pp. 185–212. Cambridge, 2009 • 신경망의 토폴로지 기반 진화연산 사용 • Hidden Layer의 구조와 연산 방법을 진화연산으로 결정 • C++/C# 라이브러리 제공(http://eplex.cs.ucf.edu/hyperNEATpage/)
- 26.
- 27.
- 28. A Hybrid Methodof Dijkstra Algorithm and Evolutionary Neural Networks for Ms. Pac-Man Agent Keunhyun Oh and Sung-Bae Cho Republic of Korea World Congress on Nature and Biologically Inspired Computing 2010
- 29. Background: Game AI •Ideal test environments for artificial intelligent • Happening random events • Huge decision space • Challenging task to find Good Strategy
- 30. Background: Ms. Pac-Man •One of the most popular area in game AI • Navigating Ms. Pac-Man without user control • Accumulating points and avoiding attacking ghosts
- 31. Motivation: Previous works •Two Methods for Ms. Pac-Man agent • Human-defined rules • GOOD: Good rules’ guarantying stable high scores • BAD: Difficulty in covering every situation because of non- deterministic ghosts • Evolutionary Computation • GOOD: Covering uncertain environment • BAD: Difficulty in designing an architecture for getting good score and consuming high costs such as time
- 32. Motivation: Hybrid Approach •To help covering each method’s advantage and covering weakness • Basically using rules based on Dijkstra algorithm • Using evolved artificial neural networks to deal with unpredicted circumstances that cannot be determined by defined rules
- 33. Ms. Pac-Man Competition •Providing the best software controller for the game of Ms. Pac-Man • Annual popular competition • IEEE CIG • IEEE CEC • The world record (2010) • Human player: 921,360 • Software agent: 17,102 (avg. of 10 games)
- 34. Ms. Pac-Man Competition(cont’) •Game version • the Microsoft Revenge of Arcade port of the Ms Pac-Man game • WebPacman: http:// www.webpacman.com/ • Software toolkit: Java and C#
- 35. Approach: Hand-Coded Rule •Designed by human experts who understand the game well • If-then rules and graph search algorithms • Tree search strategy(D. Robles and S. M. Lucas, 2009) • RAMP(A. Fitzgerald et al., 2009) • Ice Pambush(R. T. H. Matsumoto and C. Tokuyama, 2008) • Advantage: High scoring controllers • Disadvantage: Hard to consider every situation
- 36. Approach: EC • Methodsusing evolutionary computation • Problem solving using genetic algorithm • Evolved artificial neural networks and fuzzy systems that are often used • Reinforcement learning using EC(I. Szita and A. Lorenz, 2007) • Evolved neural networks(S. M. Lucas, 2005) • M. Gallagher and A. Ryan, 2008 • Advantage: Ability to deal with unpredicted situations • Disadvantage: Hard to get high scores
- 37. Overview of Ms.Pac-Man Game Agent
- 38. Sensing • Catching theinformation on the current situation • Screen capture • Input the game state to the game agent • using capture of a game interface • Pixel extractor: Finding the color of each pixel • Feature extractor • Understanding coordinates of each components • ex) Location of pills and ghosts • Information extactor • Producing information on the environment • ex) Distance between Ms. Pac-Man and a ghost
- 39. Acting • Agent’s controllingMs. Pac-Man after next direction is determined • Using keyboard hooking
- 40. Thinking based onthe hybrid method
- 41.
- 42. Evolutionary Neural Networks •To response to uncertain circumstances • Using NEAT Algorithm(K. O. Stanely and R. Miikkulainen, 2002)
- 43. Evolutionary Neural Networks •Input(#: 24): Game state • Output(#: 4): Direction of Ms. Pac-Man(Up, Down, Right, Left) Component Parameter Type The nearest ghost, The nearest edible ghost, The nearest pill, The nearest power pill Distance Float Up Bool Down Bool Right Bool Left Bool Ms. Pac-Man Up Bool Down Bool Right Bool Left Bool
- 44. The pseudo-code forevolving neural networksInput: int MAX_POPULATION, int MAX_GENERATION, int number_of_game GENE[] PacMan::EANN{ int generation=0; int number_of_population=0; int number_of_game=0; int score=0; int avg_score=0; NEAT::GENE[] POPULATION=new GENE(MAX_POPULATION); NEAT::Parameters params=new ECParameters(); LoadParameters(params); RandomPopulation(POPULATION); for(int i=0;j<MAX_GENERATION;j++){ for(int j=0;i<MAX_POPULATION;i++){ PacManSimulator( POPULATION[j],number_of_game ); } if(i<MAX_GENERATION-1){ /* Fitness sharing, Selection, crossover,and mutation */ NEAT::Generation(POPULATION, params); }else Sort_by_fitness(POPULATION); } return POPULATION[0]; // best gene }
- 45. Experiment • Simulating threeagents 10 times • 1st: Evolved neural networks • 2nd: Dijkstra based rules • 3rd: Hybrid method(Proposed method) • Ms. Pac-Man Software Kit • C# toolkit developed by J. Flensbank and G. Yannakakis • Containing Ms. Pac-Man simulator(we use it) • url: http://mspacmanai.codeplex.com/
- 46. Experiment Settings • Thresholdof Dijkstra based rules: 2 • Parameters for evolving neural networks Parameter Value Population 100 Generation 1000 The mutation rate for connection’s weight 0.2 The mutation rate to add and delete nodes 0.2 Elitism proportion 0.1 Selection proportion 0.8
- 47.
- 48.
- 49. The score ofeach controller • Max and avg score: The hybrid method is outstand! • Min score: Though designed rules are highest, the gap among the proposed methods is tiny
- 50. Statistics of decisions inthe hybrid method • The average number of decisions • The Dijkstra-based rules • Mainly used • Guarantying hish scores • The evolved neural network • Controlling her in only special environments • Covering weakness of human-designed rules • One decision’s influencing the overall game Method # of decisions % of decisions The Dijkstra-based rules 71504.6 0.99860343 The evolved neural network 100.5 0.00139657
- 51. Summary • Hybrid methodfor optimal Ms. Pac-Man agent • Using hand-coded rules and the evolved neural network • Human designed rule based on Dijkstra algorithm • Guarantying best choice in some cases • Not covering every circumstance —> using evolved neural network • Evolutionary computation • Making her go to the safety location in every environment • Helping human designed rule more safety • The method’s getting higher scores than them of rules EC separately