• Save
Chess end games using Neural Networks
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Chess end games using Neural Networks

on

  • 15,008 views

 

Statistics

Views

Total Views
15,008
Views on SlideShare
7,888
Embed Views
7,120

Actions

Likes
3
Downloads
0
Comments
3

12 Embeds 7,120

http://chessprogramming.wikispaces.com 6462
https://chessprogramming.wikispaces.com 590
http://translate.googleusercontent.com 20
http://www.slideshare.net 18
http://pprabhu.freehostia.com 12
http://www.e-presentations.us 8
http://webcache.googleusercontent.com 3
http://www.google.com 2
http://131.253.14.66 2
http://207.46.192.232 1
http://131.253.14.98 1
https://twitter.com 1
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Chess end games using Neural Networks Presentation Transcript

  • 1. - Presented By - Kavitha S. - GH 03437 Shilpa Kendre - GH 03438 Kaustubh Kolhar - GH 03439 Priyanka Prabhu - GH 03454 Implementation of Self-Learning in Chess using Neural Networks
  • 2. LEARNING
    • acquisition of new declarative knowledge
    • development of motor and cognitive skills through instruction or practice
    • organization of new knowledge into effective general representations
    • discovery of new facts and theories through observation and experimentation.
  • 3. LEARNING AGENTS
  • 4. NEED FOR LEARNING
    • unknown environments
    • system construction
    • improve performance
  • 5. LEARNING APPROACHES
    • Rote Learning
    • Learning from instruction
    • Learning by analogy
    • Learning from examples
    • Learning from observation and discovery
    More inductive reasoning
  • 6. LEARNING REPRESENTATIONS Taxonomies Procedural encodings Frames and Schemas Graphs and Networks Formal logic Production rules Formal grammars Decision Trees Parameters in algebraic expressions Data
  • 7. TYPE OF FEEDBACK
    • Supervised learning: correct answers for each example
    • Unsupervised learning: correct answers not given
    • Reinforcement learning: occasional rewards
  • 8. INDUCTIVE LEARNING
    • Simplest form: learn a function from examples
    • f is the target function
    • An example is a pair ( x , f(x) )
    • Problem: find a hypothesis h
      • such that h ≈ f
      • given a training set of examples
    • (This is a highly simplified model of real learning:
      • Ignores prior knowledge
      • Assumes examples are given)
  • 9. INDUCTIVE LEARNING METHOD
    • Construct/adjust h to agree with f on training set
    • ( h is consistent if it agrees with f on all examples)
    • E.g., curve fitting:
  • 10.
    • Construct/adjust h to agree with f on training set
    • ( h is consistent if it agrees with f on all examples)
    • E.g., curve fitting:
    INDUCTIVE LEARNING METHOD
  • 11.
    • Construct/adjust h to agree with f on training set
    • ( h is consistent if it agrees with f on all examples)
    • E.g., curve fitting:
    INDUCTIVE LEARNING METHOD
  • 12.
    • Construct/adjust h to agree with f on training set
    • ( h is consistent if it agrees with f on all examples)
    • E.g., curve fitting:
    INDUCTIVE LEARNING METHOD
  • 13.
    • Construct/adjust h to agree with f on training set
    • ( h is consistent if it agrees with f on all examples)
    • E.g., curve fitting:
    • Ockham’s razor: Prefer the simplest hypothesis consistent with data
    INDUCTIVE LEARNING METHOD
  • 14. WHY USE NEURAL NETWORKS
    • Shortcomings of Procedure approach
      • - precise knowledge of the mapping function
      • - cant learn from mistakes
      • - static, no adaptability
    • Shortcomings of Expert Systems
      • - need of expert for knowledge acquisition and rule specification
      • - space requirements intensive
  • 15.
    • Shortcomings of Decision trees
    • - Attributes need to be known
    • Shortcomings of Formal grammars and Production systems
    • -Rules need to be known
    • -Rules are not dynamic
    • -Sequential processing of rules
    WHY NEURAL NETWORKS
  • 16.
    • Constructs knowledge from examples
    • Discovers rules on its own
    • Fault tolerant
    • Parallel
    • Distributed
    • Inductive
    • Adaptible to changes in environment
    • Learns from mistakes
    WHY NEURAL NETWORKS
  • 17. LEARNING USING NEURAL NETWORKS
    • A process by which the free parameters of a neural network are adapted through a process of stimulation by the environment in which the neural network is embedded
  • 18. APPLICATIONS OF NEURAL NETWORKS Problems where intelligence and induction is required Network Type Networks Use for Network Prediction
    • Back-propagation
    • Delta Bar Delta
    • Directed Random Search
    • Self-organizing map into Backpropagation
    • Use input values to predict some output
    • Pick the best stocks in the market
    • Identify people with cancer risks etc.
    Classification
    • Learning Vector Quantization
    • Counter-propagation
    • Probabilistic Neural Networks
    • Use input values to determine the classification
    • Is the input the letter A
    • Is the blob of video data a plane
    Data Association
    • Hopfield
    • Boltzmann Machine
    • Hamming Network
    • Bidirectional associative Memory
    • Spation-temporal Pattern Recognition
    • Like Classification; also recognizes data that contains errors
    • Not only identify the characters that were scanned but identify when the scanner isn't working properly
    Data Conceptualization
    • Adaptive Resonance Network
    • Self Organizing Map
    • Analyze the inputs to infer relationships
    • Extract from a database the names most likely buyers of a product
    Data Filtering Recirculation
    • Smooth an input signal
    • Take the noise out of a telephone signal
  • 19. BENEFITS OF NEURAL NETWORKS
    • Nonlinearity
    • Input-Output Mapping
    • Adaptivity
    • Evidential response
    • Contextual Information
    • Fault tolerance
    • VLSI Implementability
    • Uniformity in Analysis and Design
    • Neurobiological analogy
  • 20. WHY CHESS?
    • a game of perfect information
    • a purely intellectual game, a perfect environment for testing AI techniques
    • synthetic, discrete, finite domain, known worldwide and defined by a few rules of the game
    • A clear definition of result - win, draw, or loss
    • Deep Blue Wins against Garry Kasparov mainly due to the enormous searching
    • capacity provided by parallel computation
    • lacked learning or adaptability
    • Kasparov’s human power analyzes the first game and accordingly plays to
    • exploit Deep Blue's evident weaknesses in the next, leads to his victory against
    • Deep Blue.
  • 21. WHY ENDGAMES?
    • Re-analyze exemplar positions and games played over the board
    • Validate and possibly repair, published problems and studies;
    • Identify new concepts, patterns and specific positions
    • Create complete theories of particular endgames, such as KRK
    • Prioritize and simplify endgame theories to make them more
    • usable for over-the-board games.
  • 22. KING-ROOK-KING ENDGAME
    • First objective is to use to the Rook to restrict files/ranks of
    • the black King
    • Use the white King to further restrict the black King’s
    • movement
    • Force the black King into an ever-decreasing area of
    • movement, towards the intended checkmate square.
  • 23. 2 PRIMARY MATING PATTERNS
    • DIRECT KING OPPOSITION:
    • 1.1. Kings aligned parallel on squares separated by one file/ rank, Rook n
    • black King in the same file:
    • White King prevents black King from moving to the three outer squares
    • in adjoining file for the six square rectangle Anatomy of Mate mating pattern.
  • 24. Black King is in a corner square. White King is blocking the black King from moving to the two outer squares of the four square Anatomy of Mate mating pattern. Rook is controlling either the back rank or the file ending in the corner square 1. DIRECT KING OPPOSITION: …contd.
  • 25. 2. MISALIGNED KING OPPOSITION: 2.1. Kings are on squares that are not aligned parallel to each other, but separated by one square White King prevents black King from moving to the three outer squares in adjoining file for the six square rectangle Anatomy of Mate mating pattern.
  • 26. PROBLEM STATEMENT
    • Current implementations use of brute force
    • Pitfalls
    • - Lack of analytical ability to learn from previous mistakes
    • - A greedy approach towards winning the game
    • Self-Learning
    • Various parameters
    • Identify a better, if not the best possible move .
    • Learning approach using a Back-Propagation neural network.
    • Train the network
    • Evaluation of each of the board positions
    • Comparison with the expected output position.
    • Error and weight adjustments.
    • Agent learns to play a better move.
  • 27. NEURAL NETWORK ARCHITECTURE
    • Back Propagation Feed-Forward Network
    • Parameters of the Neural Network
        • Number of layers: 2
        • Number of neurons in Input Layer: 18
        • Number of neurons in Hidden Layer: 34/56/80
        • Number of neurons in Output Layer: 6
  • 28. INPUT REPRESENTATION
    • Number of neurons in input layer : 18
    • 6 neurons to represent each of the 3 chess piece
        • 3 neurons for the rank of the piece (000 to 111 i.e. a to g)
        • 3 neurons for the file of the piece (000 to 111 i.e. 1 to 8)
    • Thus, 18 input neurons indicate:
    Neuron Number in the I/p Layer Representation 1-3 File of Black King 4-6 Rank of Black King 7-9 File of White King 10-12 Rank of White King 13-15 File of White Rook 16-18 Rank of White Rook
  • 29. Board position i/p representation for given board Thus, input representation for above board position 001101011010101110 INPUT REPRESENTATION …contd. Parameter Value Representation File of Black king b 001 Rank of Black King 6 101 File of White king d 011 Rank of White King 3 010 File of White Rook f 101 Rank of White Rook 7 110
  • 30. OUTPUT REPRESENTATAION
    • 6 neurons to represent chess move.
      • White King to move
    Neuron in the O/p Layer Representation 1 K/R Index 2-3 Number of squares the piece has moved 4-6 Direction of the move
  • 31.
    • K/R Index(1)
        • White King’s move or White Rook’s move
        • For white king to move, this neuron value is always “0”
    • Number of squares(2-3)
        • Number of squares the piece has moved.
        • king can move only one square, this value is always “01”
    OUTPUT REPRESENTATAION …contd.
  • 32.
    • Direction (4-6)
        • Direction of the move.
        • king can move in 8 different directions,
        • 3 neurons are used to specify it (000 to 111).
    OUTPUT REPRESENTATAION …contd. 111 101 011 100 000 001 110 010
  • 33. Initial board position After White King’s move The output representation of the above move is 001001 OUTPUT REPRESENTATAION …contd. Parameter Representation K/R index 0 No. of squares 01 Direction 001
  • 34. (b) White Rook to move OUTPUT REPRESENTATAION …contd. Neuron in the O/p Layer Representation 1 K/R Index 2-3 Direction of the move 4-6 Number of squares the piece has moved
  • 35.
    • K/R Index (1)
        • White King’s move or White Rook’s move
        • For white rook to move, this neuron value is always “1”
    • Direction (2-3)
        • Direction of the move.
        • Rook can move in 4 different directions,
        • 2 neurons are used to specify it (00 to 11).
    OUTPUT REPRESENTATAION …contd.
  • 36.
    • Number of squares (4-6)
    • Number of squares the piece has moved.
    • Rook can move over a range of one to seven squares, this value can
    • range over 001 to 111. Thus, 3 neurons are used to specify it.
    OUTPUT REPRESENTATAION …contd. 11 00 01 01 11
  • 37. Board position White rook’s move The output representation of the above move is 100101 OUTPUT REPRESENTATAION …contd. Parameter Representation K/R index 1 Direction 00 No. of squares 101
  • 38. NUMBER OF HIDDEN NODES
    • If the number of hidden layer nodes is too small, high training error and high generalization error due to under fitting.
    • If the number of hidden layer nodes is too large, low training error but high generalization error due to under fitting.
    • Reference 1: https://www.thedacs.com/
      • Upper bound on = (Number of training samples)/
      • number of hidden layers (number of i/p and o/p elements)
      • = (30000) / (16+8) = 166
  • 39.
    • Reference 2: axon.cs.byu.edu/~martinez/classes/478/slides/BP.ppt :
    • Initial no. of hidden nodes = 2n or 2 logn
        • where n is the number of input nodes
        • Here, n= 18
        • So, initial number of hidden nodes = 36
    • Reference 3: IEEE paper
    • Trained neural networks play chess endgames :
    • Paper on use of neural network in chess endgame KRK
      • Uses 34 hidden nodes for improved efficiency
    • Based on above references, we developed nets with 34, 56 and 80 hidden nodes
    NUMBER OF HIDDEN NODES …contd.
  • 40. KNOWLEDGE ACQUISITION
    • Generation of sample space i.e. sample input and output.
    • Tom Kerrigan's Simple Chess Program (TSCP)
    • It comprised of the following files:
        • defs.h - TSCP's macros and data types.
        • data.c - TSCP's global data is declared
        • protos.h - Prototypes for every function in TSCP
        • search.c – It uses minimax method along with alpha beta pruning to search for the best move.
        • main.c – This helped us in assigning pieces to various positions on the board just initializing the board.
  • 41.
    • To generate the database:
        • Modify the board positions to include all possible KRK positions
        • For each KRK board position, use the Minimax method with alpha beta pruning to generate the best possible move.
    • Sample space of 221619 board positions and moves generated
    KNOWLEDGE ACQUISITION …contd.
  • 42. BACK PROPAGATION
    • Supervised learning technique
    • Steps followed:
      • The weights of a network to be trained by back-propagation must be initialized to some non-zero values.
      • Compare the network's output to the desired output from that sample. Calculate the error in each output neuron ej = dj - oj
      • Adjust the weights of each neuron to lower the local error
    Where, δ j = φ'( v j ) x e j φ(*) is the logistic function v j is the total input to node j i.e. Σ i w ji y i
    • Repeat the steps till mean squared error reaches some low level
    Weight change learning rate local gradient input signal to node j Δ w ji = η* δ j * y i
  • 43. HEURISTICS FOR MAKING THE BP ALGORITHM PERFORM BETTER
    • Sequential versus batch update
    • Maximizing information content
      • Results in the largest training error (emphasizing scheme)
      • Is radically different from all those previously used
    • Activation function
    • Target values
  • 44.
    • Normalizing the inputs
      • The input variables contained in the training set should be uncorrelated
      • The decorrelated input variables should be scaled so that their
      • covariance are approximately equal, so that the different weights in
      • the network learn at approximately the same speed
    • Initialization of weights
    • Learning from hints
    • Learning rates
    HEURISTICS FOR MAKING THE BP ALGORITHM PERFORM BETTER …contd.
  • 45. VIRTUES OF BACK PROPAGATION
    • Feature Detection
    • Function Approximation
    • Computational Efficiency
    • Sensitivity Analysis
  • 46. LIMITATIONS OF BACK PROPAGATION
    • Convergence
      • Every adjustable network parameter of the cost function should have its own individual learning-rate parameter
      • Every learning-rate parameter should be allowed to vary from one iteration to the next.
      • Derivative of the cost function w.r.t. a synaptic weight has the same algebraic sign for several consecutive iterations of the algorithm, the learning-rate parameter should be increased
      • Algebraic sign of the derivative of the cost function changes with respect to a particular synaptic weight alternates for several consecutive interactions of the algorithm, the learning-rate parameter for that weight should be decreased
    • Local Minima
    • Scaling
  • 47. GENERALIZATION
    • input-output mapping computed by the network is correct (or nearly so) for test data never used in creating or training the network
    Training Data Non-Linear Mapping Generalization Properly fitted data
  • 48. OVERFITTING
    • When a neural network learns too many input-output examples and ends up memorizing its training data.
    • Loses the ability to generalize between similar input-output patterns.
    • E.g. By finding a feature (due to noise) that is present in the training data but not true of the underlying function.
    Training Data Non-Linear Mapping Over fitted data (poor generalization)
  • 49. FACTORS AFFECTING GENERALIZATION
    • size of the training set, and how representative it is of the environment of interest,
    • architecture of the neural network
    • physical complexity of the problem at hand.
  • 50. Approach 1
    • architecture of the network is fixed
      • 18 input
      • 56 hidden nodes
      • 6 output nodes
    • Goal is to determine the size of the training set
    No of training samples 4900 50000 Accuracy on ideal moves 11.79% 11.79%
  • 51. PERFORMANCE GRAPH
  • 52. Approach 2
    • size of the training set is fixed
    • Goal is to determine the best architecture of network
    • Training samples=?????
    • Testing Samples=39354
    Criteria 34 hidden nodes 56 hidden nodes Valid moves 59.937% 63.071% Ideal moves 6.009% 24.917%
  • 53. EFFECT OF CLUSTERING AND BIAS Clustered samples on d1 Hidden nodes=80 Total number of testing samples= 39354 Total number of training samples=???? Total number of training epochs=????? Criteria Accuracy for biased NN Accuracy for unbiased NN Valid moves 54.609% 19.822% Ideal moves 3.295% 4.200%
  • 54.
    • Clustering
      • The training samples used were not adequately randomized
      • there was a repetitive pattern in the positioning of the pieces on the board. As a result the net gave a very low accuracy.
      • Improvements can be achieved by truly randomizing the samples presented to the net for training.
    • Bias
      • better result with bias as compared to that without bias.
    EFFECT OF CLUSTERING AND BIAS …contd.
  • 55. FUTURE SCOPE
    • Distributed Programming approach
    • Architecture modification
    • - increasing the number of layers
    • - recurrent networks
    • Genetic algorithms for weight evolution
    • Learning through negative examples
  • 56. CONCLUSION
    • The KRK endgames provide the additional advantage of faster learning, as there exist only three mating patterns for the game
    • Complexity of using a database is overcome
    • The feasibility of the use of a trained network in playing the game was judged on two criterion
      • ability to make valid/legal moves
      • ability to play intelligently comparing with grandmasters’ moves
  • 57.
    • Diff variants studied w.r.t.
        • Various number of hidden nodes: As the no. of hidden nodes
        • increases, performance increases
        • Performance function (mse/msereg): Better performance
        • was obtained with regularized mean square error function
        • With and without bias: Chess endgame environment
        • favored with bias
        • Clustering: Clustered nets do not show generalized behavior
    • Thus, chess endgame is an example of learning by example
    CONCLUSION …contd.
  • 58. Thank You!