How computers play chess

1. How Computers Play Chess Wednesday, September 25, 13

2. A bit about me Carlos Justiniano (carlos.justiniano@gmail.com) Started playing chess at age 7, reached master strength in (slow) online play. Project Manager / Team Lead on the Chessmaster series. Founded ChessBrain - a distributed computing project which earned a Guinness 2005 world record as the world’s largest chess computer. Wednesday, September 25, 13

3. Overview In this talk we’ll look at how computers play chess. We’ll begin with a brief history of chess. Then we’ll look at the core modules which make up all chess programs. We’ll end with a look inside an actual chess program. Wednesday, September 25, 13

4. Chess History Chess is well over 1000 years old. Believed to have originated in India before spreading to Persia. The style of chess we play today took form in Europe during the 15th century. At about the that time chess books were written. Wednesday, September 25, 13

5. Chess History During the 18th century, Philidor wrote The Analysis of Chess (L’analyse des échecs). He was considered to be one of the World’s best chess players in his time. François-André Danican Philidor Wednesday, September 25, 13

6. It’s been said that more books have been written about chess than for all other games combined! Wednesday, September 25, 13

7. So why this fascination with Chess? Long considered an intellectual game - the game of kings, queens and generals. A game which embodies the struggles of common men to entire kingdoms. A epic battle of life and death. Wednesday, September 25, 13

8. Fast forward to the future... Wednesday, September 25, 13

9. Chess Today Today, chess is played around the globe. It’s estimated that out of 7 billion people about 700 million have played chess at one point in their lives. Computers have surpassed the best human players. Today competitive chess players use computers in order to prepare for their human opponents. Wednesday, September 25, 13

10. The creation has surpassed its creator So, although we humans have had over 1000 years to invent, study and understand the ﬁner points of chess - machines are now better at playing chess than we are! Wednesday, September 25, 13

11. Whoa! No way dude! How did this happen? Wednesday, September 25, 13

12. It didn’t happen over night As early inventors created machines, public fascination grew for some of the more clever machines exhibiting human characteristics. Early inventor, Al Jazari, described many such inventions in his book “The Book of Knowledge of Ingenious Mechanical Devices” which he wrote in 1206. Surely, it was only a matter of time before machines could play chess. Wednesday, September 25, 13

13. The Turk, Chess Playing Automaton Built in 1769 by Wolfgang von Kempelen. The Turk beat many players but lost to the strongest players of the time, such as Philidor. It won many more games than it lost and was a sensation throughout Europe. Wednesday, September 25, 13

14. The Turk, Chess Playing Automaton Sadly, however, it was an elaborate hoax. The Turk concealed a human chess master. The world would not see a true chess playing machine for another 180 years. Wednesday, September 25, 13

15. Fast forward to the late 1930s The creation of electronic computers began in the 1930s. By the late 40s, computers were used as research and military tools in the US, England, Germany and the former USSR. The ENIAC, which became operational in 1946, is considered to be the ﬁrst general-purpose electronic computer. Programmers Betty Jean Jennings (left) and Fran Bilas (right) are depicted here operating the ENIAC's main control panel. Wednesday, September 25, 13

16. Early research papers in Computer Chess In 1945, Konrad Zuse, the German pioneer of computer science ﬁrst wrote about the possibility of creating a chess program. Although Konrad developed one of the ﬁrst electronic computers, the Zuse-1, there’s no record of him actually creating a chess program. Wednesday, September 25, 13

17. Early research papers in Computer Chess Four years later, in 1949, Claude Shannon (a research scientist at Bell Labs) authored a seminal paper entitled “Programming a Computer for Playing Chess”. Many of Shannon’s ideas are still in use today! Wednesday, September 25, 13

18. Turing takes things a few steps further In 1951, British mathematician and early computer scientist, Alan Turing wrote about computer chess. He later completed a one move chess analyzer called TUROCHAMP. Turing’s chess analyzer didn’t run on an actual computer, but was rather a set of instructions he could execute by hand. Wednesday, September 25, 13

19. Turing takes things a few steps further Turing simply calculated chess moves by looking ahead one move at a time and scoring them. Although played on paper, this was the ﬁrst program to play a complete game of chess. Turing believed that games such as chess served as ideal models in which to study machine intelligence. Wednesday, September 25, 13

20. No worries! We’re not going to cover all of computer chess history. Let’s speed things up a bit... Wednesday, September 25, 13

21. The 50s - 80s In 1957, Chess programs using a 6x6 board instead of the 8x8 board began playing simple chess. By 1957, the first program to play a game of chess was developed by Alex Bernstein in the US and one by programmers in Russia. In 1961 a Russian chess program played a game against a human chess amateur. This was the first recorded game of man vs machine. Although the machine lost, it wouldn’t be long before the tables would turn. Along with advances in main-frame and mini computers, chess programs also continued to improve during the 60s and 70s. During the 70s and 80s, Joe Condon and Ken Thompson at Bell Labs created Belle, the first chess machine to reach master strength play. Side note: you may remember Ken Thompson as the creator of the UNIX operating system. Wednesday, September 25, 13

22. The 80s also ushered in low cost chess computers Wednesday, September 25, 13

23. The 80s and 90s During the 80s we also started seeing chess programs for the early personal computers as well as dedicated chess machines for consumers. The 90s were an exciting time in computer chess history as chess programs began challenging International Chess masters and later Grandmasters. This progress was fueled by faster computers, improvements in software and advances in computing science. Wednesday, September 25, 13

24. IBM’s Deep Blue By 1997 IBM’s Deep Blue chess machine beat then World Champion, Garry Kasparov by two wins against one win and three draws. This marked the ﬁrst time in human history that a machine had ever defeated a World Champion. Wednesday, September 25, 13

25. Early 2000s Deep Blue was a highly specialized machine designed to play chess. However, PCs were also getting better! In the early 2000s there were three high proﬁle matches between world-class chess players and PCs. Keep in mind that these games where played against “gamer-class” PCs and not large mini and super computers. Not bad for machines less powerful than Deep Blue. In 2002 Vladimir Kramnik and Deep Fritz competed an eight game match ending in a draw. In 2003, Garry Kasparov played Junior, The match ended a draw with 3–3. Later that year, Kasparov played X3D Junior in a match also ending in a draw. Wednesday, September 25, 13

26. In 2005, Human chess dominance ends In 2005, Hydra (a specialized Chess machine using 64 processors) defeated the 6th ranked player in the world, Michael Adams 5½ to ½ in a six-game match. In 2006 undisputed World Champion Vladimir Kramnik played Deep Fritz and lost by a score of 2 to 4. Today chess programs running on our mobile phones play better than all but the world’s top human players. Wednesday, September 25, 13

27. Today, all competitive chess players train using machines - most have no chance of winning against the machines in actual tournament play Wednesday, September 25, 13

28. Speed in perspective In the 1980s a microcomputer could execute just over 2 million instructions per second, by the 90s they were executing well over 50 million instructions per second. Today, the processors in our tablets and phone are capable of executing over a billion instructions per second. Advances in computer science and software tools have also helped considerably, but advances in raw processing speed can’t be ignored. Wednesday, September 25, 13

29. Thinking Games To understand how computers play chess it’s important to understand how they play simpler games. Wednesday, September 25, 13

30. Thinking Games We’ve all played simple games like tic-tac-toe and the 15 tile sliding game. These games require us to consider which moves will bring us closer to winning. Wednesday, September 25, 13

31. Thinking Games The process of thinking about how to win involves employing strategies which bring us closer to achieving a solution. All games have strategies. In Monopoly it’s important to obtain high priced properties, in Reversi/Othello controlling the corner square is vital. For computer programs these strategies are described using Heuristics and Algorithms. Wednesday, September 25, 13

32. Heuristics and Algorithms In computer science the term Heuristic refers to a method of solving a problem by getting closer to a solution or end goal. In contrast the term algorithm refers to a method (approach) of solving a particular problem. It’s common for Heuristics to involve one or more algorithms. Wednesday, September 25, 13

33. Most non-trivial games can’t be realistically solved with an algorithm - largely because it would take too damn long Wednesday, September 25, 13

34. Solving Chess Take chess for example: there exists a way of ﬁnding the best move at the start of the game. However, it involves looking at all possible moves and the replies to each of those moves -- followed by replies to those moves, and so on until an end is reached. Not a bad plan if it were not for the shear number of possible chess moves in a given game. Wednesday, September 25, 13

35. If we assume an average chess game of 40 moves, there are... 10^120 possible moves, or 10,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,00 0,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000 At a calculation speed of 1/1000 of a second for each move - it would take a computer 10^90 years to make the ﬁrst move. Wednesday, September 25, 13

36. 10^120 is a really really (yes really) big number! Fictional side note: It turns out that it’s considerably larger than the seven million years required for Deep Thought to arrive at the answer: 42 Wednesday, September 25, 13

37. Can’t win by brute force Brute force refers to an approach which explores all possibilities. Complex intelligent games can’t be solved by brute force. So moves have to be selectively chosen. This realization is what has driven advances in how intelligent games are built. Wednesday, September 25, 13

38. Evaluating choices So if choices have to be evaluated then we need a way of performing an evaluation. In the ﬁeld of intelligent games this is referred to as an evaluation function. The function says: Given a known move return a score. The best move in a list of choices is the move which scored the highest. Wednesday, September 25, 13

39. How a machine sees Before a chess program can evaluate a move, we must ﬁrst teach it how to represent a chess position. This is unsurprisingly known as “Board representation”. Wednesday, September 25, 13

40. How a machine sees A chess board can be represented as a numbered list of squares. Wednesday, September 25, 13

41. How a machine sees The chess pieces themselves can be represented using the numbers -1 through 6. Piece colors are indicated using positive and negative numbers. Wednesday, September 25, 13

42. How a machine sees This is by no means the only way to represent a board and pieces. Other methods exist with names such as “bitboards” and “Forsyth-Edwards Notation”. In this talk we’ll stick with our earlier method. Wednesday, September 25, 13

43. How a machine sees Following our earlier example, it’s necessary to expand upon our simplistic board representation in order to detect chess moves which fall out of bounds. Essentially we need to encode the board’s boundaries. Wednesday, September 25, 13

44. How a machine sees This results in a larger board which includes our actual chess board within. A value of -99 is used to represent out of bound areas. Wednesday, September 25, 13

45. How a machine sees Thus our numbered list of squares also changes to allow for our expanded board. Wednesday, September 25, 13

46. How a machine sees Next, we need to help the machine understand what it sees. We need to teach it how each piece moves, and ﬁnally chess rules which further constrain legal moves. For example, the King chess piece can’t move into a square which is already under attack by the opposing side. Wednesday, September 25, 13

47. How a machine sees Let’s consider the Knight chess piece. It moves in an L shaped path: two squares in a horizontal or vertical direction and then one square to either side. Wednesday, September 25, 13

48. How a machine sees With a black Knight on square 78 we can see that it can move to squares 53, 55, 64, 68, 88, 92, 101, and 103. We can encode the Knight’s movements as the difference between the square it can move to and the square its currently on. Wednesday, September 25, 13

49. How a machine sees A movement to square 53 involves a subtraction 53-78=-25 So we encode that move as -25. Moving to square 101 involves 101-78=23 so we encode that as +23. This encoding works no matter where we place the Knight. Wednesday, September 25, 13

50. How a machine sees Lets place our Knight in the lower corner of the board. Notice how most of its moves fall out of bounds. This is why we use a larger board: in order to detect piece moves which aren’t valid. Wednesday, September 25, 13

51. How a machine sees A chess programmer proceeds to encode the offset differences for each piece. This algorithm is known as a legal move generator. With a list of legal moves the program can then run the evaluation function against each resulting move to begin to isolate a best move. Wednesday, September 25, 13

52. Inside an evaluation function In chess an evaluation function would essentially weigh various desirable characteristics in a chess position and return a score. This is one of the earliest methods used in computer chess. Wednesday, September 25, 13

53. Inside an evaluation function A simple evaluation function would ask questions like: “how many moves do I have available?”, “is my King safe?”. Evaluation functions are typically implemented using a weighted sum model. This approach assigns relative values to various chess factors to arrive at a weighed score. Wednesday, September 25, 13

54. Inside an evaluation function Before we can assign values in a weighed sum function we have to agree on a basic unit of measurement. Wednesday, September 25, 13

55. Inside an evaluation function In computer chess a pawn is assigned the value of 100, a knight is assigned 300, a bishop 350, a rook 500 and a Queen is valued at 900. A king is assigned a large number because capturing the king marks the end of the game. Wednesday, September 25, 13

56. Inside an evaluation function Each factor of an evaluation function is assigned a value relative to a centipawn, that's 1/100 of a pawn. For each desired factor, a chess programmer asks, "how much of a pawn is that factor worth?". Wednesday, September 25, 13

57. Inside an evaluation function Evaluation functions also contain a measure of material balance - that is, by a show of remaining pieces - who is winning?. This is determined by adding up the value of each side’s pieces, which are already measured in centipawns. So a knight and two pawns is equal to 500 or equal in value to a Rook. Wednesday, September 25, 13

58. Inside an evaluation function Here is another example: A chess game begins with each side having two bishops. It's considered advantageous to retain both bishops for as long as possible. So an evaluation function might value the present of both bishops as equal to half a centipawn or 50. Once a player no longer has both bishops an evaluation function would cease to add 50 to its overall score. Wednesday, September 25, 13

59. Inside an evaluation function Many factors go into an evaluation function, such as king safety (can the king be attacked, is it safe?) and Piece Mobility (how many moves are available to a given piece) greater mobility often equates to more options or opportunities. The presence or absence of various factors is what determines how a position is scored. Wednesday, September 25, 13

60. Inside an evaluation function Here is a really crude evaluation function where Queen=900, Rook=500, Bishop=350, Knight=300, Pawn=100. f below is our evaluation function, the parameter p is the position to be evaluated, M is a measure of mobility. f(p) = QueenWeight X (Qw-Qb) + RookWeight X (Rw - Rb) + BishopWeight X (Bw-Bb) + KnightWeight X (Nw- Nb) + (Pw-Pb)+ 0.1 X (Mw-Mb) Wednesday, September 25, 13

61. Inside an evaluation function Simpler evaluation functions model how beginners see chess positions, while more complex evaluation functions model how very strong players see a position. Complex evaluation functions take more time to execute than simpler ones. Thus, other algorithms need to be employed to speed up the selection process. Wednesday, September 25, 13

62. Searching for a best move So now that we’ve seen how a chess set (position) can be represented and how a it can be evaluated - we’re ready to consider how good moves can be found. Wednesday, September 25, 13

63. Look ahead In order for a chess program to decide on a best move it must also take into account its opponents move - followed by its own replies and so on. This is known as looking ahead. Good chess players look ahead a few moves while great chess players have been known to look ahead a dozen or so moves. Wednesday, September 25, 13

64. Keeping track of evaluations We’ve seen how computers represent chess positions and how they evaluate them - but how do they keep track of what they’ve evaluated? Enter: game trees a tool used to graph positions (nodes) and moves (vertices) and their relative evaluations. Wednesday, September 25, 13

65. When inverted a game tree appears more like a natural tree with a trunk, branches and leaves moving upward. Wednesday, September 25, 13

66. Game Trees In computer science a tree is also known as a data structure. Common data structures include arrays and hash tables (also known as dictionary or associative arrays). Trees can be implemented as array of nodes - where each node contains both a pointer to its parent and an array with sibling nodes. Wednesday, September 25, 13

67. Wednesday, September 25, 13

68. Game Trees Game Trees, like other data structure, are useful for more than just storing data. Algorithms typically operate on data structures. For example, a sorted array may contain a list of places and an algorithm (binary search) might be used to ﬁnd a speciﬁc location. Wednesday, September 25, 13

69. Game Trees and Search Algorithms A game tree is built using a legal move generator and nodes are evaluated using an evaluation function - which was described earlier. Search algorithms navigate the game tree while looking at the score left by an evaluation function. As a search algorithm visits a node (position) it may further update other node values along the way. Wednesday, September 25, 13

70. Search Algorithms Over the years many search algorithms have been devised. Wednesday, September 25, 13

71. Search Algorithms The Minimax algorithm was the first search method used in computer chess. MAX values (positive) are assigned to moves for the first player, while MIN values are assigned to the moves of the second player. The game tree is filled with MIN and MAX values and ordered so that any given node contains the MIN value or MAX value of the best replies below it. In this way a path to the best (highest scoring) move is identified. Wednesday, September 25, 13

72. Search Algorithms Alpha-Beta Pruning is a search algorithm which improves upon the Minimax algorithm by reducing the number of nodes which need to be evaluated. It does this by discarding a move branch when it’s proven to be worse than one already identiﬁed. We humans do this when we consider a choice which is so bad that we stop considering it and move on to more promising options. Wednesday, September 25, 13

73. Alpha beta pruning on a minimax tree Wednesday, September 25, 13

74. Improved Alpha/Beta Over the years, many improvements have been made to the Alpha Beta Pruning algorithm. If you’re interested, checkout NegaScout, PVS and MTF(f). Wednesday, September 25, 13

75. Key components we’ve covered Board representation / game state: How to represent a given position. Move Generation: Given a position, determine all of the legal moves. Wednesday, September 25, 13

76. Key components we’ve covered Static Evaluation: How to assess a given position based on various factors. Search: How to locate the best move in a game tree of chess positions. Wednesday, September 25, 13

77. That’s how computers play chess The areas we’ve covered should give you a sense of how computers play chess. Naturally this talk is a gross oversimpliﬁcation but with additional research into the ideas we’ve covered most talented programmers should be able to build their own chess program. Wednesday, September 25, 13

78. Ideas in action Let’s take a brief look at an actual chess program... There are hundreds of chess programs available on the web. For this talk I choose GarboChess by Gary Linscott. GarboChess is written in simple JavaScript and can run locally on your computer. Wednesday, September 25, 13

79. Wednesday, September 25, 13

80. Now you know how computers play chess! The ideas we’ve looked at apply to a wide range of games. There are many sites online which further elaborate on the materials we’ve covered. Wednesday, September 25, 13