SlideShare a Scribd company logo
1 of 11
Download to read offline
Solving Four-in-a-Row on 9×4 and 10×4 Boards ∗
Vasileios Charatsidis
January 2, 2017
Abstract
This paper investigates the Four-in-a-Row game with
the help of the MiniMax alpha-beta pruning algo-
rithm, a pairing algorithm and a threat search heuris-
tic. Transposition tables are used to reduce the num-
ber of nodes that need to be investigated. With these
techniques 9×4 and 10×4 were possible to be solved,
both being a win for the first player.
1 Introduction
Four-in-a-Row belongs to the mnk-games family,
where m and n are the height and width of the board
while k is the number of pieces a player must make in
a row, column or diagonal to win. For example 333
is the TicTacToe game. Four-in-a-Row is played by
two players. Every empty square of the board is a le-
gal move. As the name suggests the first player that
makes four in a line (either row, column or diagonal)
wins (denoted by “score-4”).
Four-in-a-Row is solved in every board except the
boards from 9×4 to 29×4 [6, 7]. If the board is suf-
ficiently large, the game is a win for the player who
plays first (he will be called Black). Boards from 1×4
to 8×4 are draw and from 29×4 and larger are a win
for Black. For m×5 boards, the game is drawn for m
= 5 and a win for m = 6 [6, 7]. It is easy to prove
that as a consequence the game is a win for every
board larger than 6×5.
The research question is to find the turning point,
∗This thesis was prepared in partial fulfillment of the re-
quirements for the Degree of Bachelor of Science in Knowl-
edge Engineering, University of Maastricht, supervisor: Dr.
Jos Uiterwijk
that lays between 9×4 to 29×4 boards, that makes
the game a win for Black.
The game tree of Four-in-a-Row, even for 9×4, is
very large. In the starting position the branching
factor is 36 and while the game unfolds the branching
factor stays quite high for brute-force algorithms to
work, hence pruning techniques must be applied in
order to find the winning variations, given that they
exist.
1.1 Overview
Section 2 provides technical details about the imple-
mentation of the techniques that were used. In Sec-
tion 3 results and experiments are presented. In Sec-
tion 4 the conclusions are given. Finally, in Section
5 there is some proposed future work.
2 Implementation
The implementation of the program consists of the
representation of the game and the solver. The solver
consists of the move generation, the MiniMax alpha-
beta search and the pairing algorithm. Technical de-
tails considering the above parts are provided below.
2.1 Representation
The game state is represented by bit-boards. A bit-
board is a bit array of (board)height×(board)width
elements, one binary digit for each square of the
board. Each bit-board is represented by a long vari-
able. Two bit-boards are sufficient to represent a po-
sition, one for the black and one for the white stones
(all played moves are going to be called stones). In
1
Figure 1 an example is shown. A 5×4 board is repre-
sented with two longs. The longs are written slightly
differently below, so that it is made clear how they
represent the stones (starting to write from the bot-
tom rightmost 1 of the matrix and keep going from
right to left and from bottom to top, both binary
strings are attained).
Figure 1: An example position.
This is a very efficient and compact representation,
which allows operations like checking for a terminal
state or cloning the board, to be performed fast.
To play a move, firstly the coordinates of the move
are translated into an integer. Then this integer is
translated into a long, using bit shifting. Finally,
with the logical operator ”Xor”, the above long is
integrated with the long that represents either the
black or white stones, depending of whom player the
move is.
An example is the black move B1 in Figure 1
(that strikes score-4 vertically). The coordinates are
translated into the integer x = row×width+column.
With row equals 4 and column equals 1 one gets x
= 4×4+1 = 17. Note that this arithmetic opera-
tion is equal as counting (from left to right), starting
at the top left corner with 0. This integer is trans-
lated to a long by left-shifting 1 x times, thus, giving
100000000000000000.
The black stones were at this point represented by
the long 1010011000100000. The new stone is then
integrated to the black stones with the simple Xor op-
eration: 1010011000100000 Xor 100000000000000000
= 101010011000100000 (as shown in Figure 1).
2.2 Move Generation
The biggest part of pruning the game tree takes place
in the move generator class. This class, as the name
suggests, generates the moves that will be investi-
gated. The process for Black and White share the
common steps below:
1. It is checked if the active player can achieve a
score-4 and finish the game. If yes the move is
returned.
2. It is checked if the opponent threats to achieve a
score-4. If yes the move that defends is returned.
Given that one of the above conditions is true,
the move generator returns just one move. Oth-
erwise, the process continues.
3. It is checked if there exists a move that creates a
double threat (which always precedes and causes
a score-4). As it is shown in Figure 2, move C3
makes a double threat (one at C4 and one at
D3).
Figure 2: Double threat at square C3.
In all figures all the moves that are going to be
explored are indicated in gray, as a matter of
fact in Figure 2 only move C3. Important is
that, while the algorithm searches for a double
threat, it stores all the moves that pose a single
threat (called threats) in the front of the move-
list, starting from threats that are adjacent to
the last black move. Threats are prioritized since
2
they restrict the opponent into one single move.
This fact often renders them as the best move
available . As will be shown, the above detail
is very important for the efficiency of the algo-
rithm.
4. Next, the active player checks if the opponent
threats to make a double threat. If that is the
case, the available moves are the ones that stop
the upcoming opponent double threat, plus the
threats that the active player has. In Figure 3
moves A4, D7 and D4 defend to the upcoming
double threat of A4 (one at D7 and one at D4).
Moves A5, D8 and A7 are the aforementioned
white threats. If played they “buy” White one
move “time”, thus giving him the choice to de-
fend in his next move.
Figure 3: Defenses to a double threat.
5. If none of the above applies, the process contin-
ues differently for Black and White. For Black
the most reasonable moves are added. These are
moves that meet at least one of the following
conditions.
(a) They are not more than two rows far away
from the closest, row wise, black stone and
they have synergy with at least one of the
black stones.
(b) They eliminate White’s potential.
With synergy it is meant that there is potential
for a score-4 consisting of the move that is going
to be added and another black stone. With elim-
inate White’s potential it is meant moves that
are in the same group (groups are defined as lines
of four in either row, column or diagonal) with
two white stones and no other black stone. Such
a move is the move D4 in the board of Figure 4
(which happens to be the winning move!).
Figure 4: Candidate moves that are going to be ex-
plored (in gray color) after 1. A5-C5 2. A6-A7.
White numbers depict the ordering.
In Figure 4 all gray candidate moves are close
to the black stones, at A5 and A6, and have
some potential to score-4 using A5 or A6 or both.
Moves like A8, B8, D7, D5 and B3, although
close enough, are not included, since there is no
potential for score-4 with the black stones. D5 is
in the same row as A5 but there is a white stone
at the row group eliminating score-4 potential
between them. A8 is in the same column group
with A5 and A6 but, again, there is the white
stone at A7. Finally B3, D7 and B8 are not in
the same group with any of A5 or A6, hence, the
above moves have no potential. Of course moves
in the first, second, ninth and tenth rows are not
included since they are too far away from the
black stones.
Again, moves that are adjacent to the last black
move are added first, before all the other non-
threat moves. This makes sense for both players.
For Black moves adjacent to the last black move
have, most likely, the best synergy and fulfill the
purposes of it. As for White, moves adjacent
3
to the last black move are most likely the best
defenses. The above detail speeds up the con-
vergence of both, Black to the winning variation
and White to a successful defense, significantly.
For example in Figure 4 the last black move is
A6 hence the moves will be ordered as depicted
by the white numbers: A4, A3 (threats), B7, B5,
B6 (adjacent to the last black move A6), then C6
and D7 (same row as the last black move A6) and
then all the others.
Considering White, besides move ordering, all of
his legal moves must be calculated.
After the above five steps have been performed,
the part of the move generation is over and the
exploration of the generated moves starts.
2.3 MiniMax alpha-beta search
For the exploration of the game tree the Mini-
Max alpha-beta [3] search in a Iterative deepening
depth-first [4] fashion. Iterative deepening depth-first
search (IDDs) is a state-search strategy in which a
depth-limited version of depth-first search runs re-
peatedly with increasing depth limit until the goal is
found. IDDs is convenient in cases that the height of
the game tree is unknown.
The evaluation of a given board is fairly simple. It
assigns the value 300 if Black has achieved a double
threat and the value -300 if White has achieved a
double threat, every other position is assigned with
the value 0. Every won position (with a score of 300
or -300) is stored to a table and used when it occurs
again, either in later depth limits or by different move
ordering in the same depth limit.
Every time the depth limit of IDDs is exhausted
and the last move is played by White a special search,
called threat search is performed.
Threat search
The idea came out naturally (independently without
knowing its existence in the literature [1]) when it
was noticed that before a double threat, a number
of continuous threats (threat barrage) always take
place.
Threat search as the name suggests explores,
strictly, moves that are threats. This is done recur-
sively, starting with one of the possible threats checks
if by continuously threatening the opponent, a double
threat can occur. The search goes through all pos-
sible threat combinations and stops when a double
threat is found or no other threats can be made on
the board. Situations where White’s defensive move
creates at the same time a threat make the search
stop as well.
The average branching factor of, strictly, threaten-
ing moves is low, although, the researcher can adjust
the depth of threat search as he likes. The optimal
depth is discussed in Section 3. It is clarified that af-
ter a threat has been applied only related threats are
examined. Related threats are defined as treats such
that the last black move is part of them. Otherwise
unrelated filler threats, that might exist but do not
contribute to the solution, would be redundantly ex-
plored, slowing the solver by approximately a factor
of two.
As an example, consider the position in Figure 5.
When the solver is initiated the IDDs will at some
point reach depth limit 8, thus the position will be
evaluated. Since it is Black’s turn threat search will
be performed. In the following figures, the threat
Figure 5: Example starting state of threat search.
search will be demonstrated. The reader should ig-
nore gray squares since they are the moves that IDDs
would explore.
• As said, threat search will examine just threats
4
B4 and C4, shown in Figure 5. B4 is chosen.
White is forced to answer with C4 (Figure 6).
Figure 6: Threat barrage after moves 5. B4-C4.
• After ply 2, the only available threats are B2 and
B3. B2 is chosen. White forcefully answers with
B3, leading to the position in Figure 7.
Figure 7: Threat barrage after moves 6. B2-B3.
• After ply 4 (figure skipped), the threats to be
searched are C3 and A1, Black chooses C3 and
White answers A1.
• After ply 6. the threats to be considered are A5
and D2. A5 is chosen and White answers D2
(Figure 8).
Figure 8: Final state of threat barrage after 7. C3-A1
8. A5-D2.
• After ply 8 in Figure 8 a double threat is iden-
tified (with moves A6 and A3), hence a victory
for Black.
After that the starting position in Figure 5 is stored
as a victory for Black for future use in later depths
or if brought by different move ordering.
While this sequence was forced, if the threat search
at the depth limit has not been performed, many
other moves would be generated, and explored by the
iterative deepening search in later depths.
If a threat barrage that leads to victory exists, it
is guaranteed that it will be found. There is one
downside, being that there might be a shorter threat
barrage. The threat search just returns the first one
found.
Returning to the MiniMax alpha-beta search, a
worth noting implementation detail is that the search
tries to find scores ranging from 0 (draw) to 300 (win)
for Black, and -300 (loss) to 0 for White; since the
result is either a win for Black or draw. That way the
program does not waste time searching for a White’s
win.
Transposition tables and symmetry
Transposition tables were used to make the search
more efficient. The won positions are stored in a hash
table and used in later depths. All other undecided
positions are stored in another hash, since they might
5
reappear with different move ordering in the same
depth. The latter hash table is emptied after the
searching of the specific depth limit is over (since it
might proven Black’s win in latter depth).
In addition symmetric positions are identified and
only one side of them is examined.
2.4 Pairing algorithm
Pairing is a method that recognizes draw positions
[2]. If every row, column, first and second diagonal
group of the board can be covered then we have a
pairing match and the position is a draw. With cov-
ered it is meant that for every one of the above groups
there are two points such that when Black plays one
of them White will play the other, basically show-
ing that White can have a stone in every group. The
pairing match if it exists will look like Figure 9, where
one is found after one move of Black and one of White
have been played.
Figure 9: Pairing in a 4×4 position.
W stands for a white stone, B for a Black stone,
“ - ” for row cover points, “ / ” for diagonal cover
points and “ | ” for column cover points.
It is worth noticing that many boards that are ob-
viously drawing positions, like the empty 4×4 board
cannot be matched with pairs. Without the black
and white moves in the above 4×4 board there are not
enough squares for all groups to be covered (we have
16 squares and ten groups, 4 row-groups, 4 column-
groups, 1 first diagonal-group and 1 second diagonal-
group, thus 20 squares need to cover them all)!
The pairing algorithm is particularly useful when
someone wants to prove that a given board is a draw,
integrated in the alpha-beta search, simply by look-
ing for a pairing match for every position. Other
than that, it happened that this method was not par-
ticularly helpful, since it makes the algorithm much
slower, and the number of positions that are pruned,
when Black wins, are less than five per cent.
Considering the implementation of the pairing al-
gorithm, again bitboards are used. Firstly, in a given
board, like in Figure 10, one instance with its first di-
agonal groups covered is created. So in the board of
Figure 10 the “” will appear in four spots since there
are two first diagonal groups without white stones.
The board now is represented with three longs. Two
for the pieces and one for the covered first diagonal
groups. The latter long has ones when “” appears.
Then from the produced board an instance of it with
its second diagonal groups covered is created and that
again is represented with one long with ones where
“/” appears, so there are four longs now. As shown
just two of them exist in Figure 10 since only one
second diagonal group is uncovered. The same pro-
cess continues for rows. Finally, the aforementioned
instance, if it exists, is examined. If the configuration
of the left empty squares is enough to cover all the
column groups (overlapping between column groups
should be taken into consideration) there is a pairing
match. If not, the process repeats again trying all
combinations possible exhaustively.
Figure 10: Pairing in a 9×4 position.
3 Results and Experiments
For every board, there are actually just two starting
moves for Black, as shown in Figure 11.
This happens under the assumption that starting
in the middle row of the board has at least equal
or more chances to win with every other row. Al-
6
Figure 11: Starting moves for 9×4.
though the above assumption is not proven, it is in-
tuitively sound, since by playing in the middle row,
Black has the maximum space both north and south
of the board available, to achieve a score-4. Consider-
ing the starting column, the move in the first column
is different from the move in the second column but
the moves in the third and fourth are symmetrical to
the first ones respectively.
The first board that was tried was the 10×4, and
proved by the author a win for Black. At this time,
the solver was very slow, such that it was not able
to identify a win in 9×4, still the game was explored
deeply and the author’s guesstimate was that 9×4 is
a draw.
3.1 9×4 Black wins
It was proved though, by Uiterwijk [8], while this pa-
per was prepared, that the smaller winning board is
the 9×4! Afterwards the above demonstrated solver1
confirmed the result.
The only winning move is B5. Versus the most
resilient white moves Black wins within 20 moves.
The board is shown in Figure 12 left. The most
resilient white move is the diagonal C6. The right
board comes from the following variation: 1. B5-C6
2. C4-B4 3. A4-A6 4. A2-D6 5. B6-A3 6. C5-B3 7.
C3-C2 8. C7-B2 (a filler threat that was mentioned
before, it does not contribute but prolongs the game
1Both solvers have been built separately, from scratch, and
differ quite significantly, considering the techniques used.
one move until White’s inevitable loss) 9. B1-D5 10.
B7-B6 11. A7 (making a double threat D7, D4).
Figure 12: 9×4 Starting and final boards after 1. B5-
C6.
In Figure 13 are demonstrated some insightful in-
formation printed in the console as outputs, after ev-
ery depth limit is finished, while the game tree for
9×4, was being explored. D stands for the differ-
Figure 13: Console prints, while exploring the tree
for 9×4.
ent depth limit in iterative deepening search. M is
the move that has been considered as best so far,
E stands for the evaluation score of that move. T
stands for the amount of time, that has been spent
so far. The following numbers are in millions and
are cumulative. N-exp stands for the total number
of nodes explored. IDDs-exp are the nodes expanded
by iterative deepening search. Finally, TSN-exp are
the nodes explored from threat search. Note that the
prints starts at depth 8. In the first depths move A5
is considered as best but then in depth 18 move B5
appears with the score of 300, as a matter of fact the
7
search is terminated as a forced win has been found2
.
3.2 Heuristics experimentation
During the construction of the solver much experi-
menting took place considering heuristics like threat
search, pairing algorithm, different move orderings
and move generations.
Best performance
The final, fastest, version uses all parts described in
Section 2 except the pairing algorithm. As shown in
Figure 13 it solves 9×4 within 358 seconds or roughly
6 minutes after exploring 78.73 million nodes in total,
28.24 million nodes by IDDs and 55.88 million nodes
from threat search, with a speed of 219,916 nodes per
second (n/sec).
Move ordering
The move ordering used shows its impact when in an
almost similar version for 9×4, without any ordering,
the solver explored roughly 536 million nodes only in
the last depth limit! The whole search took 2 hours
and 36 minutes.
Threat search
Considering the threat search, initially it was used in-
side the move generator as a heuristic. While signif-
icantly many nodes were pruned, a lot of redundant
actions took place since MiniMax alpha-beta priori-
tize threats anyway, meaning that they are explored
pretty early in the game tree. As a matter of fact,
the solver was as slow as 600 n/sec,such that 10×4
was solved in 2 days 8 eight hours (now roughly 2
minutes) while 9×4 could not be solved at all!
However, if used correctly, threat search is a power-
ful tool. Without it 713.56 million nodes are explored
within thirty five minutes (340,764 n/sec), approxi-
mately 14 times slower. This happens since without
threat search the iterative deepening needs to reach
depth limit 21, in contrast with depth limit 18. Other
2Actually in move 18 a threat barrage that leads to victory
has been found. The threat barrage starts after move 9 in
Section 3.1 and is 10. B7-B6 11. A7- .
than that, much experimentation took place with the
depth limit of the threat search (even variable depth
has been tried) but it does not affect significantly the
speed, given that the depth limit of threat search is
larger than 2, when the game is solved in 30 minutes.
Transposition tables
For the 9×4 board the necessity of transposition ta-
bles is shown:
• Disabling both hash tables the solver explores
385.96 million nodes with IDDs and another
392.27 million nodes with threat search, 778.23
million nodes in total within exactly 41.5 min-
utes (312,542 n/sec), 5 times slower.
• Disabling the hash table that stores all moves in
every depth, avoiding the exploration of transpo-
sitions, the IDDs explores 126.58 million nodes
while threat search explores 135.5 million, 262.09
millions in total within 14 minutes (307,485
n/sec), more than 2 times slower.
• Finally, disabling the wins table produce sim-
ilar results, the IDDs explores 125.15 million
nodes and the threat search 133.21 millions in
total 258.36 millions within roughly 14 minutes
(307,142 n/sec), more than 2 times slower.
Pairing
Another method to examine is the pairing algorithm.
The current pairing implementation, using no heuris-
tic to converge faster into a pairing, makes it quite
slow. Initially it was created to prove that a board is
drawn but 9×4 proved to be a win. Nevertheless it
was tested.
Since pairing matches occur when many white
moves have been played it was enabled just after and
only for the 16th move. The result was 477,930 pair-
ings to be found causing a total reduction of 5.91
million nodes explored. All in all for 9×4 it proved
to be a neutral method, considering the time needed
for the solver to find a solution. But for 10×4, that
is solved in earlier depth, as expected pairing was
not efficient, slowing the process approximately by a
factor of 3.
8
3.3 Pairing threshold experimenta-
tion.
A research has been performed considering the
thresholds, determined by the number of moves that
must be played, for pairings to start appearing and
for the board to be drawn. In the table below the re-
sults are demonstrated. mn-1p and f-1p stand for the
move number and the percentage of the board cov-
erage for at least one pairing to exist, respectively.
Likewise mn-d and f-d stand for the move and per-
centage of the board coverage for every position to
be paired (thus the board is drawn).
Board mn-1p f-1p mn-d f-d
4×4 2 12.5% 2 12.5%
5×4 4 20% 4 20%
6×4 6 25% 8 33.33%
7×4 8 28.57% 14 50%
8×4 8 25% 18 56.25%
9×4 10 27% 223
61%
10×4 12 30% - -
11×4 14 32% - -
Unfortunately the sample is too small for a specific
pattern to be recognized.
Figure 14 shows the outputs4
while 8×4 is exam-
ined. E stands for evaluation. When the evaluation
is 1 it means that the position is undecided and when
0 that there is a pairing. Since in move 18 the best
evaluation achieved is 0, the search stops meaning
that the board is drawn. The pairings seems to fol-
low something like a sigmoid function as until depth
16 their number is insignificant, in contrast with the
nodes explored, at depth 16 they massively appear,
and at depth 18 every board can be matched to a
pairing, thus formulating a steep threshold.
3.4 The difference between 10×4 and
9×4.
10×4 proved to be Black’s victory for both starting
moves A5 and B5. In 9×4, though, there is just one
variation that holds against Black’s A5. This varia-
3With A5 as first move.
4IDDs-exp pd now stands for nodes expanded from IDDs
per depth limit.
Figure 14: Nodes expanded from IDDs and pairings
per depth in 8×4.
tion starts with the first white move at C5 and it is
demonstrated in Figure 15.
Figure 15: 10×4 board after 1. A5-C5 2. A6-A4 3.
B7- .
In 10×4 the third move of white B6 is defeated
easily with the following threat barrage: 1. A7-A8 2.
C7-D7 3. C8-D9 4. C9- that ends up in a vertical
double threat as demonstrated in Figure’s 15 right.
In 9×4, though, the 10th row is absent such that
Black cannot create the double threat, as a matter of
fact the position in Figure 15 left omitting the 10th
row, proved to be a draw for 9×4 with the help of
the pairing algorithm. Other than this variation, all
other moves lose to A5 even in 9×4, and that is why
C5 move is so special. 10×4 is a Black victory within
15 moves. The longer lasting variation start as 1. A5-
B5 2. C5- . The solution is found within 130 seconds
after 38.87 million nodes have been explored.
9
3.5 Other variations on 9×4
An interesting variation arises after 1. B5-C5. The
winning move appears to be 2. D5- . A threat barrage
is found after 13 moves. D5 looks counter intuitive
since it does not follow the synergy rule.
In the process, that moves are generated, explained
above black moves should have some synergy, that
obviously does not exist in Figure 16. This is a rare
instant that this happens, as a matter of fact, after
manually C5 was tested and proved to be the best
move there was a change in the heuristic such that
it allows moves that create the shown configuration
(more specifically: a black stone and adjacent to it,
in the same row, a white one, and then a black stone
adjacent to the aforementioned white, all three in the
same row). The above situation, constituted a teach-
ing example demonstrating the danger of heuristics
that might over-prune the tree 5
.
Figure 16: Board after 1. B5-C5 2. D5- .
All other white moves lose quite fast.
4 Conclusions
The main conclusion of this paper, considering the
research question, is that Four-in-a-Row is a win on
9×4, the threshold that makes the game a win for
Black in the range of 9×4 to 29×4. Furthermore,
10×4 is the first board that Black wins with both of
the starting moves.
5Note that the complete disabling of the synergy rule was
tried, with worse results.
Considering the implementation details the move
ordering affects the efficiency of the MiniMax alpha-
beta algorithm the most. Furthermore, many heuris-
tics, that seem useful, are proven too computationally
expensive to contribute efficiently to the algorithm,
like the pairing method.
And the final conclusion is a surprising phe-
nomenon that has been noticed. Moves that are more
than one row6
far away from the closest, row wise,
black stone do not need to be explored at all, for
Black to converge to win, unless:
• they are threats or
• they are the one point vertical “jump” from the
last black move.
With the aforementioned move selection 9×4 is
solved within 224 seconds (40% faster). Figure 17
Figure 17: Board after 1. B5-C5 2. D5-B6 .
demonstrates the above concept for Black. The only
gray moves being more than one row away from the
closest, row wise, black stone are D7 and D3 placed
at the one point vertical “jump” from the last black
move D5.
This is an important fact, showing that Four-in-
a-Row is clearly a tactical game. All moves, except
the first one, have impact locally and trigger tacti-
cal combinations instead of having a general strategic
purpose which benefits Black later on.
6In the move selection for this algorithm was used the limit
of 2 rows away from the closest black move as said in Figure 4,
but 1 row limit is enough for converging to the solutions both
for 9×4 and 10×4.
10
5 Future work
A couple of ideas have been arisen but were not im-
plemented due to time constrains.
• Does the above phenomenon, the local-move
idea, holds for every or some other mnk-games?
• Since the game can be formulated as finding a
line path of 4 from any node to any other against
an opponent an interesting question emerges. Is
there a relation between the pairing steep thresh-
old and the percolation’s steep threshold7
[5] ?
• Creating a more efficient pairing algorithm by
transforming the game from: whoever score-4
wins to whoever strikes a double threat wins.
That would allow a pairing to exist in boards
that are drawn but the standard pairing does
not. One example is 4×4, which is an obvious
draw, but a normal pairing cannot exist unless
White places a move in it. Instead of trying to
cover all row, column and diagonal groups, some-
one would try to cover all different groups in
which a double threat can exist (a double threat
needs 7 squares, hence the groups will consist of
7 squares). In a 4×4 board there are 32 such
groups, overlapping with each other, meaning
that there are not needed 32×2 squares for them
to be covered, and probably a pairing can occur.
Of course it is given that when a single threat
appears, White just answers it.
• A cut heuristic, cuts being defined as shapes
that divide the board such that moves that being
played in the north side of the cut do not affect
the south side. The position, then, can be solved
with two different trees, and the starting board
is assigned with the best score of the two smaller
ones. Even if the starting board is drawn, it is
faster to find a pairing in two smaller boards.
• Finally, the generalization of threat search into a
selective depth search that in start explores only
7In a big square grid there is almost always a path connect-
ing one side with the other if more than 59% of its tiles are
explored. On the other hand if less than 59% of its tiles are
explored almost never there is such a path.
moves local with the last black move. When a
win cannot be found, a broadening of the range
of moves takes place.
References
[1] Allis, L.V., Herik, H.J. van den, and Huntjens,
M.P.H. Go-Moku Solved by New Search Tech-
niques Computational Intelligence 12 (1996) 7–
23.
[2] Hales, A.W., and Jewett, R.I.: Regularity and
Positional Games. Trans. Amer. Math. Soc. 106
(1963) 222-229.
[3] Knuth, D.E., and Moore, R.W.: An analysis
of alpha-beta pruning. Artificial Intelligence 6
(1975) 293-326.
[4] Korf, Richard (1985). ”Depth-first Iterative-
Deepening: An Optimal Admissible Tree Search”.
Artificial Intelligence. 27: 97–109.
[5] Stauffer, D. and Aharony, A. Introduction to Per-
colation Theory, 2nd ed. London: Taylor and
Francis, 1992.
[6] Uiterwijk, J.W.H.M.: Kennisbehandeling in po-
sitionele spelen; computeranalyse van Four-in-a-
Row. Proceedings of NAIC’91, the Fourth Dutch
AI Conference (Ed. J. Treur), Stichting Informat-
ica Congressen, Amsterdam, The Netherlands;
(1991) 193-207.
[7] Uiterwijk, J.W.H.M. and Herik, H.J. van den:
The advantage of the initiative. Information Sci-
ences 122 (2000) 43-58.
[8] Uiterwijk, J.W.H.M. Personal communication
(2016).
11

More Related Content

What's hot

ACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems Review
ACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems ReviewACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems Review
ACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems ReviewRoman Elizarov
 
Data mining with differential privacy
Data mining with differential privacy Data mining with differential privacy
Data mining with differential privacy Wei-Yuan Chang
 
Graph Dynamical System on Graph Colouring
Graph Dynamical System on Graph ColouringGraph Dynamical System on Graph Colouring
Graph Dynamical System on Graph ColouringClyde Shen
 
Line Drawing Algorithms - Computer Graphics - Notes
Line Drawing Algorithms - Computer Graphics - NotesLine Drawing Algorithms - Computer Graphics - Notes
Line Drawing Algorithms - Computer Graphics - NotesOmprakash Chauhan
 
Line drawing algo.
Line drawing algo.Line drawing algo.
Line drawing algo.Mohd Arif
 
Functional Programming In Mathematica
Functional Programming In MathematicaFunctional Programming In Mathematica
Functional Programming In MathematicaHossam Karim
 
Breadth First Search (BFS)
Breadth First Search (BFS)Breadth First Search (BFS)
Breadth First Search (BFS)Dhrumil Panchal
 
Cs6702 graph theory and applications question bank
Cs6702 graph theory and applications question bankCs6702 graph theory and applications question bank
Cs6702 graph theory and applications question bankappasami
 
maXbox starter68 machine learning VI
maXbox starter68 machine learning VImaXbox starter68 machine learning VI
maXbox starter68 machine learning VIMax Kleiner
 
2.8 coord. plane 1
2.8 coord. plane 12.8 coord. plane 1
2.8 coord. plane 1bweldon
 
Lecture 1 admin & representing fcts
Lecture 1   admin & representing fctsLecture 1   admin & representing fcts
Lecture 1 admin & representing fctsnjit-ronbrown
 
Bresenham Line Drawing Algorithm
Bresenham Line Drawing AlgorithmBresenham Line Drawing Algorithm
Bresenham Line Drawing AlgorithmMahesh Kodituwakku
 
Depth First Search and Breadth First Search
Depth First Search and Breadth First SearchDepth First Search and Breadth First Search
Depth First Search and Breadth First SearchBenjamin Sach
 
Breadth first search (Bfs)
Breadth first search (Bfs)Breadth first search (Bfs)
Breadth first search (Bfs)Ishucs
 

What's hot (20)

Bfs and Dfs
Bfs and DfsBfs and Dfs
Bfs and Dfs
 
ACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems Review
ACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems ReviewACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems Review
ACM ICPC 2014 NEERC (Northeastern European Regional Contest) Problems Review
 
Data mining with differential privacy
Data mining with differential privacy Data mining with differential privacy
Data mining with differential privacy
 
Graph Dynamical System on Graph Colouring
Graph Dynamical System on Graph ColouringGraph Dynamical System on Graph Colouring
Graph Dynamical System on Graph Colouring
 
Line Drawing Algorithms - Computer Graphics - Notes
Line Drawing Algorithms - Computer Graphics - NotesLine Drawing Algorithms - Computer Graphics - Notes
Line Drawing Algorithms - Computer Graphics - Notes
 
Imo class 5
Imo class 5Imo class 5
Imo class 5
 
BFS
BFSBFS
BFS
 
Graphs bfs dfs
Graphs bfs dfsGraphs bfs dfs
Graphs bfs dfs
 
Line drawing algo.
Line drawing algo.Line drawing algo.
Line drawing algo.
 
Functional Programming In Mathematica
Functional Programming In MathematicaFunctional Programming In Mathematica
Functional Programming In Mathematica
 
Breadth First Search (BFS)
Breadth First Search (BFS)Breadth First Search (BFS)
Breadth First Search (BFS)
 
Cs6702 graph theory and applications question bank
Cs6702 graph theory and applications question bankCs6702 graph theory and applications question bank
Cs6702 graph theory and applications question bank
 
Bfs dfs
Bfs dfsBfs dfs
Bfs dfs
 
maXbox starter68 machine learning VI
maXbox starter68 machine learning VImaXbox starter68 machine learning VI
maXbox starter68 machine learning VI
 
2.8 coord. plane 1
2.8 coord. plane 12.8 coord. plane 1
2.8 coord. plane 1
 
Lecture 1 admin & representing fcts
Lecture 1   admin & representing fctsLecture 1   admin & representing fcts
Lecture 1 admin & representing fcts
 
Hprec2 5
Hprec2 5Hprec2 5
Hprec2 5
 
Bresenham Line Drawing Algorithm
Bresenham Line Drawing AlgorithmBresenham Line Drawing Algorithm
Bresenham Line Drawing Algorithm
 
Depth First Search and Breadth First Search
Depth First Search and Breadth First SearchDepth First Search and Breadth First Search
Depth First Search and Breadth First Search
 
Breadth first search (Bfs)
Breadth first search (Bfs)Breadth first search (Bfs)
Breadth first search (Bfs)
 

Similar to Thesis paper

January 13, 2014
January 13, 2014January 13, 2014
January 13, 2014khyps13
 
Minmax and alpha beta pruning.pptx
Minmax and alpha beta pruning.pptxMinmax and alpha beta pruning.pptx
Minmax and alpha beta pruning.pptxPriyadharshiniG41
 
hanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.ppt
hanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.ppthanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.ppt
hanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.pptssuser148ae0
 
Cse 402 offline b2
Cse 402 offline b2Cse 402 offline b2
Cse 402 offline b2sujoyhnkc
 
Backtracking-N Queens Problem-Graph Coloring-Hamiltonian cycle
Backtracking-N Queens Problem-Graph Coloring-Hamiltonian cycleBacktracking-N Queens Problem-Graph Coloring-Hamiltonian cycle
Backtracking-N Queens Problem-Graph Coloring-Hamiltonian cyclevarun arora
 
A* and Min-Max Searching Algorithms in AI , DSA.pdf
A* and Min-Max Searching Algorithms in AI , DSA.pdfA* and Min-Max Searching Algorithms in AI , DSA.pdf
A* and Min-Max Searching Algorithms in AI , DSA.pdfCS With Logic
 
101-maths short cut tips and tricks for apptitude
101-maths short cut tips and tricks for apptitude101-maths short cut tips and tricks for apptitude
101-maths short cut tips and tricks for apptitudePODILAPRAVALLIKA0576
 
Graph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam LermaGraph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam LermaPyData
 
Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...
Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...
Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...Fernando Rodrigues Junior
 
I. Mini-Max Algorithm in AI
I. Mini-Max Algorithm in AII. Mini-Max Algorithm in AI
I. Mini-Max Algorithm in AIvikas dhakane
 

Similar to Thesis paper (20)

quarto
quartoquarto
quarto
 
Parallel search
Parallel searchParallel search
Parallel search
 
B0730406
B0730406B0730406
B0730406
 
tic-tac-toe: Game playing
 tic-tac-toe: Game playing tic-tac-toe: Game playing
tic-tac-toe: Game playing
 
Two player games
Two player gamesTwo player games
Two player games
 
Chess engine presentation
Chess engine presentationChess engine presentation
Chess engine presentation
 
January 13, 2014
January 13, 2014January 13, 2014
January 13, 2014
 
Minmax and alpha beta pruning.pptx
Minmax and alpha beta pruning.pptxMinmax and alpha beta pruning.pptx
Minmax and alpha beta pruning.pptx
 
hanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.ppt
hanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.ppthanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.ppt
hanoosh-tictactoeqqqqqqqaaaaaaaaaaaa.ppt
 
Parallel searching
Parallel searchingParallel searching
Parallel searching
 
Cse 402 offline b2
Cse 402 offline b2Cse 402 offline b2
Cse 402 offline b2
 
Backtracking-N Queens Problem-Graph Coloring-Hamiltonian cycle
Backtracking-N Queens Problem-Graph Coloring-Hamiltonian cycleBacktracking-N Queens Problem-Graph Coloring-Hamiltonian cycle
Backtracking-N Queens Problem-Graph Coloring-Hamiltonian cycle
 
A* and Min-Max Searching Algorithms in AI , DSA.pdf
A* and Min-Max Searching Algorithms in AI , DSA.pdfA* and Min-Max Searching Algorithms in AI , DSA.pdf
A* and Min-Max Searching Algorithms in AI , DSA.pdf
 
Stratego
StrategoStratego
Stratego
 
Unit 3 ap gt
Unit 3 ap gtUnit 3 ap gt
Unit 3 ap gt
 
Graphs
GraphsGraphs
Graphs
 
101-maths short cut tips and tricks for apptitude
101-maths short cut tips and tricks for apptitude101-maths short cut tips and tricks for apptitude
101-maths short cut tips and tricks for apptitude
 
Graph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam LermaGraph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
 
Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...
Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...
Breadth-First Search, Depth-First Search and Backtracking Depth-First Search ...
 
I. Mini-Max Algorithm in AI
I. Mini-Max Algorithm in AII. Mini-Max Algorithm in AI
I. Mini-Max Algorithm in AI
 

Thesis paper

  • 1. Solving Four-in-a-Row on 9×4 and 10×4 Boards ∗ Vasileios Charatsidis January 2, 2017 Abstract This paper investigates the Four-in-a-Row game with the help of the MiniMax alpha-beta pruning algo- rithm, a pairing algorithm and a threat search heuris- tic. Transposition tables are used to reduce the num- ber of nodes that need to be investigated. With these techniques 9×4 and 10×4 were possible to be solved, both being a win for the first player. 1 Introduction Four-in-a-Row belongs to the mnk-games family, where m and n are the height and width of the board while k is the number of pieces a player must make in a row, column or diagonal to win. For example 333 is the TicTacToe game. Four-in-a-Row is played by two players. Every empty square of the board is a le- gal move. As the name suggests the first player that makes four in a line (either row, column or diagonal) wins (denoted by “score-4”). Four-in-a-Row is solved in every board except the boards from 9×4 to 29×4 [6, 7]. If the board is suf- ficiently large, the game is a win for the player who plays first (he will be called Black). Boards from 1×4 to 8×4 are draw and from 29×4 and larger are a win for Black. For m×5 boards, the game is drawn for m = 5 and a win for m = 6 [6, 7]. It is easy to prove that as a consequence the game is a win for every board larger than 6×5. The research question is to find the turning point, ∗This thesis was prepared in partial fulfillment of the re- quirements for the Degree of Bachelor of Science in Knowl- edge Engineering, University of Maastricht, supervisor: Dr. Jos Uiterwijk that lays between 9×4 to 29×4 boards, that makes the game a win for Black. The game tree of Four-in-a-Row, even for 9×4, is very large. In the starting position the branching factor is 36 and while the game unfolds the branching factor stays quite high for brute-force algorithms to work, hence pruning techniques must be applied in order to find the winning variations, given that they exist. 1.1 Overview Section 2 provides technical details about the imple- mentation of the techniques that were used. In Sec- tion 3 results and experiments are presented. In Sec- tion 4 the conclusions are given. Finally, in Section 5 there is some proposed future work. 2 Implementation The implementation of the program consists of the representation of the game and the solver. The solver consists of the move generation, the MiniMax alpha- beta search and the pairing algorithm. Technical de- tails considering the above parts are provided below. 2.1 Representation The game state is represented by bit-boards. A bit- board is a bit array of (board)height×(board)width elements, one binary digit for each square of the board. Each bit-board is represented by a long vari- able. Two bit-boards are sufficient to represent a po- sition, one for the black and one for the white stones (all played moves are going to be called stones). In 1
  • 2. Figure 1 an example is shown. A 5×4 board is repre- sented with two longs. The longs are written slightly differently below, so that it is made clear how they represent the stones (starting to write from the bot- tom rightmost 1 of the matrix and keep going from right to left and from bottom to top, both binary strings are attained). Figure 1: An example position. This is a very efficient and compact representation, which allows operations like checking for a terminal state or cloning the board, to be performed fast. To play a move, firstly the coordinates of the move are translated into an integer. Then this integer is translated into a long, using bit shifting. Finally, with the logical operator ”Xor”, the above long is integrated with the long that represents either the black or white stones, depending of whom player the move is. An example is the black move B1 in Figure 1 (that strikes score-4 vertically). The coordinates are translated into the integer x = row×width+column. With row equals 4 and column equals 1 one gets x = 4×4+1 = 17. Note that this arithmetic opera- tion is equal as counting (from left to right), starting at the top left corner with 0. This integer is trans- lated to a long by left-shifting 1 x times, thus, giving 100000000000000000. The black stones were at this point represented by the long 1010011000100000. The new stone is then integrated to the black stones with the simple Xor op- eration: 1010011000100000 Xor 100000000000000000 = 101010011000100000 (as shown in Figure 1). 2.2 Move Generation The biggest part of pruning the game tree takes place in the move generator class. This class, as the name suggests, generates the moves that will be investi- gated. The process for Black and White share the common steps below: 1. It is checked if the active player can achieve a score-4 and finish the game. If yes the move is returned. 2. It is checked if the opponent threats to achieve a score-4. If yes the move that defends is returned. Given that one of the above conditions is true, the move generator returns just one move. Oth- erwise, the process continues. 3. It is checked if there exists a move that creates a double threat (which always precedes and causes a score-4). As it is shown in Figure 2, move C3 makes a double threat (one at C4 and one at D3). Figure 2: Double threat at square C3. In all figures all the moves that are going to be explored are indicated in gray, as a matter of fact in Figure 2 only move C3. Important is that, while the algorithm searches for a double threat, it stores all the moves that pose a single threat (called threats) in the front of the move- list, starting from threats that are adjacent to the last black move. Threats are prioritized since 2
  • 3. they restrict the opponent into one single move. This fact often renders them as the best move available . As will be shown, the above detail is very important for the efficiency of the algo- rithm. 4. Next, the active player checks if the opponent threats to make a double threat. If that is the case, the available moves are the ones that stop the upcoming opponent double threat, plus the threats that the active player has. In Figure 3 moves A4, D7 and D4 defend to the upcoming double threat of A4 (one at D7 and one at D4). Moves A5, D8 and A7 are the aforementioned white threats. If played they “buy” White one move “time”, thus giving him the choice to de- fend in his next move. Figure 3: Defenses to a double threat. 5. If none of the above applies, the process contin- ues differently for Black and White. For Black the most reasonable moves are added. These are moves that meet at least one of the following conditions. (a) They are not more than two rows far away from the closest, row wise, black stone and they have synergy with at least one of the black stones. (b) They eliminate White’s potential. With synergy it is meant that there is potential for a score-4 consisting of the move that is going to be added and another black stone. With elim- inate White’s potential it is meant moves that are in the same group (groups are defined as lines of four in either row, column or diagonal) with two white stones and no other black stone. Such a move is the move D4 in the board of Figure 4 (which happens to be the winning move!). Figure 4: Candidate moves that are going to be ex- plored (in gray color) after 1. A5-C5 2. A6-A7. White numbers depict the ordering. In Figure 4 all gray candidate moves are close to the black stones, at A5 and A6, and have some potential to score-4 using A5 or A6 or both. Moves like A8, B8, D7, D5 and B3, although close enough, are not included, since there is no potential for score-4 with the black stones. D5 is in the same row as A5 but there is a white stone at the row group eliminating score-4 potential between them. A8 is in the same column group with A5 and A6 but, again, there is the white stone at A7. Finally B3, D7 and B8 are not in the same group with any of A5 or A6, hence, the above moves have no potential. Of course moves in the first, second, ninth and tenth rows are not included since they are too far away from the black stones. Again, moves that are adjacent to the last black move are added first, before all the other non- threat moves. This makes sense for both players. For Black moves adjacent to the last black move have, most likely, the best synergy and fulfill the purposes of it. As for White, moves adjacent 3
  • 4. to the last black move are most likely the best defenses. The above detail speeds up the con- vergence of both, Black to the winning variation and White to a successful defense, significantly. For example in Figure 4 the last black move is A6 hence the moves will be ordered as depicted by the white numbers: A4, A3 (threats), B7, B5, B6 (adjacent to the last black move A6), then C6 and D7 (same row as the last black move A6) and then all the others. Considering White, besides move ordering, all of his legal moves must be calculated. After the above five steps have been performed, the part of the move generation is over and the exploration of the generated moves starts. 2.3 MiniMax alpha-beta search For the exploration of the game tree the Mini- Max alpha-beta [3] search in a Iterative deepening depth-first [4] fashion. Iterative deepening depth-first search (IDDs) is a state-search strategy in which a depth-limited version of depth-first search runs re- peatedly with increasing depth limit until the goal is found. IDDs is convenient in cases that the height of the game tree is unknown. The evaluation of a given board is fairly simple. It assigns the value 300 if Black has achieved a double threat and the value -300 if White has achieved a double threat, every other position is assigned with the value 0. Every won position (with a score of 300 or -300) is stored to a table and used when it occurs again, either in later depth limits or by different move ordering in the same depth limit. Every time the depth limit of IDDs is exhausted and the last move is played by White a special search, called threat search is performed. Threat search The idea came out naturally (independently without knowing its existence in the literature [1]) when it was noticed that before a double threat, a number of continuous threats (threat barrage) always take place. Threat search as the name suggests explores, strictly, moves that are threats. This is done recur- sively, starting with one of the possible threats checks if by continuously threatening the opponent, a double threat can occur. The search goes through all pos- sible threat combinations and stops when a double threat is found or no other threats can be made on the board. Situations where White’s defensive move creates at the same time a threat make the search stop as well. The average branching factor of, strictly, threaten- ing moves is low, although, the researcher can adjust the depth of threat search as he likes. The optimal depth is discussed in Section 3. It is clarified that af- ter a threat has been applied only related threats are examined. Related threats are defined as treats such that the last black move is part of them. Otherwise unrelated filler threats, that might exist but do not contribute to the solution, would be redundantly ex- plored, slowing the solver by approximately a factor of two. As an example, consider the position in Figure 5. When the solver is initiated the IDDs will at some point reach depth limit 8, thus the position will be evaluated. Since it is Black’s turn threat search will be performed. In the following figures, the threat Figure 5: Example starting state of threat search. search will be demonstrated. The reader should ig- nore gray squares since they are the moves that IDDs would explore. • As said, threat search will examine just threats 4
  • 5. B4 and C4, shown in Figure 5. B4 is chosen. White is forced to answer with C4 (Figure 6). Figure 6: Threat barrage after moves 5. B4-C4. • After ply 2, the only available threats are B2 and B3. B2 is chosen. White forcefully answers with B3, leading to the position in Figure 7. Figure 7: Threat barrage after moves 6. B2-B3. • After ply 4 (figure skipped), the threats to be searched are C3 and A1, Black chooses C3 and White answers A1. • After ply 6. the threats to be considered are A5 and D2. A5 is chosen and White answers D2 (Figure 8). Figure 8: Final state of threat barrage after 7. C3-A1 8. A5-D2. • After ply 8 in Figure 8 a double threat is iden- tified (with moves A6 and A3), hence a victory for Black. After that the starting position in Figure 5 is stored as a victory for Black for future use in later depths or if brought by different move ordering. While this sequence was forced, if the threat search at the depth limit has not been performed, many other moves would be generated, and explored by the iterative deepening search in later depths. If a threat barrage that leads to victory exists, it is guaranteed that it will be found. There is one downside, being that there might be a shorter threat barrage. The threat search just returns the first one found. Returning to the MiniMax alpha-beta search, a worth noting implementation detail is that the search tries to find scores ranging from 0 (draw) to 300 (win) for Black, and -300 (loss) to 0 for White; since the result is either a win for Black or draw. That way the program does not waste time searching for a White’s win. Transposition tables and symmetry Transposition tables were used to make the search more efficient. The won positions are stored in a hash table and used in later depths. All other undecided positions are stored in another hash, since they might 5
  • 6. reappear with different move ordering in the same depth. The latter hash table is emptied after the searching of the specific depth limit is over (since it might proven Black’s win in latter depth). In addition symmetric positions are identified and only one side of them is examined. 2.4 Pairing algorithm Pairing is a method that recognizes draw positions [2]. If every row, column, first and second diagonal group of the board can be covered then we have a pairing match and the position is a draw. With cov- ered it is meant that for every one of the above groups there are two points such that when Black plays one of them White will play the other, basically show- ing that White can have a stone in every group. The pairing match if it exists will look like Figure 9, where one is found after one move of Black and one of White have been played. Figure 9: Pairing in a 4×4 position. W stands for a white stone, B for a Black stone, “ - ” for row cover points, “ / ” for diagonal cover points and “ | ” for column cover points. It is worth noticing that many boards that are ob- viously drawing positions, like the empty 4×4 board cannot be matched with pairs. Without the black and white moves in the above 4×4 board there are not enough squares for all groups to be covered (we have 16 squares and ten groups, 4 row-groups, 4 column- groups, 1 first diagonal-group and 1 second diagonal- group, thus 20 squares need to cover them all)! The pairing algorithm is particularly useful when someone wants to prove that a given board is a draw, integrated in the alpha-beta search, simply by look- ing for a pairing match for every position. Other than that, it happened that this method was not par- ticularly helpful, since it makes the algorithm much slower, and the number of positions that are pruned, when Black wins, are less than five per cent. Considering the implementation of the pairing al- gorithm, again bitboards are used. Firstly, in a given board, like in Figure 10, one instance with its first di- agonal groups covered is created. So in the board of Figure 10 the “” will appear in four spots since there are two first diagonal groups without white stones. The board now is represented with three longs. Two for the pieces and one for the covered first diagonal groups. The latter long has ones when “” appears. Then from the produced board an instance of it with its second diagonal groups covered is created and that again is represented with one long with ones where “/” appears, so there are four longs now. As shown just two of them exist in Figure 10 since only one second diagonal group is uncovered. The same pro- cess continues for rows. Finally, the aforementioned instance, if it exists, is examined. If the configuration of the left empty squares is enough to cover all the column groups (overlapping between column groups should be taken into consideration) there is a pairing match. If not, the process repeats again trying all combinations possible exhaustively. Figure 10: Pairing in a 9×4 position. 3 Results and Experiments For every board, there are actually just two starting moves for Black, as shown in Figure 11. This happens under the assumption that starting in the middle row of the board has at least equal or more chances to win with every other row. Al- 6
  • 7. Figure 11: Starting moves for 9×4. though the above assumption is not proven, it is in- tuitively sound, since by playing in the middle row, Black has the maximum space both north and south of the board available, to achieve a score-4. Consider- ing the starting column, the move in the first column is different from the move in the second column but the moves in the third and fourth are symmetrical to the first ones respectively. The first board that was tried was the 10×4, and proved by the author a win for Black. At this time, the solver was very slow, such that it was not able to identify a win in 9×4, still the game was explored deeply and the author’s guesstimate was that 9×4 is a draw. 3.1 9×4 Black wins It was proved though, by Uiterwijk [8], while this pa- per was prepared, that the smaller winning board is the 9×4! Afterwards the above demonstrated solver1 confirmed the result. The only winning move is B5. Versus the most resilient white moves Black wins within 20 moves. The board is shown in Figure 12 left. The most resilient white move is the diagonal C6. The right board comes from the following variation: 1. B5-C6 2. C4-B4 3. A4-A6 4. A2-D6 5. B6-A3 6. C5-B3 7. C3-C2 8. C7-B2 (a filler threat that was mentioned before, it does not contribute but prolongs the game 1Both solvers have been built separately, from scratch, and differ quite significantly, considering the techniques used. one move until White’s inevitable loss) 9. B1-D5 10. B7-B6 11. A7 (making a double threat D7, D4). Figure 12: 9×4 Starting and final boards after 1. B5- C6. In Figure 13 are demonstrated some insightful in- formation printed in the console as outputs, after ev- ery depth limit is finished, while the game tree for 9×4, was being explored. D stands for the differ- Figure 13: Console prints, while exploring the tree for 9×4. ent depth limit in iterative deepening search. M is the move that has been considered as best so far, E stands for the evaluation score of that move. T stands for the amount of time, that has been spent so far. The following numbers are in millions and are cumulative. N-exp stands for the total number of nodes explored. IDDs-exp are the nodes expanded by iterative deepening search. Finally, TSN-exp are the nodes explored from threat search. Note that the prints starts at depth 8. In the first depths move A5 is considered as best but then in depth 18 move B5 appears with the score of 300, as a matter of fact the 7
  • 8. search is terminated as a forced win has been found2 . 3.2 Heuristics experimentation During the construction of the solver much experi- menting took place considering heuristics like threat search, pairing algorithm, different move orderings and move generations. Best performance The final, fastest, version uses all parts described in Section 2 except the pairing algorithm. As shown in Figure 13 it solves 9×4 within 358 seconds or roughly 6 minutes after exploring 78.73 million nodes in total, 28.24 million nodes by IDDs and 55.88 million nodes from threat search, with a speed of 219,916 nodes per second (n/sec). Move ordering The move ordering used shows its impact when in an almost similar version for 9×4, without any ordering, the solver explored roughly 536 million nodes only in the last depth limit! The whole search took 2 hours and 36 minutes. Threat search Considering the threat search, initially it was used in- side the move generator as a heuristic. While signif- icantly many nodes were pruned, a lot of redundant actions took place since MiniMax alpha-beta priori- tize threats anyway, meaning that they are explored pretty early in the game tree. As a matter of fact, the solver was as slow as 600 n/sec,such that 10×4 was solved in 2 days 8 eight hours (now roughly 2 minutes) while 9×4 could not be solved at all! However, if used correctly, threat search is a power- ful tool. Without it 713.56 million nodes are explored within thirty five minutes (340,764 n/sec), approxi- mately 14 times slower. This happens since without threat search the iterative deepening needs to reach depth limit 21, in contrast with depth limit 18. Other 2Actually in move 18 a threat barrage that leads to victory has been found. The threat barrage starts after move 9 in Section 3.1 and is 10. B7-B6 11. A7- . than that, much experimentation took place with the depth limit of the threat search (even variable depth has been tried) but it does not affect significantly the speed, given that the depth limit of threat search is larger than 2, when the game is solved in 30 minutes. Transposition tables For the 9×4 board the necessity of transposition ta- bles is shown: • Disabling both hash tables the solver explores 385.96 million nodes with IDDs and another 392.27 million nodes with threat search, 778.23 million nodes in total within exactly 41.5 min- utes (312,542 n/sec), 5 times slower. • Disabling the hash table that stores all moves in every depth, avoiding the exploration of transpo- sitions, the IDDs explores 126.58 million nodes while threat search explores 135.5 million, 262.09 millions in total within 14 minutes (307,485 n/sec), more than 2 times slower. • Finally, disabling the wins table produce sim- ilar results, the IDDs explores 125.15 million nodes and the threat search 133.21 millions in total 258.36 millions within roughly 14 minutes (307,142 n/sec), more than 2 times slower. Pairing Another method to examine is the pairing algorithm. The current pairing implementation, using no heuris- tic to converge faster into a pairing, makes it quite slow. Initially it was created to prove that a board is drawn but 9×4 proved to be a win. Nevertheless it was tested. Since pairing matches occur when many white moves have been played it was enabled just after and only for the 16th move. The result was 477,930 pair- ings to be found causing a total reduction of 5.91 million nodes explored. All in all for 9×4 it proved to be a neutral method, considering the time needed for the solver to find a solution. But for 10×4, that is solved in earlier depth, as expected pairing was not efficient, slowing the process approximately by a factor of 3. 8
  • 9. 3.3 Pairing threshold experimenta- tion. A research has been performed considering the thresholds, determined by the number of moves that must be played, for pairings to start appearing and for the board to be drawn. In the table below the re- sults are demonstrated. mn-1p and f-1p stand for the move number and the percentage of the board cov- erage for at least one pairing to exist, respectively. Likewise mn-d and f-d stand for the move and per- centage of the board coverage for every position to be paired (thus the board is drawn). Board mn-1p f-1p mn-d f-d 4×4 2 12.5% 2 12.5% 5×4 4 20% 4 20% 6×4 6 25% 8 33.33% 7×4 8 28.57% 14 50% 8×4 8 25% 18 56.25% 9×4 10 27% 223 61% 10×4 12 30% - - 11×4 14 32% - - Unfortunately the sample is too small for a specific pattern to be recognized. Figure 14 shows the outputs4 while 8×4 is exam- ined. E stands for evaluation. When the evaluation is 1 it means that the position is undecided and when 0 that there is a pairing. Since in move 18 the best evaluation achieved is 0, the search stops meaning that the board is drawn. The pairings seems to fol- low something like a sigmoid function as until depth 16 their number is insignificant, in contrast with the nodes explored, at depth 16 they massively appear, and at depth 18 every board can be matched to a pairing, thus formulating a steep threshold. 3.4 The difference between 10×4 and 9×4. 10×4 proved to be Black’s victory for both starting moves A5 and B5. In 9×4, though, there is just one variation that holds against Black’s A5. This varia- 3With A5 as first move. 4IDDs-exp pd now stands for nodes expanded from IDDs per depth limit. Figure 14: Nodes expanded from IDDs and pairings per depth in 8×4. tion starts with the first white move at C5 and it is demonstrated in Figure 15. Figure 15: 10×4 board after 1. A5-C5 2. A6-A4 3. B7- . In 10×4 the third move of white B6 is defeated easily with the following threat barrage: 1. A7-A8 2. C7-D7 3. C8-D9 4. C9- that ends up in a vertical double threat as demonstrated in Figure’s 15 right. In 9×4, though, the 10th row is absent such that Black cannot create the double threat, as a matter of fact the position in Figure 15 left omitting the 10th row, proved to be a draw for 9×4 with the help of the pairing algorithm. Other than this variation, all other moves lose to A5 even in 9×4, and that is why C5 move is so special. 10×4 is a Black victory within 15 moves. The longer lasting variation start as 1. A5- B5 2. C5- . The solution is found within 130 seconds after 38.87 million nodes have been explored. 9
  • 10. 3.5 Other variations on 9×4 An interesting variation arises after 1. B5-C5. The winning move appears to be 2. D5- . A threat barrage is found after 13 moves. D5 looks counter intuitive since it does not follow the synergy rule. In the process, that moves are generated, explained above black moves should have some synergy, that obviously does not exist in Figure 16. This is a rare instant that this happens, as a matter of fact, after manually C5 was tested and proved to be the best move there was a change in the heuristic such that it allows moves that create the shown configuration (more specifically: a black stone and adjacent to it, in the same row, a white one, and then a black stone adjacent to the aforementioned white, all three in the same row). The above situation, constituted a teach- ing example demonstrating the danger of heuristics that might over-prune the tree 5 . Figure 16: Board after 1. B5-C5 2. D5- . All other white moves lose quite fast. 4 Conclusions The main conclusion of this paper, considering the research question, is that Four-in-a-Row is a win on 9×4, the threshold that makes the game a win for Black in the range of 9×4 to 29×4. Furthermore, 10×4 is the first board that Black wins with both of the starting moves. 5Note that the complete disabling of the synergy rule was tried, with worse results. Considering the implementation details the move ordering affects the efficiency of the MiniMax alpha- beta algorithm the most. Furthermore, many heuris- tics, that seem useful, are proven too computationally expensive to contribute efficiently to the algorithm, like the pairing method. And the final conclusion is a surprising phe- nomenon that has been noticed. Moves that are more than one row6 far away from the closest, row wise, black stone do not need to be explored at all, for Black to converge to win, unless: • they are threats or • they are the one point vertical “jump” from the last black move. With the aforementioned move selection 9×4 is solved within 224 seconds (40% faster). Figure 17 Figure 17: Board after 1. B5-C5 2. D5-B6 . demonstrates the above concept for Black. The only gray moves being more than one row away from the closest, row wise, black stone are D7 and D3 placed at the one point vertical “jump” from the last black move D5. This is an important fact, showing that Four-in- a-Row is clearly a tactical game. All moves, except the first one, have impact locally and trigger tacti- cal combinations instead of having a general strategic purpose which benefits Black later on. 6In the move selection for this algorithm was used the limit of 2 rows away from the closest black move as said in Figure 4, but 1 row limit is enough for converging to the solutions both for 9×4 and 10×4. 10
  • 11. 5 Future work A couple of ideas have been arisen but were not im- plemented due to time constrains. • Does the above phenomenon, the local-move idea, holds for every or some other mnk-games? • Since the game can be formulated as finding a line path of 4 from any node to any other against an opponent an interesting question emerges. Is there a relation between the pairing steep thresh- old and the percolation’s steep threshold7 [5] ? • Creating a more efficient pairing algorithm by transforming the game from: whoever score-4 wins to whoever strikes a double threat wins. That would allow a pairing to exist in boards that are drawn but the standard pairing does not. One example is 4×4, which is an obvious draw, but a normal pairing cannot exist unless White places a move in it. Instead of trying to cover all row, column and diagonal groups, some- one would try to cover all different groups in which a double threat can exist (a double threat needs 7 squares, hence the groups will consist of 7 squares). In a 4×4 board there are 32 such groups, overlapping with each other, meaning that there are not needed 32×2 squares for them to be covered, and probably a pairing can occur. Of course it is given that when a single threat appears, White just answers it. • A cut heuristic, cuts being defined as shapes that divide the board such that moves that being played in the north side of the cut do not affect the south side. The position, then, can be solved with two different trees, and the starting board is assigned with the best score of the two smaller ones. Even if the starting board is drawn, it is faster to find a pairing in two smaller boards. • Finally, the generalization of threat search into a selective depth search that in start explores only 7In a big square grid there is almost always a path connect- ing one side with the other if more than 59% of its tiles are explored. On the other hand if less than 59% of its tiles are explored almost never there is such a path. moves local with the last black move. When a win cannot be found, a broadening of the range of moves takes place. References [1] Allis, L.V., Herik, H.J. van den, and Huntjens, M.P.H. Go-Moku Solved by New Search Tech- niques Computational Intelligence 12 (1996) 7– 23. [2] Hales, A.W., and Jewett, R.I.: Regularity and Positional Games. Trans. Amer. Math. Soc. 106 (1963) 222-229. [3] Knuth, D.E., and Moore, R.W.: An analysis of alpha-beta pruning. Artificial Intelligence 6 (1975) 293-326. [4] Korf, Richard (1985). ”Depth-first Iterative- Deepening: An Optimal Admissible Tree Search”. Artificial Intelligence. 27: 97–109. [5] Stauffer, D. and Aharony, A. Introduction to Per- colation Theory, 2nd ed. London: Taylor and Francis, 1992. [6] Uiterwijk, J.W.H.M.: Kennisbehandeling in po- sitionele spelen; computeranalyse van Four-in-a- Row. Proceedings of NAIC’91, the Fourth Dutch AI Conference (Ed. J. Treur), Stichting Informat- ica Congressen, Amsterdam, The Netherlands; (1991) 193-207. [7] Uiterwijk, J.W.H.M. and Herik, H.J. van den: The advantage of the initiative. Information Sci- ences 122 (2000) 43-58. [8] Uiterwijk, J.W.H.M. Personal communication (2016). 11