This document describes LoGiC, a system for procedurally generating RPG worlds using a self-learning neural network. It discusses using graphs and adjacency matrices to represent game structures and rewriting them in real-time. A neural network called Eve is used to learn rewrite rules by training on examples using backpropagation and a SARSA reinforcement learning agent. The graphs are then rewritten using the algorithm milk to transform the game world based on what Eve has learned. The goal is to generate diverse yet coherent game worlds in an open-source system called logic-game.

1. LoGiC: Generating RPG Worlds using
Self-Learning Neural Networks
Marlon Etheredge
Amsterdam University of Applied Sciences
marlon.etheredge@hva.nl
December 5, 2012

2. About This Research
Fast exact graph matching using adjacency matrices
Describes a solution for the Graph Isomorphism problem
Strong focus on Procedurally Generated Content
Describes graph-based game structures rewriteable by rewrite
rules
milk: Implementation of the algorithm (open source)
Builds on work by Joris Dormans
LoGiC
Learning Game world Creator
Describes a method of creating RPG (or other sorts) game
world by using a self-learning Neural Network (NN)
Makes use of SARSA

3. A Subproblem: Genetic vs Graph Based Procedural
Content (Reiteration)
Requirement of realtime alternation of structures within the
game
Evolutionary algorithms require an arbitrary number of
generations
Undesired with the quick and direct way of transformations we
demand
Graph Based Procedural Content offers us fast and direct
modification of structures within the game
Practically any structure within the game may be represented
by a graph
Allows for the modifications of a lot of structures within the
game

4. Another Problem: Graph Isomorphism (Reiteration)
Graph Based Procedural Content requires fast realtime Graph
Transformation
Fast realtime algorithm for solving the Graph Isomorphism
problem
Existing well-known algorithms including:
VF2
Mainly focuses on large graphs
Ullmann
Too slow for usage in our project
R. Heckel
Exponential processing time
Need for a fast algorithm still offering full flexibility
Solved by milk

5. milk: Adjacency Matrices (Reiteration)
Uses adjacency matrices
Adjacency matrix generation by trivial function
For every connection in set set in a two dimensional by index
of first node and index of second node, a one
Matrices should store connection count for rows and columns,
convenient to store this at the end of the row or column
C1 E2 B3 A4 A5 B6 l
 
C1
C1 0 0 0 0 0 0 0
B6 A5 A4 E2
E2  1
 0 1 0 0 0 2
B3
B3  0
 0 0 0 0 0 0
Pattern
A4  0
 1 0 0 0 0 1
A5  0
 0 0 1 0 0 1
B6  0 0 0 0 1 0 1
l 1 1 1 1 1 0 0

6. Neural Network: Eve
Neurons and connections
Weights, determine when to fire and passthrough
Input Layer (input neurons, sensors)
Hidden Layers
Output Layer (output, actions/decisions/others)

7. Letting Eve learn
Back propagation, two phases (propagation and revise)
First phase step 1: Forward propagation, run input through the
NN, generate output
First phase step 2: Backward propagation, run the output in
reversed order through the NN and determine ∆ for all
neurons in the hidden and output layer
Second phase step 1: Multiply ∆ and input activation to
determine gradient
Second phase step 2: Define weight by reversing the gradient
and taking into account a defined Learning Rate
AND training set
Training set
Generalization (true, never seen data)
Validation (verify the training accuracy)
0 0 0
0 1 0
1 0 0
1 1 1

8. SARSA
Try to follow me
Q(st , at ) ← Q(st , at ) + α[rt + γQ(st+1 , at+1 ) − Q(st , at )]
Update the Q value of a pair of st , at taking into account a
learning rate α and error value
Q values are determined by a prediction of the next st , at ,
Q(st+1 , at+1 )
With other words: For a prediction of the next st , at ,
determine some sort of ’weight’, define the best st , at pair
See what’s happening here?

9. SARSA, continued: Backpropagation
Defined an ideal st , at pair
Act according to this st , at pair
At some point, this will lead to some result set (in case of
logic-game, the player finishing the game)
When using a SARSA agent within our NN we need to keep
track of all the possible st , at pairs for later reference
Use backpropagation and generalize the chosen st , at pair
through the NN:
Run the actual result set in reverse order through the NN
Update weights
Iterate to live another world (hopefully a bit smarter than
before)

10. The Big Picture

11. XML to Graphs to World
Using XML to represent an abstract form of our soon to be
graph-based structures
Graph-based structures are directly converted to geometry
Currently supporting two types of overlapping geometry types:
GeoEntity: May contain children (represents dividable
geometry, or space)
GameEntity: Is a leaf (represents concrete objects)
An assumption with logic is that we always supply a very basic
default world
Rewriting of game structures is performed on graph-structure
level

12. Summary
logic-game: 2d topdown RPG, engine written on top of XNA
4.0 (for convenience)
Eve: NN implementation in C
SARSA agent: State-Action-Result-State-Action, defines the
best state-action pair according to predicted result determined
by NN
Backpropagation: Used to update the internal connection in
the NN
milk: Graph-isomorphism implementation used to rewrite the
structures inside the game, rewrite rules define concrete action
sets

13. Work in progress
Components
logic-game
Eve
SARSA Agent
The Big Picture
Open source: Visit marlonetheredge.name for updates

14. Thank You Very Much
Thank you!
I’ll now show some source.
Marlon Etheredge
marlon.etheredge@hva.nl
http://marlonetheredge.name/
”Big results require big ambitions.”
Heraclitus