A Sequential Recommendation Approach for Interactive Personalized Story Generation paper presentation

Παντελιδάκης Μηνάς
Θεοδωρής Χαράλαμπος
Παρουσίαση θεωρητικής εργασίας
A Sequential Recommendation Approach for Interactive
Personalized Story Generation
Hong Yu and Mark O. Riedl
College of Computing
Georgia Institute of Technology
Atlanta, GA 30332
{hong.yu, riedl}@cc.gatech.edu
ΠΛΗ 517 – Πολυπρακτορικά Συστήματα – Χειμώνας 2017

There is a whole genre of games known as interactive
story games. To modernize and make them more
appealing, developers have been using agents (better
known as Drama Managers) that build a player profile
deriving from the player’s in game actions.
Hong Yu and Mark O. Riedl approach this issue by
constructing the player model based on the user’s
preferences (ratings) instead of their actions.
Introduction

In this paper…
● Drama Manager as surrogate for both the human designer and
the user
● Prefix-based collaborative filtering for modeling the user’s
preference profile
● Two learning algorithms, probabilistic Principal Component
Analysis (pPCA) and Non-negative Matrix Factorization (NMF)
● Validation testing on both human and simulated users

●
Drama Manager
● Story Library
● Player Model
● User
Setup

But what is a Drama Manager?
An omniscient agent that acts to
bring about a particular sequence
of plot points for the user to
experience
● Monitors the system state
● Selects the optimal next plot point from the story library

Story Library
● Plot points form story graphs after termporal and semantic
constraints are applied
●
Story graphs are later transformed into prefix graphs and stored
in the story library
● Notice that the prefix graph will be a tree or forest since any
prefix cannot have more than one parent node
Story graph prefix graph → tree

Player Model
●
Extracts the dimensions of the users’ preferences from the users’
ratings
●
It is not constrained by pre-defined dimensions
●
Assumes that people who share similar preference in the past tend to
share it again in the future, hence it utilizes a form of collaborative
filtering
●
Stories are presented plot point by plot point and a preference rating
for the story-so-far is collected after every plot point
●
Ratings are stored in the story library along with the story prefix
●
The prefix-rating matrix is obtained

Player Model+
The prefix-rating matrix
● Stars (*) represent missing ratings.
● Dimensions nxm (n prefixes, m users)
●
Input for both pPCA and NMF

Principal Component Analysis
● A technique of dimensionality reduction
● Aims at finding some correlation among data of a large
data set

Probabilistic
Principal Component Analysis
● Assumes the prefix-rating vector obeys a multi-
dimensional Gaussian distribution
● If the user-prefix matrix contains missing values, the EM
algorithm can be used to compute W and σ.

Expectation Maximization
●
Iterative algorithm with two phases
●
Starts with random Gaussian distributions
●
Phase 1: correlate data to given Gaussian distributions
●
Uses soft clustering to assign data to Gaussian distributions
●
Phase 2: estimate new Gaussian distributions given the data

Non-negative Matrix Factrorization
● Constrains data to non-negative values
● Constrains only to additive combinations
● Given t types, creates W (weights-by-types)
● and H (types-by-users) vectors.
● Uses EM to determine best W and H values, using existing data.

Player Modeling Steps
● Build the story library
● Collect the ratings and populate the matrix R
● Compute W, σ and μ for the pPCA, or W for NMF
For A new user:
● Model his preference using r, computing x for pPCA or h for NMF
● Calculate the full rating vector r’ from x
● Select the best full-length story, descendant of the current prefix
● Collect the user’s rating on the story-so-far
● Include the new rating into r and goto step 1
Training the model:

● A story library based on Choose-Your-Own-
Adventure books was transcribed
● Both human and simulated testers
Testing

● Four choose-your-own-adventure books for better story quality
● The Abominable Snowman, Journey Under the Sea, Space and Beyond,
●
The Lost Jewels of Nabooti
● Transformed into prefix graphs, containing 154 possible stories and 326 prefixes
Story Library Components

Model Training
Story Generation Testing
Experiments with human users

Model Training Phase
●
31 users, only 5 exposed to similar games
●
Each user reads and rates 10 random stories for initialization
●
Obtained a 326-by-31 prefix matrix R with ~86% ratings missing
●
Split R to 90% training set / 10% validation set randomly 10 times
●
Each time the RMSE is calculated after training
● The average of all the training data for that session suggested there
were 6 types of users

● Represents the sample standard deviation of
● the differences between predicted values and observed values
● Intuitively, finds out how good the expectation is by calculating the
distance of real datapoints from the expected function
● The smaller the RMSE, the better our expectation
Root Mean Square Error

Procedure & results
●
22 new users alongside 11 training ones
● 5 random stories read and rated
● DM chooses personalized stories based on user preference
● Final NMF and pPCA models are trained
● The stories are presented to the users and rated
●
DM serves another 5 personalized stories to the users
●
followed by 5 random stories to eliminate prejudice

Experiments with simulated users
Same steps as with humans
Many more (consistent) users

Model Training phase
● Built based upon Robin’s Law (5 gamer types)
● 5-dimensional characteristic vector
● Prefixes on library were labeled according to those types
● 120 randomized one-type-only users
● 10 stories read and rated for initialization
● A 326-by-120 prefix-rating matrix was populated

Story Generation phase
● 1000 fully random generated users tested the algorithm exactly
as the human ones.
● Baseline P and N: PPCA and NMF without prefixes, rated only at
the end.
● Vector: A vector that simulates the user, all stats are initialized at
0, get added values as the game progresses
● PPCA: The proposed PPCA algorithm
● NFMwP, NFMwoP : The proposed NFM algorithm with and without
prior knowledge of player types (Robin’s Law)

Results
● The prefix algorithms are a lot more accurate than their non-
prefix counterparts
● NMF algorithms outclass every other algorithm even without
prior knowledge

Results+
● Average accuracies as the number of story changes
● Prefix-based algorithms shine from the second story forth
● In particular, NMF algorithms are very accurate from the start

Results++
● Average RMSEs of the prefix-based algorithms with different
numbers of training users/data
● NMF with prior is the most accurate from start to finish

Conclusion
● Future work could combine structured (user preference) and
unstructured feedback (user actions) to further improve player
modeling
● Machine learning + human intuition = Successful storytelling

Congratulations!
You have endured our presentation!
We hope you found the topic as interesting as we did, and we encourage
you to engage us backstage on the topic!
Good Luck to the upcoming teams!

A Sequential Recommendation Approach for Interactive Personalized Story Generation paper presentation

Recommended

Recommended

More Related Content

Similar to A Sequential Recommendation Approach for Interactive Personalized Story Generation paper presentation

Similar to A Sequential Recommendation Approach for Interactive Personalized Story Generation paper presentation (20)

Recently uploaded

Recently uploaded (20)

A Sequential Recommendation Approach for Interactive Personalized Story Generation paper presentation