This is the 4th of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
This lecture shows how we can use mathematical analysis to classify players into stereotypes and leverage this classification into generating more successful decisions.
(Some content appears to be missing from the end of this one - I'll fix this as soon as I can)
Mathematics online: some common algorithmsMark Moriarty
Brief overview of some basic algorithms used online and across data-mining, and a word on where to learn them. Prepared specially for UCC Boole Prize 2012.
Lecture 8 - What is Game AI? Final ThoughtsLuke Dicken
This is the last lecture in the series series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
In this lecture I rehash the fundamental differences between Game AI and the traditional AI that has been taught in previous courses. It also includes a (frankly time-filling) section called the "Brain Dump" where I briefly touch on a bunch of things I was thinking about at the time.
This session was given to the small group of students at University of Strathclyde participating in the after-hours Game Development program. It covers different roles within the industry and a number of different aspects of the kinds of teams you will need in order to make a game.
This is the 3rd of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
This lecture moves beyond the Game Theoretic definition of a game, and demonstrates how algorithms can be used not only to find a single good choice, but a sequence of choices that will eventually reach a winning state.
Around three years ago I took my first steps into the games industry. Now I'm reasonably well known, recognised as an expert in my area and get to present at conferences around the world. The things that have helped me achieve that though aren't all that hard, and in this talk I discuss some of the tools I've used to become who I am, as well as talking a lot about my own insecurities and those that many other developers were able to share with me.
Influence Landscapes - From Spatial to Conceptual RepresentationsLuke Dicken
These slides are from a presentation of a paper from AISB 2011. They lay out the concept of the Influence Landscape, a technique which uses Automated Planning tools to apply Influence Map-style representations to conceptual as well as spatial representations
This is the first of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
This lecture introduces the concept of a game, and the branch of mathematics known as Game Theory.
Mathematics online: some common algorithmsMark Moriarty
Brief overview of some basic algorithms used online and across data-mining, and a word on where to learn them. Prepared specially for UCC Boole Prize 2012.
Lecture 8 - What is Game AI? Final ThoughtsLuke Dicken
This is the last lecture in the series series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
In this lecture I rehash the fundamental differences between Game AI and the traditional AI that has been taught in previous courses. It also includes a (frankly time-filling) section called the "Brain Dump" where I briefly touch on a bunch of things I was thinking about at the time.
This session was given to the small group of students at University of Strathclyde participating in the after-hours Game Development program. It covers different roles within the industry and a number of different aspects of the kinds of teams you will need in order to make a game.
This is the 3rd of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
This lecture moves beyond the Game Theoretic definition of a game, and demonstrates how algorithms can be used not only to find a single good choice, but a sequence of choices that will eventually reach a winning state.
Around three years ago I took my first steps into the games industry. Now I'm reasonably well known, recognised as an expert in my area and get to present at conferences around the world. The things that have helped me achieve that though aren't all that hard, and in this talk I discuss some of the tools I've used to become who I am, as well as talking a lot about my own insecurities and those that many other developers were able to share with me.
Influence Landscapes - From Spatial to Conceptual RepresentationsLuke Dicken
These slides are from a presentation of a paper from AISB 2011. They lay out the concept of the Influence Landscape, a technique which uses Automated Planning tools to apply Influence Map-style representations to conceptual as well as spatial representations
This is the first of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
This lecture introduces the concept of a game, and the branch of mathematics known as Game Theory.
Machine Learning Essentials Demystified part1 | Big Data DemystifiedOmid Vahdaty
Machine Learning Essentials Abstract:
Machine Learning (ML) is one of the hottest topics in the IT world today. But what is it really all about?
In this session we will talk about what ML actually is and in which cases it is useful.
We will talk about a few common algorithms for creating ML models and demonstrate their use with Python. We will also take a peek at Deep Learning (DL) and Artificial Neural Networks and explain how they work (without too much math) and demonstrate DL model with Python.
The target audience are developers, data engineers and DBAs that do not have prior experience with ML and want to know how it actually works.
Hands-On Machine Learning with Scikit-Learn and TensorFlow - Chapter8Hakky St
This is the documentation of the study-meeting in lab.
Tha book title is "Hands-On Machine Learning with Scikit-Learn and TensorFlow" and this is the chapter 8.
Knowing When to Hold 'Em, When to Fold 'Em and When to Blow 'Em UpLuke Dicken
Guest presentation given to a mixed-discipline group at the University of West of Scotland Research Students Society @ UWoS 3rd March 2010.
Topics covered : High level overview of work with AI for Poker, Ms. Pac-Man and my own research on the I2 system, concluding with some of my opinions on the current state of Academic and Industrial Game AI.
Coffee beans variants recommendation using clusteringMihirKadam3
By clustering different coffee bean variants together, we can use this information for targeted marketing. If a customer buys a variant, other similar variants can be recommended to them.
In some cases, variants from completely different clusters can also be recommended if the customer is looking for a different taste. Since connoisseurs also like to mix-match their beans, two beans from completely different clusters can also be recommended for a blend.
Introduction, Terminology and concepts, Introduction to statistics, Central tendencies and distributions, Variance, Distribution properties and arithmetic, Samples/CLT, Basic machine learning algorithms, Linear regression, SVM, Naive Bayes
This talk, delivered at the Høgskolen i Bergen (Bergen College) in Norway in October 2014. It covers some recent games and deconstructs potential AI techniques that could* be used by these games to achieve this.
* Note that the author has no knowledge of the internals of these games and this is broadly educated speculation.
What's with all the zombies in games right now? In this session, I talk about some of the reasons that Zombies are a lazy AI Engineers dream come true and what we could be doing instead
Machine Learning Essentials Demystified part1 | Big Data DemystifiedOmid Vahdaty
Machine Learning Essentials Abstract:
Machine Learning (ML) is one of the hottest topics in the IT world today. But what is it really all about?
In this session we will talk about what ML actually is and in which cases it is useful.
We will talk about a few common algorithms for creating ML models and demonstrate their use with Python. We will also take a peek at Deep Learning (DL) and Artificial Neural Networks and explain how they work (without too much math) and demonstrate DL model with Python.
The target audience are developers, data engineers and DBAs that do not have prior experience with ML and want to know how it actually works.
Hands-On Machine Learning with Scikit-Learn and TensorFlow - Chapter8Hakky St
This is the documentation of the study-meeting in lab.
Tha book title is "Hands-On Machine Learning with Scikit-Learn and TensorFlow" and this is the chapter 8.
Knowing When to Hold 'Em, When to Fold 'Em and When to Blow 'Em UpLuke Dicken
Guest presentation given to a mixed-discipline group at the University of West of Scotland Research Students Society @ UWoS 3rd March 2010.
Topics covered : High level overview of work with AI for Poker, Ms. Pac-Man and my own research on the I2 system, concluding with some of my opinions on the current state of Academic and Industrial Game AI.
Coffee beans variants recommendation using clusteringMihirKadam3
By clustering different coffee bean variants together, we can use this information for targeted marketing. If a customer buys a variant, other similar variants can be recommended to them.
In some cases, variants from completely different clusters can also be recommended if the customer is looking for a different taste. Since connoisseurs also like to mix-match their beans, two beans from completely different clusters can also be recommended for a blend.
Introduction, Terminology and concepts, Introduction to statistics, Central tendencies and distributions, Variance, Distribution properties and arithmetic, Samples/CLT, Basic machine learning algorithms, Linear regression, SVM, Naive Bayes
This talk, delivered at the Høgskolen i Bergen (Bergen College) in Norway in October 2014. It covers some recent games and deconstructs potential AI techniques that could* be used by these games to achieve this.
* Note that the author has no knowledge of the internals of these games and this is broadly educated speculation.
What's with all the zombies in games right now? In this session, I talk about some of the reasons that Zombies are a lazy AI Engineers dream come true and what we could be doing instead
Procedural Processes - Lessons Learnt from Automated Content Generation in "E...Luke Dicken
In this talk, given at the 2012 No Show Conference, and alongside long-term partner in crime Heather Decker-Davis, we talk about our game "Easy Money?" and our approach to content generation - along with the challenges they provided and the way it affected our workflow.
Game AI 101 - NPCs and Agents and Algorithms... Oh My!Luke Dicken
This is a session originally written for students at Bradley University (Peoria, IL).
It covers a very high level introduction to the concepts behind Game AI, and includes some examples of how we can begin to make characters in a game world perform actions and appear to be making intelligent decisions.
This session was the first in a series given to a group of University students of differing year groups and abilities. In this lecture, I try to highlight some of the many different aspects that need to be decided when thinking about how to make a game, and demonstrate that simply picking a genre is insufficient.
This is the 7th of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
This lecture covers ways that we can use AI to manage the experience that the player receives. Topics include Immersive Worlds, Player/Game Interactions, Interactive Fiction and "AI Directors" such as that found in Left4Dead
Lecture 6 - Procedural Content and Player ModelsLuke Dicken
This is the 6th of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
In this lecture I link together the material presented in lectures 3 and 4 on profiling players and show how this can be used to good effect with Procedural Content Generation (lecture 5). I use Silent Hill : Shattered Memories as a specific example, and discuss research using Tomb Raider, and the standard Bartle Player Types.
This is the 5th of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
In this lecture I outline some approaches that use AI techniques to automate the creation of content within game world. I make specific reference to assets such as rocks and plants, to interaction mechanisms such as weapons and to quest generating systems, in particular Skyrim's Radiant engine.
This is the 2nd of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course.
This lecture covers the fundamentals of probability theory, and is relatively basic to ensure that all students have a good grasp on the concept.
The Strathclyde Poker Research EnvironmentLuke Dicken
This presentation was given at AISB 2011 and introduces the Strathclyde Poker Research Environment (SPREE) an open tool for Poker research. Available from Sourceforge @ https://sourceforge.net/projects/spree-poker/
The Ludic Fallacy Applied to Automated PlanningLuke Dicken
This is a short talk I gave to the Strathclyde Planning Group on deficiencies I can see in the way we thing and reason about planning in non-deterministic environments. PPDDL - the accepted standard - is overly simplistic and can get us into hot water because we focus on solving the PPDDL problem, rather than the Real World problem it models.
The breakout session that followed was very useful for generating a lot of ideas about different threads we could use to attack the weaknesses of PPDDL and work being done around the edges, which I hope to summarise at some point.
Artificial Intelligence in Computer and Video GamesLuke Dicken
This lecture was given at the April meeting of the Glasgow branch of the British Computer Society on 12th April 2010. The lecture was supposed to be given by Dr. Darryl Charles, who fell ill a couple of days before the event, and I was asked to take the lecture instead.
In the presentation I cover the basics of why AI and Games are well suited and give a brief discussion of different types of AI as I see it. I discuss briefly how AI fits into the context of the game in terms of execution.
The bulk of the talk presents case studies in the format of Commercial game -> Theoretical technique used -> Research project using this technique.
It should be noted that the section on Left4Dead was omitted from the lecture as it was presented at the time due to concerns about the length
Integrated Influence - The Six Million Dollar Man of AILuke Dicken
This presentation introduces the fundamentals of contemporary AI research and highlights a significant challenge that we have still not addressed - namely that we have to trade quality of decision making against speed of decision making.
It goes on to discuss the concepts behind the "Integrated Influence Architecture", a new approach to making high-speed and high-quality decisions currently under development at University of Strathclyde.
A presentation of work undertaken for my Masters by Research degree. This presentation outlines how we can use new techniques from Automated Planning to enable agents to execute plans in a robust manner, allowing for recovery from faults with necessarily requiring a re-plan.
A brief overview of the emerging AI field of "General Games". This presentation was originally given as part of the Researchers' Digest series at University of Strathclyde on 14th Dec 2009.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
2. Monte Carlo in Poker
(Recap)
• Yesterday we saw that Monte Carlo could be used to
estimate the expected reward of an action by evaluating the
delayed reward
• We do this by simulating or "rolling out" games to their end
state.
• Assess the amount we won or lost
2
4. Random Walks
in the Game Tree
• When we walk the Game Tree at random, we pick nodes to
follow at random.
• We assume (for now) that this is an unbiased choice
• This means every choice has the same probability of being
chosen
4
5. Can We Do Better?
• Random walks are all well and good
• But a uniform distribution across action choices isn't
accurate
‣ Certain situations will make sensible players more likely to use
certain actions
• How can we bring this bias into play in the walk?
5
6. Classifying Opponents
• The way we do this is to work out what type of player
someone is.
• We observe them to get a better understanding of how they
operate.
• In Poker and other games, we can use all sorts of statistical
measures to quantify a player's type.
6
7. Action Prediction
• Once we know what kind of player someone is, we can flip
things on their head.
• We answered "what is the likelihood this player is type X
given we have seen this type of play"
• We can now answer "what is the likelihood this player will
make action Y given they are of type X"
• Remember from Bayes Theorem last week, these questions
are closely linked
7
8. Simple (Human)
Classification
• Pro Poker players try to quantify their opponents into one of
several classes based on 3 measures
‣ Voluntarily Put in Pot (VPiP)
‣ Won at Showdown (WSD)
‣ Pre-flop Raise (PFR)
8
9. Player Stereotypes
• Players can be
‣ Tight / Loose (how likely they are to play hands)
‣ Passive / Aggressive pre-flop
‣ Passive / Aggressive post-flop
9
10. Utilising Stereotypes
• If we can classify players we can use this against them
• For instance, we might discover that passive players can be
chased off by aggressive play
• Or we understand that when a super-conservative player
decides to raise, we need to be careful
• We can build heuristic rule bases around this like we saw
before.
• Or we can be much smarter
10
11. Better Classifications
• Humans are getting by on 3 dimensions
• But Poker has waaaay more statistics available than this
• We can make a lot of use of this extra data.
11
12. Poker Tracker
• Poker Tracker is a stats package specifically for Poker
• Analyses play at online casinos
• Real-time access to stats about opponents
• Allows players to review hands later
12
13. Stats in Poker
• A few slides ago - Poker has many statistics
• Poker Tracker keeps tabs on around 150 metrics
• Some of these are somewhat similar, some relate more to
the games than the players
13
14. Problem of Dimensionality
• The problem now is that we have too much information!
• Trying to learn on cluttered data can be problematic,
assuming it works at all.
14
15. Dimensionality Reduction
• Somehow we have to reduce the number of dimensions that
our data points are using.
• In many ways, getting the right data into a learning algorithm
is the biggest challenge.
• As much art as it is engineering.
• Two options
‣ Feature selection
‣ Feature extraction
15
16. Selection vs Extraction
• In Selection, you pick the dimensions you believe to be most
relevant
‣ The human players did this to get their 3 dimensional
representations
• In Extraction, you come up new dimensions that can
represent your datapoint
16
17. Principal Components
Analysis
• PCA is a common strategy for this.
• Recasts the dimensions of the datapoint into another set of
"basis vectors".
• Smushes together dimensions that have a strong correlation
‣ Some stats measures are looking at fundamentally the same thing, in
different ways
‣ E.g.Various raise frequency metrics might be treated as a single
“aggression” dimension after PCA
17
18. Principal Components
Analysis
• This was going to be a worked example.
• Honestly, that’s way to painful.
• For N observations in M dimensions X is a matrix
where each column is an observation.
• Calculate the mean and std. dev. for each row in the
matrix (each dimension)
18
19. Principal Components
Analysis
• Calculate the covariance matrix, the amount that
the dimensions vary with respect to each other.
• Calculate the eigenvectors and eigenvalues of the
covariance matrix
‣ The eigenvectors are the new basis vectors of the
reduced-dimension datapoints
‣ The eigenvalues represent how significant the
eigenvector is. Large value = significant
19
20. Principal Components
Analysis
• Pick the most significant K of the eigenvectors.
• Project the original datapoint in X onto the new
basis vectors.
20
21. Principal Components
Analysis
• Honestly, if anyone ever asks you to do this
‣ Get a textbook
‣ Use Matlab
‣ Be really careful because it’s kind of complicated
• It is possible to do it by hand.
‣ I can’t anymore...
21
22. Principal Components
Analysis
• Assuming that you finish the calculations without
mucking up.
‣ Or, you find something to work it out for you (Matlab
functions for this exist)
• What you have now is a new datapoint, that is
approximately the same information.
• Recast into fewer dimensions.
‣ Note that the dimensions will not make sense
22
24. Clustering Algorithms
• Having performed PCA, we have a much more manageable
set of datapoints, and we’ve eliminated extraneous
dimensions
• Now we need to group them together.
• Clustering algorithms are one approach.
• Tries to find a set of “clusters” of points that are grouped
together.
24
25. Clustering
50.0
37.5
25.0
12.5
0
0 7.5 15.0 22.5 30.0
Blue Peter style example - real data is rarely so neat
25
26. Clustering
• k-means is one of the most popular algorithms
‣ Others exist, fuzzy c-means, FLAME clustering and more
• Pick a value for k
‣ You can play around a bit to find good values or use
some tricks
‣ Accepted “rule of thumb” :
26
27. K-Means Algorithm
• Typically, we run the k-means algorithm as an
“iterative refinement” process
‣ Guess at some initial values, keep running the process
round and round until it stabilises
• Randomly assign datapoints to one of the k clusters
• Step 1 - Calculate centroids of the clusters
• Step 2 - Update assignment based on new centroids
• Rinse and repeat 1 and 2 until convergence.
27
28. K-Means Algorithm
• Calculating Centroids of clusters
‣ xj denotes the datapoints being sampled
‣ mi(t+1) denotes mean of cluster i at iteration t+1
‣ Si(t) denotes the set of datapoints assigned to cluster i at
iteration t
• Effectively, the average of the datapoints
28
29. K-Means Algorithm
• Assigning Datapoints to Clusters
• The set of points Si is all datapoints for which the
centroid of cluster i (mi) is the nearest centroid.
29
31. From Classification to
Prediction
• Once we have our clusters defined, we know what
datapoints constitute the type of player we are analysing
• We can use this to predict what the player will do
‣ We have a collection of “similar” players, we can use
their history.
‣ We may be able to use the raw data from the
observations directly.
• In either case, we can use the classification to predict actions
31
32. Back to Monte Carlo
• So, back to the game tree.
• We now have an idea of what type of player we are dealing
with.
• We have an idea of what actions the players are going to
take in given situations.
• Can we plug this back into the Monte Carlo simulation?
32
33. Informed Walks
in the Game Tree
• We talked earlier about Opponent nodes in the game tree
• Specifically, when we hit an Opponent node, we would use a
uniform distribution to randomly pick between the options
available.
• Now, we can bias that distribution towards selecting the
action we expect the player to take.
33
34. Does This Work?
• Intuitively, it should
• The more accurate we make the simulation, the
more accurate the results should be.
• Concern is that the prediction process will slow
things down too much
‣ Monte Carlo relies on large numbers of samples, if they
take too long, accuracy isn’t helping.
34
35. Does This Work?
• We don’t know.
• It’s been proven to aid Monte Carlo for Poker when
k=1
‣ All players are treated as a generic “player”
• This is ongoing research right now in SAIG.
• Look for papers next year. :)
35
36. What We Do Know
• We’ve previously attempted Machine Learning for
Opponent Modelling.
• Using 32 different statistical measures (reduced
down to 8 significant dimensions by PCA)
• Training data of 700,000 hands of Poker
• Successfully extracted around 28 different player
stereotypes.
36
37. The Aim of the Game
• We aren’t going to be able to make an AI that
always wins at Poker
• There’s too much chance involved
‣ Bad hands come up
‣ Mis-interpreting players
• What we want to do is make an AI that performs
better than the other players under the same
circumstances
37
38. Evaluation
• Any time we do research we are testing some sort
of scientific hypothesis.
• We need to design experiments to test whether the
hypothesis is true or not
• Science doesn't care if we're right - unbiased. Even if
we're wrong, we have learnt something.
38
39. Evaluation
• Consider a pro Poker player
• Will win some games and lose others
‣ In fact, a fundamental rule of good poker play is not even
taking part in about 80% of the games you sit through
• Measuring in terms of a single game doesn't work
‣ Need to look at the forest, not the trees
• What counts is how much money the player wins at
the end.
39
40. Measuring the Strength of
an AI
• What we need is a measure of how successful a bot
is on average.
• Poker gives a metric for this - Big Blind / 100
‣ Metric is in terms of the table limit - normalised
• Note that even for a large number of games, the
variance on this measure can be really big.
‣ Recall Black Swan events - low likelihood, high
impact. Large wins are Black Swans here.
40
41. Stable Experimentation
• We really need a way to remove the variance from
the problem.
• Ordinarily we might repeat the experimentation, take
a large number of sample, use law of averages to our
advantage.
• We talked yesterday about the state space of just the
card dealing component of Poker
‣ We know it's too large for this to be an option
41
42. Experimentation
• What if we generate experimental scenarios.
• A large number of games, with the deck already
configured.
• We can play the scenario with player A
• Then replay the exact same scenario with player B
• The results that player A and B generate are now
comparable.
42
43. Experimental Design
• Designing good experiments is really important
• Not just for AI but for all kinds of things
• Understanding sources of uncertainty means we can
find ways to factor them out
• Design fair unbiased experiments
• For Science!
43
44. Summary
• More detail on Monte Carlo in Poker
• Explanation of Opponent Modelling in Poker
‣ Dimensionality Reduction
‣ Clustering algorithms
• Exploiting Opponent Models
• Experimental Design
44
45. Next Week
• Other uses for Opponent Models
• Procedural Content Generation
• AI in Video Games
45