Candy Crush Saga is one of the biggest mobile games today with more than 1000 levels of difficulty—and users continue to ask for more. When building new content, it is extremely important to make sure that the level of difficulty is balanced and that the user does not experience crashes or problems through some unforeseen level of play. Alexander Andelkovic shows you how King is training artificial intelligence (AI) programs (bots) to test its games by mimicking human interactions. Join Alex as he discusses how King is taking testing to the next level by employing Monte Carlo Tree Search, automatic heuristic construction, and NeuroEvolution of Augmenting Topologies (NEAT) to train bots to test and evaluate difficulty levels. He discusses ways to extend and use AI bots to predict game success rates and conduct automatic performance testing. Alex explains how this AI approach can be generalized to test other applications. Learn how AI can help you with testing that's getting very difficult to master with traditional testing techniques.

1. W12
Session
10/26/2016 1:30:00 PM
Using Artificial Intelligence to Test the
Candy Crush Saga Game
Presented by:
Alexander Andelkovic
King/Midasplayer AB
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888-­‐268-­‐8770 ·∙ 904-­‐278-­‐0524 - info@techwell.com - http://www.starcanada.techwell.com/

2. Alexander Andelkovic
King/Midasplayer AB
The agile testing lead for Sweden-based King/Midasplayer AB, developer of the
popular mobile game Candy Crush Saga, Alexander Andelkovic has worked on
multiple complex test projects ranging from using session-based test
management for quality assuring MED-Tech devices for life critical systems to
establishing a world class approval process for Spotify apps used by Fortune 500
companies. Now he teams with developers in testing big data, business analytics,
and game level regression testing using AI. Alex performs both system testing and
exploratory testing with a focus on assisting teams with high-quality deliveries.
He is a frequent speaker at international conferences including both STAREAST
and STARWEST.

3. Using Artificial Intelligence to Test the Candy Crush Saga
Game

4. © King.com Ltd 2016
About us
King/Midasplayer AB
• Founded 2003
• +200 Games
• 2000 employees
• 13 Studios(Stockholm, London, Barcelona,
Seattle…)
• 409 million MAU(Q2 2016)
• Acquired by Activision/Blizzard(2016-02-23)
Page 2

5. © King.com Ltd 2016
Products
Franchise Games
Page 3

6. © King.com Ltd 2016
QA at King
King QA roles
• ATL – Agile Testing Lead
• Ninja – Exploratory tester
• QRT – Quick Regression Team
• TAE – Test Automation Engineer
• DS – Data Scientist
• Dev – Frontend/Backend
Developer
Page 4

7. © King.com Ltd 2016
QA Problem
More than 2000 levels in Candy
Page 9

8. © King.com Ltd 2016
Introduction
History of AI
Page 6

9. © King.com Ltd 2016
Introduction
So, what is a bot exactly?
Page 7

10. © King.com Ltd 2016
Introduction
Application that performs an automated task
Page 8

11. © King.com Ltd 2016
Introduction
AI-bot –
why?
Page 9
• Level designers
• QualityAssistance (QA)
• Game-domain research

12. © King.com Ltd 2016
AI-bot
How would an AI-bot actually think(heuristic,
simulation)?
Page 9

13. © King.com Ltd 2016
Heuristic
Heuristic is good, maintenance is bad!
Page 11
• Having a function that ranks moves – heuristic – could
help a lot
• Each simulation much richer
• Has to be generic
• Increasing complexity with more features
• Can we create the heuristic automatically?

14. © King.com Ltd 2016
MCTS
Simulation – Monte Carlo Tree Search
Page 12

15. © King.com Ltd 2016
MCTS
Page 13
Simulate example

16. © King.com Ltd 2016
MCTS
Page 14
Win Loss Win
Is this reliable?

17. © King.com Ltd 2016
MCTS
Page 15
36% 28% 19%
Repeat simulations – Monte-Carlo simulations

18. © King.com Ltd 2016
MCTS
Page 16
36% 28% 19%
Gradually we build a tree
=> Monte-Carlo Tree Search
40% 32%

19. © King.com Ltd 2016
MCTS
Search tree for level 13 in Candy
Page 17

20. © King.com Ltd 2016
MCTS
Available actions
Page 18

21. © King.com Ltd 2016
MCTS
Future states
Page 19

22. © King.com Ltd 2016
MCTS
Page 20
After 299 simulations

23. © King.com Ltd 2016
MCTS demo
Page 21
MCTS demo

24. © King.com Ltd 2016
MCTS - Impact
What do we have?
Page 22
• Generic method
• Works for all levels
• Works for other games

25. © King.com Ltd 2016
MCTS - summary
MCTS summary
Page 23
• Closer to human success rate
• Easy to tune with more simulation per
decision
• Time consuming, one level takes
~ 5-10 mins
• How can we improve?

26. © King.com Ltd 2016
NEAT
NEAT – NeuroEvolution of Augmented
Topologies!
Page 24
• Artificial Neural Network(ANN)
• Reinforced learning(action, learn, evolve)
• Guided playout

27. © King.com Ltd 2016
NEAT
Play Candy Crush!
Page 25
4 1 0 0

28. © King.com Ltd 2016
NEAT
NEAT – Steps
Page 26
• Get training data - Predict score (hours)
• Create random bots
• Compete (hours)
• Create new bots
• Choose the best bot - ANN
• Play Candy choosing moves in playout with
ANN

29. © King.com Ltd 2016
NEAT
The jungle law in Candy
Page 27
Focus on
blockers!
Look at
jellies!
Special candies are tasty!

30. © King.com Ltd 2016
NEAT
Making babies
Page 28
Mom Dad

31. © King.com Ltd 2016
NEAT
Page 29
Newborn
Mom Dad

32. © King.com Ltd 2016
NEAT
Product of evolution
Page 30

33. © King.com Ltd 2016
NEAT demo
Page 31
NEAT training
• Red=Fittest Bot
• Yellow=Avg Bot
fitness
• Green=# Bot’s
• Blue=Fittest Bot
ANN nodes

34. © King.com Ltd 2016
NEAT demo
Page 32
NEAT training
demo

35. © King.com Ltd 2016
Results
Guiding the
playout helps
Page 33

36. © King.com Ltd 2016
NEAT
NEAT – Performance
Page 34
• Simulation time, game engine needs to replay
each game state
• Traversing time, game engine does not
support jump to state
• More GPU’s for bot training(5-6 hours > on
commit)

37. © King.com Ltd 2016
Content QA
Content QA – No bot
Page 35

38. © King.com Ltd 2016
Content QA
Content QA – With bot
Page 36

39. © King.com Ltd 2016
Benefits
Bot benefits
Page 37
• Level development: level difficulty, less
tweaks
• Quality assistance: crash testing,
performance testing, regression testing
• Data scientists: Game domain
knowledge, fun levels, game balancing

40. © King.com Ltd 2016
Challenges
Bot challenges
Page 38
• Ownership(Knowledge, Resources, Bot
team)
• Integration(Headless mode, C++/Lua/JS,
Hackday/Sprint, Bot Api)
• Maintenance(Bot Training, Infrastructure,
Extend)

41. © King.com Ltd 2016
Q&A
alexander.andelkovic@king.c
om
Page 39
Questions?

42. © King.com Ltd 2016
Thank you!
Page 40