The journey started with an unused beta server and a relatively high amount of issues which were found after live release. In the first step we staffed the beta team with very active members who knew the game very well and had a sixth sense for bugs. We also introduced missions which gave a testing focus on specific parts of the game including a rough explanation of new features. During the time boxed missions players could report bugs and for valid bugs they got a bonus at the end. To be a valid bug, the report needed to follow a specific template to provide all necessary information and it had to be part of the scope of the mission. At this point we were already utilizing Gamification by using three different game mechanics: Discovery, Bonuses, Countdown. Gamification is the use of game thinking and game mechanics in non-game contexts to engage users in solving problems and increase users’ self contributions. As a first result the issues found before live release increased, but the turnover rate among the testers was quite high.
As there are many more game mechanics which are already a success in our games, we thought it might help for motivating our testers as well. Hence we thought about a level system with achievements where players gain points for bug reports to accomplish new levels. A leaderboard shows the progress of all participating players and also the achievements on the next level. We also introduced so called Seasons which are running for three months, where a player can get even more bonuses and achievements. After the first months we can already see that the turnover rate is decreasing.
As the journey went on, we got very dedicated beta testers who kept on testing. Also the quality of the game increased by finding and fixing more issues before the live release, and the percentage of live issues was reduced from 50 to 25 percent.
4. Introduction
-‐ Conductor
• Since
2011
working
for
InnoGames
• Currently
Senior
QA
Technical
Tester
• Projects:
Tribal
Wars,
Forge
of
Empires
5. Introduction
-‐ Passenger
• Care
for
the
customers
and
quality
of
the
product
• The
less
bugs
the
less
complaints
6. Introduction
-‐ Passenger
• Dedicated
and
motivated
players
• Like
to
play
with
a
minimum
of
nuisance
• Interested
in
new
content
before
it
goes
live
7. First
Stop
-‐ Gamification
Gamification is
the
use
of
game
thinking
and
game
mechanics
in
none-‐game
contexts
to
engage
users
in
solving
problems
and
increase
users’
self
contributions.
8. First
Stop
-‐ Game
Mechanics
Bonuses
Achievements
Countdown
Discovery
Levels
Points
Progress
Status
9. First
Stop
-‐ Team
• Strengthening
the
Beta
team
• Experienced
players
with
game
knowledge
• Prior
testing
knowledge
• Direct
communication
channels
• Faster
reaction
and
feedback
cycle
10. Construction
Zone
-‐ Mission
Goal
• Activation
and
increasing
the
beta
testing
activities
• Increase
issue
spotting
rates
for
features
with
random
content
• Gauge
reactions
and
reception
of
a
slightly
more
regulated
feedback
approach
11. Construction
Zone
-‐ Missions
Rules
• Template
usage
• No
duplicates
• No
comments,
discussions,
etc.
• Not
“by
design”
12. Construction
Zone
-‐ Missions
• Short
descriptions
• Template
Tasks
• Questions
• Problems
Testing
• Error
reporting
• Test
report
Report
13. Construction
Zone
-‐ Mission
Result
• Reduction
of
defect
reporting
after
release
by
25%
• Bug
reports
of
higher
quality
by
beta
testers
• Increased
beta
testing
activities
14. Construction
Zone
-‐ Experience
• Better
planning
certainty
in
bigger
teams
• Steady
testing
activities
through
regular
feature
updates
• High
fluctuation
among
beta
testers
