Unreal conference slides

1,113 views

Published on

Slides from today's Indie Games Summit at FIU

  • Be the first to comment

Unreal conference slides

  1. 1. REAL GOOD UNREAL TESTING CHARLES P. SCHULTZ Author “Game Testing All In One” INDIE GAMES SUMMIT
  2. 2. TEST SUBJECTS <ul><li>WHAT CAN I TEST? </li></ul><ul><li>UNREAL ENGINE VERSIONS, PORTS </li></ul><ul><li>MODS </li></ul><ul><li>LEVELS </li></ul><ul><li>WEAPONS, VEHICLES </li></ul><ul><li>PC, CONSOLE or MOBILE </li></ul>INDIE GAMES SUMMIT
  3. 3. TEST OBJECTIVES <ul><li>FIND BUGS </li></ul><ul><li>PINPOINT CAUSES </li></ul><ul><ul><li>- Environment </li></ul></ul><ul><ul><li>- Repeatability </li></ul></ul><ul><ul><li>- Frequency </li></ul></ul><ul><li>ESTABLISH WHAT WORKS </li></ul><ul><li>VERIFY FIXED BUGS </li></ul><ul><li>GIVE DEVS FEEDBACK </li></ul>INDIE GAMES SUMMIT
  4. 4. TESTER’S WEAPONS <ul><li>PARAMETERS </li></ul><ul><li>MODELS </li></ul><ul><li>USE CASES </li></ul><ul><li>METHODS </li></ul><ul><li>TOOLS </li></ul><ul><li>PLAYER STYLES </li></ul><ul><ul><li>- Achiever, Explorer, Killer, Socializer </li></ul></ul>INDIE GAMES SUMMIT
  5. 5. BOUNDARY VALUES <ul><li>0 </li></ul><ul><li>MINIMUM </li></ul><ul><li>MAXIMUM </li></ul><ul><li>MAXIMUM + 1 </li></ul><ul><li>SPECIAL VALUES </li></ul><ul><ul><li>- High Scores </li></ul></ul><ul><ul><li>February 29 th </li></ul></ul><ul><ul><li>Zone Boundaries: KOTOR example </li></ul></ul>INDIE GAMES SUMMIT
  6. 6. GAME PHASES PRE-GAME ACTIVATE GAME START Initialize system components Load maps and game assets Establish network communication Game controllers detected etc. IN-GAME POST-GAME DEACTIVATE INDIE GAMES SUMMIT
  7. 7. DEFECT TRIGGERS <ul><li>NORMAL BEHAVIOR </li></ul><ul><li>CONFIGURATION </li></ul><ul><li>STARTUP </li></ul><ul><li>EXCEPTIONS </li></ul><ul><li>STRESS </li></ul><ul><li>RESTART </li></ul>INDIE GAMES SUMMIT
  8. 8. DEFECT TRIGGERS <ul><li>NORMAL BEHAVIOR </li></ul><ul><ul><li>typically 80 to 100% of tests </li></ul></ul><ul><ul><li>most costly to find </li></ul></ul><ul><ul><li>relatively low cost to repair </li></ul></ul><ul><ul><li>less effective as project progresses </li></ul></ul><ul><ul><li>serious issues removed before ship </li></ul></ul>INDIE GAMES SUMMIT
  9. 9. DEFECT TRIGGERS <ul><li>CONFIGURATION </li></ul><ul><li>STARTUP </li></ul><ul><li>EXCEPTIONS </li></ul><ul><li>STRESS </li></ul><ul><li>RESTART </li></ul><ul><ul><li>up to 20% of tests </li></ul></ul><ul><ul><li>higher rate of defects found per test </li></ul></ul><ul><ul><li>requirements may be lacking </li></ul></ul>INDIE GAMES SUMMIT
  10. 10. TRIGGER GOALS <ul><li>NORMAL BEHAVIOR: 40% </li></ul><ul><li>CONFIGURATION: 20% </li></ul><ul><li>STARTUP : 10% </li></ul><ul><li>EXCEPTIONS: 20% </li></ul><ul><li>STRESS: 5% </li></ul><ul><li>RESTART: 5% </li></ul><ul><li>grow non-Normal test cases </li></ul><ul><li>elicit requirements from developers </li></ul>INDIE GAMES SUMMIT
  11. 11. RESTART “ You can fast swap weapons to reload the first one to make it fire faster, for example you can shoot the sniper, quickly swap to another weapon lets say hammer, swap back and shoot the sniper again as the reload time has been wiped” BUG EXAMPLE INDIE GAMES SUMMIT
  12. 12. COMBINATORIAL TABLES <ul><li>SELECT PARAMETERS TO TEST </li></ul><ul><li>- GUN, AMMO, RANGE </li></ul><ul><li>SELECT TEST VALUES FOR </li></ul><ul><li>EACH PARAMETER </li></ul><ul><li>- AMMO: 0, 1, MAXIMUM </li></ul><ul><li>PAIR EACH VALUE WITH ANOTHER </li></ul><ul><ul><li>PARAMETER’S VALUES </li></ul></ul><ul><ul><li>- AMMO=1 with </li></ul></ul><ul><ul><li>GUN=ROCKET, LINK GUN, SNIPER </li></ul></ul>INDIE GAMES SUMMIT
  13. 13. COMBINATORIAL TABLES <ul><li>START WITH FIRST VALUE OF ONE PARAMETER </li></ul>GUN AMMO RANGE ROCKET ROCKET ROCKET 0 1 MAX 1 MAX MAX+1 INDIE GAMES SUMMIT
  14. 14. COMBINATORIAL TABLES <ul><li>FOR THE NEXT PARAMETER, CREATE NEW PAIRS </li></ul>GUN AMMO RANGE ROCKET ROCKET ROCKET 0 1 MAX 1 MAX MAX+1 LINK GUN LINK GUN LINK GUN 0 1 MAX MAX MAX+1 1 INDIE GAMES SUMMIT
  15. 15. COMBINATORIAL TABLES <ul><li>CONTINUE UNTIL ALL PAIRS ARE SATISFIED </li></ul>GUN AMMO RANGE ROCKET ROCKET ROCKET 0 1 MAX 1 MAX MAX+1 LINK GUN LINK GUN LINK GUN 0 1 MAX MAX MAX+1 1 SNIPER SNIPER SNIPER 0 MAX 1 MAX+1 1 MAX INDIE GAMES SUMMIT
  16. 16. COMBINATORIALNOMICS <ul><li>For our 3x3 matrix, we covered all pair combinations with only 9 tests. A set of 13 parameters with 3 values would require 1.5 million tests for all combinations, but only requires 15 tests for pairs </li></ul><ul><li>FOR LARGE OR COMPLEX SETS OF PARAMETERS, USE A TOOL </li></ul><ul><li>ALLPAIRS </li></ul><ul><li>http://www.satisfice.com/tools.shtml </li></ul>INDIE GAMES SUMMIT
  17. 17. TEST FLOW DIAGRAMS <ul><li>Graphic model of a function or use case </li></ul><ul><li>Take the player’s point of view </li></ul><ul><li>Use loops </li></ul><ul><li>Add reverse flows </li></ul><ul><li>Can double as a design tool </li></ul>INDIE GAMES SUMMIT
  18. 18. TEST FLOW DIAGRAMS <ul><li>Start with an entry point and initial “state” </li></ul>IN AMMO FULL 1. Start INDIE GAMES SUMMIT
  19. 19. TEST FLOW DIAGRAMS <ul><li>Do something to change to another state </li></ul>IN AMMO FULL NO AMMO 2. Fire All Ammo 1. Start INDIE GAMES SUMMIT
  20. 20. TEST FLOW DIAGRAMS <ul><li>Can you go back to where you started? </li></ul>IN AMMO FULL NO AMMO 2. Fire All Ammo 3.Reload 1. Start INDIE GAMES SUMMIT
  21. 21. TEST FLOW DIAGRAMS <ul><li>Can you repeat anything? </li></ul>IN AMMO FULL NO AMMO 4. Fire Weapon 5.Reload 2. Fire All Ammo 3.Reload 1. Start INDIE GAMES SUMMIT
  22. 22. TEST FLOW DIAGRAMS <ul><li>Can you do something a different way? </li></ul>IN AMMO FULL NO AMMO 4. Fire Weapon 5.Reload 2. Fire All Ammo 3.Reload 1. Start 6. Pick up fresh weapon INDIE GAMES SUMMIT
  23. 23. TEST FLOW DIAGRAMS <ul><li>Provide an Exit point </li></ul>IN AMMO FULL NO AMMO 4. Fire Weapon 5.Reload 2. Fire All Ammo 3.Reload 1. Start 6. Pick up fresh weapon OUT 7. Exit INDIE GAMES SUMMIT
  24. 24. TEST FLOW DIAGRAMS <ul><li>Create tests from various paths </li></ul>1,2,4,7 1,2,3,2,7 1,2,6,2,7 1,2,3,5,2,7 1,5,2,3,5,2,4,6,2,7 INDIE GAMES SUMMIT IN AMMO FULL NO AMMO 4. Fire Weapon 5.Reload 2. Fire All Ammo 3.Reload 1. Start 6. Pick up fresh weapon OUT 7. Exit
  25. 25. ANY QUESTIONS?
  26. 26. EXTRAS
  27. 27. ESCAPED PROBLEMS CONFIGURATION Unreal Tournament 3 flickers after playing for some time / UT3 Flickering screen Solution A : Make sure that you are using the latest drivers. Solution B : Turn down / Turn-ff Anisotropic Filtering in UT3′s menu Unreal Tournament 3 models are the same for all players after playing the first map. Make sure that the ‘Allow custom characters’ option is checked in the options. This is because there are times that the config dont get saved or some of the items in the options menu gets randomly checked. Make sure that the UT3 config is changed after ticking the “allow custom characters” option by checking the options menu again after closing.
  28. 28. CONFIGURATION Unreal Tournament 3 Configs dont get saved / UT3 settings and cd-key not saved / cd-key problem Make sure that the MY DOCUMENTS or and UT3 folder is not set as read-only Cannot Play Unreal Tournament 3 Campaign Open the UTEngine.ini (My DocumentsMy GamesUnreal Tournament 3UTGameConfig) and change PumpWindowMessagesWhenRenderThreadStalled=False to PumpWindowMessagesWhenRenderThreadStalled=True ESCAPED PROBLEMS

×