What does ethical player testing entail? Are there things that game developers should never do to or with game testers? How can they be sure that they are being fair to players by informing them of things that might happen to them during testing, particularly if developers don't want to reveal everything about their game in advance? This presentation makes the case for ethical play testing and offers developers a practical guide for implementing ethical player testing. It starts with an overview of the history of human subjects research, provides resources for more information, and then dives into what ethical player testing can look like, and how to implement your own best practices for player testing.
A rough template to use and abuse when pitching Raw Fury (or other publishers as well, of course) Feel free to steal the structure and modify it as you see fit.
For a more in-depth explanation on How To Pitch Raw Fury:
https://rawfury.com/how-to-pitch-to-raw-fury/
Solving problems by searching Informed (heuristics) Searchmatele41
Informed Search – a strategy that uses problem-specific knowledge beyond the definition of the problem itself
Best-First Search – an algorithm in which a node is selected for expansion based on an evaluation function f(n)
Lec 3 knowledge acquisition representation and inferenceEyob Sisay
Artificial Intelligence lecture notes. AI summarized notes for knowledge reasoning and knowledge representation, its for you in order for reading and may be for self-learning, I think.
Talk by Yuriy O’Donnell at GDC 2017.
This talk describes how Frostbite handles rendering architecture challenges that come with having to support a wide variety of games on a single engine. Yuriy describes their new rendering abstraction design, which is based on a graph of all render passes and resources. This approach allows implementation of rendering features in a decoupled and modular way, while still maintaining efficiency.
A graph of all rendering operations for the entire frame is a useful abstraction. The industry can move away from “immediate mode” DX11 style APIs to a higher level system that allows simpler code and efficient GPU utilization. Attendees will learn how it worked out for Frostbite.
A rough template to use and abuse when pitching Raw Fury (or other publishers as well, of course) Feel free to steal the structure and modify it as you see fit.
For a more in-depth explanation on How To Pitch Raw Fury:
https://rawfury.com/how-to-pitch-to-raw-fury/
Solving problems by searching Informed (heuristics) Searchmatele41
Informed Search – a strategy that uses problem-specific knowledge beyond the definition of the problem itself
Best-First Search – an algorithm in which a node is selected for expansion based on an evaluation function f(n)
Lec 3 knowledge acquisition representation and inferenceEyob Sisay
Artificial Intelligence lecture notes. AI summarized notes for knowledge reasoning and knowledge representation, its for you in order for reading and may be for self-learning, I think.
Talk by Yuriy O’Donnell at GDC 2017.
This talk describes how Frostbite handles rendering architecture challenges that come with having to support a wide variety of games on a single engine. Yuriy describes their new rendering abstraction design, which is based on a graph of all render passes and resources. This approach allows implementation of rendering features in a decoupled and modular way, while still maintaining efficiency.
A graph of all rendering operations for the entire frame is a useful abstraction. The industry can move away from “immediate mode” DX11 style APIs to a higher level system that allows simpler code and efficient GPU utilization. Attendees will learn how it worked out for Frostbite.
Player Traversal Mechanics in the Vast World of Horizon Zero DawnGuerrilla
Download the original PowerPoint presentation here: http://www.guerrilla-games.com/read/player-traversal-mechanics-in-the-vast-world-of-horizon-zero-dawn
Paul van Grinsven shows what is needed to make Aloy traverse the vast world of Horizon Zero Dawn, with its complex and organic environments. Various traversal mechanics are covered from a gameplay programmer's perspective, focusing on the interaction between code and animations. The different systems and techniques involved in the implementation of these mechanics are explained, and Van Grinsven looks at the underlying reasoning and design decisions.
Talk by Graham Wihlidal (Frostbite Labs) at GDC 2017.
Checkerboard rendering is a relatively new technique, popularized recently by the introduction of the PlayStation 4 Pro. Many modern game engines are adding support for it right now, and in this talk, Graham will present an in-depth look at the new implementation in Frostbite, which is used in shipping titles like 'Battlefield 1' and 'Mass Effect Andromeda'. Despite being conceptually simple, checkerboard rendering requires a deep integration into the post-processing chain, in particular temporal anti-aliasing, dynamic resolution scaling, and poses various challenges to existing effects. This presentation will cover the basics of checkerboard rendering, explain the impact on a game engine that powers a wide range of titles, and provide a detailed look at how the current implementation in Frostbite works, including topics like object id, alpha unrolling, gradient adjust, and a highly efficient depth resolve.
Telling Story Through Sound: Building an Interactive "Radio Play"nfreakct
Video demo: https://youtu.be/FZm3d-3tgeM
Before visual media monopolized popular entertainment, radio plays were the predominant way to experience stories and adventures on a weekly basis. Enhanced by voice actors, sound designers, and musicians the audience was transported into new and exciting realms. Today, with the emergence of new listening-only devices, audio-based narrative experiences are poised for a comeback. In this session join the developers of the 'Baker Street Experience', a Sherlock Holmes interactive choose-your-own-adventure for the Amazon Echo, as they chronicle their experiences and discuss the importance of audio design in building narrative environments within the current limitations of the audio-only devices.
Takeaway
Attendees will gain insights about creating audio-only adventures in the style of classic radio plays, the importance of audio design in building narrative environments and settings, and the current limitations of the Amazon Echo/Alexa Skills Kit and similar platforms.
This talk presents the approach Frostbite took to add support for HDR displays. It will summarize Frostbite's previous post processing pipeline and what the issues were. Attendees will learn the decisions made to fix these issues, improve the color grading workflow and support high quality HDR and SDR output. This session will detail the display mapping used to implement the"grade once, output many" approach to targeting any display and why an ad-hoc approach as opposed to filmic tone mapping was chosen. Frostbite retained 3D LUT-based grading flexibility and the accuracy differences of computing these in decorrelated color spaces will be shown. This session will also include the main issues found on early adopter games, differences between HDR standards, optimizations to achieve performance parity with the legacy path and why supporting HDR can also improve the SDR version.
Takeaway
Attendees will learn how and why Frostbite chose to support High Dynamic Range [HDR] displays. They will understand the issues faced and how these were resolved. This talk will be useful for those still to support HDR and provide discussion points for those who already do.
Intended Audience
The intended audience is primarily rendering engineers, technical artists and artists; specifically those who focus on grading and lighting and those interested in HDR displays. Ideally attendees will be familiar with color grading and tonemapping.
Drew University Celebrates Its Sesquicentennial in 2017Tony Ehinger
Having served as managing director at the Switzerland-based financial services company Credit Suisse, Tony Ehinger focused his time on retirement investments, properly balancing stock portfolios, and managing low cost index funds for its clients. Outside of his interest in business and economics, Tony Ehinger enjoys farm-to-table dining and is a supporter of the student center at Drew University.
Killzone Shadow Fall: Creating Art Tools For A New Generation Of GamesGuerrilla
This talk describes the tool improvements Guerrilla Games implemented to make Killzone Shadow Fall shine on the PlayStation 4. It highlights additions to the Maya pipeline, such as Viewport 2.0, Maya's coupling with in-game updates and in-engine deferred renderer features including real-time shadow-casting, volumetric lighting, hardware instancing, lens flares and color grading.
Chapter 9: Evaluation techniques
from
Dix, Finlay, Abowd and Beale (2004).
Human-Computer Interaction, third edition.
Prentice Hall. ISBN 0-13-239864-8.
http://www.hcibook.com/e3/
Building Non-Linear Narratives in Horizon Zero DawnGuerrilla
Download the original PowerPoint presentation here: http://www.guerrilla-games.com/read/building-non-linear-narratives-in-horizon-zero-dawn
Open world RPGs require deep stories, which emphasize player choice and freedom. The bigger the games grow, the more complex systems managing these stories get. See how Guerrilla tackled the issue in Horizon Zero Dawn by creating a quest system which has non-linearity at its base.
Player Traversal Mechanics in the Vast World of Horizon Zero DawnGuerrilla
Download the original PowerPoint presentation here: http://www.guerrilla-games.com/read/player-traversal-mechanics-in-the-vast-world-of-horizon-zero-dawn
Paul van Grinsven shows what is needed to make Aloy traverse the vast world of Horizon Zero Dawn, with its complex and organic environments. Various traversal mechanics are covered from a gameplay programmer's perspective, focusing on the interaction between code and animations. The different systems and techniques involved in the implementation of these mechanics are explained, and Van Grinsven looks at the underlying reasoning and design decisions.
Talk by Graham Wihlidal (Frostbite Labs) at GDC 2017.
Checkerboard rendering is a relatively new technique, popularized recently by the introduction of the PlayStation 4 Pro. Many modern game engines are adding support for it right now, and in this talk, Graham will present an in-depth look at the new implementation in Frostbite, which is used in shipping titles like 'Battlefield 1' and 'Mass Effect Andromeda'. Despite being conceptually simple, checkerboard rendering requires a deep integration into the post-processing chain, in particular temporal anti-aliasing, dynamic resolution scaling, and poses various challenges to existing effects. This presentation will cover the basics of checkerboard rendering, explain the impact on a game engine that powers a wide range of titles, and provide a detailed look at how the current implementation in Frostbite works, including topics like object id, alpha unrolling, gradient adjust, and a highly efficient depth resolve.
Telling Story Through Sound: Building an Interactive "Radio Play"nfreakct
Video demo: https://youtu.be/FZm3d-3tgeM
Before visual media monopolized popular entertainment, radio plays were the predominant way to experience stories and adventures on a weekly basis. Enhanced by voice actors, sound designers, and musicians the audience was transported into new and exciting realms. Today, with the emergence of new listening-only devices, audio-based narrative experiences are poised for a comeback. In this session join the developers of the 'Baker Street Experience', a Sherlock Holmes interactive choose-your-own-adventure for the Amazon Echo, as they chronicle their experiences and discuss the importance of audio design in building narrative environments within the current limitations of the audio-only devices.
Takeaway
Attendees will gain insights about creating audio-only adventures in the style of classic radio plays, the importance of audio design in building narrative environments and settings, and the current limitations of the Amazon Echo/Alexa Skills Kit and similar platforms.
This talk presents the approach Frostbite took to add support for HDR displays. It will summarize Frostbite's previous post processing pipeline and what the issues were. Attendees will learn the decisions made to fix these issues, improve the color grading workflow and support high quality HDR and SDR output. This session will detail the display mapping used to implement the"grade once, output many" approach to targeting any display and why an ad-hoc approach as opposed to filmic tone mapping was chosen. Frostbite retained 3D LUT-based grading flexibility and the accuracy differences of computing these in decorrelated color spaces will be shown. This session will also include the main issues found on early adopter games, differences between HDR standards, optimizations to achieve performance parity with the legacy path and why supporting HDR can also improve the SDR version.
Takeaway
Attendees will learn how and why Frostbite chose to support High Dynamic Range [HDR] displays. They will understand the issues faced and how these were resolved. This talk will be useful for those still to support HDR and provide discussion points for those who already do.
Intended Audience
The intended audience is primarily rendering engineers, technical artists and artists; specifically those who focus on grading and lighting and those interested in HDR displays. Ideally attendees will be familiar with color grading and tonemapping.
Drew University Celebrates Its Sesquicentennial in 2017Tony Ehinger
Having served as managing director at the Switzerland-based financial services company Credit Suisse, Tony Ehinger focused his time on retirement investments, properly balancing stock portfolios, and managing low cost index funds for its clients. Outside of his interest in business and economics, Tony Ehinger enjoys farm-to-table dining and is a supporter of the student center at Drew University.
Killzone Shadow Fall: Creating Art Tools For A New Generation Of GamesGuerrilla
This talk describes the tool improvements Guerrilla Games implemented to make Killzone Shadow Fall shine on the PlayStation 4. It highlights additions to the Maya pipeline, such as Viewport 2.0, Maya's coupling with in-game updates and in-engine deferred renderer features including real-time shadow-casting, volumetric lighting, hardware instancing, lens flares and color grading.
Chapter 9: Evaluation techniques
from
Dix, Finlay, Abowd and Beale (2004).
Human-Computer Interaction, third edition.
Prentice Hall. ISBN 0-13-239864-8.
http://www.hcibook.com/e3/
Building Non-Linear Narratives in Horizon Zero DawnGuerrilla
Download the original PowerPoint presentation here: http://www.guerrilla-games.com/read/building-non-linear-narratives-in-horizon-zero-dawn
Open world RPGs require deep stories, which emphasize player choice and freedom. The bigger the games grow, the more complex systems managing these stories get. See how Guerrilla tackled the issue in Horizon Zero Dawn by creating a quest system which has non-linearity at its base.
Designing effective user research to discover the truth PeakXD
The truth doesn't cost you anything but a lie could cost you everything. Tania Lang's presentation at UX Australia's Design Research conference March 2019
Week 3 – 91417Today• 2nd Research Example• Q.docxcockekeshia
Week 3 – 9/14/17
Today
• 2nd Research Example
• Quiz #1 Debrief
• Research Ethics
• Practice
• Workplace Productivity – You are interested in the factors
that influence and have relationships with employee’s
productivity in the workplace.
Questions:
1. What are one (1) directional & one (1) non-directional
research hypothesis?
2. What are the IV and DV in this scenario?
a. What might be an operational definition for these variables?
3. Is this research being done experimentally or non-
experimentally?
a. How can you tell?
4. Is this a qualitative or quantitative research study?
a. How can you tell?
Research Scenario Example #2
Conceptual
Math Achievement
Trust (Workplace)
Depression
Anger
Operational
Score on PSSA Math
Score on the ITA (International
Trust Assessment)
Hamilton Depression Rating
Scale
Yelling, hitting, etc.
Variable Definition (IV or DV)
Quiz 1
• How’d it go?
• Questions / Concerns?
• 1 Hour?
• Let’s take a look.
Types of Variables
Continuous and discrete variables
• Continuous: Measured along a continuum at any
place beyond the decimal point, meaning that it
can be measured in whole units or fractional units
• E.g. Olympic sprinters are timed to the nearest
hundredths place (in seconds), but if the Olympic
judges wanted to clock them to the nearest millionths
place, they could
Types of Variables
Continuous and discrete variables
• Discrete: Measured in whole units or categories
that are not distributed along a continuum
• E.g. Number of brothers and sisters you have,
socioeconomic class (working class, middle class,
upper class)
Recap So Far
• Empirical Research
• Experimental vs. Non-Experimental
• Quantitative vs. Qualitative
• Hypothesis vs. Research Question
• Directional
• Independent Variable vs. Dependent Variable
• Variable Definitions
• Conceptual vs. Operational
• Investigator is liable; must ensure…
– Protection from harm
– Experiments? Survey?
– Informed consent
– Privacy
– Knowledge of results
– Potential benefit
– Debriefing
• For sponsored work, University is also liable
– Institutional Review Board (IRB)
– Offering Inducements?
– Anonymity vs. Confidentiality
Ethics in Human Subject Research
IRB Approval
Ethical Use of Deception
Deception in research
• Can be active (deliberately untruthful) or passive
(omission of key information about a study)
• For IRB to approve the use of deception:
• The deception is necessary, and the use of
nondeceptive alternatives is not feasible
• There is no reasonable expectation for causing
physical pain or severe emotional distress to
participants
• Participants are informed of the deception as early as
possible, but no later than at the end of data
collection
• Use of Placebo
• Difficult to study topics
How did we get here?
• Who?
• Prisoners, Minorities, Children, Poor, Mental Disease, Cognitive
Disabilities
• What?
• Nuremberg Code
• Study of Disease Progressio.
Introduction to research ethics for VR, from undergraduate lectures at the School of Simulation & Visualisation at The Glasgow School of Art.
Includes a basic introduction to research ethics.
With a large team from my market research class, we were to identify where the non-profit Lifeline Puppy Rescue needed improvement, and how customers viewed their operation. We created, conducted, and analyzed our findings. This file shows what we concluded and the recommendations we suggested.
Heuristics, bias and critical thinking in testing distributionMatt Mansell
This was a day long workshop I gave at the NZTester 2015 conference. Looking at psychology and cognitive science research and how to apply that to testing.
This presentation was provided by John Wilbanks of Sage Bionetworks, during the NISO Symposium, Privacy Implications of Research Data held on September 11, 2016 in conjunction with International Data Week in Denver, Colorado
Similar to GDC17 A Practical Guide to Doing Ethical Player Testing (20)
Players and Avatars: Against IdentificationMia Consalvo
Much study of MMOGs as well as other videogames presumes an affinity between players and their avatars. Gee has developed the concept of 'projective identity' and Bailensen, Yee and others have done extensive work exploring the Proteus effect, which suggests humans are deeply influenced by the avatars they choose, and likewise how such avatars become extensions of themselves in games and virtual spaces. Some of my own past work has explored how women strongly identify with female avatars, 'gender-swapping' at rates much lower than similar male players. Yet what of games that don't employ avatars, or rely on multiple or non-human avatars for players to employ? What of players who simply do not characterize game avatars as extensions of themselves? How can we speak of identification such instances? Is it still a useful concept to investigate?
This talk reviews some of my past research about players, identity and avatars, to offer a starting point for argument. But the heart of the talk explores instances of games where avatar presentation and use depart from our traditional conceptualizations --either by their absence or their opposition to humanoid facsimiles. By doing so this talk challenges game studies' easy reliance on avatars as proxies for identity in games, and asks what happens when players fail to use or access such embodiments in their gameplay. It suggests alternative ways to understand player agency and identification in games, and moves beyond avatars as the principle means for doing so.
Humanities Unlocked: The Value of Liberal Arts for your Game Design ProgramMia Consalvo
Mia Consalvo's 2011 GDC talk for the Game Education Summit.
How can disciplines like religion, history, philosophy and literature make a game development program stronger? And how can games--both the study of them as well as their creation-- make such fields more relevant to contemporary society? This talk takes as a given that you've built a successful program at your college or university. You have plenty of majors with excellent GPAs, good faculty and professional advisors, and a relevant curriculum. You've distinguished and built room for programming, art, production and design into your degree (or some subset), and students are getting jobs in the industry upon graduation. What comes next? How can a game development program go from good to great? The answer lies outside of game development classes, in more distant parts of the university such as the arts, humanities, and social sciences. Should these classes be dismissed? How are they relevant to a decent game development major or education? In this talk, Mia will argue for the essential part that they can play in a quality game development program.
Using Your Friends: Top Interaction Mechanics in Social GamesMia Consalvo
Mia Consalvo's 2011 GDC talk "Using Your Friends: Identifying the top interaction mechanics in social games & social media." Please email me for more information on the games analyzed.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
GDC17 A Practical Guide to Doing Ethical Player Testing
1. A Practical Guide to
Doing Ethical Player
Testing
Mia Consalvo
Concordia University
2. Who am I?
● Studying game players since 2000
● Use a variety of methods to better
understand play
● Wrote Cheating, Players and their Pets,
Atari to Zelda
● Run the mLab at Concordia University
3. Outline for today’s talk
● A brief history of why ethical research became a
thing at universities
● Important concepts to keep in mind for testing
● Questions to ask yourself when prepping for
testing
● What about VR?
● Q&A
4. What this talk is NOT about
● Improving your game design through
player testing
5. What this talk IS about
● Ensuring play testers are getting the best
possible experience (and want to come
back again)
● Improving the quality of testing feedback
6. A caveat
● “Following a code of ethics is not the
same as being ethical. A domain-specific
ethics code … can never function as a
substitute for ethical reasoning itself.”
8. The Milgram experiment (1961)
● Studied obedience to authority figures via
willingness to give increasingly severe
‘shocks’ to others
● 65% of participants administered final
massive 450-volt shock
● Study criticized for use of deception,
other flaws
9. Stanford Prison Experiment (1971)
● A study of the psychological effects of
becoming a prisoner or prison guard
● Conducted by Philip Zimbardo with a
group of college students
● Prisoners ‘arrested’ at their homes, driven
to prison, processed, given jumpsuit and
assigned an ID number
10. Stanford Prison Experiment
● Experiment abandoned after six days –
some prisoners had breakdowns
● Subjects could not leave voluntarily
● Zimbardo was not a neutral observer
● No debriefing was done to assess
potential harms done
13. What is consent?
● Free, informed and ongoing consent
● Free = voluntary
14. Voluntary Consent
● No Undue Influence or Coercion
● Is there a power relationship between the
Investigator and potential participants?
● Is the investigator relying on the trust or
dependency of particular people (ex: my
students)?
● Threat of harm or punishment for failure to
participate?
15. Informed Consent
● Participants know what is expected of
them during the test
● How long the test will last
● How will they be observed/recorded
● It’s okay to change their mind
16. Informed Consent
● Told of foreseeable risks and potential
benefits
● Under no obligation to participate, free to
withdraw at any time
● Whether participants will be identified
directly or indirectly
17. Informed Consent
● Contact information if they have
questions after the fact
● Information about how data will be used,
who it (may be/will be) disclosed to
● Information about payments or incentives
● Time to consider all of this, ask questions
18. Ongoing Consent
● Consent is not a ‘once and you’re good’
decision
● Consent can be negotiated if there are
different aspects to the play test
19. Withdrawing Consent
● Testers should be told they can stop the
test at any time
● The environment should support this
possibility
● There should be multiple ways to
withdraw (verbal, nonverbal/written)
20. Using incentives to participate
● Incentives can attract more potential
participants
● If incentives are very large or valuable this
may encourage participants to disregard risks
● Economic circumstances of participant pool,
age and capacity, customs and practices
21. Use of deception
● Deception can be a part of ethical
research
● What is the risk involved in the
deception?
● Does the benefit of deception outweigh
potential risks?
23. Defining Harm and Risk
● Minimal risk: probability and magnitude
of possible harms implied by participation
in the study is no greater than those
encountered by participants in those
aspects of their everyday life that relate
to the research
26. Children
● Consent usually obtained through parents/
guardians
● Consider the age range you are including and
why
● Children who are 7, 10 and 13 are very different
in terms of what they understand as risks, as
feedback, as consent
27. Children
● During testing, nonverbal communication
can be more important than what is being
said (for both testers and researchers)
28. Children
● For a detailed examination of kids and
play testing, check out Gareth Griffiths’
2014 GDC talk “Child’s Play: Playtesting
with Children in the World of Skylanders”
● http://www.gdcvault.com/play/1020348/
Child-s-Play-Playtesting-with
30. Privacy and Confidentiality
● How are you collecting data?
● How are you storing data?
● How are participants identified?
● What are you doing with the data?
● What happens to the data when the study
is over?
31. Privacy and Confidentiality
● Levels of disclosure of participant identity
● Anonymous
● Pseudonyms/Confidential
● Identifiable
● What personal information are you collecting and how
might it be linked to participants’ tests?
33. “VR poses risks that are novel, that go
beyond the risks of traditional
psychological experiments in isolated
environments & go beyond risks of
existing media technology for the
general public” –Madary & Metzinger,
2016
34. “the virtual pit”
● Subject with HMD stands on ‘ledge’ and asked to
lean over and drop a beanbag into a deep pit
● Subject stands on wooden platform 1.5” from
the ground
● Showed increased signs of stress through
increases in heart rate, skin conductance
35. Remember the Milgram Experiment?
● Re-done in VR environment
● Asked participants to administer shocks
to a virtual human performing memory
tests [they knew she was virtual]
--Slater M, Antley A, Davison A, Swapp D, Guger C, et al. (2006) A Virtual
Reprise of the Stanley Milgram Obedience Experiments. PLoS ONE 1(1):
e39. doi:10.1371/journal.pone.0000039
36. Virtual Reprise
● 2 participants emphasized correct answers while
reading words ‘in an attempt to help her’
● 8 repeated the question after receiving no
response
● Voices of some participants showed increasing
frustration with wrong answers
37. Virtual Reprise
● Often behaved in a way that only made sense if
they were responding to the virtual character as
if she were real
● Humans respond realistically at subjective,
physiological, and behavioral levels in
interactions with virtual characters
notwithstanding their cognitive certainty that
they are not real
38. Virtual Reprise
● Might be continued evidence that subjects will
be obedient to an authority figure
● May be a matter of participants being willing to
put up with their own discomfort for the sake of
honoring their agreement to be a participant in
the experiment
39. Virtual Reprise
● Might be continued evidence that subjects will
be obedient to an authority figure
● May be a matter of participants being willing to
put up with their own discomfort for the sake of
honoring their agreement to be a participant in
the experiment
40. Proteus Effect
● Subjects conform to the behavior that
they believe others would expect them to
have based on the appearance of their
avatar
● Behavior in VE can have lasting
psychological impact after subjects return
to the physical world
41. Guidelines for VR research
● Do no harm
● No real history we can use as a source for
insight
● Tautology?
42. Guidelines for VR Research
● Have an explicit statement (or explain) to
the effect that “immersive VR can have
lasting behavioral influences on subjects,
and some of these risks may presently be
unknown”
43. Guidelines for VR Research
● Tell people that they may have powerful
emotional responses to game content
whether or not they ‘believe’ it’s real
● Torture in a virtual environment “is still
torture”
44. Guidelines for VR Research
● Many new, additional kinds of data being
collected
● Eye-movements, emotions, real-time reactions, bodily
movements (mo-cap)
● One’s kinematics may be uniquely connected to one’s
identity
● Consider what data you really need to keep and
what might be erased to preserve privacy
45. Broader considerations beyond testing
● We don’t know the psychological impact
of long term immersion in VR (addiction,
manipulation of agency, unnoticed
psychological change, mental illness)
● The potential for abuse of avatars that
look like their users
49. What am I testing for?
● Do players understand what to do in the
first level?
● Do my instructions make sense or are
players floundering around?
● How long do players take to get to the
end of the level?
50. Recruitment
● Who is my player base? How can I recruit
players from that group without
exploiting trust/power relationship?
● Should I provide an incentive or reward
for play testing?
● Where am I asking them to play test?
51. Informed Consent
● How will I handle obtaining consent?
● What do I say about withdrawing
consent?
● How can I ensure testers feel comfortable
enough to leave if they really want to quit
testing?
52. Deception
● Is there anything in the game I want to be a
surprise to testers?
● Is there a way to let testers know there’s some
material they might be really bothered by
without spoiling the content for everyone?
● Is this ‘surprise’ really worth it?
53. Harm and Risk – VR Edition
● Having a ‘kill switch’
● Testers’ avatars have protective bubble
● Protected populations issues
● Find information or agencies that testers might
want to contact if they are bothered by content
in your game
54. Privacy and Confidentiality
● What should I record – gameplay, tester audio,
tester video, physiological responses?
● If I share data with anyone, am I keeping tester
identities anonymous or confidential?
● How am I storing data and keeping it secure?
● What am I doing with that data after testing is
done?
55. Questions for multiplayer testing
● Do players understand how they can interact
with other players?
● Do instructions for that make sense?
● Are players using anti-harassment tools we
created?
● What sorts of toxicity (if any) am I seeing during
play?
56. Harm and risk – an aside
● Be wary of defining what ‘harassment’
looks like/sounds like
● Ask what they might have seen done/said
about other players, not just this player
● ‘Abuse’ versus ‘Drama’
57. Putting it all together –
Consent Form Language/
Guidelines for testers to
understand and agree to
58. Example Consent Form Text
● “I understand that I have been asked to
participate in play testing of Eksa: Isle of
the Wisekind by Mia Consalvo of GAMBIT
Studios, contact information HERE.”
59. Example Consent Form Text
● “I have been informed that the purpose
of the play testing is to determine [how
clear directions are in the game, and how
challenging puzzles are in the first few
levels.]”
60. Example Consent Form Text
● “I understand that I am being asked to
play a game that isn’t finished, for about
30 minutes.”
● “I understand that the computer is video
recording my gameplay and I am being
observed by someone while I play, who is
taking written notes.”
61. Example Consent Form Text
● “I understand that I am free to withdraw
my consent and discontinue my
participation at any time without negative
consequences.”
62. Example Consent Form Text
● “There might be certain risks in
participating in this play test. These risks
include feeling frustration or anger while
playing, or hearing other players saying
negative things during gameplay which
might upset me.”
63. Example Consent Form Text
● “I understand that my participation in this
study is…
● CONFIDENTIAL (the researcher will know, but
will not disclose my identity)
● NON-CONFIDENTIAL (my identity will be
revealed in study results)”
64. Example Consent Form Text
● “I have carefully studied the above and
understand this agreement. I freely
consent and voluntarily agree to
participate in this study.”
65. Additional Resources
● For US-based developers
● https://phrp.nihtraining.com/index.php
● The Belmont Report
● http://www.hhs.gov/ohrp/regulations-and-policy/belmont-
report/index.html
66. Additional Resources
● For Canadian-based developers
● http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/
tcps2-eptc2/Default/
● For European-based developers
● http://www.eurecnet.org/index.html
67. Additional Resources
● Madary M and Metzinger TK (2016) Real
Virtuality: A Code of Ethical Conduct.
Recommendations for Good Scientific
Practice and the Consumers of VR-
Technology. Frontiers in Robotics and AI.
3:3. doi: 10.3389/frobt.2016.00003