Research Methods and data collection technique summary for Computer Science and other disciplines. The content is meant for graduate and post-graduate researchers.
This workshop was made in my MSc under the supervisor of my supervisors and with the input of the lab members around me.
2. § Creating a dialogue between humans and computers
§ Easier exchange of information
§ Optimizing interactions and tasks
§ Illustrating the importance and current limitations of the study of Human
Computer Interaction (HCI)
2
3. § Design of everyday things
§ Task-oriented Productivity
§ Expert Use Software
§ Design of Interaction Techniques
§ Entertainment Software
§ Games User Research (GUR)
3
7. § Selective attention in video game players (Bavelier, 2012)
§ Learning and video games (Bavelier, 2012)
§ Inter-hemispheric Alpha Asymmetry and User experience (Salminen, 2009)
7
8. § Characterizing and measuring user experience (Ijsselsteijn, 2007)
§ SCI model of Immersion (Ermi, 2005)
§ Sensory
§ challenged based
§ Imaginative
8
10. § Real-time data
§ Differential analyses provides us with a interpretation of the same data
§ Combine with other measures for more complete picture
10
11. § Physiological measures allow us to understand the user by investigating the
fundamental physiological reactions
§ Methodologies include:
§ Electroencephalography (EEG)
§ Cardiovascular Measures
§ Skin Conductance
§ Eye Tracking
§ Respiratory Measures
§ Muscle Activation
11
12. § EEG is collected using sensors that sit on scalp
of the participant. As neurons fire the
electrical impulses generated are recorded.
§ Data can yield information about:
§ Cognition (e.g. Cognitive Load, Errors)
§ Information Processing (e.g. Visual, Auditory)
§ Emotions (e.g. Excitement, boredom, frustration)
12
http://www.esa.int/gsp/ACT/images/bio/sleep.jpg
17. § Measures of the heart activity
§ Average Heart Rate
§ Heart Rate Variability (HRV)
§ Indicator of Stress, Excitement,
§ Physical exertion
17
http://m.eet.com/media/1115118/c0726-figure1.gif
18. § Also commonly referred to as a measure
of Galvanic Skin Response (GSR) or
Electrodermal Activity (EDA).
§ Measures the conductance of the skin which varies
in proportion to the amount sweat due to the
body’s natural sympathetic response.
§ Measures psychological Arousal (Stress, excitement).
18
http://www.hyperbody.nl/fileadmin/images/imi_51.jpg
19. § Measures what eyes look at
§ Saccades (fast movement)
§ Gaze path
§ Fixations (dwell times)
§ Attention focus
§ Pupil dilation/blink rate
§ Attention precedes gaze (200ms)
§ Used mainly to improve interface
§ Lack of 3D analysis tools
19
34. § Creating technology that adapt to the users current state
§ Offer Assistance
§ Change the level of challenge
§ Incorporate an emotional display
§ Dynamic Difficulty Adjustment in games
§ Flight and Driving Automation
§ Health and Exergaming
34
35. § Using EEG to feedback into the system
§ Monitors operator engagement
§ To prefect the approach they also test out engagement during videogame
play
§ Pope, A.T., Bogart, E.H., Bartolome, D.S., 1995. Biocybernetic system
evaluates indices of operator engagement in automated task. Biological
Psychology 40, 187–195.
35
37. § Relationship between the measures and response dictate how we can
design a system
§ Four possible relationships:
§ One-to-one
§ Many-to-one
§ One-to-Many
§ Many-to-Many
37
38. § Designing valid systems and experiments
§ Selecting stimuli carefully
§ Previous literature
§ Validation of responses
§ Mixed Measures approaches
§ Qualitative Measures
§ Multiple indicators
§ Ecological Validity
§ “In-the-Wild” testing
38
40. § Privacy
§ Constant Monitoring Situation
§ Who has access to the information?
§ Anatomy of the User
§ Control of the system
§ When should the system intervene?
§ Consequences of the System’s Actions
40
42. § Tradition of expert reviews and heuristics in the ‘traditional’ usability &
software engineering
§ Games demand specifically designed heuristics
§ Easy implementation in the game design and development process
42
44. PROS
§ Cost-efficient
§ Time-efficient
§ Can be implemented at any stage
of a project
§ identify majority of existing bugs
§ Several sets of heuristics
CONS
§ Diversification of
§ Game genres
§ Input devices
§ Goals Challenges Heuristics
§ Lack of game usability &
playability experts in the game
industry
44
45. § “Typical” playtest
§ Watch people play the game or use the
system
§ Observe their behavior
§ Simulate at-home experience
§ Goals:
§ Understand workflow of the user
§ Errors & Misunderstandings
45
46. PROS
§ + Get a feel for player interaction
with game
§ + Importance of what people do—
not what they say
CONS
§ – Presence of observers can bias
results
§ – Salient event can slant
interpretation
§ – Behavior requires interpretation
46
52. PROS
§ + Less biased responses
§ + Response validation
§ + Forced choice helpful for
revealing preference
§ + Time-based comparisons
CONS
§ – Eliminate nuance
§ – Difficulty in converting ratings to
meaningful decisions
§ – Limited solution space
52
53. § Data about behavior of players
§ In game environments
§ Anything a game engine can log
§ Examples
§ Player movement
§ Firing weapons
§ Interacting with NPCs
§ Interface interaction
53
57. § This slide set included points, slides, and references from both Lennart E.Nacke and Pejman Mirza-Babaei
§ Csikszentlmihalyi, M. (1990). The Psychology of Optimal Experience. New York: Harper and Row.
§ Fairclough, S. H. (2009). Fundamentals of physiological computing. Interacting with Computers, 21(1-2), 133–145.
http://doi.org/10.1016/j.intcom.2008.10.011
§ Fairclough, S. H., Karran, A. J., & Gilleade, K. (2015, April). Classification accuracy from the perspective of the user: Real-time
interaction with physiological computing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing
Systems (pp. 3029-3038). ACM.
§ Frey, J., Daniel, M., Castet, J., Hachet, M., & Lotte, F. (2016). Framework for Electroencephalography-based Evaluation of User
Experience. arXiv preprint arXiv:1601.02768.
§ Pope, A.T., Bogart, E.H., Bartolome, D.S., 1995. Biocybernetic system evaluates indices of operator engagement in automated
task. Biological Psychology 40, 187–195.
§ Pulsipher, L. (n.d.). Why we play. Retrieved March 10, 2016, from http://gamecareerguide.com/features/625/why_we_.php
§ Yuksel, B. F., Oleson, K. B., Harrison, L., Peck, E. M., Afergan, D., Chang, R., & Jacob, R. J. (2016). Learn Piano with BACh: An
Adaptive Learning Interface that Adjusts Task Difficulty based on Brain State. In Proceedings of the SIGCHI conference on Human
Factors in Computing Systems (2016),InPress. DOI: http://dx. doi. org/10.1145/2858036 (Vol. 2858388).
§ Yannakakis, G. N., & Togelius, J. (2011). Experience-driven procedural content generation. Affective Computing, IEEE Transactions
on, 2(3), 147-161.
57
58. § Frey, J., Daniel, M., Castet, J., Hachet, M., & Lotte, F. (2016). Framework for Electroencephalography-based Evaluation of User
Experience. arXiv preprint arXiv:1601.02768.
§ Yuksel, B. F., Oleson, K. B., Harrison, L., Peck, E. M., Afergan, D., Chang, R., & Jacob, R. J. (2016). Learn Piano with BACh: An
Adaptive Learning Interface that Adjusts Task Difficulty based on Brain State. In Proceedings of the SIGCHI conference on Human
Factors in Computing Systems (2016),InPress. DOI: http://dx. doi. org/10.1145/2858036 (Vol. 2858388).
§ Yannakakis, G. N., & Togelius, J. (2011). Experience-driven procedural content generation. Affective Computing, IEEE
Transactions on, 2(3), 147-161.
§ Stellmach, S. (2009). Visual Analysis of Eye Gaze Data in Virtual Environments. Master’s Thesis. University of Magdeburg.
§ Drachen, A., Canossa, A., & Yannakakis, G. N. (2009). Player Modeling using Self-Organization in Tomb Raider: Underworld.
Proc. IEEE CIG2009, Milano, Italy.
58