Immersive virtual environment technology as a basic research tool ...Document Transcript
Behavior Research Methods, ln.strummt.s, & Computers1999, 31 (4). 557-564 Immersive virtual environment technology as a basic research tool in psychology JACK M. LOOMIS and JAMES J. BLASCOVICH University of California, Santa Barbara, California and ANDREW C. BEALL Massachusetts Institute of Technology, Cambridge, Massachusetts Immersive virtual environment (WE) technology has great promise as a tool for basic experimental research in psychology. N E technology gives participants the experience of being surrounded by the computer-synthesized environment. We begin with a discussion of the various devices needed to im- plement immersive virtual environments, including object manipulation and social interaction. We re- view the benefits and drawbacks associated with virtual environment technology, in comparison with more conventional ways of doing basic experimental research. We then consider a variety of examples of research using IVE technology in the areas of perception, spatial cognition, and social interaction. Human history records a progression of artifacts for (Virtual reality is widely used as an alternative term, butrepresenting and recreating aspects of external reality, we prefer VE.) An immersive virtual environment (WE)ranging from language, drawings, and sculpture in earlier is one in which the user is perceptually surrounded by thetimes to the more modern artifacts of photographs, movies, VE. Ivan Sutherland (1965), one of the originators of 3-Dtelevision, and audio recordings. Relatively recently, the computer graphics, was the first person to conceive anddigital computer and its associated technologies, includ- build an immersive VD system. For the history of IVEs,ing three-dimensional (3-D) graphics, have given rise to see Ellis (1 995), Kalawsky (1 993), and Rheingold (199 1).increasingly realistic artifacts that blur the distinction be- There are two usual implementations of an IVE. Thetween reality and its representation (Ellis, 1995). first of these involves placing multiple projection screens The ultimate representational system would allow the and loudspeakers around the user. A popular design is theobserver to interact "naturally" with objects and other in- CAVE (Cruz-Neira, Sandin, & DeFantini, 1993), whichdividuals within a simulated environment or "world," an involves back-projecting the computer-generated visualexperience indistinguishable from "normal reality." Al- imagery onto the translucent walls, floor, and ceiling ofthough such a representational system might conceivably a moderately sized cubical room, in which the user is freeuse direct brain stimulation in the future, it will more to move; shutter glasses provide stereoscopic stimula-likely use digitally controlled displays that stimulate the tion, so that one sees the VE not as projections on thehuman sensory organs, the natural conduits to the brain. room surfaces, but as solid 3-D structures within and/or Displays of this type, referred to as virtual displays outside of the cube. The second and more common imple-(VDs), although far from ideal, exist today. Following the mentation of an IVE involves the use of a head-mountedterminology of others (e.g., Durlach & Mavor, 1995; Stan- display (HMD), used in conjunction with a computer andney & Salvendy, 1998), we refer to the corresponding en- a head tracker (Barfield & Furness, 1995; Biocca & De-vironment represented and stored in the computer and laney, 1995; Burdea & Coiffett, 1994; Durlach & Mavor,experienced by the user as a virtual environment (VE). 1995; Kalawsky. 1993). The head tracker measures the Virtual environnient technology (VET) refers inclusively changing position and orientation of the users head withinboth to VDs and to the VEs so created, including VEs pro- the physical environment, information that is communi-duced by using conventional desktop computer displays. cated to the rendering computer, which has stored within it a 3-D representation of the simulated environment (Meyer, Applewhite, & Biocca, 1992). At any given mo- With the support of Grant NOOO14-95-1-0573 from the Office ofNaval Research (to J.M.L.) and National Science Foundation Grants ment, the computer generates and outputs the visual andSBR 9872084 (to J.J.B.) and SBR 9873432 (to J.M.L. and J.J.B.). and auditory imagery to the users HMD from a perspectivewith additional support from the University of California, Santa Barbara, that is based on the position and orientation of the usersthe authors have developed several immersive virtual displays, use of head. The HMD consists of earphones and video dis-which has stimulated many ofthe ideas expressed here. The authors thank plays attached to a support worn on the head; the videoFlorence Gaunet and Patrick PCruch for comments on an earlier versionof the article. Correspondence concerning this article should be ad- display component is based on cathode ray tube (CRT)dressed to J. M. Loomis, Department of Psychology. University of Cal- displays, liquid crystal displays, or laser-based retinalifornia, Santa Barbara, CA 93 106 (e-mail: firstname.lastname@example.org). scanners (Barfield, Hendrix, Bjorneseth, Kaczmarek, & 557 Copyright 1999 Psychonomic Society, Inc.
558 LOOMIS, BLASCOVICH, AND BEALLLotens, 1995; Holmgren & Robinett, 1993). In many im- In the typical IVE implementation using an HMD, theplementations using HMDs, binocular (stereoscopic) vi- user is visually isolated from the real environment. Aug-sual imagery provides convergence and retinal disparity mented reality (or mixed reality) avoids this isolation bycues that contribute to the perception of egocentric dis- permitting the user to see both the real environment andtance and exocentric depth. Virtual or "spatialized sound the VE (see, e.g., Ohta & Tamura, 1999). When an HMDis created by using special-purpose hardware and soft- is used, augmented reality is most effectively accom-ware that generates binaural signals typically presented plished with see-through video techniques (Fuchs & Ack-by earphones; the result is that the user hears sounds ap- erman, 1999; Tamura, Yamamoto, & Katayama, 1999).pearing to emanate from surrounding auditory space (Be- Here, the user sees the real environment by way of videogault, 1994; Carlile, 1996; Gilkey & Anderson, 1997; cameras attached to the helmet; image processing tech-Loomis, Klatzky, & Golledge, 1999; Wenzel, 1992).With niques are used to produce natural occlusion of the vir-a properly designed system using either HMDs or pro- tual entities by real entities (from the cameras), or vicejection screens and loudspeakers, the user receives binoc- versa. Although effecting natural occlusion remains aular and binaural stimulation approximating the stimu- major challenge, achieving positional stability of the vir- lation he or she would receive if moving about within a tual entities with respect to the real environment is evenreal physical environment equivalent to that stored within more difficult, because spatial inaccuracy, noise, andthe computer. Under these circumstances, individuals temporal lag of the head tracking all result in imperfectexperience a compelling sense of immersion orpresence registration and relative movement of the virtual im-within the VE (Barfield, Zeltzer, Sheridan, & Slater, agery with respect to the objects in the real environment. 1995; Heeter, 1992; Held & Durlach, 1992; Loomis, 1992; Sheridan, 1992; Slater, Usoh, & Steed, 1994; ADVANTAGES OF IMMERSIVE Stanney & Salvendy, 1998; Steuer, 1995). VIRTUAL ENVIRONMENT TECHNOLOGY The discussion so far has focused on visual and audi- AS A RESEARCH TOOL tory stimulation. For haptic stimulation, force feedback devices and inflatable gloves provide modest degrees of In psychology, experimental researchers have tradi- kinesthetic and cutaneous stimulation of the hands and tionally confronted the choice between experimental con- fingers (Barfield & Furness, 1995; Biocca & Delaney, trol and ecological validity (see Figure I), with experi- 1995; Burdea & Coiffett, 1994; Durlach & Mavor, 1995; mental control being regarded as the sine qua non of the Iwata, 1999; Kalawsky, 1993). Eventually, advanced hap- field. Consequently, most experimentalists create con- tic interfaces might allow a person to use the hands and trolled and contrived situations in sterile artificial envi- fingers to feel and identify objects, sense the layout of sur- ronments, or laboratories, and employ specialized devices faces, and so forth. For naturalistic interaction with virtual objects, the user needs a means of manual control and even of manipula- tion. Sensors attached to the hands and fingers allow tracking of users positions and orientations (Biocca & Delaney, 1995). This information, from which manual gestures can be decoded, can be used in a multitude of ways, to allow the person to manipulate and otherwise interact with virtual objects. For social IVEs, two basic approaches afford social interaction. The first requires the computer to model in- telligent interactants other than the user. Although fairly realistic simulations of dogs, cats, and other pets are fea- sible, realistic modeling of people as interactants re- mains a major challenge in artificial intelligence. The sec- ond approach requires immersion of additional real individuals within the same VE. Although conceptually Automobile Driving Research straightforward, implementation will require surmount- ing many hardware and software challenges. For realis- tic social interaction beyond speech, one needs special- Low High ized sensors for tracking eye gaze, facial expressions, hand gestures, and body language for both (or all) inter- Ecological Validity actants (Biocca & Delaney, 1995). The graphics com- Figure 1. Experimentation in psychology entails a tradeoff be- puter then renders the nonverbal behavior of each inter- tween experimental control and ecological validity. Virtual dis- actant by way of modifications of a stored graphics model plays, especially immersive ones, afford less of a tradeoff than do of that individual. traditional approaches to psychological experimentation.
IVE TECHNOLOGY AS A BASIC RESEARCH TOOL 559allowing precise control over the delivery of stimuli. In means. Changing the shape of an object contingent on thethe early days of visual perception research, for example, subjects relative location or even the subjects behaviorthe tachistoscope represented one extreme; this was a de- provides another example. Thus, in a driving study, onevice for presenting a small number of two-dimensional can selectively alter aspects of the road ahead to deter-pictures for very brief durations. It permitted good stim- mine how much of theforward field of view influencesulus control, but at a huge cost in ecological validity. the drivers steering. In the social realm, with IVE tech-Moving about in natural settings, such as automobile driv- nology, one can manipulate even organismic variables,ing, represented the other extreme-high ecological va- such as a participants apparent gender, race, weight, andlidity, but very limited experimental control. so forth, enabling heretofore practically impossible ex- Advances in audio and video recording equipment aminations of social identity, roles, and stereotypes. Also,have allowed researchers to expand the operating char- the researcher can decouple nonverbal behavior that anacteristic depicted in Figure 1 outward, thereby decreas- interactant displays from what another interactant per-ing the degree of tradeoff between control and validity. ceives; for example, the eye gaze or facial expressions ofThree-dimensional computer graphics and digital sound interactant A can be systematically altered in the graphicsynthesis have continued this trend, with high-end flight rendering, so that interactant B sees something quite dif-simulation being the most successful application of these ferent, thus affording novel basic research on nonverbaltechnologies in recent years (Moroney & Moroney, 1999). communication and contributing to the development ofHowever, immersive VDs, by virtue of their being nearly realistic applications involving social interaction in IVEs.optimal interfaces for the human senses, have the poten- Another advantage provided by IVE technology stemstial for maximally expanding the operating characteris- from the ease with which investigators can implement andtic, thereby providing high ecological validity without conduct experiments. In the area of visual perception, forcompromising experimental control, and vice versa. Such example, IVE technology facilitates the creation of natural-a change in the operating characteristic afforded by IVE appearing VEs for studying space perception, color per-technology will very likely promote rapid advancement ception, and so forth. Although investigators can createand application of psychological science by increasing specially designed natural environments in the labora-the generalizability of experimental findings and the the- tory (e.g., Brainard, Brunt, & Speigle, 1997), such cre-ories based on them. ations entail the investment of much time and expense. In addition, VET will increase the power ofexperimen- In contrast, once one has acquired a VD system, creationtal research, because of the increases in experimental of new IVEs requires only programming~effort. addi- Inrealism-the impact of manipulations on participants. In tion, cumbersome rearrangements of a physical labora-much of cognitive and social psychology, for example, tory that can take hours, if not days, can happen practi-manipulations often involve the induction of cognitive or cally instantaneously with IVE technologyaffective states by written passages, verbal instructions, IVE technology also provides a largely new and prac-videotapes, and sounds. The effectiveness of these induc- tical data source. For example, in the area of perceptiontions, in terms of their variability across participants, is and action, one often needs to make measurements of thelimited by the attentional, motivational, and imaginative participants walking and hand trajectories, as in experi-capacities of the research participants. By more directly ments on visually controlled locomotion or visuallyeliciting the participants cognitive and affective processes, guided reaching, respectively. Because IVE implemen-IVE technology substantially augments these capacities tation requires these measurements, they are automati-by increasing the effectiveness of many experimental in- cally available for data analysis in WE-based experiments.ductions, thereby increasing experimental realism and For example, in the area of nonverbal communication,reducing the variability in the results of these manipula- IVE technololgy will actually facilitiate research on mu-tions and inductions. For example, one can more easily tual eye gaze between multiple interactants, because theelicit fear associated with acrophobia in VEs, as com- eye tracking measurements required to effect natural eyepared with imagery and other techniques, even with the gaze in IVEs will also provide the necessary data for analy-participant being fully aware of the unrealistic nature of sis of eye gaze patterns; heretofore, analysis of mutualsuch fear within a VE (Rothbaum et al., 1995). eye gaze, especially in group settings, has been greatly Importantly, IVE technology gives researchers the limited by the unavailability of such measurements.ability to perform manipulations impossible, or nearlyso, with any other means. For example, in the physical DISADVANTAGES OF IMMERSIVEworld, many stimulus variables (e.g., the egocentric dis- VIRTUAL ENVIRONMENT TECHNOLOGYtance of an object and its angular size) are highly corre- AS A RESEARCH TOOL lated. IVE technology affords the decoupling of suchvariables, In some of our research employing a VE, for Whenever one attempts to abstract the real world withinexample, we have held the angular size of an object con- the laboratory, some loss in ecological validity always stant as the subject moves relative to it, thus eliminating occurs. As was argued above, IVE technology offers the its visual angle as a distance cue (Gaunet & Loomis, 1999), possibility of increasing ecological validity above that a decou~ling most difficult to achieve with ordinarv common in laboratorv research today. At the same time,
560 LOOMIS, BLASCOVICH, AND BEALLhowever, the imperfection and high complexity of IVE EXAMPLES OF RESEARCH USINGhardware and software greatly increase the likelihood of IMMERSIVE VIRTUAL ENVIRONMENTartifacts contaminating the research findings, especially TECHNOLOGYin areas such as perception, where the sensory stimula-tion requires high accuracy. Imperfections in the render- This section presents examples of basic research, someing models (e.g., for illumination), limitations of the vi- already reported in print or at conferences. The purposesual display (e.g., low spatial resolution, small field of is to convey the unique capabilities of IVE technologyview, and fixed accommodative distance), slow graphics for furthering psychological science.update rate, and significant lags between head trackingand visual display are all sources of artifact for this type Visual Perception and Actionof research (Barfield, Hendrix, et al., 1995; Peli, 1995). The area of visual space perception has benefited andFor research concerned with more abstract processes re- will continue to benefit from the use of IVE technology.lating to cognition and social interaction, it is likely that One important reason is that, in comparison with morethe risk of artifacts will derive less from imperfections in conventional 3-D computer graphics displayed on desk-the visual display and more from the computer models top CRTs, an immersive VD, even one employing anused in rendering the environment and other interactants. HMD, can give the user the perceptual experience of be-Imperfections in the technology, whether associated with ing inside a large-scale environment (e.g., a large room,tracking and display hardware or with modeling soft- a building, an outdoor setting). This means that the re-ware, will continue to impede certain types of research searcher can investigate the perception of distance, size,until the technology improves. In view of these concerns, and motion at larger scales not permitted by conven-researchers using IVE technology need to be vigilant tional CRTs and yet have the convenience of computer-about possible limitations to the validity of their research. based research (e.g., Beall, Loomis, Philbeck, & Fikes, Although IVE technology offers increased ecological 1995; Ellis & Menges, 1997,1999; Loomis & Knapp, invalidity through more complex, but controlled, stimula- press; Proffitt, Bhalla, Gossweiler, & Midgett, 1995; Rol-tion, specification ofthe stimulation becomes formidable. land, Gibson, & Arierly, 1995; Surdick, Davis, King, &For laboratories using the same IVE implementation, the Hodges, 1997; Witmer & Kline, 1998; Witmer & Sad-sharing of software, which allows for exact replication, owski, 1998; Yang, Dixon, & Proffitt, 1999). Why HMD-can circumvent the problem. However, given that few based IVE technology permits the experience of largelaboratories will actually use the same implementation, objects and environments is itself an interesting question.the problem may persist, although we note that few, if Possibly, the HMD implementation avoids the perceptualany, physical laboratories are the same across researchers. conflict accompanying traditional screen-based vision re- The difficulty of setting up a high-quality IVE labora- search-that between the intended size of the simulatedtory is another disadvantage of IVE technology, at least 3-D object and its image size as rendered on the displayin the near term. We can anticipate turnkey systems in (for a discussion of a related issue, see Pirenne, 1970).the more distant future. Until then, the complexity of cur- An elegant way of testing this hypothesis is to simulaterent IVE implementations will demand individuals with projection surfaces, such as CRTs or movie screens, withinconsiderable skill in programming and interfacing. Con- the virtual environment (Yang et al., 1999); in this way,sequently, adoption of IVE technology will be slow to one can vary the size of the simulated 3-D object inde-meet the burgeoning interest. pendently of its projected image on the simulated projec- A final and very significant disadvantage of using IVE tion surface.technology is the common occurrence of aftereffects ex- We note, however, a significant limitation of HMD-perienced by many participants (see the excellent over- based IVEs for investigating visual space perception atview by Stanney & Salvendy, 1998). Aftereffects include near distances. Because the accommodative distance ofsymptoms of motion (simulator) sickness, disturbance of the displays is fixed (usually close to optical infinity),balance and of eye-hand coordination, drowsiness, and there is a necessary conflict between accommodation andeven instances of memory "flashbacks." Research indi- other distance cues, such as convergence (Ellis & Menges,cates that the incidence of these aftereffects depends 1997; Peli, 1995). Besides resulting in incorrect distancegreatly on characteristics of the participants and specific information for near objects, the conflict can also interfere features of the IVE implementation (e.g., display field with proper binocular fusion, especially among younger of view, tracker lag; Stanney & Salvendy, 1998). Although participants, for whom accommodation is most effective. IVE technology continues to improve, the complex eti- Acrophobia is another topic that involves visual spaceology of these aftereffects means that these aftereffects perception but goes well beyond it. The fascinating workwill continue to plague IVE research well into the future of Rothbaum et al. (1995) has shown that acrophobes, and that researchers will have to develop ways of miti- although well aware that they are in an IVE, readily ex- gating these effects, including improved selection of par- perience palpable fear when positioned at simulated high ticipants and excusing participants from the experiment locations. Moreover, this research has demonstrated the at the onset of the earliest discernible symptoms. efficacy of IVE phobia desensitization (see also Roth-
IVE TECHNOLOGY AS A BASIC RESEARCH TOOL 561baum, Hodges, & Kooper, 1997). For our present con- ing spatial cognition and review much of the researchcern, however, we point out the value of using IVE tech- that has been done.nology to better understand the process of acrophobia As is now widely recognized, spatial navigation relieselicitation. Besides the obvious manipulations of distance on two distinct processes: piloting and path integration.cues to determine the ones most important in creating Piloting relies on position fixing on the basis of environ-the impression of being up high, other cues concern sen- mental cues, such as landmarks, whereas path integra-sorimotor integration. For example, does pure visual in- tion involves updating ones estimate of current positionformation elicit acrophobia, or must appropriate head on the basis of the integration of self-motion (Gallistel,orientation, signaled by neck proprioception and vestibu- 1990; Golledge, 1999). One strategy for understandinglar cues (e.g., looking down from a high bridge) accom- human navigation ability is to first investigate the func-pany the visual information? To answer this, one can de- tional properties of the two processes in isolation andcouple the participants visual stimulation from his or then to study them together. IVE technology provides theher head orientation, so that exactly the same visual in- means for investigating the two in isolation. For example,formation is presented with the head facing downward to study path integration based on visual input, one can(normal coupling) or facing upward (decoupled). use an IVE to provide good optic flow information for the For the topic of visually controlled action, IVE technol- sensing of self-motion while eliminating any environ-ogy has enormous potential, because the coupling be- mental position cues (e.g., Chance et al., 1998; Klatzkytween the participants motor activity and the resulting et al., 1998). Conversely, one can use an IVE to investi-visual stimulation can be readily modified. In connec- gate piloting apart from any involvement of path inte-tion with the visual control of walking, for example, the gration. Gaunet and Loomis (1 999) studied the ability ofresearcher can modify the coupling between the optic participants to find their way back to a target locationflow field and the participants motor commands to the specified by several visible landmarks; by rotating andlegs and feet (see, e.g., Warren & Kay, 1997), so that the translating the virtual environment with respect to theproprioceptive inputs signaling travel direction are dis- physical laboratory on each trial, any estimate of currentcrepant with respect to the optic flow. Similarly, in eval- location based on path integration by physical walkinguating the influence of moving objects on the ability of a was rendered useless. Related research investigating seif-participant to aim his or her walking toward a target ob- localization using landmarks has been done with desktopject, one can laterally displace objects or a substantial VEs (Jacobs, Laurance, & Thomas, 1997; Sandstrom,portion of the simulated environment (like walking in Kaufman, & Huettel, 1998). An important advantage ofsurf) and measure the influence of the subsequent optic IVEs over real environments is that the experimenter canflow on the measured trajectory of the participant. The be assured that self-localization is not mediated by anyvisual control of ball catching and batting are also nat- incidental positional cues, for only those cues included inural topics for IVE technology, because the experimenter the 3-D spatial database are perceptible to the participant.can easily manipulate various aspects of the balls trajec- Cognitive mapping, the acquisition of environmentaltory, as well as the participants visual feedback about his knowledge, is another topic ideally suited for IVE technol-or her arm/hand, in order to determine the information ogy (see, e.g., Rossano & Moak, 1998; Wilson et al.,critical to performance (see McBeath, Shaffer, & Kaiser, 1997; Witmer et al., 1996). Besides making this research 1995). An exciting new development that will be a boon much easier, IVE technology also allows manipulationsto research on perceptionlaction is the recent integration that would be impossible in the real world. Not only canof eye trackers into HMDs. This will permit the moni- one readily manipulate local environmental cues, onetoring of eye fixation patterns while participants carry can also manipulate large distal features (e.g., moun-out complex spatial behaviors, all within the safety of the tains), as well the overall shape of the environment, inVE laboratory. order to assess which aspects are most important for cog- nitive mapping. For researchers wishing to have partici-Spatial Cognition pants walk or be conveyed by a real vehicle through very Spatial cognition research has already embraced VET large scale IVEs, the coupling of Global Positioning Sys-(Chance, Gaunet, Beall, & Loomis, 1998; Darken & Sib- tem receivers with portable computers is beginning toert, 1996; Gillner & Mallot, 1998; Klatzky, Loomis, make this possible (vision: Feiner, MacIntyre, & Hol-Beall, Chance, & Golledge, 1998; May, Peruch, & Sa- lerer, 1999; audition: Loomis, Golledge, & Klatzky, 1998).voyant, 1995; Peruch & Gaunet, 1998; PCruch, May, & Still another spatial cognition topic that will benefitWartenberg, 1997; Peruch, Vercher, & Gauthier, 1995; from IVE technology is memory for the locations of hid-Regian, Shebilske, & Monk, 1992; Richardson, Mon- den objects (see, e.g., Hermer & Spelke, 1996; Hutten-tello, & Hegarty, 1999; Rossano & Moak, 1998; Wilson, locher, Newcombe, & Sandberg, 1994). IVE technologyForeman, Gillett, & Stanton, 1997; Witmer, Bailey, & will greatly facilitate research dealing with 3-D spatialKnerr, 1996). Much of the appeal of VET is the ease with memory, since an unlimited number of objects can be hid-which one can create complex environments for partici- den, with each object being rendered visible when the par-pants to explore. Pkruch and Gaunet (1998) discuss in ticipant arrives within its vicinity or investigates its hidingdetail the advantages and disadvantages of VET for study- location. Moreover, manipulations, such as exchanging
562 LOOMIS, BLASCOVICH, AND BEALLhidden objects or modifying their appearance, can be ac- Social psychologists and others also study interactioncomplished with just a little programming effort. among two or more bona fide participants but do so less frequently. These types of investigations obviously pre-Social Interaction clude much control over the behaviors of the interactants Although IVE technology has already proven valuable and, thus, experimental manipulations of the interac-for perception and spatial cognition researchers, its tants actions. Although such necessarily observationalmethodological potential extends to other areas of psy- approaches to real social interaction provide importantchology and behavioral science, even to such a complex and valuable information, they provide less inferentialdomain as social behavior. Experimental social psychol- power regarding theory and hypothesis testing. IVE tech-ogists and others study social interaction in many ways but nology represents a valuable tool in these types of stud-usually employ one of two types of techniques for devel- ies as well, because IVEs allow two or more individualsoping interaction scenarios. Most frequently, they inves- to interact within the same virtual world (Benford et al.,tigate social interaction by employing a bona fide partic- 1995; Palmer, 1995; Stone, 1993). At the same time, IVEipant and a simulated (nondigital) other. Social interaction technology allows selective and systematic filtering orresearchers commonly simulate the presence of other inter- alteration of the appearance of the others actions to eachactants via verbal or written information (i.e., vignettes), participant. Thus, one can select certain behavioral fea-via voice or video recordings of supposed others (i.e., ac- tures that are theorized to influence interaction and thentors), and via live human actors or confederates. One might control or manipulate only those features. For example,assume that the more realistic the technique (i.e., live ac- in studies of nonverbal behaviors believed to be crucialtors), the greater the experimental impact of the simulated to the fabric of social interaction, such as mutual gaze orother. However, as the difficulty of precisely controlling interpersonal distance, one can systematically alter onlysimulated others increases with the use of increasingly the gaze or the interpersonal distance aspects of the vir-realistic but nondigital techniques, experimental control tual representations of participants, thereby allowingis lessened (again, see Figure 1). Given this state of af- strong tests of theorized functions of this important non-fairs, not surprisingly, most complex manipulations of verbal aspect of social interaction. An experimenter couldsimulated other characteristics employ the least realistic vary the simulated eye gaze of a male interactant fromand impactful techniques (i.e., vignettes, as opposed to gaze aversion to staring or leering, without his knowledge,live actors). The more realistic techniques even preclude and determine the influence of this manipulation on thesome types of precisely controlled manipulations. For ex- interaction between the male and a female interactant,ample, using actors, one cannot manipulate organismic both of whom inhabit the same virtual environment. Evenvariables, such as the sex or the race of the other, keep- without such filtering and manipulation, IVE technologying every other physical characteristic the same. Finally, contributes to the ease with which studies of uncontrolledof course, the more "others" that one needs to include in social interactions can be done by providing a completean experiment (e.g., small group research), the less prac- digitized record of all interactant actions and movements.tical are realistic techniques, such as those employing ac-tual confederates. SOURCES OF INFORMATION ABOUT IVE technology provides a potential solution to the IMMERSIVE VIRTUALproblems of simulating others, using traditional tech- ENVIRONMENT TECHNOLOGYniques. As we have argued, IVEs substantially increasethe experimental realism of manipulations. They provide Two edited books are good sources about IVE technol-far more impactful manipulations than those attempted ogy as it relates to various research issues in psychology.via vignettes, audio recordings, and videotape. Hence, Besides covering software and hardware technology, bothby using IVE technology to immerse participants in im- the volume by Barfield and Furness (1995) and that bypactful virtual worlds, one can precisely control the ac- Durlach and Mavor (1 995) review in detail the propertiestions and appearances of others. For example, using VET, of vision, audition, somesthesis, and kinesthesia as theyone can run interpersonal attraction studies, employing relate to advanced interface design. The book by Bioccaa wide variety of lifelike simulated others varying on and Levy (1995) is a good introduction to IVE technol-characteristics hypothesized to influence attraction. Or, ogy; in addition, many of its chapters treat an issue rarelyin studies on stereotypes, one could vary virtual others addressed by other IVE researchers-social communi-only on those characteristics important theoretically, cation. Two other excellent sources of information aboutsuch as sex, race, physical size, and so forth, while keep- IVE technology and its applications are the volumes au-ing constant all other physical features. Thus, for exam- thored by Burdea and Coiffett (1994) and by Kalawskyple, if one wanted to systematically study the effects of (1993). Finally, three recent volumes deal with auditorythe weight of a target individual on the social perceptions perception and virtual acoustic displays (Begault, 1994;of that target by others, IVE technology would allow a Carlile, 1996; Gilkey & Anderson, 1997), and a forthcom-realistic simulation in which only the weight dimension ing book is concerned with perception, cognition, humanwas varied. Finally, one can create and control virtual performance, and applications of VET (Hettinger & groups of others to study small group dynamics. Haas, in press).
IVE TECHNOLOGY AS A BASIC RESEARCH TOOL 563 Presence: Teleoperators and Virtual Environments, the BIOCCA, & DELANEY,(1995). Immersive virtual reality technology. F., B.journal most relevant to basic psychological research in In F. Biocca & M. R. Levy (Eds.), Communication in the age of vir-IVEs, is devoted to a wide spectrum of research and devel- tual reality (pp. 57-124). Hillsdale, NJ: Erlbaum. BIOCCA, & LEVY, R. (1995). Communication in the age of virtual F., M.opment of IVE technology. CyberPsychology & Behavior reality. Hillsdale, NJ: Erlbaum.focuses on VEs and multimedia and is more applied, J. BRAINARD,H., BRUNT, A,, & SPEIGLE, M. (1997). Color con- D. W.with articles devoted to psychiatry, the sociological im- stancy in the nearly natural image: I. Asymmetric matches. Journalpacts of VEs, and so forth. Human Factors, International of the Optical Society ofAmerica A , 14,2091-21 10. BURDEA, & COIFFETT, (1994). Virtual reality technology. New G., P.Journal of Human-Computer Interaction, and ACM York: Wiley & Sons. Transactions on Computer-Human Interaction are three CARLILE, (1996). Virtual auditoy space: Generation and applica- S.other journals that regularly have articles devoted to ap- tions. New York: Chapman & Hall.plied research involving VET and psychology. S. F., A. CHANCE, S., GAUNET, BEALL, C., & LOOMIS, M. (1998). Loco- J. Three regularly held conferences in the United States, motion mode affects the updating of objects encountered during travel: The contribution of vestibular and proprioceptive inputs. Presence:with published proceedings that include papers on IVE Teleopemtors & Virtual Environments, 7, 168- 178.technology relating to basic psychological research, are CRUZ-NEIRA, SANDIN, A., & DEFANTINI, V.(1993). Surround C., T. R.the IEEE Virtual Reality Symposium (formerly VRAIS), screen projection-based virtual reality: The design and implementationthe International Conference on Auditory Display (ICAD), of the cave. In Proceedings ofStGGRAPH 1993 (pp. 135-142). DARKEN, P, & SIBERT, L. (1996). Wayfinding strategies and be- R. . J.and the Annual Symposium on Haptic Interfaces for Vir- haviors in large virtual worlds. In Proceedings of the ACM CHI 96tual Environments and Teleoperator Systems. Presence: (pp. 142- 149). New York: Association for Computing Machinery. Teleoperators and Virtual Environments has a listing in DURLACH, I., & MAVOR, S. (1995). Virtual reality: Scientific and N. A.each issue of upcoming conferences relating to VEs. technological challenges. Washington, DC: National Academy Press. Finally, the Internet provides a good source of infor- ELLIS, R. (1995). Origins and elements of virtual environments. In S. W. Barfield & T. A. Furness 111 (Eds.), Virtual environments and ad-mation on IVE, with many sites describing commercially vanced interface design (pp. 14-57). New York: Oxford Universityavailable hardware and software, entertainment uses, en- Press.gineering and medical applications, and applied and basic ELLIS, R., & MENGES, M. ( 1997).Judgments of the distance to nearby S. B.human research. virtual objects: Interaction of viewing conditions and accommodative demand. Presence: Teleoperators & Virtual Environments,4,452-460. ELLIS, R., & MENGES, M. (1999). Operator localization of virtual S. B. SUMMARY objects. In Y. Ohta & H. Tamura (Eds.), Mixed reality: Merging real and virtual worlds (pp. 305-323). Tokyo: Ohmsha. IVE technology is a highly promising tool for the study FEINER, MACINTYRE, & HOLLERER,(1999). Wearing it out: First S., B., T.of basic psychological processes. Its primary advantages steps toward mobile augmented reality systems. In Y.Ohta & H. Ta- mura (Eds.), Mixed reality: Merging real and virtual worlds (pp. 363-are affording more ecological validity without compro- 377). Tokyo: Ohmsha.mising experimental control and allowing the decoupling F u c ~ sH., & ACKERMAN, , J. (1999). Displays for augmented reality: His-of variables that naturally covary. We have provided exam- torical remarks and future prospects. In Y.Ohta & H. Tamura (Eds.),ples of research from the areas of perception, spatial cog- Mixed reality: Merging real and virtual worlds (pp. 3 1-40). Tokyo: Ohmsha.nition, and social interaction, but surely many other areas GALLISTEL,R. (1990). The organization of learning. Cambridge, MA: C.of psychology will benefit as well from this technology. MIT Press. F., GAUNET, & LOOMIS, M. (1999). Selflocalization as a function of J. REFERENCES the informational content of landmarks. Manuscript submitted for publication.BARFIELD. & FURNESS, A., 111 (1995). Virtual environments and W., T. GILKEY, & ANDERSON,R. (1997). Binaural and spatial hearing R.. T. advanced interface design. New York: Oxford University Press. in real and virtual environments. Hillsdale, NJ: Erlbaum.BARFIELD, HENDRIX. BJORNESETH, KACZMAREK,A,, & W., C., 0.. K. GILLNER, & MALLOT, A. (1998). Navigation and acquisition of spa- S., H. LOTENS, (1995). Comparison of human sensory capabilities with W. tial knowledge in a virtual maze. Journal of Cognitive Neuroscience, technical specifications of virtual environment equipment. Presence: 10,445-463. Teleoperators & Virtual Environments, 4,329-356. GOLLEDGE, G. (1999). Wayfinding: Cognitive mapping and spatial R.BARFIELD. ZELTZER, SHERIDAN,& SLATER, (1995). Pres- W., D., T., M. behavior. Baltimore: Johns Hopkins University Press. ence and performance within virtual environments. In W. Barfield & HEETER, ( 1992). Being there: The subjective experience of presence. C. T. A. Furness 111 (Eds.), Virtual environments and advanced inter- Presence: Teleoperators & Virtual Environments, 1,262-271. ,face design (pp. 474-513). New York: Oxford University Press. HELD, & DURLACH, I. (1992). Telepresence. Presence: Teleoper- R., N.BEALL, C., LOOMIS, M., PHILBECK. & FIKES,T. (1995). Ab- A. J. J. M., 3. ators & Virttral Environments, 1 , 109- 1 12. solute motion parallax weakly determines visual scale in real and vir- HERMER, & SPELKE, ( 1996). Modularity and development: The case L., E. tual environments. In Proceedings ofconference on human vision, of spatial orientation. Cognition, 61, 195-232. visual processrng, and digital disp1a.v (Vol. 241 1, pp. 288-297). L. HETTINGER,J., & HAAS, W. (in press). Virtual andadaptive envi- M. Bellingham, WA: Society of Photo-Optical Instrumentation Engi- ronments. Mahwah, NJ: Erlbaum. neers. HOLMGREN, E., & ROBINETT, (1993). Scanned laser displays for D. W. D.BEGAULT, R. (1994). 3 - 0 sound for virtual reality andmultimedia. virtual reality: A feasibility study. Presence: Teleoperators & Virtual New York: AP Professional. Environments, 2 , 17 1 - 184.BENFORD, BOWERS, FAHLEN, E., GREENHALGH, S., J., L. C., MARIANI, J., HUTTENLOCHER.NEWCOMBE. & SANDBERG,H. (1994). The J.. N.. E. & RODDEN. (1995). Networked virtual reality and cooperative T. coding of spatial location In young children. Cognitive Psychology, work. Presence: Teleoperators & Virtual Environments, 4, 364-386. 27, 115-147.
564 LOOMIS, BLASCOVICH, AND BEALLIWATA, ( 1999). Feel-through: Augmented reality with force feedback. H. ROLLAND, P., GIBSON, & ARIERLY, (1995). Towards quantify- J. W., D. In Y. Ohta & H. Tamura (Eds.), Mixed reality: Merging real and vir- ing depth and size perception as a function of viewing distance. Pres- tual worlds (pp. 2 15-227). Tokyo: Ohmsha. ence: Teleoperators & Virtual Environments, 4.24-49.JACOBS. J.. LAURANCE, E., & THOMAS, G. F. (1997). Place W. H. K. ROSSANO. J., & MOAK, (1998). Spatial representations acquired M. J. learning in virtual space: I. Acquisition, overshadowing, and transfer. from comouter models: Cognitive load. orientation soecificitv and the Learning & Motivation, 28.52 1-541. acquisition of survey kno4edge. ~ r i t i s h Journal q ~ ~ s , v c h o ~ 89, ~ , ogKALAWSKY.S. ( 1993). The science of virtual reality and virtual en- R. 481-497. vironments. Reading, MA: Addison-Wesley. ROTHBAUM. 0.. B. ,R. . , HONES. L. F.. & KOOPER. (1997). Virtual realitvKLATZKY, L., LOOMIS, M., BEALL, C., CHANCE, S., & R. J. A. S. exposuretherapy ~ o u r n ao ~ i ~ c h o t h e r a ~ y lf Practice & Research, 8, GOLLEDGE, G. (1998). Spatial updating of self-position and orien- R. 2 19-226. tation during real, imagined, and virtual locomotion. Psychological ROTHBAUM, O., HODGES, F., KOOPER, OPDYKE. WII.LI- B. L. R., D., Science, 9,293-298. FORD, J., & NORTH, M. (1995). Effectiveness ofcomputer-generated M.LOOMIS, M. (1992). Distal attribution and presence. Presence: Tele- J. (virtual reality) graded exposure in the treatment of acrophobia. operators & Virtual Environments, 1, 113-1 19. American Journal o Psyclziutry, 152,626-628. fLOOMIS, M., GOLLEDGE, G., & KLATZKY, L. (1998). Navigation J. R. R. SANDSTROM,J., KAUFMAN,& HUETTEL, A. (1998). Males and N. J., S. system for the blind: Auditory display modes and guidance. Presence: females use different distal cues in a virtual environment navigation Teleoperators & Virtual Envimnments, 7, 193-203. task. Cognitive Brain Research, 6.35 1-360.LOOMIS, M., KLATZKY, L., & GOLLEDGE, L. (1999). Auditory J. R. R. SHERIDAN, 1992). Musings on telepresence and virtual presence. Pres- T. ( distance perception in real, virtual, and mixed environments. In Y. Ohta ence: Teleoperators & Virtual Environments, 1, 120- 126. & H. Tamura (Eds.), Mixed reality: Merging real and virtual worlds SLATER, USOH, & STEED. ( 1 994). Depth of presence in virtual M., M., A. (pp. 201-214). Tokyo: Ohmsha. environments. Presence: Teleoperators & Virtual Envirnnmmts, 3 ,LOOMIS, M., & KNAPP, M. (in press). Visual perception of egocentric J. J. 130- 144. distance in real and virtual environments. In L. J. Hettinger & M. W. STANNEY, & SALVENDY,(1998). Aftereffects and sense of pres- K., G. Haas (Eds.), Virtualand adaptive environments.Mahwah, NJ: Erlbaum. ence in virtual environments: Formulation of a research and develop-MAY, PERUCH, & SAVOYANT,(1995). Navigating in a virtual M., P., A. ment agenda, international Journal of Human-Computer Inter- environment with map-acquired knowledge: Encoding and alignment action, 10, 135-187. effects. Ecological Psychology, 7 , 2 1-36. STEUER, (1995). Defining virtual reality: Dimensions determining J.MCBEATH, K., SHAFFER, M., & KAISER, K. (1995). How base- M. D. M. telepresence. In F. Biocca & M. R. Levy (Eds.). Communication in ball outfielders determine where to run to catch fly balls. Science, 268, the age ofvirt~ralreality (pp. 33-56). Hillsdale, NJ: Erlbaum. 569-573. STONE, E. (1993). Social interaction and social development in vir- V.MEYER, APPLEWHITE, L., & BIOCCA, A. (1992). A survey of K., H. F. tual environments. Presence: Teleoperators& Virtual Environment.~, position trackers. Presence: Teleoperators & Virtual Environments, 2, 153-161. 1, 173-200. R. SURDICK, T., DAVIS, T., KING, A,, & HODGES. F. (1997). The R. E. L.MORONEY, F., & MORONEY, W. (I 999). Flight simulation. In D. J. W. B. perception of distance in simulated visual displays: A comparison of Garland, J. A. Wise, & V. D. Hopkin (Eds.), Handbook ofaviation the effectiveness and accuracy of multiple depth cues across viewing humanfactors (pp. 355-388). Mahwah, NJ: Erlbaum. distances. Presence: Teleopemtors& Virtual Environnzents, 6.5 1 3-53 I.OHTA, & TAMURA, (1999). Mixed reality: Merging real and vir- Y., H. SUTHERLAND, 1. ( 1965). The ultimate display. Proceedings ofthe Intrr- tual worlds. Tokyo: Ohmsha. national Federation qflnformation Processing Congress,2,506-508.PALMER, T. (1995). Interpersonal communication and virtual reality: M. TAMURA, YAMAMOTO, & KATAYAMA, (1999). Steps toward H., H., A. Mediating interpersonal relationships. In F. Biocca & M. R. Levy seamless mixed reality. In Y. Ohta & H. Tamura (Eds.), Mixedwalih): (Eds.), Communication in the age of virtual reality (pp. 57-124). Merging real and virtual wor1d.s (pp. 59-79). Tokyo: Ohmsha. Hillsdale, NJ: Erlbaum. WARREN, H., & KAY, A. ( 1997, November). Control luw switch- W. B.PELI,E. (1995, July). Real vision and virtual reality. Optics & Photon- ing during visually guided walking. Paper presented at the 38th An- ics News, pp. 28-34. nual Meeting of the Psychonomic Society, Philadelphia.P ~ R U CP.. ,& GAUNET, (1998). Virtual environments as a promising H F. WENZEL, M. (1992). Localization in virtual acoustic displays. Pres- E. tool for investigating human spatial cognition. Current Psychology o f ence: Teleoperators & Virtual Environments, 1, 80- 107. Cognition, 17, 88 1-899. WILSON, N., FOREMAN, GILLETT, & STANTON. (1997). Ac- P. N., R., D.P ~ R U CP., ,MAY, & WARTENBERG, H M., F. (1997). Homing in virtual en- tive versus passive processing of spatial information in a computer- vironments: Effects of field of view and path layout. Perception, 26, simulated environment. Ecological psycho log?^, 9, 207-222. 301-311. WITMER, G., BAILEY,H., & KNERR, W. (1996). Virtual spaces and B. J. B.P ~ R U CP., ,VERCHEK,L., & GAUTH~ER, (1995). Acquisition of H J. G. M. real world places: Transfer of route knowledge. International J o w spatial knowledge through visual exploration of simulated environ- nal qf Human-Computer Stuclies, 45.4 13-428. ments. Ecological Psychology, 7, 1-20. WITMER, G.. & KLINE, B. ( 1998).Judging perceived and traversed B. P.PIRENNE. H. ( 1970). Optics, painting, andphotography. Cambridge: M. distance in virtual environments. Presence: Trleoperators & Virtual Cambridge University Press. Environments, 7, 144-167.PROFFITT, R., BHALLA, GOSSWEILER, & MIDGETT, (1995). D. M., R., J. WITMER, G., & SADOWSKI.J.. JR.(1998). Non-visually guided loco- B. W. Perceiving geographical slant. Psychonomic B~dletin& Review, 2 , motion to a previously viewed target in real and virtual environments. 409-428. Human Factors, 40,478-488.REGIAN,W., SHEBILSKE.L., & MONK, M. (1992). Virtual reality: J. W. J. YANG, L., DIXON, W., & PROFFITT. R. (1999). Seeing big T. M. D. An instructional medium for visual-spatial tasks. Journal of Com- things: Overestimation of heights is greater for real objects than for numicution, 42. 136- 149. objects in pictures. Perception. 28.445-467.RHEINGOLD,( 199 1 ). Virt~ml H. reality. New York: Summit.RICHARDSON, MONTELLO, R.. & HEGARTY, (1999). Spatial A. E., D. M. knowledge acquisition from maps and from navigation in real and vir- (Manuscript received July 20, 1998: tual environments. Memory & Cognition. 27, 74 1-750. revision accepted for publication April 2, 1999.)