Download to read offline
Uploaded byKlen Čopič Pucihar
ICMI '13: Evaluating Dual-view Perceptual Issues in Handheld Augmented Reality : Device vs . User Perspective Rendering
The document discusses expectations related to augmented reality (AR) use from both device and user perspectives. It includes statistical results showing significant differences in expectations with previous AR use, specifically highlighting the 'magic-lens' device perspective. Various image references from Google Maps and other sources are also noted.
![5G Explained! A High Level Overview [Introduction]](https://cdn.slidesharecdn.com/ss_thumbnails/5gexplainedahighleveloverview-260119165306-cc137a3e-thumbnail.jpg?width=640&height=640&fit=bounds)
ICMI '13: Evaluating Dual-view Perceptual Issues in Handheld Augmented Reality : Device vs . User Perspective Rendering
- 2.
- 3.
- 5. Images from GoogleImage Search
- 6. Images from GoogleImage Search
- 12. Images from GoogleImage Search
- 13.
- 14.
- 15.
- 16. Expectations All 0 3 21 Deviceperspective Userperspective 0 ExpectationsNo Prev. AR Use 0 Deviceperspective Userperspective 10 • 0 Expectations With Prev. AR Use 3 11 Deviceperspective Userperspective •
- 17.
- 19.
- 20.
- 21.
- 22.
- 23. Image from SonyPlayRoom
- 24.
- 25. Map Image fromGoogle Maps
Editor's Notes
- #2 Hello everybody, my name is Klen and I come from Slovenia. It will be difficult to match the talk we had this morning by Mark Billinghurst. We herd a great overview of AR with some fantastic new work I guess we all saw for the first time.I am happy that he did not surprise me and show something what I am going to preset to you now.I will talk about the paper I wrote with dr. Paul Coulton and dr. Jason Alexander on the topic of Evaluating Dual-view perceptual issue in handheld AR.But first to put things into perspective.
- #3 That’s Slovenia.
- #4 A country beautifully located where the Mediterranean meets the alps and the place where funding that enabled this research came from.However this research was done at Lancaster University where I am currently finishing my PhD.The core topic of this research is AR and even though AR is considered by some as a multimodal interaction in itself, the work I am about to present might feel slightly out of place to some of you here at ICMI.Yet having said that, the paper published at ICMI in 2007 by Michael Rohs is a key publication relating and motivating this work and also the way I discovered ICMI conference.
- #5 Just to give a quick overview of the content of this presentation. I will start by defining the dual-view problem, explain why does it occur. All in order to introduce the concept of dual-view perceptual issue.I will continue by formulating research questions that will be answered through two user studies that follow. Finally I plan to summarize this presentation by drawing conclusions and open future research questions.
- #6 What is the problem?The work focuses on dual-view problem andI decided to illustrate the problem by analyzing the first 50 hits of a goggle image search engine using a term “Mobile Augmented Reality”. I grouped images into two groups.
- #7 This is the first set which shows how handheld AR would ideally work. Handheld devices mimic a transparent glass pane on which additional graphical information is projected. Observer how the view on the phone exactly matches direct-view of the observer (surroundings). The two views seamlessly merge.Unfortunately this is not reality. I would argue that probably all of these examples are actually mockup images because currently handheld AR looks more like this.
- #8 Here it is obvious that the surrounding context and the phone view do not match, creating a dual-view situation. To be more precise. Dual-view problem describes a situation where the augmented-view (magic-lens view) becomes disconnected from the direct-view (the observers view).Why does this happen?
- #9 The problem lies in the fact that magic-lens paradigm relies on overlaying additional graphical information on top of transparent glass pane. Unfortunately, as you mush have noticed by now, handheld devices such as mobile phones are not transparent, hence virtual transparency is required.
- #10 Virtual transparency is typically achieved by utilizing a back facing camera and a device screen. Because the device is held in the hand and not worn on the head, this setup results in perspective difference between the observer and the device camera. Additional perspective differences are introduced by the camera-screen offset and differences in FOV. Looking at the image on the left. The device camera is at an offset from screen center; Due to this blue dot is rendered at the position of red dot instead of green, where it should be if the phone acted as a transparent glass pane.Differences in FOV are demonstrated on figure on the right. Contrary to the FOV of the observer which is dynamic, device camera has a static FOV.Magic-lens implementations that render camera captured images onto device screen without any modifications are known as device-perspective lenses. Magic-lenses that modify the camera imagery in order to try and match a transparent glass pane are known as user-perspective lenses. Lets look at the video showing this two rendering types and the dual view problem.
- #11 The first magic-lens type shown is device-perspective magic-lens. A typical implementation where dual-view is a problem.The second magic-lens type shown is a simplified variation of user-perspective magic-lens we call fixed-POV user-perspective magic-lens where dual-view problem does not exist. These two rendering modes were used in the two use studies presented later on in order to evaluate dual-view problem.Now that we know what dual-view problem is, and that it only occurs in device-perspective magic-lens. lets put it into the concept of perception issue.
- #12 Perceptual issues relate to problems that arise from observing and interpreting information.This process of observation and interpretation of information is shaped around previous experiences which may take a form of hands-on and hands-off interactions. If expectations differ from system behavior, perceptual problems are bound to occur. Let’s first look at hands-on interactions within the context of handheld AR and virtual transparency.
- #13 //Take out if not enough timeThese relate to any previous AR experiences as well as to any camera based activities.If you ever tried to set up a face unlock feature available on android phones, you must have experienced the difficulty of fitting your face within the dedicated section of camera view. As reflection did not match your expectations you experienced a variation of dual-view perceptual problem mainly caused by camera-screen offset. If you have not tried this before, you have a chance to experience this perceptual problems first hand.It is important to note that in all camera based activities device-perspective view is shown. Beside hands-on experiences, perception is also shaped by hands off media exposure.
- #14 //Take out if not enough timeAs has been already shown before handheld AR implementations are portrayed as device and user perspective implementations, however to the best of hour knowledge, no commercial user-perspective solutions have been released to the general public so far. This coupled with exclusive exposure to device-perspective rendering raises a number of interesting questions such as
- #15 SLIDE
- #16 To answer these questions we designed two user studies which focus on a class of tabletop sized AR workspaces. In both studies a selection task was performed. The target was augmented on top of 23 inch interactive surfaces and users were asked to indicate target position by touching the surface.24 participants: 8 female 16 male aged 21-45 completed the studies.Both studies are within subject experiments where only one variable magic-lens type is being changed. We test1.) Device-perspective magic-lens, the standard handheld AR implementation where dual-view is a problem.2.) Fixed-POV user-perspective magic-lens where dual-view is not a problem.We used Fixed-POV user-perspective magic-lensbecause at the time of running the study, full user-perspective rendering was not yet realizable on handheld devices. Lets move on to the first study conducted.
- #17 //potentially could go out.We used Fixed-POV user-perspective magic-lensbecause at the time of running the study, full user-perspective rendering was not yet realizable on handheld devices. Fixed-POV user-perspective magic-lenstechnique produces user-perspective rendering without the need for head pose tracking. We do this by fixingthe observer’s POV above the physical center of the magic-lens screen. This imposes some restrictions on user interaction that are demonstrated by this figure.1.) User has to align their POV with screen center.2.) Hold the device parallel to his/her face.3.) Maintain Distance to table.Lets move on to the first study conducted.
- #18 Study A was designed to test users’ spatial expectations. To be more precise we want to answer the following questions:What are users’ expectations?Are expectations of new and returning AR users different?Does the dual-view problem result in a dual-view perceptual issue irrespective of expectations?In order to test users’ initial expectations all visual links between the magic-lens and the interactive surface had to be removed, including user’s hand. This is required because visual information is bound to overwrite these initial expectations we wanted to observe.Throughout the test, participants were instructed to only focus only on accuracy. We recorder the touch positions and target position on the screen at touch event.After completing the study participants were asked to summarize task completion strategy. In other words tell us how they completed the task.
- #19 By analyzing task completion strategies we classified participants into two groups:(1) the group expecting user-perspective rendering. (2) the group expecting device-perspective rendering; The results show that majority of users expect user-perspective rendering (i.e. a transparent glass-pane ). Out of 10 participants with no previous AR experience all expect user-perspective view. Out of 14 participants with previous experience, only 3 expect device-perspective view. Therefore expectations of new and returning AR users do not differ.This classification by strategy was also confirmed by quantitative classification. When we plot average target position at touch event on mobile device it is obvious that 3 aligned the target towards the bottom right corner of mobile device (red plots) before touching the interactive surface. This suggests that they assumed camera was not positioned in the center of the screen, hence they expect device-perspective imagery.
- #20 Till now, Strategies analysis and target position uncovered that the majority of people expect user-perspective rendering. Hence, as at each task completion no feedback about is provided, we expect people to perform more accurately with user-perspective magic-lens.The graph on the left confirms this. Paired sample T test shows that the difference is significant.This confirms the existence of dual-view perceptual issue which is present when interacting with device-perspective magic-lens where dual-view is a problem.However the question remains if dual-view perceptual issue exists also in the group expecting device-perspective magic-lens? In other words users’ expecting device-perspective perform better than users having user-perspective expectations when interacting with device-perspective magic-lens? The graph on the right shows that comparable accuracy was achieved by participants expecting device and user-perspective magic-lens, hence dual-view perceptual issue exists in both groups. Having said this it is important to note that using only three participants to represent a group expecting device-perspective rendering is very small. Hence generalization of this claim might be somewhat limited.Irrespective of this limitation lets try to explain this result.
- #21 The group of participants expecting device-perspective rendering did not demonstrate correct understanding of the dual-view problem. Even though this group correctly assumed the camera position was not in the center of the screen, they wrongly assumed the camera imagery was rendered at the camera position and not center of the device’s screen.In other words they expected the blue dot to render at the position of the green dot instead of the camera center.Hence no significant performance difference between the two groups is not a surprise.
- #22 From A we know:User-perspective view is expectedPrevious AR experience does not affect expectations.Dual-view perceptual issue exists irrespective of these expectations. Here we want to explore How successful are participants at dealing with dual-view perceptual issues when visual context is present?Are their expectations affecting their performance?How does performance change over time.To answer these questions we change three things in comparison to study A:1.) reintroduce visual links, however hand remains to be hidden.2.) provide feedback regarding task completion (allowing users to learn and improve their performance)3.) ask participants to complete the task as quickly as possible whilst continuing to achieve minimal task performance.We record speed and accuracy. Lets see what results have to say about these questions.
- #23 Results show that users’ are not very successful at dealing with dual-view problem. Even after they identify their initial expectations are wrong they do not manage to quickly adopt her input actions.Observe graphs on the left that show accuracy and time for each magic-lens type. Users perform significantly better with user-perspective magic-lens in bot time and accuracy domain. Results on the right show that expectations do not affect uses’ ability to deal with dual-view problem. In other words are users’ expecting device-perspective view did not perform better with device-perspective magic-lens than users having user-perspective expectations? But what bout learning?
- #24 The results show that people improve over time but slowly,Learning was only detected in error domain. As is demonstrated by graph on the left, learning only happens in device-perspective lens. This is expected as only this magic-lens suffers from dual-view perceptual issue. Interestingly though learning curve is slow. Participants continue to learn after 8 attempts. Graphs on the right show that the performance difference between the two modes decreases as participants learn how to deal with dual-view problem. Taking into account only last three attempts, significant difference continues to exist in time domain.Hence we can conclude that participants learn learn how to deal with dual-view perceptual issue but slowly.So lets draw some final conclusions.
- #25 Even though user-perspective rendering lens is preferred over device-perspective lens, significance is was reached. However significance was reached in case of workload scores where user-perspective lens scored lower. Now that I presented all of results lets look at limitations of the methodology used.
- #26 The first limitation regards the sample size. As much lower number of participants than expected proved to have device-perspective expectations, we ended up with a small sample size of this group.We used Fixed-POV user-perspective magic-lensbecause at the time of running the study, full user-perspective rendering was not yet realizable on handheld devices. Within the context of the study we believe this is not a big limitation because our results confirmed successful adoption of this. Additionally, the requirement to align observers’ POV with the predefined position may introduce rendering errors, as such alignment is not required in dynamic user-perspective rendering, the difference between device and user-perspective rendering performance is expected to grow.The last limitation is the fact that hand was not represented. This is yet again technical limitation of handheld AR. To implement geometrically correct user-perspective view, one needs an up to date model of the scene. As currently scene reconstruction on handheld devices are limited to static environments, geometrically correct user-perspective rendering of the hand is not possible. Within the context of sturdy A, this is not a problem because we focused on testing initial expectations. Hence, removing the hand was necessary as otherwise users visual information would overwrite these initial expectations we wanted to observe.In context of study B we believe that results are applicable to general AR use cases as there are many instances of AR where the other hand is not available for interaction. Additionally we believe reintroducing a visual representation of the hand would produce similar results, however this needs to be verified.Lets draw some conclusions.
- #27 SLIDEWhy is this important within Research community:Even after participants’ discovered their expectations were wrong they took considerable time to correct their input actions.This discovery is important for the handheld AR research community as it uncovers the importance of the camera position on the handheld device in tabletop sized AR setting.Repositioning the camera to the screen center would solve this problem. However, device designers have probably strong reasons for the current camera placement.This opens up the question of designing the optimal visualization to help users deal with the non-centered camera.Are these results applicable outside handheld AR context?
- #28 YESOne good example are AR systems with fixed displays and moving observers. One such recently deployed commercial system is Sony Playroom .In such AR settings the display acts as a mirror showing a reflection of the real world augmented with digital content.These systems are commonly implemented using a single static camera positioned at an offset from the TV screen center, for example at the top or at the bottom of the screen.This results in device perspective reflections, which is different to what the user would see if looking into a mirror. The mismatch between the user’s expectations and system’s is likely to reoccur.Where do we see future research opportunities?
- #29 From the perspective of the user our work encourages the design of design of novel visualization methods to help participants deal with the camera-scene offset;And petitions for a change in the hardware design of mobile handsets, by placing the camera lens into the center of the screen.Where do we see future research opportunities?
- #30 First area is related to enhancing existing studies by dynamic user-perspective rendering and the reintroduction of hand.Explore if dual-view perceptual problem exists in fixed displays and moving observer AR setup (e.g. PalyRoom).
- #31 In this work we focused on mapping of augmented content to the real world. However, the effect on the use of surrounding visual context was not explored. Previous studies indicated that surrounding contextual information is largely ignored. This is a limitation for handheld AR because we believe that effective use of the surroundings is expected to facilitate a more fluid interaction, provide greater perception of information context and aid the construction and retention of a mental model of the information space.This is bound to improve the usability and general usefulness of handheld AR interfaces. So the important qustions are:Why people ignore surrounding context?What is the role of Dual-view problem in this?How could we improve surrounding context use?It has been recently confirmed that I will present a paper answering these questions at CHI Conference in Toronto. I hope to see some of you there in April.
- #32 Thank you for your time. Any questions?