200220 Seunghyeon Choe

1. Designing AR Visualizations
to Facilitate Stair Navigation for
People with Low Vision
UIST 2019
Yuhang Zhao, Elizabeth Kupferstein, Brenda Veronica Castro,
Steven Feiner, Shiri Azenkot
Presenter : Seunghyeong Choe

2. Overview
• The first research of AR visualization for PLV’s stair navigation
 PLV : People with Low-Vision
• Two case studies of projection-based AR and smartglasses
• The effectiveness of AR visualization
 Feel confident
 Reduce walking time

3. Background Knowledge
• PLV (People with Low-Vision)
 1.2 billion people worldwide have low vision
 Cannot be corrected with eyeglasses or contact lenses
 Reluctant to use a white cane (Social acceptance)
2

4. Difficult to detect edges
Become slower, higher rates of falls and injuries
Background Knowledge
3
• Contrast Stripes

5. 4
Goals of The Study
• Aid PLV’s stair navigation with AR visualization and sonification
• Safe and speedy
• Easily perceivable, not distracting
• Social acceptance

6. Related Works
5
• Ophthalmology
• Mobility performance in low vision
• HCI
• A study of low vision people and their computing device access
• Blind navigation support system
• AR devices as a vision aid
Little study of PLV navigation
No stair navigation

7. 6
Initial Exploration
• Which AR platform would be most appropriate?
Video See-Through Optical See-Through Projection See-Through
• Switching gaze
• Hindering safety
• Not switching gaze
• Suited to public places
• Not switching gaze
• Suited to private places
• Augment large surface

8. Visualizations for Projection–Based AR
7
• Device Characteristics
 Can augment large physical surfaces
 No popular commercial devices
 Prototype : hand-held projection-based AR platforms
• Embedded depth sensors
• iPhone XR, Samsung Galaxy S10
• Embedded projectors
• Samsung Galaxy Beam

9. Visualizations for Projection–Based AR
8
• Visualizations and Sonifications
 Sonification : ding, voice, combination
 Visualization
Figure 1: Projection based AR
End highlights
Middle highlights
• Distinguish the first and last stairs
• Thick highlights on stairs
• Use yellow color (not be confused with natural light)
• Projecting thin highlights on the middle stairs
• Typically has uniform size
• Do not require much attention

10. Visualizations for Projection–Based AR
9
• Visualizations and Sonifications
 Further emphasize
• End highlight (baseline: no animation)
• Middle highlight (baseline: yellow highlight)
3. Moving edge2. Flashing edge1. Flash 4. Moving horizontal zebra 5. Moving vertical zebra
3. Blue2. Dull yellow1. Baseline

11. Visualizations for Projection–Based AR
10
• Evaluation questions
 How do PLV perceive the different visualization designs?
 How useful are the visualizations for stair navigation?
 How secure do people feel when using our visualizations?

12. Visualizations for Projection–Based AR
11
• Method
1. Participants
• Mean age=53.9
• 6 female, 6 male
• Legally blind except P3
• 녹내장
• 색맹
• 슈타르가르트병
• 망막 질환
• 뇌종양
• Snellen chart

13. Visualizations for Projection–Based AR
12
• Method
2. Apparatus
• Emergency exit staircase with eight stairs
• Prototyped design
• Use a projector at the top of the stair
• Connect to a laptop
• Visualize with PowerPoint
• Handheld simulate
• Projected only on the 3 stairs
• Hold regular phone with the back camera facing the stairs
• Control audio feedback via TCP

14. Visualizations for Projection–Based AR
13
• Method
3. Procedure
Interview
Visualization
Experience
Session
Stair
Navigation
Session
Exit
Interview
Interview
Visualization
Experience
Session
Stair
Navigation
Session
Exit
Interview
• Check visual condition
• Visual acuity test with optometrist
• Explain how to use
• Experience 3 sound feedback, 6 end highlights, 3 middle highlights
• Walk up and down with each options
• Choose the best option
1. Walk in their original way
2. Walk with visualization
• Ask the participant’s experience
• Gave Likert-scale scores for usefulness and comfort level

15. Visualizations for Projection–Based AR
14
• Results
 Effectiveness of Visualizations
“Would make life easier.”
“Having the highlights this bright is really good.”

16. Visualizations for Projection–Based AR
15
• Results
Auditory Feedback
End Highlights
Middle Highlights
Colors
• Human voice (4/12), Ding (3/12), Combination (7/12)
• Felt as an important aspect of design
• Most participants liked the original one
• Flashes and movements are distracting
• Eleven participants felt useful
• Confirm that they are still on the stairs
• Prefer yellow
• Yellow = alert
• Blue = relaxed mode

17. Visualizations for Projection–Based AR
16
• Results
Walking Time
Social Acceptance
Behavior Change
• Reduced 5~6%
• 6.59 sec → 6.17 sec
• Look down less when using visualization
• Walk up and down without holding the rail
• Facing forward
• Most were not concerned about it
• Felt as [cool]
• Might scare others

18. Visualizations for Smart Glasses
17
• Device Characteristics
 Limited FOV(Field of View)
 29° vertical FOV
 Could be dangerous potentially
 Tinted glass

19. Visualizations for Smart Glasses
18
• Visualizations and Sonifications
 Most stairs are uniform, middle stairs are less important
 Distinguish how close to change step
 7 Stages
3ft ~ 1.5~3ft ~1.5ft Stepping down Last ~1.5ft 1.5ft~

20. 19
Visualizations for Smart Glasses
• Visualizations and Sonifications
 Two visualizations
• Glow visualization
• Glow effect at the bottom of the vertical FOV
• Different colors based on the different stages
Landing Preparation Alert Middle

21. 20
Visualizations for Smart Glasses
• Visualizations and Sonifications
 Two visualizations
• Path visualization
• Shows the direction of the stairs
Landing Getting close to the first stair Middle

22. 21
Visualizations for Smart Glasses
• Visualizations and Sonifications
 One sonification
• Beep sonification
• Inform current position on the stairs
• Make different sound based on the different stages
• Start landing stage: no sound
• Preparation stages: low-frequency beep
• Alert stages: high-frequency beep
• Middle stairs: no sound
• End landing stage: verbally reports “Stair ends.”

23. 22
Visualizations for Smart Glasses
• Evaluation questions
 How do PLV perceive the visualizations on smartglasses?
 How effective are the visualizations for stair navigation?
 How secure do PLV feel when using our visualizations?

24. Visualizations for Smart Glasses
23
• Method
1. Participants
• Mean age=51.6
• 5 female, 7 male
• All Legally blind

25. Visualizations for Smart Glasses
24
• Method
2. Apparatus
• Emergency exit staircase with 14 stairs (different stairs than those in the projection study)
• HoloLens v1 (34° diagonal FOV, can be used with eyeglasses)
• Mark the position of the stairs with two Vuforia image targets
3. Procedure
• Same procedure as the projection study

26. Visualizations for Smart Glasses
25
• Results
 Experience with the smartglasses
• Tinted optics blocks environmental glare
• 3 PLVs’ visual acuity increased
• Tinted optics also made the environment darker
• 1 PLV’s visual acuity decreased
• Heaviness of the hardware

27. Visualizations for Smart Glasses
26
• Results
 Effectiveness of Visualizations

28. Visualizations for Smart Glasses
27
• Results
Smartglasses
Edge Highlights
• Tinted optics blocks environmental glare (3 PLVs’ visual acuity increased)
• Tinted optics also made the environment darker (1 PLV’s visual acuity decreased)
• Heaviness of the hardware
• Difficult to use because of the limited vertical FOV
• Had to angle their head down a lot
• Provide a preview for future steps

29. Visualizations for Smart Glasses
28
• Results
Glow
Path
Beep
• Easy to understand
• Colors and thicker/brighter are effective
• Some had difficulty in distinguishing colors
• 50% helpful
• 50% distracting, misleading
• All helpful
• Don’t have to look visualization information
• May not be distinguishable from environmental sounds

30. Visualizations for Smart Glasses
29
• Results
 Preferences for Visualizations and Sonification
• The most commonly chosen visualization : Glow
“More useful together than separate.”
“Helpful in unfamiliar place"

31. Visualizations for Smart Glasses
30
• Results
Walking Time
Behavior Change
Psychological Security
• Increased (whether using visualizations or not)
• No significant effect of Condition
• Reported feeling safer and more confident
• Move more confidently once understand the meanings
• 2 participants walked without holding the railing
• Look down less

32. 31
Discussion
• The first research of AR stair navigation
• Increased psychological security
• Prefer projection-AR more
 End highlights: stable thick yellow highlights (7/12)
 Middle highlights: yellow highlights (7/12)
 Glow and beep combination is chosen for HoloLens AR (6/12)

33. 32
Discussion
• Why walking time was increased with Smartglasses?
 Cognitive load
 Had to associate the design with the physical stairs
 Visualizations did not fixed in place
• Novelty effect

34. 33
Future works
• Navigation system should be highly accurate and fast
 Small error could lead to severe consequence
 User’s body movement
 Stair detection methods
• Real-world challenges
 Use RFID for accuracy and speed
 Face detection
• Lightweight smartglasses

35. 34
Criticism
• No consideration for color blindness
• Changing high-contrast stripes could cost less than visualization
• Sound effect for Smartglasses AR
• Challenging devices are discontinued
• Controlling visualizations

36. 35
What I’ve Learned
• Study of social acceptance
• Effective of AR visualization on normal people
• Tablet would be powerful at the aspect of projection AR
• Long-term evaluation (Novelty effect)

37. Thank you 
Any questions?