Visual Nouns for Indoor/Outdoor Navigation
Upcoming SlideShare
Loading in...5
×
 

Visual Nouns for Indoor/Outdoor Navigation

on

  • 494 views

Portable and Mobile Systems in Assistive Technology - Visual Nouns for Indoor/Outdoor Navigation - Zhu, Zhigang (f)

Portable and Mobile Systems in Assistive Technology - Visual Nouns for Indoor/Outdoor Navigation - Zhu, Zhigang (f)

Statistics

Views

Total Views
494
Views on SlideShare
494
Embed Views
0

Actions

Likes
0
Downloads
9
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Visual cues – signs, visual markers, indoor/outdoorExample – store in Times square, recognizing logos around it…
  • Icons come from US DOTSigns can be general – CAFÉ can be in different style/font
  • Most markers are static => have I been here before? => estimate camera pose
  • Pros/consSigns are readily understandable by everyone.
  • AIGA = American Institute of Graphic Arts
  • Huge repository
  • Input: sequence of video framesFeature extraction
  • Mention signs…Natural occurring signs – fire, door
  • Stitched panorama
  • EE-MSER = edge enhancedDetection is from panorama
  • Set of 2D points and corresponding 3D points Perspective N-Point (PnP) problem for extrinsic camera parameter estimationCompare single vs. panorama (for localization purposes)Red = visual nounGreen = additional sign provided
  • Single view => bad pose, 4 featuresPanorama => 4 degrees off, 4 inches offPreliminary result/work
  • Generated a lot of false positiveFuture work – providing direction based on a query Main work is NOT to do sign detection, but for navigation and orientation.

Visual Nouns for Indoor/Outdoor Navigation Visual Nouns for Indoor/Outdoor Navigation Presentation Transcript

  • ICCHP 2012Visual Nouns for Indoor/OutdoorNavigationEdgardo Molina, Dr. Zhigang Zhu, Dr. Yingli TianDept. of Computer Science and Dept. of Electrical EngineeringGrove School of Engineering, The City College of New YorkICCHP 2012 – July 11, 2012 1
  • ICCHP 2012Motivation Assist the visually impaired in navigating new locations Provide guidance to facilities and destinations  Restrooms  Exits  Elevators  Water fountains How do the sighted recall a location? Visual cues? 2
  • ICCHP 2012What are visual nouns? Signs Text Icons 3
  • ICCHP 2012Using Visual Nouns Provide contextual information  What is it telling us? Provide landmark markers  Have I been here before?  Camera pose estimation => User Orientation 4
  • ICCHP 2012Technology Alternatives RFID  Embedded info  Costly QR Codes / Bar Codes / Etc  Embedded info  Only meaningful to machines GPS  Outdoors only Signs and Text are already in our environment 5
  • ICCHP 2012Standardized Signage US DOT and AIGA signs Various Organizations have standard signage 6
  • ICCHP 2012Visual Noun Project http://www.thenounproject.com 7
  • ICCHP 2012Localizing a user1. Capture video of surroundings2. Register video image frames to the first frame (will serve as reference)3. Generate a wide field-of-view panorama4. Extract and match visual-noun features5. Localize the user in 3D space relative to the visual nouns. 8
  • ICCHP 2012Framework 9
  • ICCHP 2012Input Video 10
  • ICCHP 2012Panorama 11
  • ICCHP 2012Visual Noun Extraction Maximally Stable Extremal Regions (MSER)  Performed on a sharpened image  Use of EE-MSER could be better [Chen et. al. - ICIP’11] 12
  • ICCHP 2012Localization Localization done using PnP algorithm Compared number of features in a single view vs. a panorama 13
  • ICCHP 2012Localization Results 14
  • ICCHP 2012Summary We propose to use the signage that’s already in our environment to provide orientation and context information to visually impaired users. Low cost to print out more signage  Also beneficial to sighted users 15
  • ICCHP 2012Thank you!Questions?Work is supported by US NSF Emerging Frontiers in Research and InnovationProgram under Award No. EFRI-1137172, and City SEEDs: City College 2011President Grant for Interdisciplinary Scientific Research Collaborations. 16