Recommended
Recommended
More Related Content
Similar to Visual Nouns for Indoor/Outdoor Navigation
Similar to Visual Nouns for Indoor/Outdoor Navigation (20)
More from icchp2012
More from icchp2012 (14)
Recently uploaded
Recently uploaded (20)
Visual Nouns for Indoor/Outdoor Navigation
- 1. ICCHP 2012 Visual Nouns for Indoor/Outdoor Navigation Edgardo Molina, Dr. Zhigang Zhu, Dr. Yingli Tian Dept. of Computer Science and Dept. of Electrical Engineering Grove School of Engineering, The City College of New York ICCHP 2012 – July 11, 2012 1
- 2. ICCHP 2012 Motivation Assist the visually impaired in navigating new locations Provide guidance to facilities and destinations Restrooms Exits Elevators Water fountains How do the sighted recall a location? Visual cues? 2
- 3. ICCHP 2012 What are visual nouns? Signs Text Icons 3
- 4. ICCHP 2012 Using Visual Nouns Provide contextual information What is it telling us? Provide landmark markers Have I been here before? Camera pose estimation => User Orientation 4
- 5. ICCHP 2012 Technology Alternatives RFID Embedded info Costly QR Codes / Bar Codes / Etc Embedded info Only meaningful to machines GPS Outdoors only Signs and Text are already in our environment 5
- 6. ICCHP 2012 Standardized Signage US DOT and AIGA signs Various Organizations have standard signage 6
- 7. ICCHP 2012 Visual Noun Project http://www.thenounproject.com 7
- 8. ICCHP 2012 Localizing a user 1. Capture video of surroundings 2. Register video image frames to the first frame (will serve as reference) 3. Generate a wide field-of-view panorama 4. Extract and match visual-noun features 5. Localize the user in 3D space relative to the visual nouns. 8
- 9. ICCHP 2012 Framework 9
- 10. ICCHP 2012 Input Video 10
- 11. ICCHP 2012 Panorama 11
- 12. ICCHP 2012 Visual Noun Extraction Maximally Stable Extremal Regions (MSER) Performed on a sharpened image Use of EE-MSER could be better [Chen et. al. - ICIP’11] 12
- 13. ICCHP 2012 Localization Localization done using PnP algorithm Compared number of features in a single view vs. a panorama 13
- 15. ICCHP 2012 Summary We propose to use the signage that’s already in our environment to provide orientation and context information to visually impaired users. Low cost to print out more signage Also beneficial to sighted users 15
- 16. ICCHP 2012 Thank you! Questions? Work is supported by US NSF Emerging Frontiers in Research and Innovation Program under Award No. EFRI-1137172, and City SEEDs: City College 2011 President Grant for Interdisciplinary Scientific Research Collaborations. 16
Editor's Notes
- Visual cues – signs, visual markers, indoor/outdoorExample – store in Times square, recognizing logos around it…
- Icons come from US DOTSigns can be general – CAFÉ can be in different style/font
- Most markers are static => have I been here before? => estimate camera pose
- Pros/consSigns are readily understandable by everyone.
- AIGA = American Institute of Graphic Arts
- Huge repository
- Input: sequence of video framesFeature extraction
- Mention signs…Natural occurring signs – fire, door
- Stitched panorama
- EE-MSER = edge enhancedDetection is from panorama
- Set of 2D points and corresponding 3D points Perspective N-Point (PnP) problem for extrinsic camera parameter estimationCompare single vs. panorama (for localization purposes)Red = visual nounGreen = additional sign provided
- Single view => bad pose, 4 featuresPanorama => 4 degrees off, 4 inches offPreliminary result/work
- Generated a lot of false positiveFuture work – providing direction based on a query Main work is NOT to do sign detection, but for navigation and orientation.