Your SlideShare is downloading. ×
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Cloud Computing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Cloud Computing

847
views

Published on

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
847
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Eyesight Sharing in Blind GroceryShopping: Remote P2P Caregiving through Cloud Computing Vladimir Kulyukin, Tanwir ZamanAbhishek Andhavarapu, Aliasgar Kutiyanawala Department of Computer Science Utah State University Logan, UT, USA
  • 2. Big PictureIndependent Shopping for VisuallyImpaired (VI) and Blind Individuals
  • 3. Why Is Blind Shopping Difficult?A modern supermarket has a median area of 4 300 m2, stocksan average of 38 718 products, and services approximately 600customers per hour (6 000 to 7 000 per day)
  • 4. Hardware Evolution 2003-2005 2006-2008 2008-2010 2010 -NowRoboCart ShopTalk ShopMobile
  • 5. ShopMobile Computer Vision + Camera Alignment =Independent Eyes-Free Store Browsing on Smartpho nes
  • 6. Parallel Threads of ComputationBarcode Scanning Camera Alignment
  • 7. Automated Approaches: Problems & Alternatives● Two problems with automated approaches: – Error-prone: false positives and false negatives – Greedy power consumption● Two alternatives: – Remote caregiving – Crowdsourcing● Both can be used to augment automated approaches
  • 8. Remote Caregiving vs. Crowdsourcing● Crowdsourcing may not be suitable for time sensitive tasks and tasks that require mutual trust● Time sensitivity can be addressed through increased helper volume (unlikely to materialize for smaller disabled populations)● Trust may be a more serious issue for crowdsourcing● Remote caregiving addresses trust but requires dedicated caregivers
  • 9. Eyesight Sharing & Cloud Computing● Cloud computing infrastructures and real-time video streaming protocols make it possible for sighted individuals to share their sight with their VI friends remotely● Hearing can also be shared remotely● Amount of caregiving can be dynamically adjusted at run time
  • 10. TeleShop Caregiver Blind ShopperCaregiver Wi-Fi/3G/4G
  • 11. TeleShop ArchitectureBlind Shopper Caregiver
  • 12. A Cloud-Based Caregiving ArchitectureClient
  • 13. A Cloud-Based Caregiving ArchitectureClient Caregiver
  • 14. A Cloud-Based Caregiving Architecture Amazon EC2Client Elastic Computing Caregiver Service
  • 15. A Cloud-Based Caregiving Architecture Amazon EC2Client Elastic Computing Caregiver Service Android C2DM Cloud 2 Device Messaging
  • 16. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Android C2DM Cloud 2 Device Messaging
  • 17. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Notification Android C2DM Cloud 2 Device Messaging
  • 18. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Notification Notification Android C2DM Cloud 2 Device Messaging
  • 19. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Notification Help Notification Android C2DM Cloud 2 Device Messaging
  • 20. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Help Notification Help Notification Android C2DM Cloud 2 Device Messaging
  • 21. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service HelpClient Notification CaregiverClient Help Notification CaregiverClient Caregiver Android C2DMClient Cloud 2 Device Caregiver Messaging
  • 22. Clients & Caregivers● Clients send product images (video streams are possible but consume data plans fast)● Caregivers look at images, speak product names (type if SR does not work), send text back to clients● When caregivers cannot identify products from received images, they can request a new image
  • 23. Image Matching● SURF was used as a black box image matching algorithm● SURF was used to speed up caregivers product recognition● SURF returned top n (n=5 in our case) images and caregiver would verify the correct product name, if it is in top n, or speak/type the correct name, if it is not
  • 24. Three Experiments● A laboratory study with a blindfolded subject● Two field studies with a blindfolded subject at Fresh Market, a supermarket in Logan, UT● A field study with a blind subject was completed at Fresh Market, a supermarket in Logan, UT, after the paper was accepted at ICCHP
  • 25. Lab Study● 20 products (boxes, bottles, cans)● SURF trained on 100 images (5 images per product)● Blindfolded subject was in a lab; caregiver was in a different room● Subject given one product at at time and asked to recognize each● Subject and caregiver used Google Nexus One with Android 2.3.3● Data link was Wi-Fi
  • 26. Supermarket Experiments● Setting was at Fresh Market, a supermarket in Logan, UT● 45 products (boxes, bottles, cans) from 9 aisles● SURF was trained on 370 images● Blindfolded subject used Galaxy S2 with Android 2.3.6 was at Fresh Market● Caregiver used Google Nexus One with Android 2.3.6 was in a different building, a mile away from the supermarket● Data line was 4G
  • 27. Supermarket Experiments● In experiment 1, subject was given 16 products by assistant, one product at a time● In experiment 1, SURF was turned on● In experiment 2, subject was given 17 products by assistant, one product at a time● In experiment 2, SURF was turned off, because of poor performance in experiment 1
  • 28. Experimental ResultsEnvironment # Products Mean Time STD TOP 5 Mean SR SR FailsLab Study 16 40 .00021 8 1.1 0Store 1 16 60 .00033 0 1.2 2Store 2 17 60 .00081 0 1.1 3
  • 29. Conclusions● SR appears to be a viable option for product naming● Poor SURF performance was probably due to our limited understanding of its parameters● Basic trade-off: battery life vs. data plan consumption