Your SlideShare is downloading. ×
  • Like
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Cloud Computing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Cloud Computing

  • 842 views
Published

 

Published in Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
842
On SlideShare
0
From Embeds
0
Number of Embeds
8

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Eyesight Sharing in Blind GroceryShopping: Remote P2P Caregiving through Cloud Computing Vladimir Kulyukin, Tanwir ZamanAbhishek Andhavarapu, Aliasgar Kutiyanawala Department of Computer Science Utah State University Logan, UT, USA
  • 2. Big PictureIndependent Shopping for VisuallyImpaired (VI) and Blind Individuals
  • 3. Why Is Blind Shopping Difficult?A modern supermarket has a median area of 4 300 m2, stocksan average of 38 718 products, and services approximately 600customers per hour (6 000 to 7 000 per day)
  • 4. Hardware Evolution 2003-2005 2006-2008 2008-2010 2010 -NowRoboCart ShopTalk ShopMobile
  • 5. ShopMobile Computer Vision + Camera Alignment =Independent Eyes-Free Store Browsing on Smartpho nes
  • 6. Parallel Threads of ComputationBarcode Scanning Camera Alignment
  • 7. Automated Approaches: Problems & Alternatives● Two problems with automated approaches: – Error-prone: false positives and false negatives – Greedy power consumption● Two alternatives: – Remote caregiving – Crowdsourcing● Both can be used to augment automated approaches
  • 8. Remote Caregiving vs. Crowdsourcing● Crowdsourcing may not be suitable for time sensitive tasks and tasks that require mutual trust● Time sensitivity can be addressed through increased helper volume (unlikely to materialize for smaller disabled populations)● Trust may be a more serious issue for crowdsourcing● Remote caregiving addresses trust but requires dedicated caregivers
  • 9. Eyesight Sharing & Cloud Computing● Cloud computing infrastructures and real-time video streaming protocols make it possible for sighted individuals to share their sight with their VI friends remotely● Hearing can also be shared remotely● Amount of caregiving can be dynamically adjusted at run time
  • 10. TeleShop Caregiver Blind ShopperCaregiver Wi-Fi/3G/4G
  • 11. TeleShop ArchitectureBlind Shopper Caregiver
  • 12. A Cloud-Based Caregiving ArchitectureClient
  • 13. A Cloud-Based Caregiving ArchitectureClient Caregiver
  • 14. A Cloud-Based Caregiving Architecture Amazon EC2Client Elastic Computing Caregiver Service
  • 15. A Cloud-Based Caregiving Architecture Amazon EC2Client Elastic Computing Caregiver Service Android C2DM Cloud 2 Device Messaging
  • 16. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Android C2DM Cloud 2 Device Messaging
  • 17. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Notification Android C2DM Cloud 2 Device Messaging
  • 18. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Notification Notification Android C2DM Cloud 2 Device Messaging
  • 19. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Notification Help Notification Android C2DM Cloud 2 Device Messaging
  • 20. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service Help Notification Help Notification Android C2DM Cloud 2 Device Messaging
  • 21. A Cloud-Based Caregiving Architecture Request Amazon EC2Client Elastic Computing Caregiver Service HelpClient Notification CaregiverClient Help Notification CaregiverClient Caregiver Android C2DMClient Cloud 2 Device Caregiver Messaging
  • 22. Clients & Caregivers● Clients send product images (video streams are possible but consume data plans fast)● Caregivers look at images, speak product names (type if SR does not work), send text back to clients● When caregivers cannot identify products from received images, they can request a new image
  • 23. Image Matching● SURF was used as a black box image matching algorithm● SURF was used to speed up caregivers product recognition● SURF returned top n (n=5 in our case) images and caregiver would verify the correct product name, if it is in top n, or speak/type the correct name, if it is not
  • 24. Three Experiments● A laboratory study with a blindfolded subject● Two field studies with a blindfolded subject at Fresh Market, a supermarket in Logan, UT● A field study with a blind subject was completed at Fresh Market, a supermarket in Logan, UT, after the paper was accepted at ICCHP
  • 25. Lab Study● 20 products (boxes, bottles, cans)● SURF trained on 100 images (5 images per product)● Blindfolded subject was in a lab; caregiver was in a different room● Subject given one product at at time and asked to recognize each● Subject and caregiver used Google Nexus One with Android 2.3.3● Data link was Wi-Fi
  • 26. Supermarket Experiments● Setting was at Fresh Market, a supermarket in Logan, UT● 45 products (boxes, bottles, cans) from 9 aisles● SURF was trained on 370 images● Blindfolded subject used Galaxy S2 with Android 2.3.6 was at Fresh Market● Caregiver used Google Nexus One with Android 2.3.6 was in a different building, a mile away from the supermarket● Data line was 4G
  • 27. Supermarket Experiments● In experiment 1, subject was given 16 products by assistant, one product at a time● In experiment 1, SURF was turned on● In experiment 2, subject was given 17 products by assistant, one product at a time● In experiment 2, SURF was turned off, because of poor performance in experiment 1
  • 28. Experimental ResultsEnvironment # Products Mean Time STD TOP 5 Mean SR SR FailsLab Study 16 40 .00021 8 1.1 0Store 1 16 60 .00033 0 1.2 2Store 2 17 60 .00081 0 1.1 3
  • 29. Conclusions● SR appears to be a viable option for product naming● Poor SURF performance was probably due to our limited understanding of its parameters● Basic trade-off: battery life vs. data plan consumption