The is a technical presentation for the CSUN Assistive Technology Conference for a session occurring on Friday March 3rd 2017. It hightlights the new assistive technology for Mobile Deposits.
1. Public Information
Harmony Clauer Salyers
Voice Guided Deposits
Android Mobile Developer
LinkedIn: bit.ly/HClauerSalyers
Slides: bit.ly/CSUN17VoiceGuided
Demo: bit.ly/CSUN17VoiceGuidedDemo
03 MAR 2017
2. Public Information 2
Our Mission
The mission of the association is to
facilitate the financial security of its
members, associates, and their families
through provision of a full range of
highly competitive financial products
and services; in so doing, USAA
seeks to be the provider of choice
for the military community.
Our Core Values
Service Loyalty Honesty Integrity
Passionate
Member
Advocacy
Financial
Strength
& Wisdom
Shared
Military
Values
Our Brand Pillars
GOING ABOVE
Our Brand Promise
FOR THOSE WHO HAVE GONE BEYOND
Who We Are
As of Oct. 2014
4. Public Information 4
Agenda – Voice Guided Deposits
Mobile Accessibility Basics
Purpose
What is a Voice Guided Deposit?
Overview of the Technology
Demo
Resources
5. Public Information 5
Google Talkback
• An accessibility service that helps
blind and vision-impaired users
interact with their devices.
• Adds spoken, audible, and vibration
feedback to your device.
Apple VoiceOver
• A unique accessibility service because
it’s not a standalone screen reader.
• It’s deeply integrated in iOS.
Native Mobile Accessibility
6. Public Information 6
Provide independence to visually impaired members by providing
an accessible check deposit mobile experience.
Purpose
Released July 5, 2016 through an update to both Android & iOS applications.
7. Public Information 7
Deposit money any time via a mobile device
Deposit@Mobile
Deposit checks from
anywhere using a mobile
device
USAA Mobile app for
iPhone, iPad, and Android
8. Public Information 8
“Lift Device”
“Pull In”
“Hold Steady” & “Image
Captured”
“Push Out"
A Voice Guided What?
9. Public Information 9
Everything is in Real Time:
1. Line Detection
2. Movement Decision Algorithm
3. Native Directional Announcements
− “Move Left” or “Move Right”
− “Push Out” or “Pull In”
− “Lift Device” or “Lower Device”
− “Hold Steady”
4. Capture & Process Image
− “Image Captured”
Technology @ a Glance
11. Public Information 11
Official Press Release: bit.ly/VoiceGuidedDeposits
Additional Resources:
Editor's Notes
Inventor Developer
https://play.google.com/store/apps/details?id=com.google.android.marvin.talkback&hl=en
http://www.apple.com/accessibility/mac/vision/
The reason why I am mentioning this is because throughout the process we flip flopped between using a custom solution for our talkback over the Native Screen reader. What it boils down to though is that while we had to come up with some very clever solutions in order to get get the native features to work the way we wanted them to it really gave us more bang for our buck
Consistently changing with the OS ensures future compatibility user customization.
Voice Guided Deposits allows visually-impaired members to deposit check wherever they are and eliminates unnecessary trips to the bank or financial center or asking friends and family for help.
Voice guided deposit through the mobile app provides visually impaired members more independence and saves them time and money by allowing them to more easily perform this most basic of banking needs.
USAA is the first in the industry to provide this deposit experience (USAA patent pending).
Voice Guided Deposits allows visually-impaired members to deposit check wherever they are and eliminates unnecessary trips to the bank or financial center or asking friends and family for help.
Voice guided deposit through the mobile app provides visually impaired members more independence and saves them time and money by allowing them to more easily perform this most basic of banking needs.
USAA is the first in the industry to provide this deposit experience (USAA patent pending).
Member opens D@M and selects to add a check. Camera is opened and voice guidance prompts member for image capture, then begins navigation using simple directional cues.
This is an extremely visual slide, I have a demo of what is being depicted here in just a few moments so bare with me while I talk through the process.
This is a very basic demonstration of the steps that a user might go through when using Voice Guided Deposits. We initially inform the user that they should start off by placing their phone on the check. This was really just to help the user really understand where their check was in relation to there phone. We don’t want them to start too far away until they are really used to using the software…
We actually have this as a fall back as well if we find that the user might be lost then we ask them to place their phone back down.
A lot of work was put in when determining the best direciton to move the device. We found in a lot of scenarios we would minimize the number of instrucitons by 2 or 3 by zooming further out before giving additional direcitonal queues.
Everything is done in real time frames per second processing
Line detection via simple image processing algorithm
Movement determination algorithm based off of line detection outputs
Leverages Native accessibility (Talkback & VoiceOver) for directional announcements:
“Move Left” or “Move Right”
“Push Out” or “Pull In”
“Lift Device” or “Lower Device”
“Hold Steady”
Capture and process check image (Ensures clarity of the image and reliability for deposit)
“Image Captured”
Image on bottom right:
Blue is the camera view, White represents a check
Announces “Move Left”
For a really long time we conditioned out users to use a specific set of instructions. These instructions were actually recently removed. So in order to accommodate a new feature for the subset of accessibility users we put in this custom alert only visible when accessibility is turned on.
This was a huge lesson learned and is still a topic for debate. I need to double check if the bug was actually ever fixed on Android but we actually found that in some cases the built in call for determining if talkback was enabled could sometimes return true even if talkback wasn’t enabled. TODO find the link for the Google ticket.
While ideally, imo and based of guidelines, you shouldn’t provide separate experiences. However in some instances in order to proved the best experience for those using accessibilty and not using it imo I feel there are a few exceptions to rule
Since this is a new technology we provided some basic guidance on how it should be used.
Acknowledgements:
Marty Hathorn & Patrick Sturdivant
Thomas Backlund
Adam Newman
Jacob Ackerman
Miguel Salazar
John Thomas
Innovation Partners
Of course thank you all for giving me your time today and listening to me talk about something that is super close to my heart and what I am passionate about.