Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
ICORR 2013 - Seattle TEC2011-29106
Cristina Urdiales
Metrics and benchmarking for
assisted wheelchair navigation
Grupo ISI...
ICORR 2013 - Seattle TEC2011-29106
Wheelchair metrics
There are no universally accepted metrics/procedures to evaluate nav...
ICORR 2013 - Seattle TEC2011-29106
Wheelchair metrics
There are well known questionnaires that can be applied to any situa...
ICORR 2013 - Seattle TEC2011-29106
Condition metrics
Questionnaires can be (and usually are) complemented with clinical sc...
ICORR 2013 - Seattle TEC2011-29106
Course-related metrics
●Wheelchair propulsion, assessed in terms of period of time, a f...
ICORR 2013 - Seattle TEC2011-29106
Path-related metrics
● Deviation with respect to the canonical trajectory (often quanti...
ICORR 2013 - Seattle TEC2011-29106
Task metrics
●Task success (or) degree of success
●Task completion time
●Number of coll...
ICORR 2013 - Seattle TEC2011-29106
Psych metrics
Psych metrics aim at (indirectly) evaluating the user’s (mental) state. T...
ICORR 2013 - Seattle TEC2011-29106
Benchmarking
●It consists of trying to find how a generic user would respond to every f...
ICORR 2013 - Seattle TEC2011-29106
Data mining
Given a large enough sample space, we split it into classes where users
res...
ICORR 2013 - Seattle TEC2011-29106
User profiling and benchmarking
Using the proposed prototype user we can ...
Subject 14...
ICORR 2013 - Seattle TEC2011-29106
Conclusions and future work
(Standard) Benchmarking procedures and metrics are of key ...
ICORR 2013 - Seattle TEC2011-29106
Cristina Urdiales
Grupo ISIS (UMA)
Metrics and benchmarking for
assisted wheelchair nav...
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
TedX Malagueta: Robolución (Robolution)
Next
Download to read offline and view in fullscreen.

1

Share

Download to read offline

Metrics and Benchmarking for Assisted Wheelchair Navigation

Download to read offline

A brief survey on the different metrics that can be used in the field of assistive wheelchair navigation, including usability questionnaires, clinical scales, task and psych metrics, etc. It also includes obstacle courses and a benchmarking proposal.
An invited presentation to ICORR'13 workshop CanWheel (http://www.canwheel.ca/events/upcoming/icorr-2013-workshops/)

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Metrics and Benchmarking for Assisted Wheelchair Navigation

  1. 1. ICORR 2013 - Seattle TEC2011-29106 Cristina Urdiales Metrics and benchmarking for assisted wheelchair navigation Grupo ISIS (UMA)
  2. 2. ICORR 2013 - Seattle TEC2011-29106 Wheelchair metrics There are no universally accepted metrics/procedures to evaluate navigation There are benchmarking procedures in established obstacle courses BUT they do not provide information on everyday navigation at preferred environments for example ... The Valutazione Funzionale Mielolesi (VFM) Tufts Assessment of Motor Performance (TAMP) Wheelchair Skills Test (WST)
  3. 3. ICORR 2013 - Seattle TEC2011-29106 Wheelchair metrics There are well known questionnaires that can be applied to any situation; e.g.: ● TLX (General): overall workloadl task difficulty, time pressure, performance, mental/sensory effort, physical effort, frustration level, stress level, fatigue, activity type. ● PIADS (Assistive devices): self-esteem, well-being, quality of life, embarrassment, eagerness to try new things … ● MPT (Assistive technology): people’s condition, ADL, opinion on technology ● FEW (wheelchair users): ergonomics ● QUEST (wheelchair users): usability, weight or dimensions of the device, service delivery, repair and servicing ... … but they are usually subjetive and too general i.e. users give opinions and they do not need to provide information on their specific condition QUEST PIADS
  4. 4. ICORR 2013 - Seattle TEC2011-29106 Condition metrics Questionnaires can be (and usually are) complemented with clinical scales to characterize the user ; e.g.: ● (MMSE) or Folstein (Cognitive): quantitative measure of cognitive status in adults ● GDS (Cognitive): constructed to assess depression and general well-being in the elderly ● Barthel Index (functional): measures how a person’s copes with the activities of daily living ● CIRS (physical): cumulative effect of diverse conditions … but they only provide information about the state of the user at the moment the scale was obtained and each covers specifics, not the global picture. These questionnaires can be complemented with biometrics sensors to obtain information in real time, but wearables are often not comfortable enough to carry for prolongued times and only a few provide stable readings during activity (e.g. pulsioximeters)
  5. 5. ICORR 2013 - Seattle TEC2011-29106 Course-related metrics ●Wheelchair propulsion, assessed in terms of period of time, a fixed distance or longest distance possible. ●Transfer from and to the wheelchair, usually examining the performance in different transfers. ●Negotiation of kerbs, with kerb’s height ranging from 0.025 to 0.15m, and some requiring ascending and descending the kerb. ●Ascending slopes, defined in terms of inclination (ranging from 1 to 11 degrees) and length (ranging from 3 to 21m). ●Traversing tracks, i.e. slalom, figure of eight and obstacle course. ●Sprinting over a fixed distance (ranging from 6.5 to 30m). ●Performing a wheelie. ●Managing brakes, negotiating doors or loading the wheelchair into a car. Typically measured while the user completes a certain preestablished course. Usually very specific and objetive.
  6. 6. ICORR 2013 - Seattle TEC2011-29106 Path-related metrics ● Deviation with respect to the canonical trajectory (often quantified into a few bins as ok, acceptable and wrong) ●Success/failure Typically measured when there is a pre-established visual path to track. Very objetive. Can be combined with task metrics (see next slide)
  7. 7. ICORR 2013 - Seattle TEC2011-29106 Task metrics ●Task success (or) degree of success ●Task completion time ●Number of collisions ●Distance travelled All quantitative measures that can be objectively related to the user’s performance with the wheelchair while performing a given task Global (the task needs to be finished to obtain them) ●Smoothness, related to sharp direction changes ●Safety, related to obstacle proximity ●Directness, related to the skill of keeping directions Local (can be obtained at any given location) ●Time and Distance to Stop (Ts, Ds) ●Time and Distance to Collision (Tc, Dc) ●Obstacle clearance (distance when an obstacle is avoided) Dynamic (the task needs to be finished to obtain them)
  8. 8. ICORR 2013 - Seattle TEC2011-29106 Psych metrics Psych metrics aim at (indirectly) evaluating the user’s (mental) state. These metrics are very dependent on interfaces and ergonomics and, consequently, have been widely used in those fields. ● Interaction number (with the interface)/ amount of information exchanged between user and machine (usability) ● Number of error situations (usability) ● Joystick Variation (more than 10%) (mental workload) ● Intervention level, defined as the portion of time that the user moves a joystick (mental workload) ● Disagreement, defined as the angle between the user command and the wheelchair motor command (frustration, stress) ● Inconsistency, defined as the variation of the user output when facing similar situations (cognitive skills)
  9. 9. ICORR 2013 - Seattle TEC2011-29106 Benchmarking ●It consists of trying to find how a generic user would respond to every faced situation •Generic users do not exist, but they can be modelled from a large enough number of subjects via data mining •If situations are defined at high level, there are infite variations. If they are defined at local level, there are only a few1 1 J. Minguez, J. Osuna, and L. Montanor, “A divide and conquer strategy based on situations to achieve reactive collision avoidance in troublesome scenarios,” IEEE Trans. on Robotics, 2009.
  10. 10. ICORR 2013 - Seattle TEC2011-29106 Data mining Given a large enough sample space, we split it into classes where users respond similarly to similar situation. Their prototypes are equal to the average of all samples within the bin weighted by their respective efficiency, i.e.the best solution taken by the largest number of users
  11. 11. ICORR 2013 - Seattle TEC2011-29106 User profiling and benchmarking Using the proposed prototype user we can ... Subject 14 (Guillain-Barre, MMSE=23, Barthel=28) … see what skills any given user lacks with respect to the average user … see what efficiency a given user has in a task in comparison with the average user Subject 813 (rep. Brain stroke, MMSE=20.3, Barthel=29)
  12. 12. ICORR 2013 - Seattle TEC2011-29106 Conclusions and future work (Standard) Benchmarking procedures and metrics are of key importance in wheelchair navigation Typically we need to combine usability questionnaires, clinical indexes and several metrics to see the bigger picture Profiling helps greatly the analysis of results through personalization Can we use real time metrics to (indirectly predict potential health issues (alarms)? Can we use profiles for better personalization and adaptation of assistance?
  13. 13. ICORR 2013 - Seattle TEC2011-29106 Cristina Urdiales Grupo ISIS (UMA) Metrics and benchmarking for assisted wheelchair navigation
  • ParisiLuca

    Sep. 18, 2014

A brief survey on the different metrics that can be used in the field of assistive wheelchair navigation, including usability questionnaires, clinical scales, task and psych metrics, etc. It also includes obstacle courses and a benchmarking proposal. An invited presentation to ICORR'13 workshop CanWheel (http://www.canwheel.ca/events/upcoming/icorr-2013-workshops/)

Views

Total views

781

On Slideshare

0

From embeds

0

Number of embeds

8

Actions

Downloads

11

Shares

0

Comments

0

Likes

1

×