Improving video rating performance – focusing on user & interface

Improving video rating performance – focusing on User & Interface Geoffrey Yu, ARRB Group Ltd Interaction Design Association (IxDA) Brisbane 21 st June 2011

Welcome and who are we?
Geoffrey Yu – Consultant , Safe Systems, QLD/NT branch
ARRB Group Limited
Note: Formerly known as Australian Road Research Board
Not Assassination Records Review Board (see Wikipedia.org )
Function provide research, consulting and information services to the road and transport industry Expertise road safety , asset management, pavement /structural analysis, car parking, data collection, etc. Clients government/ road authorities (local, state, federal, overseas), int’l aid agencies (World Bank), transport (logistics, mining), etc.

Overview
Context – project domain
Task – project work
User – what is video rating
Interface: Excel – overview of original system
Interface: Raven – overview of new system
Findings – learnings from new system
Ideas? – future thoughts!

Context: road safety
Global issues
About 1.3 million people are killed on world's roads each year
Expected to increase to 1.9 million by 2020
90% occur in developing countries
Source: www.decadeofaction.org , UN’s Decade of Action 2011-2020
Local issues
Last 10 years about 17,000 people killed on Australian roads
Additionally about 220,000 were hospitalised (serious injuries)
Average: 5 deaths and 60 serious injury cases everyday
Source: www.ausrap.org , Australian Road Assessment Program

Task – road assessment
Various methods of assessing road parameters :
Pre => use design plans, analyse historical data
In-situ => onsite inspections and observations
Post => data collection of site, process back in office
Modern data collection equipment allows for efficient and safer assessment of roadways (and is repeatable !)
Typical data collected:
Multi camera (4-5) setup with high resolution (>1080p)
Geotagged video (~10 cm accuracy)
Other equipment (optional/advanced):
Pavement laser scan (road profiling – rutting and roughness)
Environment laser scan (object detection using 3D point cloud)

Hawkeye 2000 Network Survey Vehicle

User – video rating
Use a video viewer to examine/observe road parameters:
Hawkeye Processing Toolkit (ARRB)
DVR Viewer (QDTMR)
AssetLoc (RTA NSW)
Or any geotagged video/images
Parameters are assessed according to project scope :
Crack Trees Poles Asset Safety

Interface – Excel (Original)

Interface – Excel Input form Lookup tables

Design – Excel
Target user and expected usage:
Low computer-level skills
Limited resources (e.g. time) to conduct a risk assessment
Small number of sites or “hot spots” would be conducted
Utilised as a field tool (in-situ inspection) at each site
Market and project changes after initial development:
Road authorities saw benefits in assessing their major roadways
Risk assessments increased in popularity = increased demand
More assessments = more data input = more data processing
New expectations to integrating (e.g. GIS) and presenting results

Primary issues – Excel
Usability
data input errors (input raw calculation values using lookups!)
i.e. high utilisation of user memory
lack of task scope (what roads to rate, completion status)
i.e. poor communication of system status and task
Result => User overwhelmed! => Quality & Quantity reduced
Data management
Multiple raters = multiple spreadsheets (manual sync of data!)
i.e. high levels of data manipulation and handling
Data must be manually validated (e.g. typos not auto checked!)
i.e. data can easily be corrupted (e.g. 1.8 and 18, difference?)
Result => Data is double-handled! => Efficiency reduced

Scope for improvement – Raven
A more efficient and reliable solution was needed due to the increased market demand for risk assessments
Business case was put forward, with internal project approved
Total of 4 months (part-time load) for delivery given
Initial design – 1 month
Coding and development – 2 month
Beta testing – 1 month
Solution was to tackle the primary issues , trying to solve the most problems with minimal effort / least cost (80:20 rule?)
Designed for in-office tool assessing roads at 100 m intervals

Design – Raven System process……Brainstorm issues/solutions……System architecture Etc……

System – Raven Video Server Rater Rater Results

Interface – Raven
Main screen

Interface – Raven
Rating form – midblocks

Key features – Raven
Usability
Familiar standard, business (neutral) looking interface
User task-oriented layout of UI elements
Inclusion of status bar and completion status
Number of perceived rating forms reduced (5 to 2 types)
Input options are explicit and predefined (no lookup tables!)
Data management
Data validation is implicit by providing predefined options
Centralisation of multiple rating sources
Duplicated rating work is reduced (1 rater per road section)
Rating task is verified against a (predefined) rating list (scope)

Effects of using Raven over Excel
User performance
Reduced typos and other user input issues => increase quality
Reduced feeling of being overwhelmed (i.e. memory and layout)
More confident in performing (repetitive) task (system trust)
More confident in task delivery => increase task visibility
Increased user satisfaction (don’t want to use Excel anymore!)
Project performance
Increased rating output 2x (consistently better than Excel)
Reduced data issues (i.e. user errors, duplication and corruption)
Project manager has better control and awareness of task
Reduced overheads/resource waste => increases profit margin!

Learnings & future enhancements
Currently Raven has been used to assess the majority of QLD state and local (regional significance) roads => ~100,000 km !!
Small changes focusing on user task and process issues can positively affect user performance and project outcomes
Raven was a prototype / proof of concept
can rating performance be improved (primary issues)?
Raven has minor bugs => minimal affect on usability and project outcomes (for now!!)
Other features/improvements that were not implemented:
Mass data editor (table/spreadsheet to fix data issues)
Highlight (colour) rating issues (increase user communication)
Help / reference system (rating guide / issues)
Reporting engine (risk charts, risk maps, etc.)

Other ideas?
Integration of other data
GIS (Shapefiles, Google maps, etc.)
Traffic data (crash, volume, etc.)
Template rating form => flexible for other assessments
Highlight image regions for assessment (rating assistance)
Automatic video/image detection (machine vision)

Road ends here? Thank you and safe driving!

Improving video rating performance – focusing on user & interface

by Ben Melbourne

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Improving video rating performance – focusing on user & interface Presentation Transcript