Nikolaos Kaklanis - PhD presentation

University of Surrey
Department of Computing
A COUPLED USER AND TASK MODELLING
METHODOLOGY FOR ACCESSIBLE
PRODUCT DESIGN
PhD Thesis
Nikolaos Kaklanis
Information & Communication Systems Engineer, MSc

• Disability is part of the human condition
– Almost everyone will be temporarily or permanently impaired
at some point in life.
• Aging is strongly connected with difficulties in
functioning
• The lowest estimate, based on the currently defined
disablement categories, estimates their total number at
around 74 Million persons, in the European Union.
• Many products & services are inaccessible for people
with functional limitations
The need: Accessible & ergonomic products and
services for the elderly and disabled
THE NEEDTHE NEED

TYPICAL DEVELOPMENT CHAINTYPICAL DEVELOPMENT CHAIN

DEVELOPMENT CHAIN USINGDEVELOPMENT CHAIN USING VIRTUAL USER MODELSVIRTUAL USER MODELS
The
Context of the introduced research

A new coupled user-task modeling methodology
is needed, which will enable
– the detailed description of users with disabilities
– the development of Virtual User Models (VUMs)
representing specific population groups
PROBLEM IDENTIFICATIONPROBLEM IDENTIFICATION

• A novel Virtual User Model (VUM) enabling the detailed description
of the physical, cognitive & behavioural user characteristics with special
focus on the elderly and people with functional limitations.
• An innovative methodology (based on statistical analysis) for the
development of Virtual User Models that represent specific disabled
population groups.
• A framework for the adoption of virtual user models within
accessibility simulation and ergonomy assessment processes of virtual
designs.
• Design and development of tools for supporting and putting the
aforementioned methodologies and frameworks into practice.
RESEARCH CONTRIBUTIONRESEARCH CONTRIBUTION

STATE OF THE ART IN VIRTUAL USER MODELINGSTATE OF THE ART IN VIRTUAL USER MODELING
• Ontology-based models
– OntobUM (Razmerita et al., 2003)
– GUMO (Heckmann, 2006)
– VICON User Model (Pierre et al., 2011)
• XML-based models
– UserML (Heckmann, 2003)
– GUIDE User Model (Biswas & Langdon, 2012)
– User Models using relational databases
– MyUI User Model (Peissner et al. 2012)
• Personas (Cooper, 1999; Goodwin, 2002; Nielsen, 2002 & 2003; Pruitt & Grudin, 2003)
• Task modeling approaches
• ConcurTaskTree (Paternò, 1999)
• Hierarchical Task Analysis (HTA) (Annett & Duncan, 1967)
• UsiXML (Vanderdonckt, 2005)

LIMITATIONS OF EXISTING APPROACHESLIMITATIONS OF EXISTING APPROACHES
Existing approach/methodology Limitations
Personas Not expressed in a machine-readable format
Focused on specific body parts
Cannot represent in detail physical, cognitive &
behavioural characteristics
VUMs in machine-readable
format
Disabilities not supported
Focus on the progression of the disabilityUser modelling methodologies
Cannot create VUMs representing specific
population groups
User is defined independently from tasksUser & task modelling
methodologies
It’s not clear which tasks are affected by the
disabilities

• Introduction
•An innovative user
modeling
methodology focused
on the elderly and
disabled
• A methodology for
creating virtual user
models representing
specific population
groups
• The proposed user
modeling methodology
in practice
• Conclusions
An innovative user modeling
methodology focused on the
elderly and disabled

The proposed framework is based on the following seven
major building blocks:
• Abstract User Models
– high level description of potential user models
• Generic Virtual User Models
– a Generic Virtual User Model (GVUM) describes a set of users having a
specific set of disabilities
• Instance of a Generic Virtual User Model
– an instance of a virtual user (e.g., Persona).
AN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLEDAN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLED

• Primitive Tasks
– primitive human actions
• Task Models
– actions that are being systematically performed in the context of the virtual
prototype to be tested
• Multimodal Interaction Models
– alternative ways of a primitive task’s execution
• alternative modalities
• assistive devices
• Simulation Models
– simulation scenario
AN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLEDAN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLED

ARCHITECTURE OF THE PROPOSED FRAMEWORKARCHITECTURE OF THE PROPOSED FRAMEWORK

• Primitive tasks
– define primitive human actions
– are related to the disability
category
• limited but sufficient number
of primitive tasks
PRIMITIVE TASKSPRIMITIVE TASKS
Primitive task’s
category
Primitive
task
Motor Push
Motor Grasp
Cognitive Select
Cognitive Read

• User tasks are divided in two categories:
– a) primitive (e.g., grasp, pull, walk, etc.)
– b) complex (e.g., driving, telephone use, computer use, etc.)
• For each complex task, a Task Model is developed
– to specify how the complex task can be analyzed into
primitive tasks.
TASK MODELSTASK MODELS
COMPLEX TASK PRIMITIVE TASK BODY PART OBJECT
CLOSE CAR DOOR WHILE
SEATED
REACH ARM DOOR
GRASP HAND INTERIOR DOOR
HANDLE
PULL HAND INTERIOR DOOR
HANDLE
PUSH HAND LOCK BUTTON

• UsiXML’s taskModel
TASK MODELS - IMPLEMENTATIONTASK MODELS - IMPLEMENTATION
<?xml version="1.0" encoding="UTF-8"?>
<taskmodel>
<task id="st0task0" name="Close_car_door_while_seated" type="abstraction">
<task id="st0task1" name="Reach_door_with_arm" type="interaction"/>
<task id="st0task2" name="Grasp_interior_door_handle_with_hand" type="interaction"/>
<task id="st0task3" name="Pull_interior_door_handle_with_hand" type="interaction"/>
<task id="st0task4" name="Push_lock_button_with_hand" type="interaction"/>
</task>
……
</taskmodel>
UsiXML source code

• Alternative ways of a primitive task’s execution with respect to
• the different target user groups
• the replacement modalities
• the possible use of assistive devices.
MULTIMODAL INTERACTION MODELSMULTIMODAL INTERACTION MODELS
Task Body
part
Modality Task
object
Disability Alternative
task(s)
Body part Alternative
modality
Alternative
task object/
assistive
device
Pull Hand Motor Interior
door
handle
Upper limb
impaired
Speak Mouth Voice control Voice
activated
doors
Push Hand/Elbo
w
Motor Button that
closes door

MULTIMODAL INTERACTION MODELS - IMPLEMENTATIONMULTIMODAL INTERACTION MODELS - IMPLEMENTATION
<taskmodel>
<task id="st0task0" name="Pull_interior_door_handle" type="abstraction">
<task id="st0task1"
name="Pull(modality:motor)(means:hand)(object:interior_door_handle)" type="interaction"/>
<task id="st0task2"
name="Speak(modality:voice)(means:mouth)(object:voice_activated_doors)" type="interaction"/>
<task id="st0task3"
name="Push(modality:motor)(means:hand,elbow)(object:button_that_closes_door)" type="interaction"/>
</task>
<deterministicChoice>
<source sourceId="st0task1"/>
<target targetId="st0task2"/>
</deterministicChoice>
<deterministicChoice>
<source sourceId="st0task2"/>
<target targetId="st0task3"/>
</deterministicChoice>
</taskmodel>
UsiXML source code

• A high level description
of potential user models
ABSTRACT USER MODELSABSTRACT USER MODELS
Disability
category
Disability
Short
description
Quantitative disability metrics
Functional
limitations
(ICF
Classification)
Age-
relate
d
Motor Spinal
cord
injuries
(Thoracic
injuries)
Spinal cord
injuries
cause
myelopathy
or damage to
nerve roots
or
myelinated
fiber tracts
that carry
signals to
and from the
brain.
Temporal gait variables:
-Gait Cycle (sec):2.17 (1.05)
-Cadence (steps/min): 65.0 (23.1)
-Double support (%): 42.8 (10.2)
-Stride (m): 0.48 (0.13)
-Velocity ((m/sec)/height): 0.27 (0.13)
S120 Spinal
cord and
related
structures,
S1200
Structure of
spinal cord,
S12000
Cervical spinal
cord,
s12001
Thoracic spinal
cord,
s12002
Lumbosacral
spinal cord,
s12008
Structure of
spinal cord,
other specified,
s12009
Structure of
spinal cord
unspecified,
Could
be
Kinematic variables:
-Hip excursion (˚): 39.3 (9.0)
-Knee excursion (˚): 38.1 (13.2)
-Ankle excursion (˚): 25.0 (4.9)
-Hip velocity (˚/sec): 38.2 (17.5)
-Knee velocity (flexion) (˚/sec): 64.1 (41.8)
-Knee velocity (extension) (˚/sec): 83.8 (54.2)
-Ankle velocity (˚/sec): 48.1 (30.8)

• Refers to a class of virtual users exhibiting one or more
specific disabilities
• Describes the tasks affected by the disabilities and their
associated disability-related parameters.
GENERIC VIRTUAL USER MODELSGENERIC VIRTUAL USER MODELS
Disability
category
Disability Affected primitive
tasks
Affected primitive tasks’ parameters
Grasp The user is able to grasp objects, with
size <= 3cm x 3cm x 3cm
Pull The user can pull an object with max_Force: 5N
Gait velocity ranges from 0.18 to 1.03 m/sec
Motor Hemiplegia
Walk
Abnormal step rhythm

GENERIC VIRTUAL USER MODELS – SUPPORTED CHARACTERISTICSGENERIC VIRTUAL USER MODELS – SUPPORTED CHARACTERISTICS

GENERIC VIRTUAL USER MODELS - IMPLEMENTATIONGENERIC VIRTUAL USER MODELS - IMPLEMENTATION

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BASIC CONTAINERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BASIC CONTAINERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEAD ELEMENT)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEAD ELEMENT)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (DISABILITY MODEL)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (DISABILITY MODEL)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PREFERENCES)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PREFERENCES)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (ANTHROPOMETRIC PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (ANTHROPOMETRIC PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (UPPER LIMB PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (UPPER LIMB PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (LOWER LIMB PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (LOWER LIMB PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (NECK PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (NECK PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPINAL COLUMN PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPINAL COLUMN PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GAIT PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GAIT PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (VISUAL PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (VISUAL PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEARING PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEARING PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPEECH PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPEECH PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (COGNITIVE PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (COGNITIVE PARAMETERS)

GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BEHAVIOURAL PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BEHAVIOURAL PARAMETERS)

INSTANCES OF GENERIC VIRTUAL USER MODELSINSTANCES OF GENERIC VIRTUAL USER MODELS
• A specific virtual user with specific disability related parameters including:
– disabilities
– affected primitive tasks
User ID Disability
category
Disability Affected primitive
tasks
Affected primitive tasks’ parameters
Grasp The user is able to grasp objects, with
size <= 2.5cm x 2.5cm x 2.5cm
Pull The user can pull an object with
max_Force: 3N
Gait velocity : 0.9 m/s
User 1 Motor Hemiplegia
Walk
Abnormal step rhythm

INSTANCES OF GENERIC VIRTUAL USER MODELS - IMPLEMENTATIONINSTANCES OF GENERIC VIRTUAL USER MODELS - IMPLEMENTATION
<userModel xmlns="http://www.usixml.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-
instance" xsi:schemaLocation="http://www.usixml.org/spec/UsiXML-ui_model.xsd"
id="User_Model" name="Adam Brown" creationDate="2013/02/02
20:31:06" schemaVersion="1.8.0">
<head>
<version modifDate="2010/10/17 20:31:06">0.0</version>
<authorName>Automatically generated byethe User Model Generator</authorName>
<comment>This model has been generated using the User Model Generator</comment>
</head>
<disabilityModel>
<disability name="Spinal cord injuries" ageRelated="true" typesICF="b147, b760,
b7603, b780">
<disabilityDetails>Spinal cord injuries cause myelopathy or damage to
nerve roots or myelinated fiber tracts that carry signals to and from
the brain. Depending on its classification and severity, this type of
traumatic injury could also damage the grey matter in the central part
of the cord, causing segmental losses of interneurons and motorneurons.
</disabilityDetails>
<affectedTasks>
<affectedTask idTask="walking_ID" type="motor" name="walking"
details="inability to effectively transfer weight
between
legs, abnormal step rhythm, excessive plantar flexion during swing
phase, falling during activities" />
<affectedTask idTask="driving_ID" type="motor"
name="driving"
details="automatic transmission and buttons on the
steering wheel
instead of pedals for brake and accelerator needed" />
</affectedTasks>
</disability>
</disabilityModel>
<capabilityModel>
<general>
<gender>male</gender>
<ageGroup>30-49</ageGroup>
</general>
<generalPreferences />
<anthropometric>
<weight measureUnits="kgr" value="81.023056"/>
<stature measureUnits="cm" value="174.586060"/>
<headLength measureUnits="cm" value="19.533623"/>
…
GVUM: range of values
Instance of a GVUM: absolute values
while…

COMPARISON BETWEEN THE PROPOSED VUM & EXISTING MODELSCOMPARISON BETWEEN THE PROPOSED VUM & EXISTING MODELS
Number of variables
of the proposed VUM
Number of variables
of the GUIDE VUM
Number of variables
of the MyUI VUM
Number of variables
of the VICON VUM
• total: over 250 • total: 27
• included in the
proposed VUM: 21
• total: 30
• included in the
proposed VUM: 10
• total: 33
• included in the
proposed VUM: 11
Why some variables of the others VUMs do not exist in the proposed
VUM?
• they describe qualitative (not quantitative) characteristics and, thus,
cannot be directly simulated (e.g., ability to move the hand precisely)
• describe the result of the simulation process (e.g., number of successful
interactions with the system)
• are not actually user characteristics (e.g., amount of ambient light at the
users place)

• The scenario to be followed during the simulation process
• A Simulation Model may include three different types of tasks:
– a) abstract tasks
– b) interaction tasks
– c) application tasks
SIMULATION MODELSSIMULATION MODELS
Scenario Main tasks Subtasks
Pull handbrakeUse handbrake
Release handbrake
Open storage
compartment
Automotive simulation:
assess the accessibility of the
handbrake and the storage
compartment Use storage
compartment
Close storage
compartment

SIMULATION MODELS - IMPLEMENTATIONSIMULATION MODELS - IMPLEMENTATION

• Introduction
modeling
methodology focused
on the elderly and
disabled
models representing
specific population
groups
in practice
• Conclusions
• The proposed methodology offers
• detailed definition of
• user (including disabilities)
• user tasks
• simulation scenarios
• Mapping/linkage between all these
definitions

• Introduction
focused on the elderly
and disabled
models representing
specific population
groups
in practice
• Conclusions
A methodology for creating
virtual user models
representing specific
population groups

1. Selection of disability parameters
– trying to model the most characteristic functional limitations (Abstract User
Model) caused by each disability
• e.g., disability parameters for Gonarthrosis (osteoarthritis of the knee) include: knee
flexion/extension and gait parameters, like cadence, step length, etc.
2. Acquire raw data (real measurements)
3. Estimate the probability density function (pdf) of each disability
parameter
• Parametric regression
• Non-parametric regression
• An innovative hybrid regression method able to handle small sample sizes
• All the above regression methods are applied and the one with the smallest Mean
Squared Error (MSE) is chosen as the most suitable
4. Validation and optimization of the regression model
– Bootstrap method (Adèr et al., 2008)
MODELING A DISABILITY – GENERAL STEPSMODELING A DISABILITY – GENERAL STEPS

STEP 2: MEASUREMENTS CONDUCTEDSTEP 2: MEASUREMENTS CONDUCTED
• Measurements were obtained from total 164 subjects
• Parkinson’s disease (17 subjects)
• Stroke (36 subjects)
• Multiple Sclerosis (20 subjects)
• Cerebral Palsy (17 subjects)
• Coxarthrosis (7 subjects)
• Gonarthrosis (5 subjects)
• Lower Limb Amputation (3 subjects)
• Elderly people (59 subjects)

STEP3: PARAMETRIC REGRESSIONSTEP3: PARAMETRIC REGRESSION
2
2
2
)(
2
2
1
)( σ
µ
πσ
−
−
=
x
exf
Distribution Type PDF Example
Gaussian Hip flexion of
the elderly
Distribution Type PDF Example
Gaussian mixture Fingers’ extension
of people with
stroke
Distribution
Type
PDF Example
Exponential Gait cycle
of stroke
patients
Distribution
Type
PDF Example
Lévy Double support
of stroke
patients
Distribution
Type
PDF Example
Uniform Gait velocity of
patients with
cerebral palsy
∑=
−
−








⋅=
n
i
x
i
i
i
i
ewxf
1
2
)(
2
2
2
2
1
)( σ
µ
πσ
θ
θ
x
k
ex
kk
xf
−
−
Γ
= 1
)()(
1
)(
2
3
)(2
)(2
)(
µπ
µ
−
=
−
−
x
ec
xf
x
c




∈
−=
elsewhere
bax
abxf
,0
],[,
1
)(

• PDF is unknown
STEP3: NON-PARAMETRIC REGRESSIONSTEP3: NON-PARAMETRIC REGRESSION
Kernel regression
Knee angular velocity of the stroke patients
Local polynomial multiple regression
Step width of the elderly
•Seems to follow a Lévy distribution.
•This can be proved by examining the MSE resulting
from parametric regression:
•Lévy: MSE: 0.00204
•Gaussian: MSE: 0.00256
•Gaussian mixture: MSE: 0.00290
•Uniform: MSE: 0.00367
•Exponential: MSE: 0.07297
•Seems to follow a Gaussian distribution.
•This can be proved by examining the MSE resulting
from parametric regression:
•Gaussian: MSE: 0.94464
•Gaussian mixture: MSE: 1.02365
•Uniform: MSE: 1.26263
•Exponential: MSE: 1.29327
•Lévy: MSE: 2.69961

• Sample size is important!
• BUT…the task of taking real measurements from disabled people is
– very difficult
– needs much resources
• Estimating the PDF of a disability parameter using a limited set of
subjects could result in great variance.
We introduce the concept of hybrid regression that exploits
– information found on the literature
– real measurements
– virtual measurements
STEP3: HYBRID REGRESSIONSTEP3: HYBRID REGRESSION
Solution

• The probability distribution followed by a disability parameter is
defined using an ε-contaminated class of priors, namely Γ.
– The contaminated class controls the balance of the determinate
priors, , and other possible priors, , through an ε-level of weight.
• The ε-contaminated class is formulated as:
– where D is our initial data set and
– ε denotes the level of uncertainty
HYBRID REGRESSION –HYBRID REGRESSION – Ε-Ε-CONTAMINATED CLASS OF PRIORSCONTAMINATED CLASS OF PRIORS
s'0π
Parametric regression
on the original sample
User defined
We tested the following values:
0, 0.01, 0.05, 0.10, 0.20
Gaussian

• Let variable Y represent the real measurements
• Assumption: Y is be the response of the simple linear regression
model
– where is the predictor value
– is the intercept,
– is the slope and
– are independent and identically distributed random variables from
HYBRID REGRESSION – LINEARITY ASSUMPTIONHYBRID REGRESSION – LINEARITY ASSUMPTION
iii XY εββο ++= 1
iX
0β
1β

• Our initial data set consists of measurements , where:
– each is a real measurement
– ‘s are calculated as follows:
• from Gaussian pdf found in the literature
– to keep a relevance between ‘s and ‘s, we get random samples ( ‘s) that meet
• When there is no available information in the literature, we calculate the ‘s using
the following formula:
With this process, ‘s are actually coming from a random movement (within a
specific range) of the initial samples ‘s.
HYBRID REGRESSION – INITIAL DATA SETHYBRID REGRESSION – INITIAL DATA SET
),( 2
litlitN σµ
iY iX iX
4
10−
≤−−− YYX litii µ
iX
kY
kYY
X
i
i
⋅
⋅−
=
iX
iY
Random value
within [0.95, 1.05]

• A virtual data set consisting of m pairs is constructed
as follows:
– ‘s are random samples from the pdf of ‘s.
– ‘s are random samples of , assuming that is a Gaussian
distribution with
and
where and
HYBRID REGRESSION – VIRTUAL SAMPLESHYBRID REGRESSION – VIRTUAL SAMPLES
A
D
A
jX iX
A
jY Γ 0π
AA
Y
XYA
0
1
0
0 ββµ +== ∑=
−−==
n
i
iiY
xy
n
DMSEA
1
20
1
0
0
2
)(
1
)( ββσ
∑ ∑
∑∑∑
= =
===






−
−
=
n
i
n
i
ii
n
i
i
n
i
i
n
i
ii
X
n
X
YX
n
YX
1
2
1
2
1110
1
1
1
β XY 0
1
0
0 ββ −=
),( A
j
A
j XY
User-selected value
We tested the following values:
0, 0.5n, n, 2n, 3n

• At this stage, we have available:
– The initial data set
– A virtual data set
• Both data sets are used for the fine tuning of the parameters of
, which describes the distribution of disability parameter .
• Now, for the definition of , we perform Gaussian parametric
regression on the real measurements (Yi’s).
HYBRID REGRESSION – FINE TUNING OF PARAMETERS OFHYBRID REGRESSION – FINE TUNING OF PARAMETERS OF ΓΓ
A
D
D
Γ y
0π
This is what we are
searching for…

• The maximum likelihood estimate taken into account both the
original and the virtual sample sets ( and , correspondingly)
is given by:
• Using the EM algorithm for parameter estimation concludes after
k iterations with and , which are used for the optimisation
of the Gaussian distribution .
– More specifically,
• now becomes
• and becomes
HYBRID REGRESSION – EM ALGORITHMHYBRID REGRESSION – EM ALGORITHM
D A
D
∏∏
+
+==
=
mn
ni
A
i
n
i
iMLE DyDy
11
, )|()|(maxarg ππβ
β
π
k
0β k
1β
0π
AA
Y
XYA
0
1
0
0 ββµ +== AkkA
Y
XYA 10 ββµ +==
∑=
−−==
n
i
iiY
xy
n
DMSEA
1
20
1
0
0
2
)(
1
)( ββσ ∑=
−−==
n
i
i
kk
iY
xy
n
DMSEA
1
2
10
2
)(
1
)( ββσ

• The combination of
– ε (uncertainty) and
– m (number of virtual samples)
that results to the minimum MSE is used to form
the final Γ.
HUBRID REGRESSION – RESULTEDHUBRID REGRESSION – RESULTED ΓΓ

HYBRID REGRESSION EXAMPLE - STEP WIDTH OF THE ELDERLYHYBRID REGRESSION EXAMPLE - STEP WIDTH OF THE ELDERLY
Purple line: data taken from the MSP
Green line: virtual samples generated considering the MSP data.
Yellow line: virtual samples generated considering the data coming from the literature
Light blue line: pdf reported in the literature
Dark blue line: represents all the samples (both real and virtual)
Red line: final estimated pdf
Regression type MSE
Parametric – Gaussian 0.74377
Parametric – Lévy 2.73860
Parametric – Gaussian
mixture
0.91292
Parametric – Exponential 1.46577
Parametric – Uniform 1.13034
Nonparametric – Kernel 0.5102
Nonparametric –
Polynomial
0.6414
Hybrid 0.00642

• The definition of , enables the calculation of
disability parameter values for different population
groups.
– e.g. , to find the value of wrist flexion that corresponds to the
90% of people with arthritis, we have:
CALCULATING THE DISABILITY PARAMETER VALUES FROM THE ESTIMATED PDFCALCULATING THE DISABILITY PARAMETER VALUES FROM THE ESTIMATED PDF
∫ −=
b
a
jijiijijji aFbFdxxf )()()( ,,,
jiF ,
9.0)0()( ,, =− onwristFlexiarthritisonwristFlexiarthritis FkF
PDF of parameter j of
disability i
Finite integral:
The value of wrist flexion
corresponding to the 90% of
people with arthritis

REGRESSION MODELS - INDICATIVE RESULTS (STROKE PATIENTS)REGRESSION MODELS - INDICATIVE RESULTS (STROKE PATIENTS)
Parameter
Name
Before validation & optimisation After optimisation
Regression
Type
Distribution Type
PDF
Parameters
Distribution Type
PDF
Parameters
Gait Cycle Parametric
Levy
μ = 0,918
c = 0,29893
Gaussian
μ = 1,81818
σ = 0,609692
Push
Force
Hybrid
Π0
(Gaussian):
μ=65,47
σ=67,54
q
(Exponential):
λ=0,02
ε = 0,05
Gaussian
μ = 78,5959
σ = 61,9811
Hip
Extension
Right
Parametric
Gaussian Mixture
μ1=32,81111
σ1=5,10935
w1=0,34615
μ2=15,82353
σ2=4,21936
w2=0,65385
Gaussian Mixture
μ1 = 32,597
σ1 = 5,682615
w1 = 0,343074
μ2 = 15,86191
σ2 = 4,316446
w2 = 0,656926

• Introduction
modeling
methodology focused
on the elderly and
disabled
models representing
specific population
groups
modeling
methodology in
practice
• Conclusions
The proposed user modeling
methodology in practice

• VERITAS simulation
platform
• Automatic simulated
accessibility and
ergonomy assessment
of designs
• Its development was
based on the proposed
user modeling
methodology
ACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNSACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNS
Physical characteristics Normal values Elderly (60 to 84) Rheumatoid arthritis
Hand maximum pull force (N) 335 76.8
Wrist extension (°) 0 - 60
Wrist radial deviation (°) 0 - 27.5 0 – 19
Wrist ulnar deviation (°) 0 – 35 0 – 26
Forearm supination (°) 0 – 85 0 – 74
Forearm pronation (°) 0 – 85 0 – 71
Elbow flexion (°) 0 – 142.5
Elbow hyper-extension (°) 0 – 10 0 – 4
Shoulder flexion (°) 0 – 160 0 – 10
Shoulder abduction (°) 0 – 85 0 – 67 0 – 15
Shoulder internal rotation (°) 0 – 80 0 – 63
Shoulder external rotation (°) 0 – 45 0 - 15
Spinal column flexion (°) 0 - 90 0 – 23.6
Spinal column extension (°) 0 – 30 0 - 17
Spinal column left lateral flexion (°) 0 – 25 0 – 19
Spinal column right lateral flexion (°) 0 – 25 0 - 20

ACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNS – STAPLER USEACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNS – STAPLER USE
Stapler torque-resistance (Nm) Normal Elderly Rheumatoid arthritis
2.5 Pass Pass Pass
5.0 Pass Pass Pass
10 Pass Pass Pass
15 Pass Fail Pass
20 Pass Fail Fail
25 Pass Fail Fail
30 Fail Fail Fail

• DIAS tool: Impairments
simulator
• It has been extended to
support the proposed VUMs.
IMPAIRMENT SIMULATION OVER JAVA, MOBILE AND WEB APPLICATIONSIMPAIRMENT SIMULATION OVER JAVA, MOBILE AND WEB APPLICATIONS

• ACT-R: a cognitive
architecture showing how
human cognition works.
• The proposed VUM includes
ACT-R parameters.
• Reading task example
– a non-stressed user
• visual-attention-latency: 0.085 sec
• reading task – total time: 3.57 sec
– a user with acute stress
• visual-attention-latency: 0.0978 sec
• reading task – total time: 3.726 sec
COGNITIVE SIMULATION – THE PROPOSED VUMs WITHIN ACT-RCOGNITIVE SIMULATION – THE PROPOSED VUMs WITHIN ACT-R

• Introduction
•A new User Model
able to describe
elderly and disabled
people
focused on the elderly
and disabled
models representing
specific population
groups
in practice
• Conclusions
Conclusions – Future Work

1. A new Virtual User Model was introduced, able to describe a variety of
user characteristics, including physical, cognitive and behavioural.
2. This Virtual User Model is a part of an innovative coupled user and task
modeling methodology that aims to be used mainly in simulation
frameworks performing accessibility assessment of virtual designs.
3. A methodology for creating virtual user models able to represent
specific disabled population groups
4. Tools for putting the proposed user modeling methodology into practice
CONCLUSIONSCONCLUSIONS

• The proposed VUM goes one step beyond the state-of-the-art, as it
enables the detailed description of the functioning of the elderly and
disabled
• The proposed coupled user & task modeling methodology can be a
valuable tool for the designers/developers towards the development of
accessible products
• The proposed methodology for creating virtual user models able to
represent specific disabled population groups offers designers with
the opportunity to define an accessibility threshold for their designs.
IMPORTANCE OF THE RESEARCH CONTRIBUTIONIMPORTANCE OF THE RESEARCH CONTRIBUTION

• The accuracy of a VUM representing a specific population group
depends on the sample size.
• However, the proposed methodology can result to the production
of very accurate personas.
CRITICAL DISCUSSIONCRITICAL DISCUSSION

• Fine-tuning of the resulted regression models using
• more data coming from the literature
• more real measurements
• More disabilities could be modeled
• A coupled physical-cognitive user modeling approach could be
considered
• Continue the dissemination through
– the VUMS cluster of projects (http://www.veritas-project.eu/vums/)
– the activities of the W3C MBUI Working Group
FUTURE WORKFUTURE WORK

1. Kaklanis, N., Moschonas, P., Moustakas, K., Tzovaras, D., “Virtual User Models for the elderly and disabled
for automatic simulated accessibility and ergonomy evaluation of designs”, Universal Access in the
Information Society, Special Issue: Accessibility aspects in UIDLs, Springer
2. Giakoumis, D., Kaklanis, N., Votis, K., Tzovaras, D., “Enabling user interface developers to understand
accessibility limitations through visual, hearing, physical and cognitive impairment simulation”, Universal
Access in the Information Society, Springer
3. Kaklanis, N., Votis, K., Tzovaras, D., ”Open Touch/Sound Maps: A system to convey street data through
haptic and auditory feedback”, Computers & Geosciences, Elsevier (accepted for publication)
4. Kaklanis, N., Votis, K., Tzovaras, D., “An online multimodal audio-haptic framework for visually impaired
users accessing OpenStreetMap data”, Universal Access in the Information Society, Special Issue: 3rd
generation accessibility: Information and Communication Technologies towards universal access, Springer
(under review)
5. Kaklanis, N., Biswas, P., Mohamad, Y., Gonzalez, M.F., Peissner, M., Langdon, P., Tzovaras, D., Jung, C,
“Towards Standardization of User Models for Simulation and Adaptation Purposes”, Universal Access in the
Information Society, Special Issue: 3rd generation accessibility: Information and Communication
Technologies towards universal access, Springer (under review)
6. Koutkias, V., Kaklanis, N., Votis, K., Tzovaras, D., Maglaveras, N., “An Integrated Semantic Framework
Supporting Universal Accessibility to ICT”, Universal Access in the Information Society, Special Issue: 3rd
generation accessibility: Information and Communication Technologies towards universal access, Springer
(under review)
7. Kaklanis, N., Stavropoulos, G., Tzovaras, D., “Modeling disabilities through regression analysis for the
development of accurate virtual user models”, User Modeling and User-Adapted Interaction, The Journal of
Personalization Research (under review)
PUBLICATIONS – SCIENTIFIC JOURNALSPUBLICATIONS – SCIENTIFIC JOURNALS

8. Kaklanis, N., Votis, K., Moustakas, K., Tzovaras, D. 3D HapticWebBrowser: Towards Universal Web
Navigation for the Visually Impaired, 7th International Conference on Web Accessibility, W4A, 26-27 April
2010, Raleigh, North Carolina, USA
This paper won the "Judges Award” at the “Web Accessibility Challenge” (sponsored by Microsoft)
9. Kaklanis, N., Moustakas, K., Votis, K., Tzovaras, D. A framework for accessibility testing of virtual
environments based on UsiXML”, ACM SIGCHI Symposium on Engineering Interactive Computing
Systems, UsiXML-EICS 2010, Berlin, June 2010
10. Kaklanis, N. Moschonas, P., Moustakas, K., Tzovaras, D., Enforcing accessible design of products and
services through simulated accessibility evaluation, CONFIDENCE 2010, Jyväskylä, Finland, 9-10 December
2010
11. Kaklanis, N., Moustakas, K., Tzovaras, D. A framework for automatic simulated accessibility assessment in
virtual environments, HCI International 2011, Orlando, Florida, USA, 9-14 July 2011
12. Kaklanis, N., Votis, K., Moschonas, P., Tzovaras, D. HapticRiaMaps: Towards Interactive exploration of
Web World maps for the Visually Impaired”, W4A 2011, Hyderabad, India, March 2011
13. Oikonomou, T., Kaklanis, N., Votis, K., Kastori, G.E., Partarakis, N., Tzovaras, D. WaaT: Personalised Web
Accessibility Evaluation Tool, W4A 2011, Hyderabad, India, March 2011
14. Oikonomou, T., Kaklanis, N., Votis, K., Tzovaras, D. An Accessibility Assessment Framework for
Improving Designers Experience in Web Applications, HCII 2011, Orlando, Florida, USA, July 2011
15. Partarakis, N., Doulgeraki, C., Antona, M., Oikonomou, T., Kaklanis, N., Votis, K., Kastori, G.E., Tzovaras,
D. A Unified Environment for Accessing a Suite of Accessibility Evaluation Facilities, HCII 2011, Orlando,
Florida, USA, July 2011
16. Moschonas, P., Kaklanis, N., Tsakiris, A., Moustakas, K., Tzovaras, D., “An open simulation framework for
automated and immersive accessibility engineering”, 6th Cambridge workshop on universal access and
assistive technology, CWUAAT 2012, Fitzwilliam College, University of Cambridge, March, 2012
PUBLICATIONS – INTERNATIONAL CONFERENCESPUBLICATIONS – INTERNATIONAL CONFERENCES

17. Korn, P., Kaklanis, N., Votis, K., Tzovaras, D., “Using the AEGIS OAF: making an accessible RIA”, 27th
Annual International Technology and Persons with Disabilities Conference, CSUN 2012, March, 2012, San
Diego, CA
18. Moschonas, P., Kaklanis, N., Tzovaras, D., “Novel Human Factors for Ergonomy Evaluation in Virtual
Environments Using Virtual User Models”, 10th International Conference on Virtual Reality Continuum and
its Applications in Industry, VRCAI’ 2011, December, 2011, Hong Kong, China
19. Kaklanis, N., Moustakas, K., Tzovaras, D., "An extension of UsiXML enabling the detailed description of
users including elderly and disabled", in International Workshop on Software Support for User Interface
Description Language, Interact 2011, Lisbon, September, 2011
20. Mohamad, Y., Biswas, P., Langdon, P., Peissner, M., Dangelmaier, M., Jung, C., Wolf P., Kaklanis, N.,
“Standardisation of user models for designing and using inclusive products”, Joint Virtual Reality Conference
(JVRC), 20-21 September, 2011, Nottingham, UK
21. Kaklanis, N., Moustakas, K., Tzovaras, D., 2012. A methodology for generating virtual user models of
elderly and disabled for the accessibility assessment of new products. In Proceedings of the 13th
international conference on Computers Helping People with Special Needs - Volume Part I (ICCHP'12),
Klaus Miesenberger, Arthur Karshmer, Petr Penaz, and Wolfgang Zagler (Eds.), Vol. Part I. Springer-Verlag,
Berlin, Heidelberg, 295-302. DOI=10.1007/978-3-642-31522-0_44
http://dx.doi.org/10.1007/978-3-642-31522-0_44
22. Moschonas, P., Tsakiris., A., Kaklanis, N., Stavropoulos, G., Tzovaras, D., “Holistic accessibility evaluation
using VR simulation of users with special needs”, UMAP 2012, Montreal, Canada, July, 2012
23. Kaklanis, N., Mohamad, Y., Peissner, M., Biswas, P., Langdon, P., Tzovaras, D., ”An Interoperable and
Inclusive User Modelling concept for Simulation and Adaptation”, UMAP 2012, Montreal, Canada, July,
2012
24. Kaklanis, N., Votis, K., Tzovaras, D., “Touching OpenStreetMap Data in Mobile Context for the Visually

25. Kaklanis, N., Votis, K., Tzovaras, D., “A Mobile Interactive Maps Application for a Visually Impaired
Audience”, The Paciello Group Web Accessibility Challenge - W4A 2013, 10th International Cross-
Disciplinary Conference on Web Accessibility, 13-15 May 2013, Rio de Janeiro, Brazil (accepted)
26. Tsakiris, A., Kaklanis, N., Paliokas, I., Stavropoulos G., and Tzovaras, D. Cognitive Impairments Simulation
in a Holistic GUI Accessibility Assessment Framework, 12th European AAATE Conference, 19-22
September 2013, Vilamoura, Algarve, Portugal (under review)

27. Kaklanis, N., Moustakas, K., Tzovaras, D. Haptic Rendering of HTML components and 2D maps included in
web pages, in G. Ghinea, F. Andres, S. Gulliver (Eds) Multiple Sensorial Media Advances and Applications:
New Developments in MulSeMedia, 2010
28. Biswas, P., Kaklanis, N., Mohamad, Y., Peissner, M., Langdon, P., Tzovaras, D., Jung, C., “An Interoperable
and Inclusive User Modeling concept for Simulation and Adaptation”, A Multimodal End-2-End Approach to
Accessible Computing, Springer
PUBLICATIONS – BOOK CHAPTERSPUBLICATIONS – BOOK CHAPTERS

THANK YOU!
Next...
video presentations

Nikolaos Kaklanis - PhD presentation

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (15)

Similar to Nikolaos Kaklanis - PhD presentation

Similar to Nikolaos Kaklanis - PhD presentation (20)

Recently uploaded

Recently uploaded (20)

Nikolaos Kaklanis - PhD presentation

Editor's Notes