Prof Peter Sawyer, Professor of Software Systems Engineering, University of Lancaster:sensing cognitive health through monitoring computer interaction and through ambient technology – this is a most exciting (and I guess to some a bit scary) new area of research where you can sense someone’s cognitive health from how they use smartphones and other devices! http://dhaca.org.uk/dhaca-day-vi-presentations-and-brief-summary/

1. Passive'Sensing'of'Cogni.ve'
Health'
Pete'Sawyer'
School'of'Compu.ng'and'Communica.ons'
Lancaster'University'

2. Demen.a'in'the'UK'
• c.'900,000'people'affected'in'the'UK'
– Projected'to'reach'over'1'million'by'2021'
– Annual'cost'currently'c.'£23'billion'
• Only'44%'of'people'receive'a'diagnosis'
– Diagnosis'is'oNen'late'
• Being'able'to'monitor'the'progression'of'demen.a'
from'the'early'‘preclinical’'or'‘prodromal’'(e.g.'MCI)'
stage'is'of'poten.al'benefit'for'prognosis'of'how'the'
condi.on'is'likely'to'develop''
• It'also'opens'up'the'possibility'of'intervening''
with'diseaseXmodifying'therapies,'which'may''
slow'the'progression'

3. Two'Projects'
• SAMS'X'SoNware'Architecture'for'Mental'health'Self'
management'
– EPSRC'Working(Together(project'EP/K015796/1'
• MODEM'X'Monitoring'Of'Demen.a'using'Eye'Movements'
– EPSRC'Sensing(and(Imaging(for(Diagnosis(of(
Demen5as'project'EP/M006255/1'

4. Partners:)
EPSRC working together project EP/K015796/1'

5. What'SAMS'does'
• SAMS'monitors'people'as'they'use'their'home'
computer'
• SAMS'looks'for'signs'of''
cogni.ve'decline''
over'.me'
• If'decline'is'consistent'with''
decline'from'healthy'to'MCI''
or'early'demen.a'SAMS'will''
prompt'the'user'to'take'a''
followXup'test'and/or'see'their'GP'

6. Why'monitor'computer'use?'(1)'
• When'we'use'a'computer,'we'use'a'range'of'
cogni.ve'domains'
– Motor'control;'execu.ve'func.on;'memory'
recogni.on'&'recall;'language;'visioXspacial'reasoning'
• The'development'of'demen.a'will'lead'to'deficits'
in'at'least'some'of'these'same'cogni.ve'domains''
– Typically'these'are'what'are'tested'at'a'memory'clinic'
by'(e.g.)'Mini'Mental'State'Examina.on'(MMSE)'
• So,'if'we'are'finding'it'harder'and'harder'to'use'
our'computer,'it'might'be'because'our'cogni.ve'
health'is'declining'

7. Why'monitor'computer'use?'(2)'
• Opportunism:'it’s'increasingly'normal'for'
older'people'to'use'a'computer'for'keeping'in'
touch'with'family,'shopping,'banking,'etc.'
• It'gives'us'ecological'validity'if'we'simply'
monitor'peoples’'rou.ne,'daily'use'of'their'
computer.'
• We'hope'it'will'help'persuade'people'to'refer'
themselves'to'their'GP'who'might'otherwise'
not'have'done'so.'

8. The'challenge'
• Instrumen.ng'the'computer'to'collect'user'
data''
• Interpre.ng'the'collected'data'in'terms'of'
cogni.ve'health'
• Valida.ng'SAMS’'interpreta.on'
• Overcoming'the'barriers'to'adop.on'

9. Instrumen.ng'the'computer'(1)'
• This'means'wri.ng'soNware''
that'collects'data'as'the'user''
interacts'with'their'computer'
– It'needs'to'be'completely'unobtrusive'
– But'the'user'needs'to'be'aware''
that'they'are'being'monitored,''
so'they'can'turn'it'off'if'desired''

10. Instrumen.ng'the'computer'(2)'
• What'do'we'instrument?'
– The'opera.ng'system'for'general'housekeeping'
ac.vity'(MS'Windows'7,'8,'10)'
– MicrosoN'Office'applica.ons'(Word,'Outlook,'
Excel,'..)'
– Browsers'and'webmail'(IE'&'Gmail)'

11. Instrumen.ng'the'computer'(3)'
• What'do'we'collect?'
– A'range'of'stuff,'e.g.:'
• Mouse'moves,'tracking'the'cursor'as'the'user'moves'it'
from'one'part'of'the'screen'to'another'
• Selec.on,'drag,'resize'ac.ons'
• Authored'text'

12. Instrumen.ng'the'computer'(4)'

13. Interpre.ng'the'data'(1)'
• We'need'to'collect'data'that'tells'us'
something'about'the'health'of'the'user’s'
different'cogni.ve'domains.'
– e.g.'we'can'collect'mouseXmovement'data,'but'
what'does'that'tell'us'about'motor'control,'
execu.ve'func.on,'etc.?''
– Can'we'get'enough'data?'Can''
we'get'sufficiently'frequent'data?'
– Can'we'infer'user'intent?'

14. Interpre.ng'the'data'(2)'
• For'example:'
– Easy(ish)'
• Reduc.on'in'vocabulary,'idea'density'X'language)
• Hun.ng'for'commonlyXused'menu'items'X'memory)
– Not'so'easy'
• Failing'to'complete'common'sequence'of'tasks,'e.g.'email'X'
Memory'or'execu3ve)func3on)or'both?'
– [reply,'compose,'but'no'send]'OR'[reply,'no'compose,'send]'
• Needs''
– Expert'reference'group'to'help'us'iden.fy'the'most'fruipul'
user'ac.ons'to'dataXmine'
– Uncertainty'inference'
– PostXhoc'mining'of'data'looking'for'paqerns'

15. Valida.on'
• SmallXscale'pilot'study'(c.'10'ppl)'
• CrossXsec.onal'study'(30'MCI/early'AD,'30'
healthy'controls.'All'65+)'
– In'lab,'iden.cal'computer'setXup,'paperXbased'test'
baqery'then'set'of'computer'tasks'
• Longitudinal'study'(12'months,''
c.'60'ppl'All'65+)'
– SAMS'soNware'installed'on'par.cipants’'
'home'computers;'data'(anonymized'&'
'encrypted)'uploaded'periodically'to'LU''
We'are'here'

16. Barriers'to'adop.on'
• Why'would'anyone'want'to'run'SAMS?'
• How'can'they'be'sure'we'won’t'(e.g.)'read'
their'passwords?'
• If'SAMS'thinks'there’s'something'wrong,'how'
can'we'get'the'user'to'take'ac.on?'

17. MODEM'
• Growing'evidence'that'inhibitory'saccades'are'
impaired'by'Alzheimer’s'disease'
– Can'be'tested'using'eyeXtracking'
• EyeXtracking'systems'on'verge'of'stepXchange'
from'lab'instrument'to'widely'deployed'
sensor'
• MODEM'will'inves.gate'ambient'eyeXtracking'
for'demen.a'detec.on''
Crawford'TJ,'Higham'S,'Mayes'J,'Dale'M,'Shaunak'S,'Lekwuwa'G'(2013).'The'role'of'working'memory'and'aqen.onal'disengagement'
on'inhibitory'control:'effects'of'aging'and'Alzheimer's'disease.'AGE,'35,'5,'1637X1650'

18. Thanks'for'listening'