DIGITAL COLLEAGUES FOR SMART AGING
Patrice D. Tremoulet, PhD
Director, Applied Informatics Group (AIG)
• What is a Digital Colleague (DC)?
– Human Augmentation history (abridged)
• 1st Use Case for DCs: Aging Workforce
– Scope of initial effort (& parallel projects)
– Approach & Team
• Questions / Feedback
Digital Colleague (DC) Vision
A Digital Colleague augments human performance, by serving as:
1. an expert cog
2. a personal work assistant
3. a personal health coach
Like coworkers who can “read” each other, DCs discretely monitor
employees and provide customized assistance, tailored to the
current context, including the work environment, physical and
cognitive states, and ongoing task demands.
DCs alert employees to relevant new research, news, or products,
suggest tools and strategies that can enable better performance
and provide guidance to maintain and improve health.
1. A growing community is tackling the challenges
associated with building cognitive assistants
• Identify and share domain knowledge, customized based
upon understanding a particular employee’s interests/needs
2. Inexpensive wearable and environmental sensors
make it possible to reliably assess human physical and
• Human Performance Augmentation
• Cognitive supports for people with disabilities
• Personalized health support & health education
Human Augmentation History
As We May Think
to one’s memory”
will be coupled
together very tightly”
calls for “improving the
effectiveness of the
being” with computers
in a wide variety
• Electroencephalograph (EEG)
• Electrocardiograph (EKG)
• Galvanic skin response (GSR)
• Pupilometry / Eyetracking
Augmented Cognition overview
• Goal: Maximize operator cognitive performance
in dynamic, complex operational environments
• Approach: Physiological-data based
assessment of operator cognitive state
– Detects, predicts, avoids overload to reduce
operator error and maximize effectiveness
• Benefit: Mitigate
negative effects of
– Increase task
– Improve critical
Next-Gen Wearable Sensors
- Real-time physiological monitoring of body chemistries
Body chemistries and signature analysis through analytics will enable a wealth of
physiological and cognitive assessments not previously possible
Orexin A: alertness
Dopamine & Norepinephrin: performance
•Flexible, conformal, unobtrusive form factor
•Real-time biomarker measurements
•Correlations to physical and cognitive states
that affect performance
1st use case: Aging Workers
• Deep fund of work-relevant knowledge.
• Can help mentor younger employees.
• May be willing to work part time, saving employer costs.
• Health costs reduced when people stay cognitively active.
• High levels of engagement.
• Slight cognitive declines begin in the 50s
• Physical limitations may require accommodations
• User acceptance, usability of both wearable electronics and
Aging Workforce in US
• The Government Accountability Office projected in 2006 that
20% of the workforce would be aged 55 or older by 2015 .
• Population research indicates that over 75% of baby boomers
are planning to work past retirement age .
Aging Workers: Need for DCs
• Employers want to retain valuable older employees, but both
lack knowledge about tools and strategies that can help
identify and accommodate for cognitive and physical changes.
• US universities in particular need to be prepared to
support an aging workforce.
2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050
Percentage aged 60 or over
World Average United States of America Repblic of Korea
countries are also
DC for Smart Aging Opportunities
• Seoul, Korea government officials are interested in
supporting Smart Aging applied research
• NSF Partnerships for Innovation: Building Innovation
• Smart Aging Service System (SASS) provides
older workers with personalized, private
recommendations designed to help reduce health
risks and maintain or improve job performance.
• Enables older employees with valued skills and
experience to continue working.
• Reduces health costs by helping employees
take ownership of their healthcare.
• Provides a framework for developing
personalized health recommender systems.
Drexel’s multi-pronged approach
1. Faculty develop prototype that detects and mitigates against
most common changes in cognitive function (under NSF
2. CS PhD student builds Material Science Cognitive Assistant
3. Undergrad student(s) chart out the space of assistive
technologies that can support individuals with intellectual
4. Faculty propose to develop new sensors, smart aging
metrics, data repository and analysis tools with IBM and Korea
Digital Colleagues for Smart Aging: Initial effort
Goal: Keep aging University workforce healthy and engaged
even when faced with changes in any of three cognitive
functions: memory, attention, processing speed.
Approach: Wearable and environmental sensors feed an
intelligent “Digital Colleague” that a) recommends
accommodations and strategies to enable continued
contributions, and b) provides health alerts and reminders.
Initial Prototype Sensor Suite
Commercially available products to be used in initial proof-of-concept prototype
Digital Colleagues have four
major software components:
1. The robust data collection
system integrates outputs
of sensors into a data store
that supports rich queries.
2. The knowledge base
backend holds facts and
3. The Digital Colleague Algorithms component translates information
about an employee into recommendations for assessments and/or
4. The dialog interaction module communicates with the employee.
• Drexel faculty/staff:
– Yvonne Michael, Assoc. Prof, School of Public Health
– Ibiyonu Lawrence, Clinical Prof., Drexel Univ. College of Medicine
– Marcello Balduccini, Asst Prof, College of Computing & Informatics
– Gaurav Naik, Sr Research Scientist, Applied Informatics Group
• Industry Partners:
– Cognitive Compass (CEO, Madelaine Sayko)
– General Electric (Research scientists, Luis Tari & Alfredo Gabaldon)
– Independence Blue Cross (Sr. VP & CIO, Somesh Nigam)
– Evoke Neuroscience (CEO, David Hagedorn)
– Abilities Inc. subsid of Viscardi Center (CEO, Jessica Swirsky)
DC for Smart Aging Team
There will nevertheless be a fairly long interim during
which the main intellectual advances will be made by
men and computers working together in intimate
association… those years should be intellectually the
most creative and exciting in the history of mankind.
Licklider, J.C.R. (1960), “Man-Computer
Symbiosis” IRE Transactions on Human
Factors in Electronics, volume HFE-1,
1. Tishman, F.M., Van Looy, S., and Bruyère, M. 2012 “Employer Strategies for Responding to an Aging Workforce”,
National Technical Assistance and Research Center 2012 Report.
2. Saad, L. (2013). “Three out of Four US Workers Plan to Work Past Retirement Age,” Gallup / Economy Report, May
23rd, 2013. http://www.gallup.com/poll/162758/three-four-workers-plan-work-past-retirement-age.aspx
3. TIAA Cref - Aging Workforce Series, Health Fitness and the Bottom Line, July 2012 https://www.tiaa-
4. Levin, Sharon G. and Paula E. Stephan, “Age and research productivity of academic scientists,” Research in Higher
Education, 1989: 30(5): 531- 549.
5. Sörensen, L.E., Pekkonen, M.M., Männikkö, K.H., Louhevaara, V.A., Smolander, J., Alén, M.J. “Associations between
work ability, health-related quality of life, physical activity and fitness among middle-aged men,” Applied Ergonomics, Nov,
2008: 39(6):786-91. http://www.ncbi.nlm.nih.gov/pubmed/18166167.
DC HCI Challenges – Sensor Technologies
• Unobtrusive, comfortable, but ruggedized form-factors
– Robust when dusty, wet, tapped/hit, etc.
• Largely autonomous operation. Should be able to forget
you are wearing sensors…but:
• Could it be useful to cue wearers into a potential
health problem? Under what circumstances?
• Replacement notifications and failure indicators –
how delivered and to whom?
• Wearer “Reset” option?
• Data download notification – to whom? Privacy
Additional DC HCI Challenges to explore:
• Do participants know about aging-related cognitive decline
and how it may impact work?
– How would participants want to interact with their own
– What sorts of privacy safeguards would they expect?
• How and when would they like to provide information about
themselves that sensors can’t currently capture (e.g. job
• What factors should influence how information is requested
and guidance is presented (e.g. individual preferences,
current capabilities, the types and degrees of limitations,
types of recommendations), and how?
• Determine pre-mission readiness
• Maintenance of safety during mission
health of squad
Accelerated Learning / HMI design
• Customize training based on cognitive
states of trainees
• Develop interfaces that support low
Dynamic modifications to training
exercises to speed learning
• Early Medical Problem Detection
• Accident Medical SA
• Recording of injury event & treatment
for use at all echelons of care
Pilot showing symptoms of Hypoxia
Developing technologies to collect and take action on
health and readiness data of individuals
Medic SA Sys.
Learning & HMI
-With Near-Zero Footprint