Sirris Symposium: Human Factors and
Technologies for Pro-active, Contextaware and Data-intensive Applications

Human-­‐Sys...
Agenda	
  
Ø  A	
  bit	
  about	
  me	
  and	
  Design	
  Interac7ve,	
  Inc.	
  
	
  
Ø  Human	
  Augmenta7on:	
  Essen...
A	
  bit	
  about	
  me	
  and	
  Design	
  Interac9ve,	
  Inc.	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

17	
  October	...
DI	
  Divisions	
  
DIVISION'

MARKET'POSITIONING'STATEMENT'

	
  
	
  
Defense&Solutions&
	
  
	
  
	
  
	
  
	
  	
  

O...
Emerging	
  DI	
  Products	
  –	
  Low	
  Cost	
  EEG	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
 
	
  

	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  

Emerging	
  DI	
  Products	
  
-­‐	
  New	
  Training...
Emerging	
  DI	
  Products	
  -­‐	
  Lessons	
  Learned	
  Tool	
  
Playbook is a rapid authoring tool that can be
	
  
us...
DI	
  Products	
  –	
  easyGaze	
  and	
  GazeWare	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
Emerging	
  DI	
  Products	
  –	
  STRAP	
  
STRAP vest communicates a haptic
language based on military hand signals
	
  ...
Emerging	
  DI	
  Products	
  –	
  New	
  Evalua9on	
  Tools	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
Emerging	
  DI	
  Products	
  –	
  SIMI	
  Sensor	
  Suite	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
Rela9ng	
  DI’s	
  SIMI	
  to	
  the	
  ASTUTE	
  Project	
  
Build	
  an	
  EEG-­‐based	
  Measure	
  of	
  Situa9on	
  A...
Human	
  Augmenta9on:	
  

Essen9al	
  Emerging	
  Transforma9onal	
  Technology	
  
Human	
  Augmenta9on:	
  	
  Essen9al	
  
Emerging	
  Transforma9onal	
  Technology	
  
	
  

As	
  natural	
  human	
  ca...
Human	
  Augmenta9on	
  Essen9al	
  for	
  Gaining	
  
Benefits	
  of	
  Emerging	
  Technology	
  Advances	
  	
  
	
  	
 ...
Emerging	
  Technologies	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

Human Augmentation
Emerging Technology
Emerging	
  Technologies	
  Priority	
  Mix	
  2013	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

Human	
  
Augmenta7on	
  
...
ASTUTE	
  Focused	
  on	
  Human	
  Augmenta9on	
  
Ø  ASTUTE	
  is	
  focused	
  on	
  the	
  transforma7onal	
  
	
   e...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on	
  
Adap9ve	
  Automa9on	
  &	
  Augmented	
  Cogni9on	
  
Past	
  Approaches	
  to	
  Human	
  Augmenta9on	
  
Ø  Mission	
  cri7cal	
  systems	
  put	
  immense	
  pressure	
  on...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Adap9ve	
  Automa9on	
  
Ø  Rouse’s	
  (1998)	
  adap7ve	
  automa7...
Workload	
  Matched	
  Adap9ve	
  Automa9on	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

Source:	
  hep://www.docstoc.com/d...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Adap9ve	
  Automa9on	
  
Ø  Several	
  closed-­‐loop	
  solu7ons	
 ...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Adap9ve	
  Automa9on	
  
Ø  Adap7ve	
  automa7on	
  o`en	
  7mes	
 ...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Adap9ve	
  Automa9on	
  
Ø  Some	
  issues:	
  
	
  
	
  
	
  
	
  ...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Adap9ve	
  Automa9on	
  è	
  Augmented	
  Cogni9on	
  
Ø  Notwiths...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Augmented	
  Cogni9on	
  
Ø  Augmented	
  cogni7on	
  has	
  provid...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Augmented	
  Cogni9on	
  –	
  The	
  Objec9ve	
  
	
  
	
  
	
  
	
 ...
Past	
  Approaches	
  to	
  Human	
  Augmenta9on:	
  
Augmented	
  Cogni9on	
  –	
  The	
  Challenge	
  

Ø  Augmented	
 ...
Augmented	
  Cogni0on:	
  
Some	
  History	
  
When	
  we	
  started	
  the	
  Augmented	
  Cogni9on	
  
Program:	
  We	
  asked	
  -­‐	
  do	
  we	
  have	
  it	
  wron...
All these designs rely on WIMP interfaces –
Windows, Icons, Menus, Pointing Devices
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
Is	
  it	
  9me	
  for	
  new	
  HSI	
  paradigms?	
  
	
  	
  

	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

	
  While	
  curr...
Evolution of HSI Paradigms
Command-­‐Line	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

•  Pure	
  text	
  interface	
  
•  ...
Augmented Cognition
Ø  A	
  new	
  HSI	
  paradigm	
  	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

§  Intui7ve	
  coupli...
AugCog Sensors & Measures	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

Source:	
  hep://www.spawar.navy.mil/s7/publica7ons/...
AugCog Sensors & Measures	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

Source:	
  hep://www.spawar.navy.mil/s7/publica7ons/...
AugCog Sensors & Measures	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

ü ASTUTE’s	
  proac7ve	
  systems	
  could	
  

des...
AugCog Adaptive Strategies
	
  
AugCog Adaptive Strategies
Adapt presentation of information
	
  

Modality augmentation
	
  
	
  
(redundancy, switching)...
AugCog Adaptive Strategies
Adapt scheduling of information
	
  
	
  
Pacing
	
  
	
  
	
  
	
  
Sequencing
	
  	
  

Hold ...
AugCog Adaptive Strategies
Adapt system autonomy
	
  
	
  
Delegate
	
  
	
  
	
  
	
  
Mixed Initiative
	
  	
  

Transfe...
Potential Future Adaptive Strategies
	
  
	
  
	
  
	
  
	
  
	
  
Innova9ve	
  Adap9ve	
  
	
  	
  

Techniques	
  

New	...
 
	
  
	
  
	
  
	
  
	
  
	
  	
  
 
Results	
  of	
  Augmented	
  
	
  
	
  
Cogni9on	
  Program	
  
	
  
	
  
Summarized	
  in	
  AugCog	
  
	
  
	
  	
  
...
 
	
  

Where	
  we	
  were	
  at	
  end	
  of	
  
Augmented	
  Cogni9on	
  
Program:	
  
	
  	
  
Need	
  for	
  broadeni...
Next Steps:

Develop	
  plethora	
  of	
  sensors	
  &	
  
measures	
  that	
  drive	
  adap9ve	
  
systems	
  
Sensors	
 ...
Extensions	
  aber	
  Augmented	
  Cogni9on	
  
Where	
  DI	
  has	
  taken	
  AugCog	
  R&D	
  
Appling	
  SIMI	
  to	
  Image	
  Analysis	
  
SIMI	
  for	
  Image	
  Analysis	
  
DI	
  used	
  EEG	
  and	
  eye-­‐tracking	
  to	
  develop	
  real-­‐7me	
  	
  	
  ...
SIMI	
  for	
  Image	
  Analysis	
  
Depic7on	
  of	
  all	
  fixa7on	
  points	
  on	
  the	
  image	
  
	
  
drawn	
  on	...
SIMI	
  for	
  Image	
  Analysis	
  
•  Fixa7on	
  dura7on	
  
	
  	
  	
   (ms)	
  in	
  addi7on	
  to	
  
	
   the	
  lo...
Appling	
  SIMI	
  to	
  Instrument	
  	
  
Flight	
  Panel	
  Training	
  
Sensors:	
  Eye	
  Tracking,	
  EEG,	
  &	
  Performance	
  
Measures:	
  Visual	
  Aeen7on	
  Alloca7on,	
  Controls	
  E...
Performance	
  Feedback	
  
Look	
  at	
  Error	
  Details!	
  

ü ASTUTE’s	
  proac7ve	
  

systems	
  should	
  consider	
  
linking	
  errors	
  t...
Error	
  Detail	
  Screen	
  Cont.	
  
Error	
  Detail	
  Screen	
  Cont.	
  
Error	
  Detail	
  Screen	
  
Error	
  Detail	
  Screen	
  
Diagnosis:	
  	
  Looking	
  at	
  Al7meter;	
  Skill	
  Aeained…	
  Now:	
  Arousal	
  Issue	
  –	
  
Time	
  to	
  move	...
Appling	
  SIMI	
  to	
  Baggage	
  Screening	
  
SIMI	
  for	
  Baggage	
  Screening:	
  
ScreenADAPT	
  
Individualized	
  training	
  for	
  visual	
  search:	
  
	
  
	...
Sensors	
  &	
  Measures	
  
Performance:	
  Error	
  Classifica7on	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

–  Hit,	
  ...
Two	
  Adap9ve	
  Training	
  Techniques	
  
	
  
Exposure	
  training	
  used	
  to	
  strengthen	
  object	
  detec7on	
...
SIMI	
  Applied	
  to	
  Emergency	
  Dispatch	
  
Which	
  sensors,	
  measures,	
  diagnoses,	
  and	
  adap9ve	
  strat...
SensorIT	
  –	
  MeasureIT	
  -­‐	
  DiagnoseIT	
  
then	
  Proac9vely	
  ADAPT!	
  
• 

	
  
	
  
	
  
	
  
	
  
	
  
	
 ...
SIMI	
  Applied	
  to	
  Emergency	
  Dispatch	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
SIMI	
  Applied	
  to	
  Emergency	
  Dispatch	
  
Gamma	
  waves	
  reveal	
  high	
  
engagement	
   	
  
	
  
Beta	
  w...
Augmented	
  Cogni9on:	
  Remaining	
  Hard	
  Problems	
  
Ø  Sensors	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

§ 

R...
Augmented	
  Cogni9on:	
  Remaining	
  Hard	
  Problems	
  
Ø  Designing	
  seamless	
  adapta7on	
  techniques	
  	
  
	...
Augmented	
  Cogni9on:	
  Remaining	
  Hard	
  Problems	
  
Ø  Designing	
  seamless	
  adapta7on	
  techniques	
  	
  
	...
The	
  Future	
  of	
  HSI	
  

An	
  R&D	
  agenda	
  to	
  direct	
  the	
  HSI	
  field	
  through	
  2050	
  	
  
Where	
  to	
  from	
  here?	
  
Augmented	
  Cogni7on	
  –	
  field	
  started	
  ~2000	
  
	
  
	
  
	
  
	
  
	
  
	
  
...
First:	
  	
  Iden9fied	
  HSI	
  “Enablers”	
  

Other	
  HSI	
  Enablers	
  Beyond	
  Human	
  Augmenta9on	
  
Enhancing	
  Human	
  Performance	
  
HSI	
  Enabler:	
  Augmenta9on	
  
	
  	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

...
Enhancing	
  Human	
  Performance	
  
HSI	
  Enabler:	
  Big	
  Data	
  
	
  	
  	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
...
Enhancing	
  Human	
  Performance	
  
HSI	
  Enabler:	
  Autonomy	
  
	
  	
  	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
...
Enhancing	
  Human	
  Performance	
  
HSI	
  Enabler:	
  Transgenics	
  
	
  	
  

Gene7cally	
  alter	
  the	
  
human	
 ...
Second:	
  	
  Iden9fied	
  HSI	
  “Eras”	
  
What’s	
  beyond	
  the	
  digital	
  era?	
  
Enhancing	
  Human	
  Performance	
  
HSI	
  Eras	
  

Ø  To	
  map	
  out	
  an	
  HSI	
  R&D	
  agenda	
  through	
  20...
Enhancing	
  Human	
  Performance	
  
HSI	
  Eras	
  

Ø  Individualism	
  Op7miza7on:	
  
	
  
	
  
	
  
	
  
	
  
	
  
...
Enhancing	
  Human	
  Performance	
  

HSI	
  Eras:	
  Individualism	
  Op9miza9on	
  –	
  Brain	
  Era	
  
Seeks	
  to	
 ...
Enhancing	
  Human	
  Performance	
  

HSI	
  Eras:	
  Individualism	
  Op9miza9on	
  –	
  Physical	
  Feat	
  
Seeks	
  t...
Enhancing	
  Human	
  Performance	
  

HSI	
  Eras:	
  Collec9vism	
  Op9miza9on	
  –	
  Human	
  	
  	
  	
  	
  	
  
Qua...
Enhancing	
  Human	
  Performance	
  

HSI	
  Eras:	
  Collec9vism	
  Op9miza9on	
  –	
  Ecological	
  
Quantum-­‐Entangle...
Emerging	
  HSI	
  Eras:	
  Big	
  Data	
  HSI	
  Enabler	
  
Extends	
  Info	
  Highway	
  
	
  
	
  
	
  
	
  
	
  
	
  ...
HSI	
  Emerging	
  Concepts:	
  HSI	
  Eras	
  &	
  Enablers	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  
HSI	
  R&D	
  Agenda	
  through	
  2050	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  	
  

ü ASTUTE	
  can	
  enhance	
  

human...
 
	
  

	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  

The	
  future	
  of	
  
HSI…	
  

	
  
	
  
	
  
	
  ...
Conclusions	
  
Ø  ASTUTE	
  aims	
  to	
  implement	
  proac7ve	
  systems	
  that	
  
increase	
  SA	
  and	
  enhance	...
Acknowledgments	
  
Special	
  thanks	
  goes	
  to	
  Innoviris	
  for	
  their	
  support	
  of	
  this	
  presenta-on.	...
Acknowledgments	
  
This	
  material	
  is	
  based	
  upon	
  work	
  supported	
  in	
  part	
  by	
  the	
  Office	
  of	...
Upcoming SlideShare
Loading in …5
×

Astute symposium 2013-10-10_keynote_kay_stanney

324 views

Published on

Astute symposium 10/10/2013 - Keynote Kay Stanney

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
324
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Astute symposium 2013-10-10_keynote_kay_stanney

  1. 1. Sirris Symposium: Human Factors and Technologies for Pro-active, Contextaware and Data-intensive Applications Human-­‐Systems  Integra0on  in   Adap0ve  Mission  Cri0cal  Systems     Kay  Stanney,  Ph.D.,  C.H.F.P.   Design  Interac7ve,  Inc.,  President  &  Founder   University  of  Central  Florida,  Courtesy  Appt.     October  10,  2013  
  2. 2. Agenda   Ø  A  bit  about  me  and  Design  Interac7ve,  Inc.     Ø  Human  Augmenta7on:  Essen7al  Emerging       Transforma7onal  Technology       Ø  Past  Approaches  to  Human  Augmenta7on         §  §  Adap7ve  Automa7on   Augmented  Cogni7on   Ø  The  Future  of  Human  Systems  Integra7on   Ø  Conclusions  
  3. 3. A  bit  about  me  and  Design  Interac9ve,  Inc.                   17  October  2013   Design  Interac-ve,  founded  in  1998,  is  a  human   factors  engineering  firm  that  helps  clients   overcome  their  most  pressing  human  performance   challenges.  Unlike  most  firms,  we  use  deep   behavioral  and  physiological  diagnos-cs  to             design  adap-ve,  engaging  solu-ons  that             op-mize  performance  and  profoundly                             enhance  the  user  experience.    
  4. 4. DI  Divisions   DIVISION' MARKET'POSITIONING'STATEMENT'     Defense&Solutions&             Our&Defense&Solutions&Division&provides&operational&analysis,&performance&assessment,&and& advanced&technology&solutions&to&Department&of&Defense&clients&who&aim&to&enhance&training& effectiveness&and&efficiency.&&We&use&deep&behavioral&and&physiological&diagnostics&to&deliver& adaptive,&meaningful,&and&intuitive&learning&experiences&for&the&Warfighter.&& Medical&Innovations& Our&Medical&Innovations&Division&provides&innovative&personal&health&solutions&for&medical&care& providers&and&consumers.&We&combine&unobtrusive&biomonitoring&technology&with&adaptive& assessment&solutions&that&continuously&analyze&collected&data&to&offer&preventative&and&corrective& measures&in&any&setting.&& Emerging&Markets& and&Technologies& & Our&Emerging&Markets&Division&specializes&in&userDcentered&design&and&usability.&We&leverage&our& cuttingDedge&military&R&D&to&develop&innovative&design&and&evaluation&tools,&human/machine& interfaces,&and&smart&mobile&solutions&that&empower&users&and&enhance&the&user&experience.&& Across  our  Divisions,  DI’s  solu0ons  save  lives,  reduce  cost,  enhance  the  user  experience,  and  op0mize  human   performance  -­‐  while  defining  the  future  of  human-­‐systems  integra0on.   17  October  2013  
  5. 5. Emerging  DI  Products  –  Low  Cost  EEG                  
  6. 6.                             Emerging  DI  Products   -­‐  New  Training  Solu9ons                          
  7. 7. Emerging  DI  Products  -­‐  Lessons  Learned  Tool   Playbook is a rapid authoring tool that can be   used to capture, publish, and share operational   observations, insights, and lessons (OIL).         Playbook provides an easy-to-use platform to     record and share personal experiences quickly and effectively.
  8. 8. DI  Products  –  easyGaze  and  GazeWare                  
  9. 9. Emerging  DI  Products  –  STRAP   STRAP vest communicates a haptic language based on military hand signals           Demonstrated rapid       retention rates learning and high
  10. 10. Emerging  DI  Products  –  New  Evalua9on  Tools                  
  11. 11. Emerging  DI  Products  –  SIMI  Sensor  Suite                  
  12. 12. Rela9ng  DI’s  SIMI  to  the  ASTUTE  Project   Build  an  EEG-­‐based  Measure  of  Situa9on  Awareness   Sensor:    EEG   MeasureIT   Measure:    EEG  Alpha  &  Theta   Diagnose:  High  Theta  &  Low   Alpha  =    Low  SA  
  13. 13. Human  Augmenta9on:   Essen9al  Emerging  Transforma9onal  Technology  
  14. 14. Human  Augmenta9on:    Essen9al   Emerging  Transforma9onal  Technology     As  natural  human  capaci-es  become  increasingly     mismatched  to  data  volumes,  processing  capabili-es,     and  decision  speeds,  augmen-ng  human  performance       will  become  essen-al  for  gaining  the  benefits  that  other       technology  advances  can  offer.        Technology  Horizons:  A  Vision  for  Air  Force  Science  &   Technology  During  2010-­‐2030   Dr.  Werner  J.A.  Dahm   United  States  Air  Force  Chief  Scien7st   May  15,  2010  (p.  58)    
  15. 15. Human  Augmenta9on  Essen9al  for  Gaining   Benefits  of  Emerging  Technology  Advances                         Robo7cs  &   UAVs   VR  &   Virtual   Assistants   Predic7ve   &  Content   Analy7cs   Human   Augmenta7on   Biochips;  Health   Monitoring   Augmented   Reality;   Wearable  UIs  
  16. 16. Emerging  Technologies                   Human Augmentation Emerging Technology
  17. 17. Emerging  Technologies  Priority  Mix  2013                   Human   Augmenta7on   Transforma7onal  
  18. 18. ASTUTE  Focused  on  Human  Augmenta9on   Ø  ASTUTE  is  focused  on  the  transforma7onal     emerging  technology  of  human  augmenta7on                 Ø  Pro-­‐ac7ve  systems  are  one  approach  to  human   augmenta7on   Ø  ASTUTE’s  goals  for  proac7ve  systems  are  to:   §  §  §  Measure  user  state  and  relate  it  to  context   Provide  pro-­‐ac7ve  sugges7ons  based  on  these  user   state  and  context  data   Thereby  realizing  adap7ve  HMIs                                                                                   that  increase  situa7onal                                                                                             awareness,  improve  decision                                                                                               making,  and  augment  other                                                                                               aspects  of  human  performance  
  19. 19. Past  Approaches  to  Human  Augmenta9on   Adap9ve  Automa9on  &  Augmented  Cogni9on  
  20. 20. Past  Approaches  to  Human  Augmenta9on   Ø  Mission  cri7cal  systems  put  immense  pressure  on     human  cogni7on     Ø  These  context  demand  swi`er,  highly  accurate,  and       ever  more  resilient  capabili7es           §  §  §  NASA  proposed  adap7ve  automa7on  as  a  means  to   address  such  demands   DARPA  proposed  augmented  cogni7on  as  new  HSI   paradigm  through  which  to  achieve  gains  in  mission   cri7cal  performance   These  efforts  can  inform  ASTUTE’s                                                     proac7ve  systems  efforts  
  21. 21. Past  Approaches  to  Human  Augmenta9on:   Adap9ve  Automa9on   Ø  Rouse’s  (1998)  adap7ve  automa7on  theory     suggested  that  both  user  and  system  should  be  able       to  ini7ate  changes  in  the  level  of  system  automa7on     in  response  to  situa7onal  demands       Ø  Systems  implemented  based  on  these  early  theories       generally  followed  a  binary  (on/off)  approach  to   adap7ve  automa7on   §  §  Some  relied  on  physiological  measures  of  operator   state  to  trigger  automa7on   Others  relied  on  task  or  context  based  measures  (e.g.,   cri7cal  events;  operator  performance;  task  models)  to   trigger  automa7on   ü ASTUTE’s  proac7ve   systems  plan  to  leverage   both  operator  state  &   context  
  22. 22. Workload  Matched  Adap9ve  Automa9on                   Source:  hep://www.docstoc.com/docs/99925451/Adap7ve-­‐Automa7on-­‐Matched-­‐to-­‐Human-­‐Mental-­‐Workload  
  23. 23. Past  Approaches  to  Human  Augmenta9on:   Adap9ve  Automa9on   Ø  Several  closed-­‐loop  solu7ons  evolved,  many  of  which     controlled  the  levels  of  task  automa7on  based  on       physiological  indices:             §  §  §  EEG  measures  (e.g.,  theta,  alpha,  beta,  and  gamma   band  ac7vity  to  develop  an  engagement  index)     Cardio-­‐circulatory  measures  (e.g.,  HR,  HRV)   Combina7on  of  physiological  indicators  (e.g.,  EEG,  ERPs,   and  HRV;  EEG,  HR,  respira7on  interval,  and  eye  blinks/ interblink  intervals)  
  24. 24. Past  Approaches  to  Human  Augmenta9on:   Adap9ve  Automa9on   Ø  Adap7ve  automa7on  o`en  7mes  substan7ally     improves  human  performance:                 §  §  §  44%  reduc7on  in  tracking  task  errors  and  a  33%   reduc7on  in  error  rates  on  resource  management  tasks   (Wilson  &  Russell,  2003)   50%  improvement  on  UAV  opera7ons  (Wilson  &   Russell,  2007)   300%  improvement  in  throughput  and  detec7on  in   image  analysis  tasks  (Bloom,  et  al.,  2009)  
  25. 25. Past  Approaches  to  Human  Augmenta9on:   Adap9ve  Automa9on   Ø  Some  issues:                   §  §  §  §  §  §  §  Unbalanced  mental  workload   Mistrust   Overreliance   Complacency     Insensi7ve  physiological  measures   Reduced  situa9on  awareness   Decision  biases   u ASTUTE’s  proac7ve   systems  seek  increased  SA   and  enhanced  decision   making  
  26. 26. Past  Approaches  to  Human  Augmenta9on:   Adap9ve  Automa9on  è  Augmented  Cogni9on   Ø  Notwithstanding  challenges  and  barriers  presented  by  real-­‐7me     psychophysiological  monitoring,  early  work  in  adap7ve     automa7on  demonstrated  immense  poten7al  derived  through       systema7c  integra7on  of  operator  state  and  system  state     Ø  Further,  possibili7es  of  leveraging  these  synergies  for  more  than         adap7ve  automa7on  soon  became  evident,  as  they  provided  a   means  by  which  to  augment  cogni7on  and  thus  extend  the   human  poten7al  
  27. 27. Past  Approaches  to  Human  Augmenta9on:   Augmented  Cogni9on   Ø  Augmented  cogni7on  has  provided  many  of  the     theories,  principles,  and  prac7ces  needed  to  realize       proac7ve  systems       Ø  Augmented  cogni7on  R&D  has  primarily  focused  on         mission  cri7cal  systems  that  put  immense  pressure   on  human  cogni7on     §  §  Such  context  have  common                                                                                 ground  with  Emergency                                                                               Dispatching  domain   Thus,  lessons-­‐learned  via                                                                         Augmented  Cogni7on  R&D                                                                                                 can  inform  the  ASTUTE  Project   ü ASTUTE’s  proac7ve   systems  can  benefit  from   AugCog  lessons-­‐learned  
  28. 28. Past  Approaches  to  Human  Augmenta9on:   Augmented  Cogni9on  –  The  Objec9ve                   Problem:  The  one-­‐two  punch  of   informa7on  overload  and   mul7tasking   •  Increased  volume  of   informa7on  available  in   command  centers,  while   staffing  levels  remain  constant   -­‐  must    to  try  to  do  more,  and   do  it  faster   Augmented  Cogni9on  Objec9ve:    Develop  technologies  capable  of   extending,  by  an  order  of  magnitude  or  more,  the  informa7on   management  capacity  of  individuals  working  with  21st  century   compu7ng  technologies.  
  29. 29. Past  Approaches  to  Human  Augmenta9on:   Augmented  Cogni9on  –  The  Challenge   Ø  Augmented  cogni7on  presents  a  pro-­‐ac7ve     paradigm  through  which  to  achieve  human     performance  gains  in  mission  cri7cal  context                                   such  as  Emergency  Dispatching  domain       –  Challenge  is  to  real-­‐7me  sensor,  measure  and       diagnose  the  collec7ve  human  state  (cogni7ve,   physical,  affec7ve)  and  then  use  theory-­‐based                                                 algorithms  to  proac7vely  adapt  to  and  augment   innate  human  abili7es   –  Making  the  unobservable  –                                                   observable    
  30. 30. Augmented  Cogni0on:   Some  History  
  31. 31. When  we  started  the  Augmented  Cogni9on   Program:  We  asked  -­‐  do  we  have  it  wrong?                   Where  has  it  been  established  that  such  WIMPs   (windows,  icons,  menus,  pointers)  are  the  ul-mate  HSI   design?    What  is  appealing  about  the  WIMP     and  internet  browser  as  the  interface  for  the  human  to   the  computer?    Are  we  falling  into  a  least  common   denominator  trap?    Can’t  we  do  beYer?     ADM  (ret.)  Lee  Kollmorgen    
  32. 32. All these designs rely on WIMP interfaces – Windows, Icons, Menus, Pointing Devices                
  33. 33. Is  it  9me  for  new  HSI  paradigms?                        While  current  HSI  paradigms  have   empowered  computer  users  of  varying   ability,  they  alone  cannot  handle  many   of  the  challenges  of  today’s  opera9onal   environments:   • Mul9-­‐tasking   • Mul9ple  informa9on  streams   • Varying  contexts        We  need  systems  that  ac9vely  integrate   the  human  and  provide  proac9ve   support  based  on  individual  capabili9es   and  limita9ons:   • Alleviate  cogni9ve  bo^lenecks   • Support  situa9on  awareness     • Enhance  decision  making   • Account  for  individual  differences  
  34. 34. Evolution of HSI Paradigms Command-­‐Line                   •  Pure  text  interface   •  Individual  or  batched   commands   •  Predetermined  sequence     à  User  feeds  system   GUI   AugCog   •  Graphical  interface   •  System  awareness   • Metaphorical   representa7ons   • Sta7c  implementa7on   •  Object-­‐oriented   • “Direct  manipula7on”   •  Reac7ve  (event-­‐based)   • To  user  input  (behavioral)   • To  system  event   • Standardized  reac7ons   •  Baby-­‐steps  towards   dynamic  interac7on   • Personaliza7on   • Smart  menus     à  System  acts  on  user   behavior   • User  state   • User  behavior   • System  state   • Task  context   •  Proac7ve  -­‐  adapts  to   changes  in:   • User  performance   • Task-­‐context  requirements   •  Provides:   • Dynamic  and  adaptable   representa7ons   • Individualized  response   •  Act  on  operator  intent     à  System  proac7vely  adapts   to  user,  task,  and  context  
  35. 35. Augmented Cognition Ø  A  new  HSI  paradigm                     §  Intui7ve  coupling  between  human  and  machine     §  Providing  the  right  informa7on  -­‐  at  precisely  the   right  7me  -­‐  in  the  right  format  to  amplify  human   capabili7es   Ø  An  augmented  cogni7on  system  has  three  main   components:   §  §  §  Sensors:  Neurophysiological,  physiological,                                                                 and  behavioral  sensors   Measures:  Cogni7ve,  physical,  and                                                                                   affec7ve  user  state  measures     Adapta7on  Strategies:  Proac7ve                                                                                                       techniques  to  alleviate  situa7ons  of         overload,  inaeen7on,  stress...    and                                             improve  human  performance  
  36. 36. AugCog Sensors & Measures                   Source:  hep://www.spawar.navy.mil/s7/publica7ons/pubs/tr/tr1940vicond.pdf  
  37. 37. AugCog Sensors & Measures                   Source:  hep://www.spawar.navy.mil/s7/publica7ons/pubs/tr/tr1940vicond.pdf  
  38. 38. AugCog Sensors & Measures                   ü ASTUTE’s  proac7ve  systems  could   design  future  sensors  and  measures   focused  on  increased  SA  and  enhance   decision  making   Source:  hep://www.spawar.navy.mil/s7/publica7ons/pubs/tr/tr1940vicond.pdf   ü ASTUTE’s  proac7ve  systems   could  learn  from  AugCog   sensors,  measures,  and   classifica7on  methods  
  39. 39. AugCog Adaptive Strategies  
  40. 40. AugCog Adaptive Strategies Adapt presentation of information   Modality augmentation     (redundancy, switching)     Transposition       Add or change mode of information presentation Change information type from verbal to spatial or vice versa Cueing Augment display to capture attention of user Decluttering Reduce amount/complexity of information displayed Context-sensitive help Provide information specific to system state at time help is needed
  41. 41. AugCog Adaptive Strategies Adapt scheduling of information     Pacing         Sequencing     Hold low priority information until current high priority tasks completed Simultaneous events converted into sequential form Decompose tasks into smaller portions and re-arrange subtasks
  42. 42. AugCog Adaptive Strategies Adapt system autonomy     Delegate         Mixed Initiative     Transfer tasks to fully-automated system Provide operator with most appropriate level of control for situation; both operator and system can adjust system autonomy ü ASTUTE’s  proac7ve   systems  could  use  AugCog   Adap7ve  Strategies  
  43. 43. Potential Future Adaptive Strategies             Innova9ve  Adap9ve       Techniques   New  Adapta9on   Objec9ves   -­‐  today:  Cogni7ve  boelenecks   -­‐  today:  Reduce  distrac7ons   -­‐  tomorrow:  Increase  SA,  reduce  confusion…   -­‐  tomorrow:  Reduce  inaeen7veness,  fear…   -­‐  Use  ambient  environment   -­‐   Music    tempo/genre  change   -­‐  Empathe7c  proac7ve  system   -­‐  Use  predic7ve  cogni7ve,   physical,  and  affec7ve  state   ü ASTUTE’s  proac7ve  systems   could  design  future  adap7ve   strategies  focused  on   enhanced  SA  and  DM   Individual  Cogni9ve   Profiles   -­‐  Aiding  novice  dispatchers   -­‐  Suppor7ng  seasoned   dispatchers  
  44. 44.                
  45. 45.   Results  of  Augmented       Cogni9on  Program       Summarized  in  AugCog         Prac99oner's  Guide   ü ASTUTE’s  proac7ve   systems  can  benefit  from   AugCog  lessons-­‐learned  
  46. 46.     Where  we  were  at  end  of   Augmented  Cogni9on   Program:       Need  for  broadening  scope  of   human  state  assessment                                                  
  47. 47. Next Steps: Develop  plethora  of  sensors  &   measures  that  drive  adap9ve   systems   Sensors  to   Measure   Cogni7ve  State   Workload,   Uncertainty,   Confusion,  etc.       Adap7ve  Smart   Glasses   Adap7ve  Smart   Phone   Sensors  to   Measure   Physical  State   Sensors  to   Measure   Affec7ve  State   Body  Temp.,         O2  Level,       Physical  Fa7gue   Adap7ve  Smart   Home   Adap7ve  Smart   Tablets   Anxiety,  Fear,   Stress,   Confidence   Adap7ve  Smart   Cars  
  48. 48. Extensions  aber  Augmented  Cogni9on   Where  DI  has  taken  AugCog  R&D  
  49. 49. Appling  SIMI  to  Image  Analysis  
  50. 50. SIMI  for  Image  Analysis   DI  used  EEG  and  eye-­‐tracking  to  develop  real-­‐7me                                         neurophysiological  indicators  of  ‘interest’  during                                                                                                     image  analysis     Sensor:  Eye  tracking       Measure:           Ø  Capture  parameters  to  assist  in  determining  ‘interest’  (e.g.,  fixa7on   dura7on,  pupilometry)     Sensor:  Electroencephalography  (EEG)   Measures:   Ø  Previously  used  to  indicate  individual’s  workload,                                                       arousal,  aeen7on,  drowsiness,  percep7on  of  events   Ø  Image  level  analysis:  iden7fy  images  that     contain  one  or  more  points  of  interest   Ø  FLERPs:  Fixa7on  level  analysis:  iden7fy  ‘interest’     at  specific  fixa7on  points  within  an  image   ü ASTUTE’s  proac7ve   systems  should  consider   use  of  FLERP’s  
  51. 51. SIMI  for  Image  Analysis   Depic7on  of  all  fixa7on  points  on  the  image     drawn  on  an  Area  Of  Interest  (AOI)  layer                  
  52. 52. SIMI  for  Image  Analysis   •  Fixa7on  dura7on         (ms)  in  addi7on  to     the  loca7on                 Fixa7ons  greater  than  700ms   •  Define  Hits,  Misses,   Correct  Rejec7ons,   False  Alarms   Behavior-­‐based  classifica7ons  
  53. 53. Appling  SIMI  to  Instrument     Flight  Panel  Training  
  54. 54. Sensors:  Eye  Tracking,  EEG,  &  Performance   Measures:  Visual  Aeen7on  Alloca7on,  Controls  Engaged,  Errors,  Cogni7ve  Measures   Diagnosis:    Skill  Deficiencies  –  Not  looking  at  Al7meter;  Workload  High   ERROR Altitude out of range HIGH WORKLOAD
  55. 55. Performance  Feedback  
  56. 56. Look  at  Error  Details!   ü ASTUTE’s  proac7ve   systems  should  consider   linking  errors  to   physiological  data  for   more  in  depth  diagnos7cs  
  57. 57. Error  Detail  Screen  Cont.  
  58. 58. Error  Detail  Screen  Cont.  
  59. 59. Error  Detail  Screen  
  60. 60. Error  Detail  Screen  
  61. 61. Diagnosis:    Looking  at  Al7meter;  Skill  Aeained…  Now:  Arousal  Issue  –   Time  to  move  on  to  next  training  objec7ve   ERROR DETECTED! Altitude out of range LOW WORKLOAD/ LOW AROUSAL
  62. 62. Appling  SIMI  to  Baggage  Screening  
  63. 63. SIMI  for  Baggage  Screening:   ScreenADAPT   Individualized  training  for  visual  search:                   –  Sensors:  U7lizes  eye  and  EEG-­‐based  sensor  technology     –  Measures:  Real-­‐7me  cogni7ve  state  and  performance  evalua7on   –  Diagnosis:  Non-­‐op7mal  cogni7ve  state,  exper7se  level,  and   deficiencies/inefficiencies  in  screening  performance     –  Adapts  in  real-­‐7me  to  op7mize  training:   –  Tailored  feedback  and  training   »  Exposure  training   »  Discrimina7on  training   –  Image  generator:     »  Allows  instructor  upload  of  new  threats,     distractors,  bags   »  Produces  endless  combina7ons  of  image     components  to  avoid  image  repe77on     Provides  individualized  training  for  visual  search  skills  
  64. 64. Sensors  &  Measures   Performance:  Error  Classifica7on                   –  Hit,  Miss,  Correct  Rejec7on,  False  Alarm   –  Recogni7on  error  -­‐  looked  at  threat,  didn’t  flag  threat   –  Scanning  error  -­‐  didn’t  even  look  at  threat   Cogni7ve  State  via  EEG   –  Readiness  to  Learn     •  Workload  and  drowsiness   Eye  Tracking   –  Gaze  paeerns   –  Recogni7on  Error       ü ASTUTE’s  proac7ve     systems  should  consider   use  of  recogni7on  and   scanning  error  diagnos7cs  
  65. 65. Two  Adap9ve  Training  Techniques     Exposure  training  used  to  strengthen  object  detec7on  ability  when  trend     of  False  Alarms  is  iden7fied                   –  Includes  both  immediate  and                                                                                                                                                                 delayed  feedback  to  support                                                                                                                                                                                         prac7ce  and  training         Discrimina7on  training  used  to  strengthen  object  recogni7on  when  trend   of  Misses  is  iden7fied   ü ASTUTE’s  proac7ve   systems  should  consider   use  of  adap7ve  feedback   based  on  performance   trends  
  66. 66. SIMI  Applied  to  Emergency  Dispatch   Which  sensors,  measures,  diagnoses,  and  adap9ve  strategies?  
  67. 67. SensorIT  –  MeasureIT  -­‐  DiagnoseIT   then  Proac9vely  ADAPT!   •                  ASTUTE’s  objec7ves  are  to  increase  situa7onal   awareness  and  improve  decision  making   •  Situa7onal  Awareness:   •  •  •  •  •  Decision  Making:   •  •  •  •      How  are  you  sensing  SA?   How  are  you  measuring  SA?   How  are  you  diagnosing  SA?   How  are  you  adap7ng  to  SA?   How  are  you  sensing  decision  making?   How  are  you  measuring  decision  making?   How  are  you  diagnosing  decision  making?   How  are  you  adap7ng  to  decision  making?   proac7ve   ü ASTUTE’s   systems  should  consider   how  to  sensor,  measure,   diagnose,  and  adapt  to  SA   level  and  DM  performance      
  68. 68. SIMI  Applied  to  Emergency  Dispatch                  
  69. 69. SIMI  Applied  to  Emergency  Dispatch   Gamma  waves  reveal  high   engagement       Beta  waves  reveal  over-­‐   arousal               EEG  Captures  Data   Link  to  Context  –  Find   Decision  Error  –  over   arousal  and  high   engagement  were   due  to   CONFUSION   Add  clarifying  info  to   address  decision   error  
  70. 70. Augmented  Cogni9on:  Remaining  Hard  Problems   Ø  Sensors                   §  Real-­‐7me,  noninvasive,  highly  sensi7ve  and  reliable   sensors  to  gather  neuro/physiological  data  rela7ng  to   human  cogni7ve,  physical,  and  affec7ve  state   Ø  Building  generic  human  state  classifiers   §  §  Determining  which  technology  yields  the  best  results   (e.g.  AI  vs  neural  networks  vs  machine  learning)   Using  the  specific  and  extrapola7ng  to  the  'generic’   Ø  Measures  –  measures  –  measures   §  Developing  valid,  reliable  measures  of  a  plethora                                   of  cogni7ve,  physical,  and  affec7ve  state  
  71. 71. Augmented  Cogni9on:  Remaining  Hard  Problems   Ø  Designing  seamless  adapta7on  techniques                     Ø  When  to  adapt:   Ø  Ø  Ø  Valid  and  reliable  classifiers  used  to  gauge  when  to  adapt     Ø  Classifiers  available  today  are  isolated  single  measures  of  1   state  (e.g.,  arousal,  workload)  -­‐  live  tutors  take  in  the  user   experience  as  a  whole  –  classifiers  need  to  be   mul7dimensional   Threshold  that  triggers  when  to  adapt   Ø  Range  of  performance  within  which  to  adapt  at  vs.  above/ below  a  given  threshold   Ø  ‘Generalized  theory’  [if  there  is  one]  that  can  drive   adapta7on  triggers  -­‐  want  to  avoid  the  yo-­‐yo  effect  of   'mi7ga7on  on',  'mi7ga7on  off'  paeern   Level  of  granularity  where  adapta7on  takes  place  -­‐  target   'paeerns  of  error'  or  individual  instances  of  error?  
  72. 72. Augmented  Cogni9on:  Remaining  Hard  Problems   Ø  Designing  seamless  adapta7on  techniques                     Ø  How  to  adapt:   Ø  Ø  Ø  How  to  transi7on  between  mi7ga7ons  -­‐  when  one  supersedes   another  and/or  mul7ple  mi7ga7ons  may  be  used  -­‐  how  to   effec7vely  insert  mi7ga7on  without  distrac7ng  the  user   Validated  mi7ga7on  strategies  proven  to  improve  opera7ons  or   training  for  a  given  diagnosis  (e.g.,  is  mi7ga7on  different  if  error   was  found  due  to  frustra7on  vs.  boredom?)   Determining  how  to  leverage  different  interfaces  to  achieve  the   same  results  (text  versus  speech  versus  earcons  versus  hap7c   language)   ü ASTUTE  could  tackle  many   of  these  challenges  as  they   develop  their  proac7ve   system  concept  
  73. 73. The  Future  of  HSI   An  R&D  agenda  to  direct  the  HSI  field  through  2050    
  74. 74. Where  to  from  here?   Augmented  Cogni7on  –  field  started  ~2000                   –  Paradigm  shi`  from  ‘dumbing  down  interac7ons’  via   WIMP  interfaces  to  dissolving  the  user  interface   through  direct  brain-­‐computer  interfaces   –  Brought  about  interdisciplinary  teams  focused  on   monitoring  and  mi7ga7ng  human  processing   limita7ons  within  opera7onal  environments   –  Focused  on  revolu7onizing  human-­‐system  integra7on   a`er  decades  of  being  “locked”  in  the  WIMP  paradigm     Aber  a  decade  of  AugCog,  it’s  9me  to  ask  again…  Where   should  HSI  go  from  here?  
  75. 75. First:    Iden9fied  HSI  “Enablers”   Other  HSI  Enablers  Beyond  Human  Augmenta9on  
  76. 76. Enhancing  Human  Performance   HSI  Enabler:  Augmenta9on                       Augment   Eliminate   Increase  dynamic  range  and  number  of   couplings  through  next  genera7on   neuroadap7ve  systems  that  achieve  synergis7c   coopera7on  among  human  physical,  cogni7ve,   and  affec7ve  states   ü ASTUTE’s  proac7ve   systems  are  incorpora7ng   human  augmenta7on  
  77. 77. Enhancing  Human  Performance   HSI  Enabler:  Big  Data                         Iden7fy  how  best  to  leverage  big  data  to   simplify  user  decision  making  without   overwhelming  a  user’s  analy7cal   capabili7es   Big  Data   Simplify   Augment   Eliminate   ü Is  ASTUTE  considering  use   of  big  data?  
  78. 78. Enhancing  Human  Performance   HSI  Enabler:  Autonomy                         Iden7fy  how  best  to  design  autonomy  into   a  mission  such  that  performance  is   op7mized  and  unintended  opera7onal   consequences  are  avoided   Combine   Autonomy   Big  Data   Simplify   Augment   Eliminate   ü Is  ASTUTE  considering  use   of  autonomy?  
  79. 79. Enhancing  Human  Performance   HSI  Enabler:  Transgenics       Gene7cally  alter  the   human  via  gene  therapy,   gene7c  breeding,  and   gene7c  engineering                   Resequence   Transgenics   Combine   Autonomy   Simplify   Big  Data   Eliminate   Augment   ü Is  ASTUTE  considering  use  of   transgenics  (e.g.,  gene7c   engineering  to  enhance  heat   acclima7on)?  
  80. 80. Second:    Iden9fied  HSI  “Eras”   What’s  beyond  the  digital  era?  
  81. 81. Enhancing  Human  Performance   HSI  Eras   Ø  To  map  out  an  HSI  R&D  agenda  through  2050,  the  current  and   future  eras  that  human-­‐systems  interac7on  will  traverse       through  were  considered               81  
  82. 82. Enhancing  Human  Performance   HSI  Eras   Ø  Individualism  Op7miza7on:                   –  Brain  era  seeks  to  achieve  “super-­‐intelligence”   –  Physical-­‐feat  era  seeks  to  achieve  “super-­‐humans”   through  symbio7c  coupling  of  human  and  machines  to   overcome  universal  human  limita7ons   Ø  Collec7vism  Op7miza7on:   –  Human  quantum-­‐entanglement  era,  which  will  support   human-­‐human  communica7on,  where  crea7on  of  human   “superorganisms”  is  the  end  goal     –  Ecological  quantum-­‐entanglement  era,  which  will  support   synergy  between  humans  and  their  environment,  where   “super-­‐symbiosis”  is  the  end  goal    
  83. 83. Enhancing  Human  Performance   HSI  Eras:  Individualism  Op9miza9on  –  Brain  Era   Seeks  to  achieve  super-­‐intelligence                   –  Ar7ficial  intelligence  becomes  an  exocortex  to   eliminate  need  for  brainpower   –  Big  data  algorithms  transform  data  into   intelligence   –  Extend  innate  intelligence  with  cogni7ve   prostheses   –  Resequence  our  brains  to  fundamentally  improve   intellectual  capacity   ü ASTUTE  is  considering  use   of  tablet-­‐based  and  PDA   cogni7ve  prostheses  
  84. 84. Enhancing  Human  Performance   HSI  Eras:  Individualism  Op9miza9on  –  Physical  Feat   Seeks  to  achieve  super-­‐humans                   –  Exoskeletons  and  psychos7mulants  used  to   enhance  human  physical  ability   –  Cloud  used  to  monitor  human  ac7vi7es  to  fuel  big   data  algorithms  that  can  realize  vast  expansion  of   human  physical  poten7al  and  op7mize  health   –  Cyborgs  become  reality  –  implants,  prosthe7cs,   psycho-­‐pharmacological  agents   –  Gene7c  muta7ons  op7mize     human  physical  capacity  to     protect  against  disease   ü Is  ASTUTE  considering   means  to  enhance  physical   ability  and  resilience?  
  85. 85. Enhancing  Human  Performance   HSI  Eras:  Collec9vism  Op9miza9on  –  Human             Quantum-­‐Entanglement  Era   Seeks  to  achieve  superorganisms                   –  Robots  synergis7cally  augment  human  collec7ve   –  Big  data  algorithms  combine  carbon  and  silicon-­‐ based  intelligence  into  a  single  collec7ve   consciousness   –  Human  to  human  quantum  entanglement  for   collec7ve  intelligence   –  Gene7cally  alter  humans  to  allow     for  chemical  communica7on,     telepathy,  and  other  means     to  support  communica7on   ü Is  ASTUTE  considering   means  of  achieving   collec7ve  intelligence?  
  86. 86. Enhancing  Human  Performance   HSI  Eras:  Collec9vism  Op9miza9on  –  Ecological   Quantum-­‐Entanglement  Era   Seeks  to  achieve  super-­‐symbiosis                   –  Biosphere  morphs  to  op7mize  human  environment   collec7ve   –  Directly  sense,  measure,  and  understand  molecular   processes  in  collec7ve  environment   –  Sensor-­‐enabled  ecological  scavengers  to  predict  and   adapt  the  environment  to  op7mize  symbiosis     –  Gene7cally  alter  human  such  that     they  are  beeer  suited  for  their     environment,  thereby  elimina7ng     heat  sensi7vity  and  other                                                               maladapta7ons   ü ASTUTE  is  collec7ng  and  leveraging   environmental  data;  are  they  considering   use  of  adaptable  biospheres?  
  87. 87. Emerging  HSI  Eras:  Big  Data  HSI  Enabler   Extends  Info  Highway                       Brain  Era  populates  the  Neurosphere   Physical  Feat  Era  populates  the  Physiosphere   Human  Quantum-­‐Entanglement  Era  populates  the   Noosphere   Ecological  Quantum-­‐Entanglement  Era  populates  the   Biosphere   ü Is  ASTUTE  collec7ng  and   coordina7ng  data  from   neurosphere,  physiosphere,   noosphere,  and  biosphere?    
  88. 88. HSI  Emerging  Concepts:  HSI  Eras  &  Enablers                  
  89. 89. HSI  R&D  Agenda  through  2050                   ü ASTUTE  can  enhance   human  performance  by   considering  4  HSI  Eras  and   4  HSI  Enablers  
  90. 90.                             The  future  of   HSI…                          
  91. 91. Conclusions   Ø  ASTUTE  aims  to  implement  proac7ve  systems  that   increase  SA  and  enhance  decision  making     Ø  Lessons-­‐learned  from  adap7ve  automa7on  and       augmented  cogni7on  R&D  can  inform  the  design  of     ASTUTE’s  proac7ve  systems     Ø  ASTUTE  should  also  look  to  future  HSI  emerging         concepts:   Ø  Consider  HSI  enablers  beyond  human   augmenta7on,  to  include  autonomy,  big  data,  and   transgenics   Ø  Consider  advances  in  other  HSI  eras  beyond  the   brain  era,  to  include  physical-­‐feat,  human   quantum-­‐entanglement,  and  ecological  quantum-­‐ entanglement  eras  
  92. 92. Acknowledgments   Special  thanks  goes  to  Innoviris  for  their  support  of  this  presenta-on.    Any   opinions,  findings  and  conclusions  or  recommenda-ons  expressed  in  this     material  are  those  of  the  author  and  do  not  necessarily  reflect  the  views  or     the  endorsement  of  Innoviris.                
  93. 93. Acknowledgments   This  material  is  based  upon  work  supported  in  part  by  the  Office  of  Naval   Research  (ONR)  under  contracts  N0001413M0047  and  N00014-­‐09-­‐M-­‐0385,     Department  of  Homeland  Security  (DHS)  under  contracts  N10POC20028     and  D11PC20053,  Defense  Advanced  Research  Projects  Agency  (DARPA)       under  contracts  W31P4Q-­‐06-­‐C-­‐0041  and  W31P4Q-­‐07-­‐C-­‐0214,  and  the  Air     Force  Research  Laboratory  (AFRL)  under  contracts    FA8550-­‐06-­‐C-­‐0151  and     FA8550-­‐06-­‐C-­‐0151.    Any  opinions,  findings  and  conclusions  or       recommenda-ons  expressed  in  this  material  are  those  of  the  author  and  do   not  necessarily  reflect  the  views  or  the  endorsement  of  ONR,  DHS,  DARPA,   and  AFRL.    

×