ISCRAM Summerschool 2009 Lecture - Teaming With Machines (Martijn Neef)

Teaming with Machines
The role of autonomous systems in
collaborative environments

Martijn Neef
TNO Defence, Safety and Security

My name is…

Martijn Neef
Networked Organizations Group
Business Unit Information and Operations
TNO Defence, Security and Safety
The Hague, The Netherlands
e-mail: martijn.neef@tno.nl

• Artificial Intelligence
• Collaborative Decision Making
• Networked Organisations
• Human – Machine Systems

1 Teaming with Machines ISCRAM Summerschool 2009

Lecture overview

• Part I: Teaming with machines
• On teams, machines and autonomy
• From automation to joint cognitive system design

• Part II: Designing Human – Machine teams
• Case study: Augmented Team design,
• A practical example from our work at TNO Safety and Security

• Part III: Implications and Discussion
• Summary and implications
• Points to ponder


Part I: Teaming with Machines
On teams, machines and autonomy

Martijn Neef

We are surrounded by machines

• Our working and living environments are filled with machines,
systems and networks

• communication devices, computers, sensors, networks, information
sources, actuators, displays, and so on…
• we are connected with and dependent on humans and machines in
many ways..


We are being surrounded by smarter machines

• Technological developments produce smarter systems
• systems that exhibit smart behaviours
• new response options through smart coupling of capabilities
• more prominent and active role for intelligent, autonomous systems
• artificial systems are being granted more autonomy.

• Lots of examples in security, safety and space
• robots and unmanned vehicles
• information management systems and intelligent agents
• traffic management systems
• safety and self-defense systems


We need a new way of thinking about teams

• Technology will:
• become more capable, adaptive and reliable
• be given more autonomy to define their own behaviour

• Increase of system autonomy leads to different ways of working
with machines:
• technology role goes from supplement to active participant
• task groups will gradually develop into hybrid teams
• we need for a new understanding of the dynamics of hybrid teams

task allocation between man and machine
interaction patterns between man and machine
coordination and communication
authorization and responsibility…


team
team

Human – Machine teams

• What is a human – machine team, what is not?
• what is a team?
• what must an artificial system be capable of?
• what are the features and implications of a human – machine team?

• Why is this an important area of research?
• we need new capabilities to face modern challenges
• we need to learn to work together with advanced systems
• tight collaboration between humans and machines yields operational
benefits… complement each other’s strengths and weaknesses.


What is a team?

Defining Characteristics of Teams

• Two or more individuals • Specialized member roles and
• Multiple information sources responsibilities
• Meaningful task • Task-relevant knowledge
interdependencies • Intensive communication
• Coordination among members • Adaptive mechanisms
• Common, valued goals • Hierarchically organized

(Salas, 1995)
requires a lot of mutual understanding
requires a common grounding
requires training and trust
usually only used for human teams


Teaming with machines

• humans working together
with artificial actors


Human – Machine Teams


Human – machine teams


‘Man – computer symbiosis’

Man-computer symbiosis is an expected development in
cooperative interaction between men and electronic computers. It
will involve very close coupling between the human and the
electronic members of the partnership.
The main aims are 1) to let computers facilitate formulative thinking
as they now facilitate the solution of formulated problems, and 2)
to enable men and computers to cooperate in making decisions
and controlling complex situations without inflexible dependence
on predetermined programs.

J.C.R. Licklider (1915-1990), Man-Computer Symbiosis
IRE Transactions on Human Factors in Electronics,
volume HFE-1, pages 4-11, March 1960


From automation to socio-technical design
make technology more capable

make the joint human – machine team
Based on Chalmers (2001)
more capable

The Substitution Myth

The Substitution Myth It is a common myth that artefacts can be value neutral in the
sense that the introduction of an artefact into a system only has the intended and
no unintended effects. The basis for this myth is the concept of interchangeability
as used in the production industry, and as it was the basis for mass production –
even before Henry Ford. Thus if we have a number of identical parts, we can
replace one part by another without any adverse effects, i.e. without any side-
effects. …

While this in practice holds for simple artefacts, on the level of nuts and bolts, it
does not hold for complex artefacts. A complex artefact, which is active rather
than passive, ie, one that requires some kind of interaction either with other
artefacts or subsystems, is never value neutral. In other words, introducing such
an artefact in a system will cause changes that may go beyond what was
intended and be unwanted

Joint cognitive systems: foundations of cognitive systems engineering,
Erik Hollnagel, David D. Woods.


Task division and coordination

• MABA-MABA: Men Are Better At,
Machines Are Better At.. typical human

knowledge intensiveness
tasks
• Task division steers coordination.
• Coordination renders tasks.
collaborative
• Challenge: find types of tasks that tasks
are suitable for each actor, and
designate tasks that can be typical agent
performed by more than one. tasks

system
tasks


Fitts’ List (1951)

Machines are constrained in that: Machines need people to:
Un-Fitts’ list
Sensitivity to context is low and is Keep them aligned to context. Hoffman (2002)
ontology-limited.
Sensitivity to change is low and Keep them stable given the
recognition of anomaly is ontology- variability and change inherent
limited. in the world.
Adaptability to change is low and is Repair their ontologies.
ontology-limited.
They are not ‘aware’ of the fact that Keep the model aligned with the
the model of the world is itself in the world.
world.
People are not limited in that: Yet people create machines to:

Sensitivity to context is high and Help them stay informed of
is knowledge- and attention-driven. ongoing events.
Sensitivity to change is high Help them align and repair their
and is driven by the recognition perceptions because they rely on
of anomaly. mediated stimuli.
Adaptability to change is high and Effect positive change following
is goal-driven. situation change.
They are aware of the fact that the Computationally instantiate their
model of the world is itself in the world. models of the world.


Levels of automation

Parasuraman (2000)


Ten Challenges for Making Automation a "Team Player"
in Joint Human-Agent Activity
1. To be a team player, an agent must fulfil the requirements of a Basic Compact
to engage in common grounding activities
2. To be an effective team player, agents must be able to adequately model the
other participant’s intents and actions vis-à-vis the state and evolution of the
joint activity – e.g. are they having trouble? Are they on a standard path
proceeding smoothly? What impasses have arisen? How have others adapted
to disruptions in the plan?
3. Human-agent team members must be mutually predictable
4. Agents must be directable
5. Agents must be able to make pertinent aspects of their status and intentions
obvious to their team-mates
6. Agents must be able to observe and interpret signals of status and intentions.
7. Agents must be able to engage in goal negotiations
8. Support technology for planning and autonomy must enable a collaborative
approach
9. Agents must be able to participate in the management of attention
10. All team members muist help control the costs of coordinated activity

Klein (2004)

Autonomy
• Autonomy: to have control over own internal state and behaviour
• Challenge: control the autonomy of autonomous systems

(Bradshaw, 2003)

Example: Human – Agent – Robot Teams

• Work from Institute for Human and Machine Cognition
• Jeffrey Bradshaw

• Demonstrate the feasibility of a human – machine team for a
security task – intruder detection and apprehension by a team of
humans and robots.

• Smart use of agreements and coordination support to optimize
joint performance.

• (movie of HART in action)


Part II: Designing Human – Machine teams

Case: Augmented Teams for Security
Missions
Martijn Neef

Augmented Teams:
Assembling Smart Sensors, Intelligent Networks and
Humans into Agile Task Groups

Martijn Neef
Networked Organizations Group
Business Unit Information and Operations
TNO Defence, Security and Safety
The Hague, The Netherlands

Martin van Rijn – Distributed Sensor Systems Group
Jan Willem Marck – Distributed Sensor Systems Group
Danielle Keus – Modelling, Simulation and Gaming Group


Augmented Teams

We are exploring design principles for augmented teams.

An augmented team consists of a collective of sensors, actuators,
information processing systems and humans
• that are interconnected by a intelligent network
• that collaborate in a close and adaptive fashion, and
• that, by presence of the artificial actors, augment the capabilities of
the human actors alone.


Design challenges

Can we come up with a design concept for augmented teams..

• .. with adaptive role- and tasking capabilities between human and
artificial actors
• Essential for agility and resilience. The team must be able to cope with
changing circumstances, e.g. by changing the behaviour or structure of
the organization.

• .. that is suitable for the current and future state of technology
• Prevent technology bias. Limit the influence of the current state or
technology on the approach as much as possible.


Design challenges

information
control

human team hybrid team with self organising
with sensor network clear divison of labour adaptive team

Basic challenges
• It must be possible to change the organization structure
• The team must be configurable at run-time
• All elements must be network-connected
• Actors must be able to represent themselves
• The information flow must self-organize


Approach

• Three main ingredients:

• Functional model
• Provides a functional blueprint for augmented teams

• Organization modeling framework
• Provides means to structure interactions and responsibilities

• Social and interaction contracts
• Provides a way to specify community rules and collaboration demands


Functional Model
Situation Command
Awareness & Control
• Networked Adaptive Interactive
Hybrid Systems model (NAIHS)
Level 3
Impact Impact • Blueprint for networked cognitive
assessment management
systems, grounded in the JDL
Situation Situation model
Level 2
• Elements fulfill functional
components.
Level 1 Object Object

• Three steering dimensions:
Signal Signal
Level 0
• level of information abstraction
• timescale of effects
Physical Data
• physical structure
Actions
Level collection

Environment


Organizational Model
• Based on OperA (Virginia Dignum, Utrecht University, NL).
• framework for the specification of multi-agent organizations
• uses a formal specification language


Social contracts, Interaction contracts
• Social contracts
• General agreements that need to be adopted to become part of the
organization (job contracts)
• organizational aspects (norms and policies, coordination scheme,
organization structure)
• social rules, administrative rules, communication language …

• Interaction contracts
• collaborative agreements between actors per task
• interaction behaviour (relation between parties, task division)
• format and conditions


Putting it all together…

a) a functional model to structure the general system

b) three levels of abstraction to represent
dependencies and interactions

c) social and interaction contracts to put
the organization into practice


Adaptivity and agility
Organization levels give means to express adaptive measures

Level Changes Description Impact

1 Real-time adjustments • Elements change their interaction agreements to Low
adapt to a certain situation.
in the Interaction • Example: Two elements decide to use a different
Model form of communication in response to new
circumstances.

2 Real-time adjustments • A role is transferred from one element to another Medium
element that is better qualified.
in the Social Model. • Example: the ‘coordinator’ role is transferred from
the actor in the control room to an actor in the field,
because he is in a better position to coordinate other
actors.

3 Real-time adjustments • The organization is redesigned to some degree. Severe
This might involve added or deleting roles, changing
in the Organizational objectives or behavior rules. Changes on this level
Model might necessitate changes in the Social and
Interaction Model too.
• Example: Because several elements have stopped
working, the objectives can no longer be reached. In
response, new objectives are set with the remaining
set of elements.


Adaptivity and agility
Approachs allows for gradual introduction of new elements
For instance: introduce artificial actors at higher functional levels

artificial actors at lower artificial actors at higher
functional levels functional levels


autonomy and
task allocation
choices
between actors
coordination
and control
methods
information
needs and
accessibility


FieldLab Indoor Safety and Security
• Fieldlab Indoor Safety
and Security

• Components
• Wireless positioning
system
• Network of smart
cameras
• Communication devices
• Tracking and position
prediction service
• Information fusion
services
• Command center with
common operational
picture

• Basic scenarios: intruder
apprehension and
incident management


Fieldlab Indoor Safety and Security


Experiments

• Network built around a service oriented network (RESTlet)
• Human contracts are still just ‘on paper’, but used strictly (before
and during an experiment)

• Some initial studies
• transfer of the coordinator role from the central position to a mobile
guard (role transfer among humans)
• transfer intruder tracking role from the camera network to coordinator
(task transfer from system to human)
• transfer tracking from positioning system to coordinator (task
transfer from system to human)

• (Movie)


Roles changes

B


Some observations

• Observations
• Using organizational models and contracts seems worthwhile to
express adaptivity and interaction dynamics in man – machine
organizations

• Functional model helps to make basic allocation choices

• it is easy to lose control over the situation after role change,
especially for the coordinator role – even in the case of human –
human role transfer.

• Need for periodical synchronization of organization awareness and
situation awareness is evident


Concerns

• Design concerns

• Define who is responsible for role and task allocation
• Set boundaries for dynamic allocation
• Ensure observability of attributes and responsibilities
• Make the type of adaptivity a design choice
• Prevent issues caused by multi-level or multi-role allocation
• Prevent communication and interaction issues after role change
• Prevent loss of situation and system awareness among humans
• Counter complacency and skill degredation
• Prevent unneccessary increase of mental workload
• Gradually build up user acceptance


VIS
Scenario 2009
B B B

detectie

• Incident observation and
reconnaissance B
SA
• Incident assessment and plan Alarm

creation Taak

Alarm
• Incident mitigation and DTA
evacuation

• Large set of virtual and actual B

sensors
command center
• Players get adaptive
B
communication devices and
new task coordination tools.

B B B B


Further developments

Current developments:
• Further formalize interaction contracts and contract management,
especially between human and artificial actors
• Explore dynamic task allocation schemes
• Explore ways to balance self-organising capabilities and procedures
• New series of experiments with additional technology (new services,
new mobile devices, more sensors) and an extensive scenario
(incident management and evacuation, larger set of human actors)

Other applications under development:
• Damage control teams aboard naval frigats
• Various distributed sensor network applications
• Training environments for civil firefighter teams


Part III: Implications and Discussion
Points to ponder

Martijn Neef

Implications

• Machine teaming is not the stuff of science-fiction movies. It’s
already here.. even though in simple forms..

• Technological advances will require us to rethink collaborative
work.

• We need to pay attention to autonomy, ethics and
accountability.

• Watch out for automation surprises and uncontrollable
adaptivity.


Discussion

• Do you have any personal experiences (good or bad) with
technology that resemble man – machine teaming? Or clearly
show the need to rethink man – machine collaborations?

• Can you imagine man – machine teaming scenarios for crisis
response scenarios?
• What would be the impact of such developments on crisis
management organisations? Would the organisation actually
change, or work differently?


thank you for your attention!


References (relevant own)

• Neef, Martijn (2006), A Taxonomy of Human - Agent Team Collaborations. In
Proceedings of the 18th BeNeLux Conference on Artificial Intelligence (BNAIC
2006), 5-6 October 2006, Namur, Belgium, pp. 245-250.

• Neef, M., van Rijn, M., Keus, D., Marck, J-W. (2009), Organizing smart networks
and humans into augmented teams. Proceedings of the 13th International
Conference on Human-Computer Interaction, 19-24 July 2009, San Diego, CA,
USA, Springer-Verlag: Lecture Notes on Computer Science, Berlin Heidelberg.

• Neef, M., van der Vecht, B. (2009), Agility Through Adaptive Autonomy,
Proceedings of the 14th International Command and Control Research and
Technology Symposium (ICCRTS 2009), 15-17 June 2009, Washington D.C.,
USA.

• Neef, M., Maanen, P.-P. van, Petiet, P., Spoelstra, M. (2009), Adaptive Work-
Centered and Human-Aware Support Agents for Augmented Cognition in Tactical
Environments. In: Proceedings of International Conference on Augmented
Cognition, Jointly held with International Conference on Human-Computer
Interaction, 2009


References (others)

• Salas, E., Dickinson, T. L., Converse, S. A., and Tannenbaum, S. I. (1992),
Toward an understanding of team performance and training. In R. W. Swezey &
E. Salas (Eds.), Teams: Their training and performance. Ablex, Norwood, USA,
pp. 3-29.
• Licklider, L.C.R. (1960), Man-Computer Symbiosis. IRE Transactions on Human
Factors in Electronics, v.HFE-1, pp. 4-11.
• Chalmers, B.A. (2001), Design frameworks for computer-based decision support.
DREA Technical Memorandum, TM 2001-210, Defence Research Establishment
Atlantic, Ottawa, Canada.
• Fitts, P. M. (1951), Human Engineering for an Effective Air Navigation and Traffic
Control System. National Research Council, Washington, D.C., USA.
• Hoffman, R.R., Feltovich, P.J., Ford, K.M., Woods, D.D., Klein, G., Feltovich, A.
(2002), A Rose by Any Other Name...Would Probably Be Given an Acronym.
IEEE Intelligent Systems 17(4): 72-80 (2002)
• Parasuraman, R., T. B. Sheridan, et al. (2000), A Model for Types and Levels of
Human Interaction with Automation. IEEE Transactions on Systems, Man, and
Cybernetics 30(3), pp. 286-297.


References

• Hollnagel, E. & Woods, D. D. (2005). Joint cognitive systems: Foundations of
cognitive systems engineering. Boca Raton, FL: CRC Press / Taylor & Francis.
• Klein, G., Woods, D.D., Bradshaw, J.M., Hoffman, R.R. and Feltovich, P.J.
(2004), Ten Challenges for Making Automation a "Team Player" in Joint Human-
Agent Activity. IEEE Intelligent Systems 19(6), pp. 91-95.
• Bradshaw, J. M., Sierhuis, M., Acquisti, A., Feltovich, P., Hoffman, R., Jeffers, R.,
Prescott, D., Suri, N., Uszok, A., and Van Hoof, R. (2003), Adjustable autonomy
and human - agent teamwork in practice: An interim report on space applications.
In H. Hexmoor, R. Falcone, & C. Castelfranchi (eds.), Agent Autonomy,
Dordrecht, The Netherlands: Kluwer, pp. 243-280.
• Kester, L.J.H.M. (2008), Designing Networked Adaptive Interactive Hybrid
Systems. Proceed-ings of the IEEE International Conference on Multisensor
Fusion and Integration for Intelli-gent Systems 2008 (MFI2008), 20-22 August
2008, Seoul, Republic of Korea, pp. 516-521.
• Dignum, V., Dignum, F., Meyer, J-J.Ch. (2004), An Agent-Mediated Approach to
the Support of Knowledge Sharing in Organizations. Knowledge Engineering
Review, Cambridge Uni-versity Press, 19(2), pp. 147-174.


ISCRAM Summerschool 2009 Lecture - Teaming With Machines (Martijn Neef)

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (12)

Similar to ISCRAM Summerschool 2009 Lecture - Teaming With Machines (Martijn Neef)

Similar to ISCRAM Summerschool 2009 Lecture - Teaming With Machines (Martijn Neef) (20)

Recently uploaded

Recently uploaded (20)

ISCRAM Summerschool 2009 Lecture - Teaming With Machines (Martijn Neef)