1) The document discusses the development of AI and automation, particularly in military contexts, through the lens of philosophy on technicity and the human-technology relationship. 2) It critiques proposals to fully automate lethal robotic systems, arguing that this would excessively delegate human functions and responsibilities without accounting for human judgment. 3) The document advocates for a more nuanced, dynamic view of human cognition influenced by technology, rather than models of pure calculation or perception-action loops, and cautions against automating war prosecution without consideration for human fictional and provisional nature.

1. Bath University AI Research
Seminar
AI, Technicity and Souls:
Military AI and Automation

2. Context
• Media/cultural studies, film, animation
• Abiding interest in war as part of
culture/media
• And philosophical work on
technology and its part in
human culture, media, war etc.

3. Aim
• PART 1 Situate development of AI/automated robotics
generally as human technological development – philosophy of
‘technicity’
• PART 2 Some comments on war/military technoscience (and
AI)
• PART 3 A critique of Ronald
Arkin’s propositions
concerning the automation
of lethal robotic systems
(more a ‘diagnosis’ of his
thinking – and I promise to
say something about souls)

4. PART 1 – ‘TECHNICITY’

6. Andre Leroi-Gourhan
• ‘Exteriorisation’ – exporting of functions (hunting,
gathering, shelter) to external objects
• Non-organic, quasi-
evolutionary dynamic
Ethno-cultural not genetic
evolution;
‘epi-phylogenetic’ not
phylogenetic…

7. Stiegler
• The long dance of the ‘who’ and the ‘what’:
Both ‘lead’ and both ‘follow’ so much so it is
impossible to say once and for all who or what
came first
The human ‘being’ is a becoming.
Human being is a being-in-default of an
essential nature….

8. • Show clip from The Ister…. Of the ancient
Greek myth of the origin of humanity.

9. Technicity
= this condition of not having a stable, essential condition.
• We become in relation to technological dynamics that are
not simply our tools to make our life easier, our labour
more efficient. But nor are they independent of us
completely
• Organology: study of relations of the 2 kinds of “organs”:
the biological (brain, body, group) and the technical
(Ancient Greek organon – tool, instrument )

10. Eg. war
• (war and technological modernity go hand in
glove)
Mass production and arms
manufacture
Mass production of killing

12. Ballistics calcs, then
H-bomb modelling and simulation
And these things…..

13. War is ‘fictional’
• Strife over ways of living (= ways of adopting
technically enabled possibilities)
• Who are we/what are we to become (or
remain? Or ‘go back to’?)
• ALL FICTIONS (which is not to say
unimportant: on the contrary! Pay strict
attention to your ‘stories’ and ‘histories’)
• And remember they are technically
implemented and conditioned/shaped

14. Some of the adoptions of these
technological developments

15. These developments are double-
edged, not neutral tools but dynamics

16. PART 2: AI (‘Cognitive simulation’)
Arthur Samuels’ Checkers program for IBM 1956
Stan Ulam H-Bomb researcher team’s ‘Los Alamos Chess’
program on the MANIAC at Los Alamos National Lab, 1956

17. AI ‘paradigms’
• This
is the model of this
(symbolic manipulation)
• Embodied intelligence (material process, emergent
quality of lower level operations)
Marco Dorigo’s Swarmanoid project

18. Technicity is missing
• Classic AI: yes of course the digital computer is
a model of thinking. But not the first, the only
or the ultimate one.
and
• Thinking has already been coloured/directed
technically and technologically in a long
history of different histories of cultural
beliefs/concepts/ways of living

19. Other models of thinking
Above: Watt & Boulton’s Steam engine
governor;
Left: Indigenous Australian sand-painting
ceremony

20. Colouring of ‘thought’
Jacquard Loom

21. And…
Peter Galison: the “Manichean
sciences” of dealing with the
“calculating enemy other”

22. Technocultural Programme
• Leroi-Gourhan: culture amounts to a pro-
gramme for behaviour, ways to live, that is
technical and conceptual, exterior and interior
• Stiegler: this programme is a response to the
‘who are we?’ question of human being, the
being without an essential identity.
• It programs our thinking…..

23. AI
• A programming development “programmed”
by the Western technocultural programme.

24. PART 3: ARKIN’S DEADLY AI
or, ‘artificial stupidity’

25. In the future….
Advances in computing speeds and capacity will change how
technology affects the OODA loop [Observe, Orient, Decide
and Act]. Today the role of technology is changing from
supporting to fully participating with humans in each step of
the process. In 2047 technology will be able to reduce the
time to complete the OODA loop to micro or nano- seconds.
Much like a chess master can outperform proficient chess
players, UAS will be able to react at these speeds and
therefore this loop moves toward becoming a “perceive and
act” vector. Increasingly humans will no longer be “in the
loop” but rather “on the loop” – monitoring the execution of
certain decisions. Simultaneously, advances in AI will enable
systems to make combat decisions and act within legal and
policy constraints without necessarily requiring human input
(41).

26. Actually, already
Samsung SGR-1

27. Arkin
Governing Lethal Behavior in Autonomous Robots, Boca Raton FL, Chapman &
Hall/CRC, 2009)

28. ‘Ethical governor’
• ‘inspiration’ from Watt’s
mechanical governor that
was ‘intended to ensure that
the mechanism behaved
safely and within predefined
bounds of performance’
(128)
• A ‘bolt-on component
between the hybrid
architectural system
[robot/software/human
operator] and the actuators’
(127)

29. Critiques and opposition

30. Exteriorisation unbounded
• Delegation of functions/capacities that will
wreck the dynamic to and fro between:
‘human’ and technical extension/prosthesis
Thought/judgment/consideration/care/responsi
bility/self-reflection
And
Action, procedure, task, duty, system of
operations, Technocultural ‘programme’

31. Forgetting…
• Our fictional character
• Our provisional, contingent character
• That war is a failure to negotiate these fictions
peacefully
• Automating war’s prosecution is the worst
imaginable response to this failure

32. A better (if older) model of thought
• Not Calculation—decision, or ‘perceive and
act’ models
• But intermittent emergence of higher
‘intelligence’ out of the constant interactions
of the lower ‘souls’
• Aristotle’s De Anima – soul as animating
force/principle

33. 3 souls of living beings
• Vegetative soul: capacity to nourish and
reproduce (eg. plants)
• Sensitive soul: capacity for sense perception
and movement/action in response (eg.
animals)
• Noetic soul (for Aristotle, only the human
posesses this): capacity for self-reflection,
thought

34. Stiegler: read Aristotle dynamically
• The capacities are overlapping and in dialogue
• The ‘noetic’ is a potential of the human, that
appears intermittently
• Always the potential of ‘regression’ to less
reflective, more ‘automatic’ modes of acting
And
• The human is always a human-technical
composition, so ‘our’ noetic, intelligent
potential is technically conditioned,

35. As is our
‘regression’
DARPA Long-range anti-ship
missile
‘LOCUST’ system– US Office of Naval
Research