The document summarizes a presentation on unmanned maritime systems. It discusses trends toward greater autonomy at sea, including remote-controlled and autonomous vessels. It also explores how new human-machine interfaces using simulation, gaming, augmented and virtual reality could train future sailors to work with autonomous systems. The presentation considers both benefits and risks of replacing humans and stresses the need for careful testing and training to maximize the strengths of both humans and machines.

1. of ###
1
Digital Sailor
Andy Fawkes
13 May 2021
SMi 4th Annual Unmanned
Maritime Systems
Technology Conference

2. The
Presenter
Andy Fawkes
FIMechE
Twitter
@andyfawkes1
LinkedIn
andyfawkes
email
andy.fawkes@thinke.co.uk
Before Now

3. Earlier Presentations/Publications

4. Harmony
Human(s)
Simulation
Autonomous
Systems

5. My Feelings on Unmanned Systems/Automation

6. Outline
• First Thoughts
• It’s a Trend… The Good, the Bad,
and the Ugly
• New Sailors, New Human/
Machine Interfaces?
• Simulation, Gaming and Remote/
Autonomous Systems
• Final Thoughts

7. First Thoughts

8. Battlesbury
Barracks,
Warminster
UK

9. VRLT Main Training Audience
Dismounts
Warrior
1
Warrior
2
Warrior
3
Challenger 2
Tank 31

10. The nature of warfare
War is chaotic
No battle plan
withstands
contact with
the enemy
Humans are
Very Adaptable

11. Why
Replace
the
Human?
•Dull
•Dirty
•Dangerous
•Dear

12. Autonomy and Communications

13. It’s a Trend…
(The Good, the Bad, and
the Ugly)

14. RN Frigates – Crew per Displacement Tonne
?

16. Remote Pilots
Gaumont British Newsreel (Reuters)
Radio Controlled “Queen Bee” (1936)

17. Sea Cat Anti Air Missile (1963)

18. Some Military Autonomous Systems
US Army
Manned-
Unmanned
Teaming
(MUM-T)
US Navy
Autonomous
Craft
US Navy
X-47B
US F-18 &
103 Drone
Swarm

19. Boeing MQ-25 Stingray (Aerial Refueling Drone) – Dec 2020

20. NASA’s Ingenuity Helicopter – First Flight – 19 April 2021
Live Video From Telemetry Data

21. Redundant
Pilots?
21
2016 2020

22. “AlphaGo”
(2016)
&
“AlphaGo
Zero”
(2017)
• AlphaGo beats top
Human Go Player
• The next
incarnation was
“AlphaGo Zero”,
with a new neural
net design
• Trained in 3 days,
no human
intervention except
Go rules
• AlphaGo Zero beats
AlphaGo – 100 to 0

23. Google
DeepMind
2019

24. Edge AI Computing

25. Data Misjudgements
Google Street View

26. BeamNG Vehicle AI (2021)

27. Monitoring Systems

28. Boeing 737 MAX – Sensors, Automation & Humans
Ethiopian
Airlines Flight
302

29. Paul M Fitts (1951)
Human Engineering for an Effective Air Navigation and Traffic Control System
• “great caution be exercised in assuming that men can successfully monitor
complex automatic machines and take over if the machines break down”
• “engineers should seriously consider systems in which machines would
monitor men and prevent them from making dangerous mistakes”

30. Test & Train
Test & Train
Test & Train
…..
Test & Train
Human(s)
Simulation
Autonomous
System

31. New Sailors, New
Human/Machine
Interfaces?

32. Sailors of the Future?
•Who will they be?
•What will they do?
•Where will they be?
•How will they train?
•Will we need them?

33. The Next Workforce
Wikipedia

34. RN Age Distribution
50% of the RN is < 30
and 80% < 40
UK Defence Personnel Statistics - CBP7930, 12 June 2018
Broadly, 2/3rds of the RN
now are Digital Natives
and 90% by 2030

35. 93% of Children Play
Video Games
Children's Commissioner for England (2019)

36. “teenagers playing on
their Xbox at home
should take their
gaming to the next
level and join the RAF
to operate real
drones”
Air Chief Marshal
Sir Stephen Hillier
Jan 18

37. Human
Machine
Interfaces

38. XR

39. VR-Based
Training at
Scale
(2019)
• 18,500 Lufthansa cabin crew practice
safety-related training required by the
authorities in 18 VR cabins located in
Frankfurt and Munich

40. Mixed Reality
in Operations

41. See What the
Machine Sees
Through AR
(Augmented
Reality)
Iviz

42. VR and Divers?
NASA Astronaut Trainer (2018)
MIT Media Lab (2016)

43. Voice
Controlled
Autonomous
Systems
UK
43
www.cervusdefence.com/case-
studies/exploring-voice-
controlled-autonomous-systems

44. Brain
Interface
44

45. Telexistence (Tachi Lab)

46. VR RHIB Simulator
RN working with Novatech, the
University of Portsmouth, and
Bohemia Interactive Simulations
• 3-DoF simulator chair
• Replica control console
• Gaming style steering wheel
• Oculus Rift headset
• VBS3 Software

47. Simulation, Gaming and
Remote/Autonomous
Systems

48. Simulation Now - Land, Sea, Air, Anywhere

49. Military Access to Simulation
1990 2000
Numbers
10’s
100’s
1,000’s
2010 2020

50. Simulation
(VBS3)
Experimentation
Autonomous
Systems

51. Unity Games Engine
Machine Learning Agents
AI Training

52. Gaming Innovation
Apex Legends (Team-Based Battle Royale)
• Released 2019 - 50 Million Players Joined in One Month
• A novel ping system allows teammates to mark supplies,
locations, and enemies

53. • Epic Games exploits Amazon Web Services (AWS)
• Fortnite servers around the world to reduce latency
• Each month Epic Games collects two petabytes of data
• Data can be used to make games more “compelling”
Fortnite Hosting

54. Stovepipes of Data

55. What about “Digital Twins”?
Polarkonsult A/S

56. Final Thoughts

57. Ender’s Game Film - 2013

58. Ender’s Game
Leadership
Remotely-Operated/Autonomous Agents
Intuitive Interfaces

59. Maximise
the
Strengths
of Both

60. Test & Train
Test & Train
Test & Train
…..
Test & Train
Human(s)
Simulation
Autonomous
System

61. Questions?
(the slides will be made available)