The document discusses the potential dystopian threats posed by AI from a hardware perspective, emphasizing the need for stringent controls on self-replication, self-repair, and energy access for AI systems. It argues for establishing hard boundaries between different levels of intelligence and trust to manage risks effectively. The author suggests that by taming potential threats, AI can be harnessed for protection rather than danger.

1. Can AI Become a Dystopian
Threat to Humanity?
A Hardware Perspective
Igor Markov
Google and The University of Michigan
The views expressed are those of my own and do not represent my employers

5. Threats to humanity?

6. Threats to humanity?

7. Threats to humanity?

8. Why are we so obsessed?

10. How did the humanity survive?

13. How did the humanity survive?
• by being smart
• by knowing the adversary
• by controlling physical resources
• by using the physical world to advantage

15. Now, back to the dystopian AI myth
• AI may become smarter than us
• Possibly malicious
• The physical embodiments are unclear

16. Black Death killed 50M people in XIV c

17. Intelligence – hostile or friendly –
is limited by physical resources

19. Computing machinery is designed
using an abstraction hierarchy
• From transistors to CPUs to data centers
• Each level has a well‐defined function

21. Introduce hard boundaries between
different levels of intelligence, trust
• Can toasters and doorknobs be trusted?
• Who can use weapons?
• Each agent should have a key weakness

22. Limit self‐replication, self‐repair
and self‐improvement

23. Limit AIs access to energy
• Firmly control the electric grid
• No long‐lasting batteries, fuel cells
or reactors

24. Tame potential threats
and use them for protection

25. Constraints on AI to intercept dystopian threats
1. Hard boundaries between levels of intelligence and trust
2. Limits on self‐replication, self‐repair and self‐improvement
3. Limits on access to energy
4. Physical and network security of critical infrastructure
Tame potential threats and use them for protection