The document discusses extreme simulation scenarios (ESS) involving whole brain emulation, virtual autonomous zones, and utility fog. It summarizes each ESS concept and then addresses criticisms of ESS from technical, moral, metaphysical, and unpredictability perspectives. It concludes that assessments of ESS are subject to two types of biases: worldview bias where conclusions are influenced by prior beliefs, and competence bias where flaws in judgment cause erroneous conclusions despite open-mindedness.

1. Extreme Simulation Scenarios Thinking about the promise, risk, and plausibility of AI & VR AmonTwyman, DPhil

2. A) Transhumanism, simulation, VR & AI B) ESS: Extreme Simulation Scenarios Whole Brain Simulation (AKA Uploading) Virtual Autonomous Zones Utility Fog C) Historical precedent D) Criticisms of ESS Technical arguments Moral arguments Metaphysical arguments Convergence, chaos, & unpredictability E) Assessment of ESS: Two types of bias Worldview bias Competence bias Errors of Judgment & Decision Making (JDM) Conflation of factors 1. Promise 2. Risk 3. Technological plausibility 4. Congruence with belief system

3. Transhumanism, simulation, VR & AI

4. A) Transhumanism, simulation, VR & AI B) ESS: Extreme Simulation Scenarios Whole Brain Simulation (AKA Uploading) Virtual Autonomous Zones Utility Fog C) Historical precedent D) Criticisms of ESS Technical arguments Moral arguments Metaphysical arguments Convergence, chaos, & unpredictability E) Assessment of ESS: Two types of bias Worldview bias Competence bias Errors of Judgment & Decision Making (JDM) Conflation of factors 1. Promise 2. Risk 3. Technological plausibility 4. Congruence with belief system

5. ESS: Extreme Simulation Scenarios Whole Brain Emulation (AKA Uploading) Replication of CNS activity in a simulation at one or more of several functional levels. Virtual Autonomous Zones (VAZ or ‘Polis’) A VR which hosts AI or uploaded minds and acts as a sovereign “state” (terms from Hakim Bey & Greg Egan). Utility Fog Distributed nano-scale manipulators which, among other things, could instantiate VR in the “physical” world.

7. Large-scale neural aggregates

8. Individual neurons

9. Molecular or atomic modelling

11. Utility fog Nano-robotic “swarms” which can rapidly reconfigure to create physical objects or environments, but apparently disappear when evenly distributed through a space. “Utility fog” & “Foglets” were terms coined by Dr John Storrs Hall, although the idea of nano-swarms has been around since at least 1964. Utility fog could, in principle, be the realization of Ivan Sutherland’s “Ultimate display”.

12. A) Transhumanism, simulation, VR & AI B) ESS: Extreme Simulation Scenarios Whole Brain Simulation (AKA Uploading) Virtual Autonomous Zones Utility Fog C) Historical precedent D) Criticisms of ESS Technical arguments Moral arguments Metaphysical arguments Convergence, chaos, & unpredictability E) Assessment of ESS: Two types of bias Worldview bias Competence bias Errors of Judgment & Decision Making (JDM) Conflation of factors 1. Promise 2. Risk 3. Technological plausibility 4. Congruence with belief system

13. Historical precedent 1950s-1960s: Turing & the Dartmouth Conferences Ivan Sutherland (“The Ultimate Display”) Happenings & Installation Art Timothy Leary & Psychedelia 1990s-2000s: End of the “AI Winter” Video & Net.Art VPL & Autodesk Leary Redux

14. A) Transhumanism, simulation, VR & AI B) ESS: Extreme Simulation Scenarios Whole Brain Simulation (AKA Uploading) Virtual Autonomous Zones Utility Fog C) Historical precedent D) Criticisms of ESS Technical arguments Moral arguments Metaphysical arguments Convergence, chaos, & unpredictability E) Assessment of ESS: Two types of bias Worldview bias Competence bias Errors of Judgment & Decision Making (JDM) Conflation of factors 1. Promise 2. Risk 3. Technological plausibility 4. Congruence with belief system

15. Criticisms of ESS Technical arguments Is this really a possible technology? Moral arguments Even if we can do this, should we? Metaphysical arguments Souls, wrath of the FSM, and other problems Convergence, chaos, & unpredictability Linear forecasting is no forecasting at all

16. Technical arguments How can we be sure we’re not leaving vital information out of a Whole Brain Emulation? Do molecular, atomic, or sub-atomic processes matter? In what sense could a VAZ be be both technically feasible and autonomous? Is it based on existing networks, in which case regulation and network security become issues. If not, what are we talking about? A lump of “cold” nano-machinery buried under Antarctic ice? How would you set up a Utility Fog that is impervious to the equivalent of “bluejacking”? Anything less could easily be fatal.

17. Moral arguments Moral arguments tend to focus on risk, but often are not concerned with wholly physical risks. Perhaps the best example is bio-conservative Francis Fukuyama. Fukuyama is among those who claim that the current human condition is closely related to human dignity. Therefore enhancement = loss of dignity (alluding to metaphysical and, ironically, libertarian arguments). These kind of critics, being inherently conservative, invariably focus on near-term biological enhancement scenarios.

18. Metaphysical arguments This kind of thing also falls under the category of opposed or incompatible worldviews. Metaphysical objections can range from the subtle (e.g. is there a “soul” that would be missing from an Upload?) to the fundamental (e.g. God would disapprove – a variant of this being the “hubris argument”). And don’t forget the Simulation Theory… the idea that we already live in a vast simulation of some sort. Might such a reality impose its ownconstraints upon simulation? The answer to these is a combination of Occam’s Razor and a healthy dose of willingness to Question Authority.

19. Convergence, chaos & unpredictability Hans Moravec, Ray Kurzweil, and a number of others have described the Law of Accelerating Returns. This suggests extreme technological change in a short time frame. Even without resorting to chaos theory,we can see that technological and cultural change feed into each other. Actual technological development and implementation are almost certainly not as predictable as some transhumanists seem to think.

21. A) Transhumanism, simulation, VR & AI B) ESS: Extreme Simulation Scenarios Whole Brain Simulation (AKA Uploading) Virtual Autonomous Zones Utility Fog C) Historical precedent D) Criticisms of ESS Technical arguments Moral arguments Metaphysical arguments Convergence, chaos, & unpredictability E) Assessment of ESS: Two types of bias Worldview bias Competence bias Errors of Judgment & Decision Making (JDM) Conflation of factors 1. Promise 2. Risk 3. Technological plausibility 4. Congruence with belief system

22. Assessment of ESS: Two types of bias Worldview bias Assumptions or preferences of the assessor may discourage them from accepting particular conclusions. 2. Competence bias The assessor may be unable to draw correct conclusions from the evidence despite being willing to do so.

23. Worldview bias An assessor may be logically impeccable and working with good evidence, and yet be unwilling to accept particular conclusions because of a priori beliefs. This is not necessarily a case of “pig-headedness”… the assessor may not even be aware of the larger belief system lying behind their specific objections.

24. Competence bias Various biases or logical flaws may cause an assessor to draw erroneous conclusions from the evidence to hand, despite a general openness to accepting whatever conclusions are produced. Such biases are often considered part of Judgment and Decision Making (JDM) research.

26. What is Decision Making?

27. Advice and Trust

28. Heuristics & Biases

29. Heuristics that make us smart

31. 1. Promise Life and death no longer mediated by biology. An end to biological disease and scarcity. New vistas for exploration 1: Space exploration no longer limited by human “payload” factors. New vistas for exploration 2: Reconfiguration of perceptual apparatus will allow exploration of a broader psychological “reality space” (i.e. interpretation or phase space).

33. Grey Goo (requiring Blue Goo)

34. Totalitarian control

35. Terminator/SkyNet scenarios

36. Ecological disaster

38. 4. Congruence with belief systems This is essentially the inclusion of Worldview Biases as a factor when assessing the value of a potential technology. What is your perspective, and how might that be colouring your assessment? Such explicit acknowledgment of personal perspective has been advocated within philosophy of science since WWII.

39. Thanks!