The Pervasive Experience - project review July 2010


Published on

This document reviews the Pervasive Experience project. In this project the driving assumption is that increasingly pervasive, networked technologies are impacting our lives. The research question is: How is Pervasive Computing changing you?

Published in: Technology

The Pervasive Experience - project review July 2010

  1. 1. The Pervasive Experience will a second Cognitive Revolution turn us into d-zombies... Project Review : July 2010 / @nambor photo by club silencio
  2. 2. what the hell does that mean?!
  3. 3. what is the Pervasive Experience? Ubiquitous Computing was pioneered by Weiser (1991) at Xerox Parc. “Ubiquitous computing names the third wave in computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives.” Mark Weiser That was obviously from a 90's perspective. This project proposes a structural and ethical argument that Pervasive is a more appropriate and Humanistic term than Ubiquitous. So the Pervasive Experience is a new, truly modern Human Condition that is now possible in today's anytime, anywhere internet world.
  4. 4. what is a Cognitive Revolution? From Descartes (1637) and the Enlightenment of the 1600's through to the 1950's and 60's, Positivism was the dominant epistemology in scientific thought and Western Philosophy. Built upon this world view, Pavlov (1904) and Skinner (1938) defined Behaviourism within the field of experimental psychology. Yet starting in the 1950's the dominant epistemology began to turn towards Realism. A reaction against Behaviourism became a movement known as the Cognitive Revolution. This movement recognised our internal phenomenological experiences as a valid object of research.
  5. 5. what was that about Zombies? Chalmers (1995) is an interesting Philosopher that has contributed to the field of Consciousness. In order to explore and discuss the concepts surrounding the phenomenological aspect of Consciousness he has proposed what has become known as the Philosophical Zombie thought experiment. In its simplest form it poses the question “if people existed that behaved exactly like a normal human, but didn't experience the phenomenon of internal representation – are they still human?”. These meaty automata like creatures are called Philosophical Zombies or p-zombies for short. It can also be adapted to explore similar questions about the concept of the Soul. Hypothetical humans without Souls are called s-zombies.
  6. 6. so what's the big idea then? Pervasive Computing could be creating a second Cognitive Revolution. It is changing the geometry of our sensory input and our presence. It is changing the structure of our sphere of control. Our very Body Image. I believe Pervasive Computing is adding an interesting new dimension to Chalmers' Philosophical Zombie thought experiment...lets give the theoretical zombies in a Pervasive Computing Experience a name. Lets call them Digital Zombies or d-zombies These are people whose experience of the phenomenon of internal representation is more than a certain percentage digital, in some way. People whose sensory experience is digitally mediated beyond a defined d-zombie threshold.
  7. 7. this raises some interesting questions Does this d-zombie threshold exists at all. When we reach a point where our sensory input is more digitally mediated than solely biologically mediated, can our conscious subjective experience somehow become less Human and more like a d-zombie? Can it lead us to become d-zombies in ANY way at ANY point at all? What if our attention itself became incrementally digital? (See the Pervasive Reality Engine defined below)
  8. 8. contents The Project : An introduction and brief history Expanding my view : The project's new focus Reality Engines : The key model proposed Philosophical context : The related body of knowledge Ubiquitous vs. Pervasive : Pinning down key terms Reading the Web : Seeing the web as a digital prosthetic The Pervasive Experience : Reflections on this project What's next? : Where to from here Experimental Technology : Rubber hands and AR helicopters
  9. 9. The Project photo credit
  10. 10. The Pervasive Experience Project Hold your hand up and away from you. Imagine your fingertips are dipped in vomit! Feel how dirty and contaminated your fingers are. Can you sense changes in how you hold your body image? Take a moment to feel how this part of your body holds your attention. Even if you look away it seems as if your hand is tagged with meaning and still to be attended to. This positive, lead hallucination provides you with your own subjective experience of a traditional "real world" Body Image. Biological, grounded, open to symbolic manipulation...yet essentially "real". By contrast, this document reviews the Pervasive Experience project. In this project the driving assumption is that increasingly pervasive, networked technologies are impacting our lives. The research question is: How is Pervasive Computing changing you? This applied research explores how Pervasive Computing may impact our Philosophy of Mind. It looks at our perception of space and self, and their underlying Neurobiology. It aims to integrate theories from these four broad areas in a high-level, structural way. Its aim is to explore through practice and reflection how these theories may be able to be reshaped by the pervasive diffusion of digital networked technologies. Based on this analysis, an integrated, pragmatic, working model of the end-to-end process is synthesised - from the environment, body, nervous system and brain to sensory input, representation, consciousness and cognition. This model represents a Reality Engine. This document aims to frame this model within the existing body of knowledge, then extend it to support the rapidly growing, digital environment of Pervasive Computing. I believe this model opens a new perspective on network enabled devices where the web itself can be seen as a form of digital prosthetic that is increasingly pervading our experience and becoming more organ-ised. So after completing this review I now believe the most likely answer to this research project's question may be: By driving rapid Body Image modification. This could well be the defining structural process of Pervasive Computing...
  11. 11. Methodologies Over the past two years I have been exploring the impact of Pervasive Computing upon my sense of space using Action Research and reflecting through the lenses of Diffusion of Innovations and Distributed Cognition. Action Research: As defined by Dick (1999), Action Research is a "cyclic or spiral process" that "alternates between action and critical reflection". This review document represents the output of one of these critical reflection cycles. Diffusion of Innovations: Initially defined by Tarde (1890) the theory of Diffusion of Innovations was crystalised in the seminal work by Rogers (1962). It aims to describe and analyse why and how ideas and technologies diffuse through cultures, especially the rate of diffusion. Distributed Cognition: As defined by Hutchins (1995), Distributed Cognition is a framework involving individuals, artefacts and the environment. It is strongly based upon Vygotsky's (1920's) work that led to the development of Activity Theory. Distributed Cognition that takes as its unit of analysis a culturally constituted functional group rather than an individual mind. This theory is concerned with how information is propagated through a system in the form of representational states of mediating structures. These structures include internal as well as external knowledge representations, (knowledge, skills, tools, etc.). This approach permits us to describe cognitive processes by tracing the movement of information through a system and characterize the mechanisms of the system which carry out the performance, both on the individual and the group level. Hutchins (1995)
  12. 12. Expanding my view photo credit
  13. 13. The bigger picture This research has now been extended to include the lenses of Situational Awareness, Neuroplasticity and Sensorimotor Theory. This change has expanded my research focus to include “consciousness” and "self" and their underlying "Neurobiology". However this creates such a broad and technically deep domain that the goal is simply to “integrate current theories from these areas at a purely structural level in relation to the Reality Engine”. I have also been aware of Gibson's (1992) work on Visual Perception for a long time now but have struggled to integrate it fully into this work. Recently I came across O'Regan's (2010) work on Sensorimotor Theory, which seems to develop a situated aspect of Gibson's work within the modern context. To me this work provides a strong foundation for a wide range of discussions from Action Research through to Philosophy of Mind. I have also been researching Bach-y-Rita's (1969) work on Sensory Substitution that lead to the development of the field of Neuroplasticity. This work leads to some amazing predictions that can be tested through direct experience as shown by Bach-y-Rita, Ramachandran (2007) and Ehrsson (2010). As you will see below I have found that what I am exploring appears to be built upon a form of Neo-Cartesian Dualism. Decartes' statement in Principals of Philosophy, “I think, therefore I am” (Cogito ergo sum) are a compelling fit for the traditional Body Image experience of our reality. Yet, in the Pervasive, Neruoplastic, digitally networked Body Image experience of our proximate future we may be better supported by the philosophical statement “I am what I think” (EGO sum quis EGO reputo?) or perhaps more explicity “I am the sum of my thoughts”. Not in a frivilous “positive thinking” kind of way, but a “my thinking alters digital hardware and software” kind of way. This is not contradicting Descartes work, but reflecting on it from this new angle. If convergence/divergence continues to develop so our sensory input is increasingly digitally mediated then the relevance of this statement should become even more significant over time. Distributed, interactive software and hardware literally becomes part of our Extended Mind. This is where the d-zombie questions come in.
  14. 14. definition: Situational Awareness Situational Awareness is the awareness of what's going on around you. How you make sense of information, events and actions. How well you understand how they'll effect your goals and objectives. Related concepts such as Sensemaking take a more historical perspective and Situational Awareness tends to have a now and near future perspective. Inadequate Situational Awareness is seen as one of the key factors in accidents through human error. Especially in roles where there are serious consequences (e.g. drivers, soldiers, pilots, doctors, etc.). diagram credit
  15. 15. definition: Neuroplasticity The concept was initially proposed by James (1907), yet the term Neuroplasticity wasn't coined until (1948) by Konorski. Neuroplasticity was first experimentally demonstrated by Bach-y-Rita (1969). His work in Sensory Substitution suggested that the brain can rewire input from one sensory mode into internal representations in another mode. Not just in a form of synaesthesia, but by completely re-wiring the cortical maps. So at a time when the Cartesian homunculus appears to be identifiable within our physical neurobiology in the form of the cortical maps, we also appear to be finding that this homunculus is also plastic, malleable and changeable based on action and even thought alone. Neuroplasticity is also revolutionary because in some ways it brings cutting edge neurological research within the reach of anyone with a working brain. Adding in the Augmented Reality technology I work with everyday and you have a powerful set of research tools. photo credit
  16. 16. definition: Sensorimotor Theory "The sensorimotor view conceives of feel ("subjective experience") as a way of interacting with the environment. The quality of feel is simply an objective quality of this way of interacting." O'Regan (2010) I believe this provides support for the view that the self is the mental model that integrates input from other models and senses. It is a by-product of the creation of attention. By it's very definition, the sensorimotor view “requires” attention. I believe the emergent property of this is the self. A situated, attentive feedback loop. This perspective is expanded further below. diagram credit
  17. 17. Reality Engines photo credit
  18. 18. Descartes' Reality Engine To provide a starting point lets look at Descartes proposal for a dualistic reality. “I have a clear and distinct idea of myself as a thinking, non-extended thing, and a clear and distinct idea of body as an extended and non-thinking thing.” So, Descartes argues, “The mind, a thinking thing, can exist apart from its extended body. And therefore, the mind is a substance distinct from the body, a substance whose essence is thought.” As we will see, this initial proposition has fragmented into a wide range of Dualisms with different structures and theories. This project takes an Interactionist Dualism as it's starting point. Descartes' Dualism Diagram diagram credit
  19. 19. The Reality Engine elements of the diagram a/b = concepts self = center of attention environment = mental model of the world Body Image = mental model of your body X <- you are here ROM = rest of mind brain = the biological organ ROB = rest of body ROW = rest of world ROB <-> brain = nervous system brain <-> Body Image/a/b = perception Body Image <-> self = attention sensory perception is dualistic ● sensory input is one world ● perception is the other This diagram describes the single integrated Reality Engine model proposed by this project. A working system whose feedback loop creates a cognitive conscious self that interacts with the perceived Real World. Our brains seem to crave sustainable models that turn sensory input into actionable behaviour. Feedback loops that integrate all the aspects of sensory input (internal and external!?) and physical action into an attentive self. To me they appear from the outside to be a rich ecosystem of self-reflexive Hidden Markov Models (ironic?).
  20. 20. The Reality Engine extended additions to the diagram a = subconscious interactive concept a1 = subconscious mental concept a2 = subconscious body concept X <- you are here b = conscious interactive concept b1 = conscious mental concept b2 = conscious body concept so what is the “self”? ● reality is rendered through attention ● attention is the integration of sensing, reacting, remembering, thinking, doing ● the by-product of this is the self... because attention needs a point of origin A concepts are subconscious and can't be accessed by the self. B concepts are conscious and may be accessed by the attentive self. 1 concepts (A1 & B1) are mental only and have no direct relationship with the body. 2 concepts (A2 & B2) are concepts that are pre-formed within the body itself before the sensory data is sent through the nervous system to the the brain. This includes things like reflexes and muscle memory.
  21. 21. Mixing Realities Milgram's Virtuality Continuum If the input for this Reality Engine is substituted with completely digitally produced vision and sound, etc. then this becomes a Virtual Reality Engine (VR). And if we only overlay digital elements onto our perception of the Real World then this becomes an Augmented Reality Engine (AR). While VR and Artificial Intelligence were an aspirational focus of the original Cognitive Revolution, it seems likely from todays perspective that AR will continue to pervade our experience...but in folded and bent ways. In this case, pixels (or other sense based equivalents e.g. sound) can provide a good measure of how much digital sensory mediation has pervaded a person's Body Image experience. That is, “What percentage of our total sensory input is digitally mediated?”. In Augmented Reality the pixels that make up the overlaid information can be counted to calculate this ratio based on our total field of visual sensory input. Within this data is an important sub-set that is not just how much of our environmental sensory input is digitally augmented, but how much of our attention is digitally mediated. Through the lenses of Distributed Cognition, and Situational Awareness it's natural to ask “how much of our attention, and therefore self, can be Digital without turning us into d-zombies?”. Is there any limit to this digitally Extended Mind?
  22. 22. Multiplayer Realities Multiple Reality Engines A useful set of terms breaks the world into the Geosphere (physical earth/planets), the Biosphere (living things) and the Noosphere (symbolic or thought related things). The Pervasive Experience is transforming the Noosphere in terms of how interconnected it is. The relationship between space and even the point of origin of thoughts and symbols is being completely reformed. While in the traditional world each “self” has been cacooned within their own Cartesian isolation chamber, the Pervasive Experience may be opening the door to change this. Shannon's (1948) work on a Theory of Communication has some interesting application here and is an application of Hidden Markov Models. Symbolic communication between two Reality Engines seems to involve building a Theory of Mind of the “other” and creates a pragmatic use of encoding/encryption in a Shannon Information sense.
  23. 23. Body Image Based upon the full Reality Engine model proposed above, the term Body Image can be used to define the specific mental model used to represent our body. It can also more flexibly be seen to represent all the mental models that go in to making up our entire “self” and the broader context we exist within. Our entire Self Image and the contexts that it implies. Often, philosophical dualistic thought experiments and discussions isolate “special” dis-embodied beings to explore or prove a point. From the perspective of this project this is a non-sense. A “dis-embodied experience” is an altered form of consciousness outside the Sensorimotor world view. Even hallucination or meditation upon “mental only” (e.g. B1) concepts are spatially related to the Body Image. The word “internal” is often used to describe these types of phenomenon. In this way attention is a situated experience (or feedback loop) that requires (or perhaps generates) a point of origin. Interestingly, early research into Attention during the first Cognitive Revolution was focused upon “divided attention” (e.g. dichotic listening). By contrast, the experiments of this research project are focused on in some senses the opposite. The goal here is to integrate objects from two (or more) places/times into a single coherent attention, instead of attempting to split attention into two (or more) separate streams. I believe this is the complete reformation of the concept of Body Image that the Pervasive Experience is enabling.
  24. 24. Philosophical context photo credit
  25. 25. Dualisms In order to relate my reflections from this research review to the existing body of knowledge I have developed the following map of what appears to be the related philosophical terrain. I am also open to new perspectives in this area and reviewing new concepts and frameworks drives a lot of my reflection. Here's the map so far. Cartesian Dualism: This philosophy of mind separates reality into a dual system of mind and body. Descartes' work was an extension of thought that dates back to Plato and Aristotle. This separation into physical and non-physical is also often bound up with a spiritual discussion of the soul as well. Interactionist Dualism: Interactionism is the form of Dualism that proposes that the mind and body can interact bi-directionally in a causal way. While it is often stated in the literature that it “is difficult” to establish how this may occur, I find it almost self evident. fMRI's, etc. clearly show that brain activity is tied to symbolic phenomenal experience. Neuroplasticity takes this one step further stating that the phenomenal experience alone can persistently change neruobiology. Interactive Dualism: There is also an area of Interaction Design/CHI where Theory of Mind is related to interactions with any dynamic system. For example Norman (1988). diagram credit
  26. 26. Neurobiology Neurobiology is an amazingly deep and technically complex area. This literally includes “brain surgery” and is dealing with the most complex structure in the known universe. This project focuses on the perspective of Cognitive Neuroscience and is primarily focused upon the theoretical frameworks available that show how cognitive processes can impact the biological structures within the brain. Neuroplasticity is the key lense used for this reflection and literature review. I believe it is important to include this from at least a high-level in order to fully explore the Interactionist Dualism foundations of the Reality Engine. I think it is also very relevant for the phrasing of the research question “How is Pervasive Computing changing you?”. There is also likely to become more closely integrated into the overall Pervasive Computing domain with the rise of Computer/Mind interfaces. This is also closely related to the Digital Prosthetics content discussed below. Interestingly there are some mechanical devices/technologies (e.g. TMS) that are widely available and are actively being used to map and alter specific neruobiologies. Personally I find TMS quite frightening and think that this project specifically raises some serious ethical issues that relate directly to it's use. (e.g. can mechanically changing the operation of certain Neurobiological structures change your consciousness to the point of creating a d-zombie? Could it be used as a literally de-humanising experience?). Not to mention the unknown long term impacts of high-intensity EMF on delicate and critical cell structures.
  27. 27. The Mind Philosophy of Mind: This is a very broad and rich area. Dennett's (1991) work breaking the philosophy of mind into two problems “consciousness” and “content” is interesting. His Neural Darwinism perspective is also useful and relevant for this project. Chalmers' work on consciousness, especially his p-zombie thought experiment is obviously relevant. Theory of Mind: The ability to form models that attempt to predict the behaviour of objects, animals and other behaviours is a critical conscious activity. Interestingly, the fact that people form Theories of Mind implies the belief that a representation system inside another's mind can causally interact to create actions. Cognitive Revolution: Essentially this was a rebellion against an automata style view of behaviourism and a recognition of the phenomenological aspects of the human experience. Interestingly, in 1950 Turing published "Computing machinery and intelligence" in the journal “Mind”. His proposal was that you could use questions and answers to test machines for intelligence is often included as part of the Cognitive Revolution. His view here appears to be that we can use our own innate Theory of Mind models (B1 concepts) to interrogate an “other” via a text based interface to determine if it is Human or synthetic. This highlights a consciousness continuum (or perhaps even triangle) between biological entities and synthetic/mechanical entities.
  28. 28. Functionalist Hidden Markov Models Functionalism: Philosophical Functionalism provides a post-Cartesian perspective that is highly relevant to the Pervasive Experience and it's digital-isation. However, the Reality Engine provides a perspective where Functionalism and Interactionist Dualism need not be mutually exclusive. I believe it is possible for Functionalism to exist at the physical/material Neurobiological level to create a subjective experience that is Dualistic and Interactionist in nature. In fact I believe that the two are perfectly fitting sides of the same coin and that the nature of attention/self requires both. Hidden Markov Models: The Cartesian cacoon that the Reality Engine suggests we all live within also suggests that the Hidden Markov Model is a useful metaphor. This creates a nice, ironic, self-recursive Solipsistic loop Shannon's Theory of Communication: Shannon's (1948) work on a “Mathematical Theory of Communication” is also highly relevant to this discussion. For example his “Fig 1.” is almost identical to the Multiplayer Reality Engine diagram above. diagram credit
  29. 29. Ubiquitous vs. Pervasive photo credit
  30. 30. Weiser's Ubiquity My starting point for the whole project was a diagram created by Mark Weiser in (1996) - see above. My interpretation of this diagram is that the left and right elements on the first row contrast non-Interactionist Cartesian Dualism (left) with Situated Cognition (right). The labels beside clearly state that Weiser believed a non-Interactionist Dualism was "wrong". The second row of left and right elements contrast Current Computing Technology with Ubiquitous Computing Technology. My interpretation here is that Weiser was showing how Computing Technology will pervasively spread throughout more and more aspects of our Situated reality, as opposed to the unipresent Desktop Computing model that dominated the 1980's and 90's.
  31. 31. Spatial Maps Review Edward T. Hall et al's work on Personal Space. Weiser's diagram is obviously focused upon the spatial Relate this to the reality engine. relationship between you and the world. All Cartesian like dualisms appear to have this spatial aspect. The spatial models we hold can also be broken down into finer grained regions. At the highest level of the physical world is Inside and Outside our body. This implies a Body Image and surrounding that, and inter-meshed within that, is our overall sense of space. Beyond proprioception are our exteroceptive senses that collect stimulus from the outside world. From this we build models about these spaces and relationships. Hall's (1966) work on Personal Space provides a useful framework for a User Centred world view. These concepts also have a pragmatic relationship to Situational Awareness too. The Reality Engine aims to ensure Hall's work integrates with Cartesian Dualism, Extended Mind, Distributed Cognition and Situational Awareness.
  32. 32. Divergence Applying this type of spatial analysis to the concept of Convergence has also led me to some interesting reflection. I believe Divergence is the by-product of Object:Network Convergence. As the distance between an Object and the Network passes a point of Convergence the distance between the “Situated Point of Origin” of that Object and “Control” of that Object Diverges. In other words, once you connect an Object to the Network you're free to remove its “Physical Interface” and simply “Control” it via the Network. This is exemplified by the marketing term “Anywhere, Anytime”.
  33. 33. Putting YOU at the centre I think that the framework above, especially the Pervasive Web integrated into the Reality Engine, highlights why Pervasive is a more appropriate term than Ubiquitous. Ubiquitous suggests that the technology is “everywhere”. While it may at first seem like that's the case, in reality I believe we are discussing how this technology is soaking into our attention. If you look at mobile network coverage maps they don't cover the whole world, just the parts where the majority of the population are likely to be. This is a natural evolutionary pattern that allows for the conservation of energy. Pervasive suggests the technology is soaking into “your” subjective experience - with you at it's heart. Ubiquitous on the other hand suggests technology is soaking into the universe - whether you are there or not. To me this appears to be an important structural and ethical distinction. The next section aims to share some insights that should clarify why this may become even more important as convergence/divergence continues to develop. Where distributed, interactive software and hardware literally becomes part of our extended self. If the impact of Moore's law and the Network Effect (Metcalfe's law) keep driving the Pervasive Network then I believe we need to make these technologies an extension of our “self”, or risk becoming diluted and de-Humanised. Again, this is where the d-zombie questions come in.
  34. 34. Reading the Web photo credit
  35. 35. Prosthetics Prosthetics are traditionally defined as an addition to the body (coming from the greek word for addition) that replaces a missing body part. If it's related to mobility or a directly sense-able part of our Body Image then it is likely to be a limb. If it is to do with sub-conscious processes then it is likely to be an organ and may include real or artificial organs. This analysis is blurred even further when the prosthetic is a sensing organ that impacts your Body Image. photo credit
  36. 36. Digital Prosthetics Now, there also exists the real opportunity to utilise Digital Prosthetics that enhance a person's sensory input and even cognitive capabilities like memory. This obviously heads us into the well populated cyborg territory. While the traditional focus of Prosthetics was upon repair and replacement, the Digital Prosthetic view of the Pervasive Reality Engine is squarely focused upon “beyond repair” and towards “enhancement”. Jamais Cascio presents a perspective that many people will be facing over the next few decades as their options turn from repair to enhancement. This is also an interesting convergence of Digital Technologies, Medical Miniaturisation and an Aging Population.
  37. 37. The Pervasive Web It's amazing to see how the web has expanded from a single point on the planet (CERN) out to becoming a pervasive cultural concept. In fact it could be seen to have initially existed as just a model within the Reality Engine of Tim Berners-Lee. Now it seems to be a model that exists in almost everyone's Reality Engine and it has distorted how they use and think of space, place and time. Pervasive Web devices can then be seen as a form of Digital Prothstetic and the Web itself is pervading throughout our planet and our experience. Literally soaking into our consciousness. However, it's in a unique class of mental models that not only cement themselves into Reality Engines, they provide networked access and become the point of exchange for other models too.
  38. 38. The Pervasive Experience photo credit
  39. 39. First Reflections The heart of my recent reflection has been upon Neuroplasticity and Situational Awareness and how they relate to the concept of Personal Space and Theory of Mind in light of the Reality Engine. Extend this to interacting with an increasing number of web applications and web enabled devices and you have a Pervasive Reality Engine. The addition of Pervasive Computing to the Reality Engine creates a structural change to the models that defined the Cognitive Revolution. While the original Cognitive Revolution was focused upon what happened within the brain and mind, the Pervasive Reality Engine is focused upon what happens to us as our self moves out of our brain and possibly even out of our mind if that is achievable (imagine extending the A2 & B2 concepts in the Reality Engine using remote sensors). To use buzz terminology, this is Cognitive Revolution 2.0. So it appears we have a relatively mature conceptual framework for the individual mind. At the other extreme it also appears that the impact of interactive networks have been explored broadly from the macro, national scale. Now seems like the perfect time to integrate these two perspectives into an exploration of the impact of pervasive networks on the mind. This exploration leads to some interesting questions. If consciousness could be a property achieved by any system that can create the right type of attentive self awareness feedback loop, then we as humans could “safely” extend our Human experience using digital technology. This would benefit us by allowing us to tag along with rapidly evolving digital technology. A chance to keep up with any possible proximate Technological Singularity. If not, then this has obvious implications for the ethics of technology and the discussion of the d-zombie threshold and the possibly de-Humanising nature of technology. However, if it were true then this has implications for the ethics of machine intelligence and even biology. If consciousness were an achievable property of the right type of attentive loop then any “thing” that displays that type of loop is essentially “equivalent to a self conscious human being” - at least at a category level. Interestingly, either way there is an impact.
  40. 40. Content Reflection Pervasive Computing is a movement driven by the structural reformation of our Reality Engines through convergence/divergence. I believe it can be shown to enable the following measurable changes: – rapid Body Image modification – flexible Body Image design and re-engineering/sculpting – dynamic multi-POV Body Image experiences I am confident now that the content outlined above provides a robust theoretical and practical framework for imagining, designing and building rich, interactive and "world changing" Pervasive Experiences. This model obviously needs to be reviewed, tested and revised on a regular basis, however it provides a workable starting point for some truly innovative experimentation.
  41. 41. Context Reflection My personal experience of this Action Research project has been an interesting, stimulating and very challenging one. The breadth of content and reading has been quite challenging, as has been keeping up with the constant pace of change on the technology side. Luckily, my day-to-day work keeps me focused upon this type of change. There are a number of limitations of my work that have been pointed out by the people that have been kind enough to review it. The first is that it is almost exclusively from a western thought perspective. The world is obviously full of a broad range of interesting and insightful perspectives that don't fit into that bucket. I hope to at least start to address this in the next phase. I have also received feedback stating that this work appears strongly philosophically monistic. Personally, I don't believe that the output is quite that simple. However the density of the content here masks some of the subtleties. I hope I can communicate this aspect of the project more clearly in the next phase and aim to make my stance here more transparent. I have also receive feedback on how overwhelming and sometimes un-approachable this document can be. I've made a number of revisions that have hopefully addressed some of this, but this is a continual challenge and I am continuing to attempt to distill this down to a simpler set of key messages without diluting the depth of the work. While this work is obviously strongly academic in nature, I am completing it independently from any academic institution as an ongoing attempt to develop my practice. I have learned and grown a lot through the process, but obviously a more formal context may have helped me stay more focused. Luckily I've been able to seek constructive feedback and support from some academic friends that I'll thank in due course. On the whole I feel I've achieved something of real value and I'm comfortable that I now have a good framework with which to begin my experiments. This is where the project gets really interesting.
  42. 42. Ethical Reflection There is an obvious commercial bias I have in my world view. I work as a Technologist for a Research Lab that helps it's clients commercialise new technologies. While my company (MOB) are happy to support this research project, both the company and I obviously benefit from the output of this project. However naive this may be, my underlying perspective is not so much that “I do this research because my job is closely related”, but more “I am good at my job because I am naturally drawn to a research project like this”. While this may be a subtle difference, I believe it has significant implications to the overall work. The questions raised by this research project are essentially Humanist in nature and are exploring the continuum between today's common Human Condition and the other extreme of Artificial Intelligence. While traditional AI starts at a synthetic point and heads towards consciousness, this project starts with the User and pushes the boundaries in the other direction. The Ethical focus of Humanism sets out the framework for this reflective process. The logical conclusion of this reflection is the structural proposition for a new terminology with Pervasive being more appropriate and Humanistic than Ubiquitous. Further and deeper reflection upon the ethical aspects of this project as it relates to the following topics is also planned. These obviously covers a very broad set of domains which will only be touched upon to gather insights. - Power - Digital Divide and Abandonment/Neo-luddites - Environmental - Cultural/social - Economic
  43. 43. What's next? photo credit
  44. 44. Dance of the bees... When bees return to the hive they communicate the location of pollen rich plants to other bees through a structured dance that apparently transmits a usable mental map. This is perhaps the only known symbolic language outside mankind. The Reality Engine model proposed above sets out a dualism between the physical and the symbolic worlds and suggests that our conscious self is a product of our attention through our neurobiology and sensory input. Pervasive Computing is literally giving us the ability to sculpt and dynamically re-design our sensory input, attention and through that neurobiology. To reshape our consciousness/self through Pervasive Experiences. But the content and answers outlined in this review just open deeper questions. e.g. Does the increasingly Digital nature of our new attention/self make us any less human at some definable point? At that point, do we turn into d-zombies? Or could it possibly make our experience even more human and humane? NOTE: Multi-POV is a structural change to our existing empathy model and may have significant implications for our Theory of Mind. The goal now is to continue to explore this model and the d-zombie thought experiment through applied experimentation. The focus is upon the subjective experience of using Pervasive Computing technologies and the methodology is based upon Action Research and reflection. The method involves creating and using Pervasive Experiences and measuring how, if at all, they change the user's Body Image. The specifics of this study are the human equivalent of the “dance of the bees” and the aim is to model the user's internal mental maps by observing and manipulating external behaviours and technologies.
  45. 45. Experimental Technology photo credit
  46. 46. The rubber hand The starting point for the planned experimentation is the rubber hand illusion which uses a mirror to mechanically/ physically bend light to change your perception. This altered sensory input is used to fool our Reality Engine into associating a foreign object into our Body Image. This literally changes our sense of who and where we are. The Pervasive Experience also allows us to flexibly distort our sensory input to other places and points in time through networked devices. Using networked cameras, microphones and user interfaces we can digitally bend sound, light and actions. Using digital sensors we can bend other forms of electromechanical energy to give us an expanded range of sensory input beyond the normal human range. The experiment this project is pursuing attempts to merge these two aspects. To use digitally altered sensory input to fool our Reality Engine into associating foreign & digital objects into our Body Image. One interesting tool that we are starting to explore for this experimentation is the AR Drone. By exploring this in different ways it is hoped that the limits of this new form of Extended Mind (as Chalmer's put it “panprotopsychism”) can be mapped. To contribute to establishing where the limits to this extension may lay.