Post-Biological Hypersurfacing Embodied Mixed Reality Data Transfer Julian Stadon Department of Art Department of Internet Studies Curtin University Perth, Australia firstname.lastname@example.org Abstract— This paper focuses on the (d)evolving interface embodiment, thus contributing to the redefinition ofbetween cyberworlds and the real world; what Gabriella aesthetic representation and embodiment.Giannachi has called the hypersurface . This fusion of real Here, I describe two of my recent projects, Prometheanand representation, linking cyber and real worlds constitutes Alchemist and Terra(socio)sonica which demonstrate whatmixed reality interaction as experienced by humans in the Giles Deleuze has called, the ‘deterritorialisation’  of thephysical world, their avatars, agents, and virtual humans. human body and its’ ‘reterritorialising’ effect on posthumanCurrent mixed reality XML RPC (Remote Procedure Call) digital identity construction. Such work furthers theinterfaces and real-time data transfer enhance the experience definition of ‘post-biological’ digital identity through citingof the hypersurface for the audience beyond any previous innovative actualisation examples. Through increasing thevirtual media types, such as, hypertext, HTML, VRML, level of physical involvement that viewers have in thevirtual reality, etc. Previous research in mixed reality andinteractive workspaces that use the concept of a bridge for creation of a mixed reality system of exchange, these worksdata transfer have largely inspired this research and I aim to demonstrate a new model of representation and experiencecontinue the development of new knowledge in this field. of the hypersurface that stretches, disperses and merges the position of both artwork and viewer in order to question Keywords-virtual worlds; virtual humans and avatars; posthuman identity and the body.human-computer interaction; social networking; networkedcollaborations; cyberworlds II. PROMETHEAN ALCHEMIST I. INTRODUCTION Perhaps most explicitly exemplifying this mixed reality structure is Promethean Alchemist. This project was Unlike traditional sites for communication and cultural conceptualized during the Liminal Screens Co-Productionexchange, virtual platforms rely on actions and Residency at the Banff New Media Institute (BNMI), Theconversations to shape not only the social and cultural Banff Centre, Canada, and is currently under development.environments, but also the spatial environment. XML RPC Promethean Alchemist is a mixed-reality, interactive datainterfaces further this to (re)include the physical transfer system that engages participants in mythologicalenvironment, from the perspective of the viewer. My work creation and DNA sequencing. Biological DNA is translatedscopes unique autonomous relationships between authors into computer data using DNA sequencing technology,and agents, through augmenting biological and virtual which is further manipulated into 2D and 3D visual graphic(natural and technical) information into mixed reality states appearance in the ARToolkit. The visualization form is thethat allow for their mediation through networked systems classic DNA double-helix, which re-enters the physicalinterfacing. Such systems allow participants to physically world through an XML RPC (Remote Procedure Call)interact with virtual biological components through real- interface. When two sections of DNA are brought into closetime data transfer and mediate through physical proximity, the system recognizes their proximity andengagement, rather than entering traditional text or ‘splices’ (combines) the data set into one.numerical based data sets and command sequences. Thus, wearing a HMD (Head Mounted Display) and Unfortunately, the majority of current research in mixed using these cards, participants are able to pick up segmentsreality focuses on specific technical aspects of the field of DNA in mixed-reality space and throw them into theencompassing computer science, cognitive science, robotics, ‘mixing pot’ (visualized as a brazier burning withetc. with arts, media and humanities only recently Promethean Fire) to create constantly evolving artificial lifecontributing to a mixed reality knowledge base. My research forms. Participants are able to create life in an augmentedapplies cultural and philosophical discourse to recent environment and then retrieve their creation in the virtual.developments in computer science to propose a new mode Here they can export the form to 3D printing or animationof representation that concerns itself with the affective platforms.capacities of art in order to articulate a sense of dispersed
Promethean Alchemist is an artistic rendition of allotting suitable powers to all mortal creatures.the implications of DNA code translated into the Deleuzian Epimetheus did the initial distribution, which was then to‘fold’. That is, the in-betweenness of spaces; “able to be reviewed by Prometheus. Epimetheus set about givingrepresent dialectical opposites, such as organic and creatures equal shares of positive and negative traits (smallinorganic, inside and outside.” The viewers’ beings were given flight, weaker beings speed, etc); “Thusparticipation in the layers of code which constitute the he made his whole distribution on a principle ofwork implicates them in a complex exchange between compensation, being careful by these devices that noorganic and inorganic information where the differences species should be destroyed.”  However, whenbetween the two dissolve. The Promethean Alchemist is Prometheus came to inspect the work, he realized thattherefore an example of Giannachi hypersurface: “Able to Epimetheus had forgotten humans and had nothing left topresent dichotomous relationships, between representation bestow upon the naked, unshod, and unarmed creatures.and matter, inside and outside, organic and inorganic, the Distraught, Prometheus stole from the Gods the gifts ofhypersurface is the site of virtual performance.”  My skill in the arts and fire, for without fire there was no wayparticipation in the exchange is the threefold translation to use the skill. Thus through kinship with the Gods byfrom organic DNA to computational code to visual their possession of such God-like powers, humans wererepresentation in the ARToolkit. The viewer pushes this able to create things such as clothing, shoes and weaponsfurther by becoming the ‘alchemist’ stirring a virtual and could obtain food from the earth.primordial soup. While the viewer in firmly planted in thephysical realm, they still experience the dissolve between Thus what constitutes humanity is something that is outsidethe organic and inorganic, humanity and technology. of humanity itself: imagination, discovery and realization through technics; what could be called Promethean Fire.Examples from computer science that engage the While animals are granted predestination in their origin,hypersurface include, for example, Augmented Reality, humans must create their qualities, “Humanity is withoutButz et al.  which proposed a drag and drop technique qualities, without predestination: it must invent, realize,between an augmented reality space to a screen space produce qualities, and nothing indicates that, oncewithin the EMMIE system. Using a mirror metaphor, produced these qualities will bring about humanity thatvirtual objects would change representation and they will become its qualities; for they may rather becomedimensionality by passing through screen boundaries, their those of technics.”  In other words, humanity is itsapproach focused on transferring documents Recently Lang inventions, its tools, its technology. Promethean Alchemistet al. from Georgia Tech University modified Second Life demonstrates this concept in that each entity that exists into create mixed reality experiences . This example the virtual environment of the system is a translation and anbridged reality states in a way that facilitated a further archive of a particular moment in the evolution of humanityinquiry into the socio-cultural implications of such systems, mediated between genetic and computational code.but was never addressed in the research publications. TheVTT Technical Research Center Finland has also recently It is through this definition of being in the divine gift ofworked with hypersurfacing Second Life avatars within Promethean Fire that humans experience their mortality.physical experience through the Meeting Avatars joint “To partake in the lot of immortals means to endure one’sproject with IBM and Nokia . By using the Second Life mortality by the fact of being in (privative) relation withengine, virtual avatars had the same appearance and immortality.”  Indeed, Promethean Alchemist represents,behavior as in the virtual world but in their context be in a literal way, the desire of many scientists andrepresented in a physical meeting room. These research philosophers to locate immortality in humanity’sexamples articulate a range of different solutions that have technological existence. The translation of DNA code intobeen proposed for technological developments in the field computational code and re-created in the augmentedof computer science, and generally neglect the environment of the digital realm, demonstrates an instancephilosophical and theoretical impact of such technologies of, ‘transcendence through technology.’ [Hayles,on human subjectivity, identity and social discourse. Posthuman] The ability to manipulate DNA mediated through the hypersurface represents the hope of manyTo address this problem, Promethean Alchemist pays scientists to, in essence, dupe Zeus once again by locatinghomage both visually and conceptually to the Ancient the essence of humanity within that which defines it: inGreek creation myth of Prometheus and Epimetheus, which technics, thereby attaining immortality.philosopher Bernard Stiegler references in his text Technicsand Time 1.  His text seeks to define technics -a This hope has practical grounding, for example, intechnical entity arising out of, but distinct from, mechanical conservation biology. Currently, many scientists areand biological entities- and to delineate the relationship exctracting DNA samples from endangered animal speciesbetween technics and humans. In the Greek creation story, bred in captivity and preserving them for a future timePrometheus and Epimetheus were given the task of when the organism might be safely reintroduced into its
restored habitat, or even into another ecosystem on another of movement into sound in both the physical and virtualplanet. The assumption is that organisms can be preserved realms, the work explores the notion of unspoken ‘silentintact in DNA to be re-substantiated in physical form at a dialogues’ under a paradigm of sociallater time; the organism is thought to exist in essence in its engagement. Individuals and large clusters of peopleDNA. Immortality is gleaned through the ability to store produce amplified sounds and shadows based on theirand preserve the organism in this way indefinitely. oscillating movements within a defined social landscape.Promethean Alchemist actively uses the kind of technologythat further abstracts and compresses the organism. That The work uses real-time motion-tracking technologies with a unique pipeline application to create a mixed-realitythe viewer can only retrieve their creation through the soundscape. This audio environment is mediated throughlimited methods of 3D printing or animation platforms, interactions between the viewer, the physical environment,rather than re-substantiating into living tissue is a testament and other participants within a hypersurfaced mixed-realityto the limitations of translating from biological to virtual feedback loop. As visitors negotiate a traditional publicenvironments. environment, the entrance and surround to the Somerville Auditorium at UWA, data regarding their movements andAlchemy is, ostensibly, the precursor to modern medicine interaction with others present is gathered and translatedand science, though its methods may have been closer to into sonic outputs, both in the physical and virtualmagic than scientific method. Sixteenth and Seventeenth environments. In the physical, the output is via stereoCentury scientists and swindlers dabbled for years in an speakers installed in the space and in the virtual, a threealchemical obsession, the search for the philosopher’s stone dimensional representation shadows and echoes sonic and— the long-sought agent for transmuting lead to gold and visual traces of the real-time dialogues into UWAs Secondunlocking other material and spiritual secrets. The stone Life Environment.was the unified theory of everything in that time, includingthe search for immortality and ways to keep the bodyyoung.  In Promethean Alchemist the viewer becomesthe alchemist searching for immortality in the primordialsoup of virtual DNA, combining and recombiningcombinations of computational ‘DNA’ code to createimmortal virtual life forms. The metaphor of the moderndigital alchemist points to the ways in which fruitlessoccult pursuits can sometimes evolve into transformativepractices unforeseen in the former utopian, self-deludedview. III. TERRA(SOCIO)SONICA: POUVOIR/PUISSANCE Figure 1. Second Life view of virtual ‘shadows’ from physical data Massively Multi-User Online (MMO) Worlds are openvirtual environments that allow for the contribution to and The individual experiences an intimate interaction with themanipulation of private and publicly owned virtual space work and social environment where they control thethrough a variety of methods, be it adding textures, models, soundscape through their actions, thereby conducting theirscripts, hacked interfaces etc. The content of these own personal song. Meanwhile, each community that formsenvironments is dependent on the participants, due to this also produces unique tones. Movements of individualsopen interaction, and therefore relies on the quality of between groups results in a sonic symphony of socialinformation transfer methods being used. Through interaction that shifts dynamically according to the socialcollaborative creative production MMOs facilitate social dialogues that occur in the space. Stepped tonal outputs areengagement and further collaborative production by its produced by audience movement in space: a scroll to theparticipants. Spatial developments define the environments right will cause the pitch to drop with each step. The pace ofand the (real or virtual) individuals inhabiting such spaces the movement determines the speed of the notes; lingeringthrough their participation in and response to them. The conversations produce long lingering sounds while the rushcollective construction of such virtual meeting sites, for of busy passersby results in fleeting melodies that come andremote interpersonal interaction acts as an instrument of go just as quickly.location and orientation, referential to the real world ofknowledge. The work is inspired by Deleuzian notions of deterritorialisation, with particular regards to second-levelTerra(socio)sonica: Pouvior Puissance is a mixed-reality cybernetic feedback systems (within the context of mixedinterface which realises a sonic soundscape constructed via reality social interaction). Deleuze and Guatarri discussthe movements of communities that inhabit two current deterritorialisation in terms of dispersed resemblance andlandscape realities that constitute the University of Western identity. In Difference and Repetition Deleuze introducesAustralia (UWA) cultural precinct. Through the translation the notion of deterritorialisation (through dispersion) as a
“dark precursor” that “relates heterogeneous systems and I believe Deleuze’s concept of intuitive thought is in lineeven completely disparate things” . In order for with Bernard Steiglers notion of technics defining what it isdeterritorialisation to occur there must be some form of to be human. As I mentioned earlier, Steigler has posited theagent that can remain constant and self-referent. important contribution to metaphysical thinking that it is technics -the artificial realm of symbols, systems, tools, etc.- that makes humans functional, speaking, meaning-making creatures; that is, what makes humans human. It is in Deleuzes distinction between different degrees of power that I see Bergsonism, that is, the intuitive method, and Steiglers technics coming together. In looking at power one must distinguish between the two French words puissance and pouvoir. "In social terms, puissance is immanent power, power to act rather than power to dominate another; we could say that puissance is praxis (in which equals clash or act together) rather than poiesis (in which others are matter to be formed by the command of a superior, a sense of transcendent power that matches what pouvoir indicates for Deleuze)."  Our intuitive interaction with technology is, as demonstrated by the hypersurface, the enactment of puissance, that is, immanent power to create meaning through our interactions as opposed to the passing- down of pre-established ideas through transcendent power (pouvoir). Intuitive interaction challenges the academy model, which essentially treats education as transcendent power, the passing on of information by solving problems that already have answers. Instead, intuitive interaction creates opportunity for true knowledge production; an instance of power that puissance indicates for Deleuze. It is through observing human intuitive interaction with technologies that one might theorize a plausible post-human or postbiological digital identity. As summarized by Daniel Smith and John Protevi, "Here we see the empiricist theme Figure2. Technical Specifications Diagram for Terra(socio)sonica of the ‘externality of relations’: in an assemblage or consistency, the ‘becoming’ or relation of the terms attainsIn practice, the work models global deterritorialisation by its own independent ontological status. In Deleuzes favoritedestabilising established social groups through intuitive example, the wasp and orchid create a “becoming” orinteraction with virtual space and reconfiguring them in the symbiotic emergent unit." In terra(socio)sonica, humansoundscape. The work also investigates the relationship interaction with the hypersurfaced mixed-reality feedbackbetween traditional (postcolonial) socio-technic hierarchies, loop becomes an emergent unit in the soundscape, physicaland those developing in current evolving cyber cities. and virtual worlds.Conceptually, the work attempts to explore BernardSteiglers historicisation of technical objects and western IV. CRITICAL THEORETICAL CONSIDERATIONSphilosophy in a way that embraces Deleuzes translation of An avatar represents an identity, usually that of itsBergsonism, particularly regarding intuitive method. For author and acts as an agent, through which users can engageDeleuze, intuitive thought is more like a narrative than an with virtual platforms. Promethean Alchemist andinstantaneous ‘eureka’ moment. In order to understand Terra(socio)sonica: Pouvior/Poussance are uniqueDeleuzian intuition it is important to situate it within the autonomous systems where participants can physicallyBergsonian departure from a structuralist notion of reality in interact with a virtual deterritorialised ‘self’ and mediate itterms of space, in favour of the metaphysical idea that through physical engagement. The dispersion of multiplereality should be understood in terms of time. In contrast to autonomous virtual agents via mixed reality constructs inDescartes instantaneous glance or moment of clarity- this research, aims to expand deterritorialisation to includedistinctness that separates the intuitive idea, Deleuzian reterritorialisation, by facilitating a dispersive relationshipintuition is rather a "progressive description of the whole," between the body and its virtual self-referent. In the samemore like a multi-lateral trajectory than a moment of way that a digital device deterritorialises and reterritorialisesillumination. For Deleuze, intuition "reconcatenates thought information through binary code, the augmentation of anto beings as the co-presence of a being of the simulacrum autonomous agent into a shared space with the body, createsand of a simulacrum of Being."  new opportunities for investigation into technology, the body and identity.
existence within such networked systems of social andHaraway relates the body’s augmentation through digital cultural exchange. The ability to incorporate real-time datetechnology to the notion of the cyborg. In A Cyborg transfer methods into such systems redefine our veryManifesto she argues that the body can be viewed as a understanding of what constitutes humanity.conglomerate where its components can be separated,combined with new elements and put together again in ways ACKNOWLEDGMENTthat violate its traditional boundaries (Haraway, 1991, The author would like to acknowledge the support of thep.149-181). This rhetoric implies a fractured identity that Banff New Media Institute (BNMI), HITLab NZ, Thearticulates a ‘cyborg’ reality. In Chaos Bound Hayles refers University of Western Australia Cultural Precinct, Curtinto the notion of dispersed self in light of virtual bodies and University and The SymbioticA Center for Excellence.narrative, arguing that by turning bodiless information into Thanks also to Raphael Grasset and Steve Berrick for theirnarratives, the teleology of disembodiment is replaced with ongoing programming support.contests with ambiguous outcomes.“As I have argued, human being is first of all embodied REFERENCESbeing, and the complexities of this embodiment mean that  G. Giannachi, Virtual Theatres: An Introduction. London and Newhuman awareness unfolds in very different ways than York: Routledge, 2004.intelligence in cybernetic machines” (Hayles, 1990, p.3).  Deleuze and Guatarri, A Thousand Plateaus: Capitalisim andMy research demonstrates embodied autonomous agents in Schizophrenia, 1980shared reality states that posses the social autonomy  J. Clerk Maxwell, A Treatise on Electricity and Magnetism,required to create a collaborative engagement, in order to 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68–73.challenge the notion of bodiless information and to develop  A. Butz, T. H¨ollerer, S. Feiner, B. MacIntyre, and C.outcomes that confront the body’s relationship with the Beshers. Enveloping users and computers in a collaborativematerial and the virtual. Creative practice offers something 3d augmented reality. In IWAR’99: Proceedingsof the 2nd IEEE and ACM International Workshop on Augmented Reality,unique to this area because it creates a zone of research and page35,Washington,DC,USA,1999.development without the confines of a tight quantitative  T. Lang, B. MacIntyre, and I. J. Zugaza. Massively multiplayerimperative. online worlds as a platform for augmented reality experiences. In IEEE VR’08, 2008 V. CONCLUSION  Kantonen T., Woodward C., Katz N., "Mixed reality in virtual Mixed Reality data transfer allows for the oscillation world teleconferencing", Proc. IEEE Virtual Reality 2010 Waltham, Massachusetts, USA, March 20 - 24, pp. 179-182between different realities in a way that is seamless and  Bernard Stiegler, Technics and Time, 1: The Fault of Epimetheus,intuitive as it incorporates traditional paradigms of physical trans. Richard Beardsworth & George Collins Stanford Universityengagement to occur, situated though, in an entirely digitally Press, 1998mediated environment. The projects discussed serve as  I. S. Jacobs and C. P. Bean, “Fine particles, thin films and exchangepractical research outcomes that present an investigation anisotropy,” in Magnetism, vol. III, G. T. Rado and H. Suhl, Eds.into the social and cultural impacts of such systems, through New York: Academic, 1963, pp. 271–350.the utilisation of traditional explanations of how we  Y. Yorozu, M. Hirano, K. Oka, and Y. Tagawa, “Electronintegrate ourselves as individuals in a greater social context. spectroscopy studies on magneto-optical media and plastic substrate interface,” IEEE Transl. J. Magn. Japan, vol. 2, pp. 740–741, AugustThe works serve to function as social experiments that 1987 [Digests 9th Annual Conf. Magnetics Japan, p. 301, 1982].propose questions without answers, in order to allow  D. Kornack and P. Rakic, “Cell Proliferation without Neurogenesisviewers to investigate these ideologies from a personally in Adult Primate Neocortex,” Science, vol. 294, Dec. 2001, pp.discursive position. Cyberworlds exist as mirrors for society 2127-2130, doi:10.1126/science.1065467.and as such should represent the current state of human