This document discusses ways to digitally capture and encode experiences and sensory information. It presents several precedents including the Triops Camera, Google Street View, QuickTime VR, MIT's Sixth Sense project, and works by Steve Mann and Pipilotti Rist. It proposes developing a "Sponge" device that would hard document primary experiences by encoding sensory metadata that could be used to recreate or share those experiences.

1. Common Sense
Object Mini-Project
Conway Liao
Spring 2009 - Major Studio 2
MFA Design + Technology
Parsons School of Design
11 February 2009

2. Domain Map
Experience Documentation
Soft
Perspective History
Hard
Communication
Memories Primary
Emotions Video
Photo
Somatosensatory
Visual Time Capsule
Auditory
Secondary
Thermoception
Traditions
Blog
Body Language Olfactory
API Mapping
Databases
Data Visualization
Semiotics Gestures

3. Primary-Hard
Can senses be digitally packaged?

4. The Sponge

5. The Sponge
• While senses are nerve impulses that the
brain interprets, experiences are unique
to each individual.
• Device that “hard” documents “primary”
experiences and encodes it into
“metadata”

6. Sight Sound Temperature Orientation Time
Metadata

7. Metadata
• Any data that helps describe the content
or characteristics of a file.
• Descriptive information embedded
inside a file.
• Adobe XMP (Extensible Metadata
Platform) = Cryonics?

8. Precedents
Past works / Inspiration / Research

9. Triops Camera

10. Triops Camera
• Panoramic Digital Camera
• Equipped with three fisheye lenses
• Experience New Perspectives
• Responds to sound or motion but also
has a manual shutter
• Shoots panoramas

11. Triops Camera

12. Google Street View

13. Google Street View

14. QuickTime VR
• Moves the photographic image from 2D
into 3D space.
• Enables viewers to explore virtual
worlds.
• Click Here

15. MIT Sixth Sense

16. MIT Sixth Sense
• Wearable computer that turns any surface
into an interactive display screen
• Harness the internet to create a “digital sixth
sense”
• In the tactile world we use our five senses
to take in information and respond to it.
• A lot of information that helps us
understand and respond to the world
doesn’t come from our senses.
• Click Here

17. Steve Mann

18. Steve Mann
• Wearable Computer
• Photographic memory
• Shared memory
• Enhances quality of life
• Personal safety

19. Geocache
• Treasure hunting game played using GPS
enabled devices
• Geocacher can place a geocache (treasure)
anywhere in the world by entering its GPS
coordinates.
• Other geocachers can utilize their own GPS
to locate geocache
• http://geocaching.com is where people
share stories and photos online.

20. Secondary-Soft
How can metadata be used to create an experience?

21. Pipilotti Rist

22. Pipilotti Rist
• “Pour Your Body Out” @ MOMA
• Immersion media
• Speakers underneath the seating
• 25’ high video projections
• Click Here

23. MIT Virtual
Constructs

24. DAIM - Real-time
Grafiti

25. DAIM - Real-time
Grafiti
• Smears virtual paint across an empty gallery
space
• Utilizes ARToolKit (Augmented Reality
Toolkit)
• Handheld markers
• Multi-camera scheme
• Click Here