This talk discusses experimental techniques to interactively manipulate space and time in moving images through the metaphor of time as a tangible substance, including early devices that reproduced the illusion of motion, expanded cinema experiments that deconstructed the cinematic medium and code, and space-time experiments using both analog techniques like chronophotography and slit-scan photography as well as digital techniques enabled by random-access recording and the "video cube" paradigm allowing a god's viewpoint virtual reality perspective.

1. Workshop at Animatronica/Microwave New Media Art Festival,
(www.microwavefest.net), Hong Kong, 4-15 Nov. 2006
Time Delayed Cinema
by Alvaro Cassinelli, Assistant Professor
Ishikawa-Komuro-Namiki Laboratory, Meta Perception Group
(www.k2.t.u-tokyo.ac.jp/), University of Tokyo

2. This talk:
Experimental techniques to interactively scramble Space & Time in
the moving image, thanks to the
metaphor of time as a tangible substance
...and resulting interesting visual & narrative implications.

3. Plan
I) Early devices to reproduce the illusion of motion
• Victorian “optical gadgets”
• The rise of the Cinematograph
• Breaking the rules
II) Expanded Cinema (device and codes)
• Deconstruction of the medium & experimental cinema
III) Space-Time Experiments:
• Early analog techniques
• Digitally Era & the Video Cube paradigm
IV) Khronos Projector experiments
• Analog techniques
• Digitally Era & the God’s Viewpoint paradigm
V) Considerations and Conclusion

4. I. Early devices recreating the
illusion of motion

5. Early motion-illusion devices
chinese
• Shadow puppets…
javanese
• Magic Lantern (probably 2d century, China):
precursor of slide projector
• Victorian Era (second half of 19th century). Systematic study of basis of
the moving picture ：
afterimage and stroboscopic effects

6. thaumatrope
zoetrope
phenakistiscope
kineograph
praxinoscope
kinetoscope
…And next?

7. • thaumatropes (only two images / motion, metamorphose) [W. Herschel, 1824]
• Phenakistoscope (or phantascope, or stroboscope) [J. Plateau, 1832]
• zoetrope (or “daedalum”) [G. Horner, 1834 …or Ting Huan ( 丁緩 ), 180 A.D.]
• praxinoscope (uses mirrors) [Ch. E. Reynaud, 1877]
• kineograph (just a flip-book!) [J. B. Linnet, 1886]
• Kinetoscope (a cinema… for one person) [Th. A. Edison, 1888]
…No more than Victorian Era
optical “gadgets”??
…Really? Purkinje (physiologist, persistence of vision, 1818):
predicted that one day it was going to be possible
to tell whole stories with moving images…

8. The rise of the cinematograph
Cinematograph, Léon Bouly/
Freres Lumiere (~1892).
Photography well developed by the end of 19th…
• Earliest use: lantern slides (real places and people)
• Eventually it became possible to take and display
The introduction of
pictures fast enough to reproduce the illusion of
the Kodak camera motion...
in 1888 was a
dramatic event.
Cinematograph (Lumière, 1892)
Lots of limitations! (no stereo vision, no panoramic, linear narrative)…
…but powerful enough as to create a new form of art!
• narrative films
Seventh Art • documentary film
of filmmaking • animation film…
ma…
ental cine
• experim
The Arrival of a Train at the station (1895),
Freres Lumiere

9. [ An early divide… ]
Double legacy (photography / magic lantern) of the cinematographic
hardware creates a permanent tension:
• Faithful depiction of reality (scientific purposes &
documentaries). From modernist realism to neo-realism. “Myth
of total cinema” revived today with VR immersive tools?
Vittorio De Sica's Ladri di biciclette (1948).
Non-professional actors, hardships of working-class
• Powerful tool for (fictional) storytelling.
Artistic / political uses. Persuasive (and pervasive in pre-
TV times).
Battleship Potyomkin (1925), Sergei Eisenstein:
power of editing and montage.
Whatever way, Hollywood “entertainment code” dominant in the 1920s.
Continuity of editing (invisible camera and sound editing)

10. Breaking the rules
Beyond/besides Hollywood: pre-WWII avant-garde / post-
WWII “Experimental Cinema”, Video Art and Digital Era.
Pre-WWII modernist (European) Avant-Garde:
• Influenced by visual-arts movements of the time (Surrealism, Un chien andalou , 1929 (Luis
Buñuel and Salvador Dalí)
Dadaism , Cubism, Futurism, Suprematism, Constructivism)
• Expanded cinematographic code (through montage)
Maya Deren,
Meshes of the
Post-war avant-garde (50s, 60s): Afternoon (1943)
• Experimental Cinema emerges as independent genre,
(New American Cinema)
• Explosion of the independent filmmaking
• Expanded Cinema experiments:
• extended cinematographic code (Structuralists,
Fluxus, minimalist, conceptual, etc)
• but also extended medium. Precursor to today’s
“new media arts”…
David Lynch,
Lost Highway
(1997)

11. Video revolution & Video Art (70s):
• Electromagnetic support and sophistication of special effects
• Influences: post-structuralism & deconstruction, minimalism, pop art,
Fluxus experiments (film & musical score, repetitions, etc.)
• Emergence of Video Art ≠Experimental Cinema. Interactiveness, loose
or no narrative.
• Closed-circuit and interactive installations break the traditional
“spectator-spectacle” relationship.
Nam June Paik,
Magnet TV, 1965.
Digital revolution & digitally expanded cinema (80s~):
• cheap and popular technology
• virtual reality
John Maeda
• links with computer game industry Nature, 2005
• Computer graphics industry
• Virtual Cinema
• Digital Video Art (algorithmic image, Programming languages
for the visual arts such as DBN – Design by Numbers and
Processing…)
MACHINIMA (machine-animation):
3D games and first-person shooters
as a platform for production and film
genre itself.

12. II. Expanded Cinema Revolution
& the “Media Arts”
End of “cinematograph reign” (filmic code and apparatus):
• Expansion of cinematographic code (≠invisible edition)
• Medium expansion (techniques, material detournements…)

13. Expanded Cinema Experiments
• Material experiments
• Retro-toys
• Multi-modal imagery
• Narrative experiments
EXPERIMENTS • Sound experiments
• Found sound and images (form of found art)
• Space experiments
• Time experiments
• Space-Time Experiments
… 60’s experiments resumed in today’s digital Media Arts

14. Material experiments
Challenging the convention of the filmic machine
Scratched, punched, painted the
celluloid…
Replacement of the film by a thread…
Rohfilm, B. & W.
… or the light beam by a rope! Hein, 1968
P. Weibel, Lichtseil, 1973
Turning the whole mechanism upside-down…
…New recording mediums (film, electromagnetic…)
J. Maire, Demi-pas, 2004
In search of the origins (media-
archeology idiom)…

15. Retro-toys and the archeological idiom
Exploring the medium: back to origins & re-appropriation
Julien Maire’s Demi-pas (2004) or the
moving magic lantern revisited…
Philip Worthington, Shadow Monsters,
2005
Toshio Iwai (3D zoetrope)
Toshio Iwai, Morphovision – distorted house, (2005)

16. Space experiments
Challenging the “painting” paradigm: image is
liberated from its frame/screen
Daniel Sauter
• panoramic shooting and projection ( ⇒ immersiveness) Light Attack, 2004
• support of the projection (screen: flat, curved, multiple,
mobile, people, water, smoke, wood…)
• projection environment (beyond the “dark room”, multi-
theatres, building, road from cars…)
Dan Graham, Cinema, 1981 (model)
Cinelabyrinth, Radúz
Çinçera, (1990)
Josef Svoboda, Polyekran, 1967
Robert Whitman’s Prune Flat, 1965
(projection on people)

17. Time experiments (we are getting closer… )
Challenging the notion of classical narrative time & montage
• “Real time” in the movies. Experimental cinema, but
prefigured by neo-realists & french nouvelle vague.
• Manipulated time in the movies. delayed or compressed
time Psychological time / psychedelic connotations. Film
shortening and lengthening, repetition.
Warhol, Empire State Building
(1963), 8 hours film
Douglas Gordon, 24 Hour Psycho (1993)

18. Time-Lapse / Speed-Up photography
Edgerton rapatronic cameras could
take a million frames per second ( 1945,
Los Alamos first Atomic Bomb test)
Scientific use:
• slow phenomena (ex: biology, plants, cells)
• fast phenomena (biomechanics, explosions)
© Andrew Dunn
© E.M. Kinsman
Bill Viola
© GraniteBay Software The quintet of the astonished (2000)
Extreme form of slow-motion & fast-forward:
• see the unseen / enhanced perception, fleeting emotions
• time distortion ≈ psychedelic experiences with drugs…

19. [ Hyper-Slow motion: the “Bullet-Time” special effect ]
Bullet-Time effect achieved by setting dozens of still cameras surrounding the
subject. These are usually triggered at once or sequentially (very similar to
Muybridge’s 1878 setup…)
…“Virtual Bullet-Time is much easier to do in computer graphics (virtual camera).

20. III. Space-Time experiments
a) Analog techniques for mixing Space & Time
b) The digital Era & the God’s Viewpoint
“[…]the distinction between past, present, and future
is only a stubbornly persistent illusion “
Albert Einstein

21. a) Analog techniques for mixing Space & Time
Preserving the integrity of the image on the screen:
• Non-linear editing can scramble the chronology, while preserving the
integrity of the images on the screen…
Scrambling space and time on the same screen:
• Discrete: polyptych cinema
P. Greenaway, The Pillow book,
Continuous!
• Long exposure & Marey’s chronophotography 1996 multi-screened scene
• Slit-Scan photography…
Example of a long
exposure shot with
moving subject

22. Chronophotography
• Jules Marey “Chronophotography”:
pictures on the same plate using a
chronophotographic “gun”
Flying pelican
captured by Marey
(around 1882)
Zoopraxiscope
disk
• Eadweard Muybridge’s “batteries
of camera” setup is the precursor
of bullet-time special effect
(virtual camera) - the difference is
that in bullet-time, all pictures are
taken at the same time (a sort of
ultra-slow chrono-photography…)
Muybridge's The Horse in Motion (1878)
•Fine art and Medical applications (study of gait, biomechanics)
•Imagery reminiscent of Futurism (representation of speed, motion…)

23. Slit-Scan Photography
A moveable slide with a slit
replaces the shutter
(Effect known as focal plane shutter
distortion in a normal camera)
Hammer thrower, George Silk 1960
• Can dramatically enhance perception of action
• Imagery reminiscent of early Analytic Cubism (1912): different (time) views of the
same object are merged on the canvas…
Henri Lartigue photography of a
race car: use of focal place
Bob Mumford distortion while panning
Streak Photography
Robert Doisneau, controlled slit-scan
(couple spinning on a turntable )

24. Slit-Scan film (analog!)
• Slit-Scan during shooting? (Similar to slit-scan
photography, but with continuous film?)
I don’t know any example of this…
(would need “optical buffers”!) Christian Hossner
Slit Scan Movie, 35mm film (2000)
• Post-produced Slit-Scan (record everything on
film, delay sections of the image while printing):
• time-delaying mask most likely fixed.
• extreme case of “screen” Expanded Cinema
(screen smoothly broken into time-clusters)
Zbig Rybczynski, The Fourth Dimension (1988),
27-minute, 35mm color film
...edition becomes invisible again: it appears as if objects
themselves are warped (special effect?)

25. (already explored in the pre-war avant-garde!)
• Subject broken up, analyzed, and re-assembled in an abstracted form.
• Subject depicted simultaneously from a multitude of viewpoints.
Analytic Cubism
Cubist/Futurist
Synthetic Cubism
(i.e. “papier collé”)
The first Synthetic Cubist work, Picasso's
Still life with chair caning (1911-12)
Marcel Duchamp,
Georges Braque, Nu descendant un Escalier (1912)
Woman with a guitar, painted 1913

26. Putting a little order…
shooting reproduction
Single Place (SP) Multiple Places (MP)
SP MP SP MP
Pictorial SPATIAL Collection of
Single Time (ST)
Still,
motif CUBISM stills, and if
ST portrait ST
(repetition, (Brake, same subject:
a la Warhol) Picasso) Panorama
Slides, and if
same subject:
MT MT bullet-time
special effect
SP MP SP MP
SPACE-TIME
Multiple Times (MT)
Marey’s Marey’s
Chronophotogra CHRONO- CUBISM Muybridge’s
phy & Time- PHOTOGRAPHY (maximum CHRONO-
ST based CUBISM (jumps, ST mess… or PHOTOGRAPHY
Not easily
(Bacon, Duchamp machines) Synthetic (Muybridge’s
horses) categorized as a
& Futurists) Cubism) cinematographic
technique or a
POLYSCREEN visual genre
Early CINEMA Traveling, pan,
(fixed camera), tilt, dollying, CINEMA
MT MT craning etc in (Radok,
TIME-LAPSE
classical Svoboda)
photography
CINEMA

27. Working definition: “time-based Cubism”
• Simultaneous viewpoints are points in TIME instead of views in SPACE
• Marey’s Chronophotography (~1880): time-evolution in a single photographic plate…
• Aesthetics related to Futurism movement (representation of speed)
Umberto Boccioni, The Charge
of the Lancers, 1915
"…I see every image all the time in a shifting way and almost in shifting sequences."
Francis Bacon, Three Studies for Self-Portrait, 1974
Marcel Duchamp.
Nu descendant un Escalier (1912)
In “analog times”, painting / collage were the only ways to
arbitrarily blend multiple point of views on the canvas…

28. b) The Digital Era & the Video Cube paradigm
Total recording on cheap random access memory enables:
• Frame by frame non-linear edition
• Total recording & total control of reproduction (⇒ any case of
table can be filled a posteriori)
• “God’s viewpoint”: total cinema, immersive cinema, virtual reality…
• Real-time interaction (Live Cinema: at the image level -
A. Garcia adaptation (2006) of Bioy
Casares “Morel’s invention” (1940)
pshychadelic VJeeing, or at the narrative level)
Video Cube
But mass of information needs new interfaces: (from E.Elliott
Video Streamer)
• Multi-track non-linear editing software…
• Video as a volume that can be sculpted
…mass of information saturates cognition (infinite possible storylines invisible)
Still, we can appreciate this bunch of data as an object in space

29. Graphical-User-Interface (GUI) based software
• Post-production tools…
• Art? Special-effect tools? Eddie Elliott
Video Streamer (1992)
Michael Cohen et al.
Stylized Video Cubes (2002)
Sidney Fels, Kenji Mase & Eric Lee
Video Cubism (1999)
Martin Reinhart & Virgil Wildrich
tx-transform (1998)

30. Explicitly treated and displayed as an object (not for editing purposes)
Tamás Waliczky & Anna Szepesi,
Time Crystals, (1997)
Sidney Fels, Kenji Mase & Eric Lee
Video Cubism (1999)
Alvaro Cassinelli, Khronos Projector (2005)
Deformable screen and video volume sculpture
Joachim Sauter & Dirk Lüsebrink
The Invisible Shape of Things Past (1995)
Video volume mapped in real space!

31. [ Toshio Iwai’s tour de force ]
The video-cube is not only mapped in real space:
the “3D buffer” is the real object!
(this is a kind of 3D zoetrope)
Toshio Iwai with NHK Science & Technical
Research and Labs.
Morphovision: Distorted House (2005)

32. Live Slit-Scan (software invisible treats video as an object)
• live video-feed… Metaphor of the spatio-temporal
• no physical interaction… WARPING MIRROR
T. Iwai Another Time, Another Space (1993)
Steina Vasulka, Bent Scans (2002)
Jussi Ängeslevä & Ross Cooper
Last Clock (2001)
Bill Spinhoven
The Time Stretcher (1988-1994)
Daniel Sauter & Osman Khan
Christian Kessler, Transverser (1998) We interrupt your regularly scheduled program (2003)

33. Interactive space-time sculpting (non GUI-based)
Metaphor of TIME AS A SUBSTANCE (oil, water-like…)
Khronos Projector (2004) , A. Cassinelli
Camille Utterback, Liquid Time (2000)
Tangible interface enhances the
physicality of the act:
Painting with Time TIMESCAPE (2006), A. Cassinelli & H.
Naito (Tangible + LIVE input)

34. IV. Khronos Projector experiments
(details of the project at the artist talk, 18:00-18:30)

35. Possible Khronos Projector contributions to cinematic expansion…
Support of projection Video as a 3D
object
• tangible, deformable
• images on real space
• sensual interface
(≠ “cold digital technology”)
Interactive
Cubism:
“Found media” processor
• Smooth blending of
• Recycling video as thousands of images
rough material for
sculpting space-time
• Beyond the DVD player.
Time as a physical
substance
Deformable
“time-mirror”
Interactive
Paintings

36. Some examples (click on images to launch video)
<www.k2.t.u-tokyo.ac.jp/members/alvaro/Khronos/>

37. Interactive, “instant” Cubism
Francis Bacon, “Three Studies for Self-Portrait”, 1974
“Three (Khronos) interactive snapshots of M. Bressaglia”, 2005

38. Combinations: Chromatic-Time Arrow + LIVE video:
Chromatic-Time Arrow:
mixing Time and Color
spaces
.. A “live” Francis
Bacon-like
portrait!

39. Further experiments with the Khronos Projector
• Large screen & story that implies physical effort and
displacement of the spectator
• Producing 3D prints of the sculpted video cube
• Mixing dimensions: color-space, gesture-space (face
expressions could “freeze” time around the actor), etc.
• Full-body interaction screen (dancers inside the box).
• Inversion the interaction principle: gestures in real space
modify the shape of the projection screen
• The Volume Slicing Display: virtual volumes intersect real
space…
• Spatialized sound…
(more on this at artist talk)

40. IV) Considerations & Conclusion
Important Questions:
• Despite interactivity, most of these “time-warping installations” look like
contemporary versions of Victorian optical gadgets playing “tricks” on people’s
perception of reality…
• Can these interfaces lead to a form of performative cinema based on real-time
space-time manipulation?
• And in general, how much today’s Media Art installations are no more (and no less)
than 21st century “tricks”?
• Can we really do more than “tricks”? (i.e., generate a full-fledged interactive /
cinematic code).
• And do we need to be concerned by that???
Remember: the cinematograph was perceived as “an invention
without future” by the Lumière Brothers. We may be just in the
age of the experiments (perhaps the most fun, let’s enjoy…)

41. Some References
Expanded Cinema:
• Expanded Cinema by Gene Youngblood, Publisher E.P. Dutton (1970)
•Online: <www.vasulka.org/Kitchen/PDF_ExpandedCinema/ExpandedCinema.html>
• Future cinema: the cinematic imaginary after film, Jeffrey Shaw and Peter Weibel [Eds.],
• Book: The MIT Press, Cambridge, MA, 2002.
• Online: <http://www.zkm.de/futurecinema/index_e.html>
Slit-Scan photography and video (and more):
•Levin, Golan. An Informal Catalogue of Slit-Scan Video Artworks, 2005
<http://www.flong.com/writings/lists/list_slit_scan.html>
Time Lapse photography:
•Software for Canon cameras: <www.granitebaysoftware.com/Product_gbt.aspx>
•Plants in Motion: <plantsinmotion.bio.indiana.edu/plantmotion/flowers/flower.html>
… and also Khronos Projector website:
• Online: <www.k2.t.u-tokyo.ac.jp/members/alvaro/Khronos/>
Khronos Projector in
“cellular video” mode