For the visual backdrop of a concert dance performance that was not only beautifully mood-appropriate, but also dramatically responsive, the motion detection features of the software library OpenCV were an integral part of the solution. Live imagery created with OpenCV's OpticalFlow motion detection object in Processing blended with video imagery controlled by the Isadora application to provide a rich palette of visuals for the "Loom" performance. Echoes of the dancers movement mixed with video imagery and dancers' shadows comingle with vivid stage lighting for a stage environment that complements the dance without overwhelming it.

1. Loom: Shadow Stage Visuals
for Dance Performance
a.k.a. A cool thing I did during my
not-a-vacation sabbatical year
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 1

2. Media arGst goals for Loom
• Support mood created by choreography and music
• Incorporate live echoes of dancer's bodies into the
imagery
• Make student dancers look marvelous
• Reasonable setup for existing equipment and
performance space
Motivating worry: “image overwhelms– distracts from the real thing”
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 2

3. What wasn’t working
• ConGiguring to code anything for
Kinect 2 in OpenFrameworks in
BootCamped Windows partition
• Webcam image processing in Isadora
(want to emphasize their movement,
not their color or luminosity)
Isadora is a software application for interactive,
real-time manipulation of digital media designed
for live performance environments. It is my
workhorse app.
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 3
Apply
head
here

4. What might work
• Had counseled many students about the
great results earlier students had with the
OpenCV library for the Processing language
• Had used the Syphon inter-app frame
sharing library successfully with Isadora
and Processing months earlier at an Inst-Int
workshop
Reader, it worked.
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 4

5. Processing OpenCV library demo
OpGcalFlow
“What does the drawOpticalFlow() method code
look like?”
W. Michelle Harris wmhics@rit.edu • RIT
IGM/MAGIC
5
OpGcal flow is the
paUern of apparent
moGon of image
objects between two
consecuGve frames
caused by the
movement of object
or camera. It is a 2D
vector field where
each vector is a
displacement vector
showing the
movement of points
from first frame to
second.
-OpenCV docs

6. W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 6
Processing OpenCV library demo
OpGcalFlow
“What does the drawOpticalFlow() method code
look like?
Let me play.”

7. Flow lines and Glow lines with time decay/blur
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 7

8. Flow ellipses and Glow ellipses with time decay/blur
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 8

9. Flow ghost trails and trails with time decay/blur
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 9

10. Demo capture
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 10

11. Processing sends frames to
Isadora via Syphon, Isadora
sends keyboard commands back
to Processing via OSC.
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 11

12. Video paleUe
(plus several
gradients)
Two key approaches:
1. Layer Glow image over gradient or video
2. Reveal video pixels according to Glow
image pixels (luminance)
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 12

13. W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 13
and dancer shadows
Projector with webcam taped on top

14. https://www.youtube.com/watch?v=qTHCWj1ff8E&feature=youtu.be
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 14

15. Credits
“Loom” choreography by Missy Pfohl Smith with the dancers
Original score, “Lift” composed by Garret Reynolds
Lighting design by Rose Pasquarello Beauchamp
Musicians: Billy Petito, guitar, Anna Dunlap, harp, Ben Magruder, viola,
Stephanie Chen, cello
Dancers: Sarah Bjornland, Rachael Hsu, Fulei Peng, Zhonghui Sun, Shuang
Wu
Sabbatical hosting and HD Projector courtesy University of Rochester
College of Arts, Sciences, and Engineering
Performance and rehearsal stills from video by Missy Pfohl Smith
Performance recorded April 21, 2017 at Spurrier Dance Theater,
University of Rochester
W. Michelle Harris wmhics@rit.edu • RIT IGM/MAGIC 15