We present our work on the development of a device by which a dancer may wirelessly transmit bodily motion to a MIDI-capable device or computer in order to produce or alter sound, creating music that is immediately integrated with and inseparable from the dance. To begin we briefly consider the history of movement mapping and dance notation. Moving into more recent history, we then present the technology employed (Arduino). An accelerometer measures the motion. The x/y/z components are scaled and inserted into a MIDI message, which is then transmitted to a receiver and can be interpreted by any MIDI device. The motions can be mapped to parameters such as filters, pitch, etc., allowing the dancer to affect any sound that can be created electronically. Several short vignettes will be used to demonstrate the device, followed by a three-minute piece showing the techniques working together as a whole.

1. Tricia Postle – Artistic Director, Majlis Art Garden { majlisarts.com }
Leif Bloomquist – Programmer & Composer { schemafactor.com }
New Adventures In Sound Art
TransX Transmission Art Symposium – Toronto, ON, Canada
May 19th, 2013
The Dancer From The Dance:
Mapping Motion With Sound
Via Radio Transmission

2. The Dancer From The Dance
O chestnut-tree, great-rooted blossomer,
Are you the leaf, the blossom or the bole?
O body swayed to music, O brightening glance,
How can we know the dancer from the dance?
William Butler Yeats
(from The Tower, 1928)
Source: folkadvance.org

3. The Dancer From The Dance
1. Introduction
2. The Past: Movement Mapping and Dance Notation
3. The Present: Arduinos, XBees and the MotionMIDI device
4. The Future: What’s next?
5. Other work in this area
6. Vignettes
7. Performance

4. Introduction
• How to add visual interest to a live electronic music
performance?
• lights, lasers, strobes
• projected visuals
• cool outfits
• etc
• What about dance?
• How to integrate the dancers with the performance?
Source: www.thissongissick.comDeadmau5

5. The Past: Movement Mapping and Dance Notation
Beauchamp-Feuillet Notation, c. 1680

6. Movement Mapping and Dance Notation
Friederich Zorn’s system
from his “Grammatik der
Tanzkunst” (1887)

7. Movement Mapping and Dance Notation
Labanotation/Kinetography
first developed in the 1920s
Source: Wikipedia

8. 8
Flexibility within the piece + constraint within the form
8
flamenco: llamada kathak: hastak
skokie public library
Movement Mapping and Dance Notation

9. Movement Mapping and Dance Notation
Motion capture... optical, inertial, mechanical, magnetic

10. The Present: Introducing Arduino
• Arduino is an open-source electronics prototyping platform
based on flexible, easy-to-use hardware and software. It's
intended for artists, designers, hobbyists, and anyone
interested in creating interactive objects or environments.
• Named for Arduin of Ivrea, King of Italy (1002-1014)
Source: www.arduino.cc

11. Arduino Continued
• Based on Atmel Microprocessors and the Processing language
(a simplified version of C++)
• Designs are “Open” – free to use, modify, and create derivatives (hardware
and software)
• Strong community focus
• Countless variations and sizes
• Some (i.e. Arduino Lilypad) can be sewn into clothing!
• Countless add-ons, “shields”, kits, sensors, actuators, other
devices…like radios! – ZigBee (XBee), 802.11 (Wifi), Bluetooth…
• SparkFun Electronics { www.sparkfun.com }
• AdaFruit Industries { www.adafruit.com }
• Creatron (right here in Toronto!) { www.creatroninc.com }
• Many many more

12. The MotionMIDI Prototype
• Version 1 presented at Toronto Mini-Maker Faire in May 2011
• Version 2 presented at KwartzLab Makerspace in October 2012
Accelerometer
Arduino Uno
Transmitter
XBee
Receiver
XBee
USB
9V Battery

13. The MotionMIDI System
XBee Radio
Receiver
FTDI Serial-to-
USB Adaptor
Digital Audio
Workstation
FL
Studio, Ableton, Cubase
, Reason etc.
Virtual
COM Port
Serial to MIDI
Adaptor/Driver
) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) )
Accelerometer
(ADXL335)
Arduino
Uno
XBee Radio
Transmitter
Analog Voltages
(x, y, z)
MIDI Messages
@ 38400 baud
Wireless Data
MIDI Messages
@ 38400 baud
USB
MIDI Yoke
MIDI Messages
@ 38400 baud
Movement and
Gravity
Or MIDI to
Synths,
Keyboards, etc.

14. Some Vector Math
• Acceleration forces in 3D can be
represented as 3 components:
x, y, and z.
• This includes gravity!
• This gives you the direction and
magnitude of the acceleration.
• Can determine orientation of the
device (i.e. smartphones)

15. MIDI Assignments
Source: www.djtechtools.com
•

16. This is the *entire* sketch (program)!
#define CENTER 371 // Same for x, y, and z
#define RANGE 100
#define MAX (CENTER+RANGE)
#define MIN (CENTER-RANGE)
#define LED 13
void setup()
{
// Direct connect or XBee. Use 31250 for Raw MIDI
Serial.begin(38400);
pinMode(LED, OUTPUT);
}
void loop()
{
int x, y, z;
byte x1, y1, z1;
digitalWrite(LED, HIGH); // set the LED on
x = analogRead(0); // read analog input pin 0
y = analogRead(1); // read analog input pin 1
z = analogRead(2); // read analog input pin 2
x1 = AccelToCC(x);
y1 = AccelToCC(y);
z1 = AccelToCC(z);
sendMIDI(0xB0,20,x1); // Channel 1, CC#20
sendMIDI(0xB0,21,y1); // Channel 1, CC#21
sendMIDI(0xB0,22,z1); // Channel 1, CC#22
digitalWrite(LED, LOW); // set the LED off
delay(100); // wait 100ms for next reading
}
// Send a MIDI message
void sendMIDI(byte cmd, byte data1, byte data2)
{
Serial.write(cmd);
Serial.write(data1);
Serial.write(data2);
}
// Map an input between MIN and MAX to 0 to 127
byte AccelToCC(int in)
{
// Bound
if (in > MAX) in=MAX;
if (in < MIN) in=MIN;
// Change zero-offset
in = in-MIN;
// Scale between 0.0 and 1.0
float temp = (float)in/((float)MAX-(float)MIN);
return temp*127;
}
{ https://github.com/LeifBloomquist/MotionMIDI }
[MIT License]

17. Future Plans
• Look into commercial and hobbyist sensors that could be employed (magnetic,
inertial).
• Use the sensor inputs to generate notes, directly or as seeds to a fractal
generation algorithm.
• Work on more pieces, involve and get feedback from the dance community
(i.e. Coexisdance)
• Small production run?
• Explore and have fun!

18. Acknowledgements
Special thanks to:
• Seth Hardy (Site3 coLaboratory) for XBee radio programming help.
• Gauri Vanarase for previous dance history input.

19. Related work
• Imogen Heap and “The Gloves”
{ www.imogenheap.com, theglovesproject.com }
• Loretta Faveri and SonicWear { www.sonicwear.ca }
Commercialization of OCADU project similar to ours – in discussions about
collaboration.
• Kinectar { www.kinectar.org } Microsoft XBOX Kinect with MIDI devices.
• The LEAP Motion { www.leapmotion.com } Handsfree 3D input device
• Many music-related apps in development (coming July 2013)
• Many more!

20. Vignette #1: sumi-e
• Using the Image-Line (FL Studio) “Harmor” generator VST with a synthesized
Gong preset.
• X-Axis is mapped to the instrument pitch

21. Vignette #2: sweet synth strings
• Using the FL Studio “Plucked!” virtual strings synthesizer.
• Y-Axis CC is mapped to arpeggiation rate
• X-Axis CC is mapped to tonal color

22. Vignette #3: oontz
(The world’s shortest electronic body music [EBM] piece)
• Force magnitude CC is mapped to the cutoff frequency for the low-pass filter
on the main synth line
Source: glamslamentertainments.com

23. Any questions?

24. Performance

25. Biography – Tricia Postle
Tricia Postle is the artistic director of Majlis Art Garden, a
multidisciplinary seasonal space in Queen West presenting
poetry, music, dance and storytelling.
This summer Majlis will host a salon-style evening on the
intersection of music and technology. For further
information please visit majlisarts.com .
Tricia is a lyric mezzo and poet, and has been known to
play the hurdy-gurdy, psaltry, and qanun. She has recently
started setting contemporary poetry as art song. She
holds a BA in Medieval Studies and Music from the
University of Toronto.

26. Biography – Leif Bloomquist
Leif Bloomquist has been creating computerized sounds
since the days of the Commodore 64. Trained in clarinet
and percussion, he now composes using sequencing
software and homebuilt hardware.
His music can be heard in environments such as gothic
nightclubs, CBC Radio 3, ambient festivals, and churches.
He has released five albums to date through his Schema
Factor and Interweaver projects. For further information
please visit www.schemafactor.com .
When not creating experimental music, Leif is a senior
engineer at MDA, an aerospace company best known for
their work on the Canadarm. He holds a BASc in Systems
Design Engineering from the University of Waterloo.