The Dancer From The Dance:  Mapping Motion With Sound Via Radio Transmission
Upcoming SlideShare
Loading in...5

The Dancer From The Dance: Mapping Motion With Sound Via Radio Transmission



We present our work on the development of a device by which a dancer may wirelessly transmit bodily motion to a MIDI-capable device or computer in order to produce or alter sound, creating music that ...

We present our work on the development of a device by which a dancer may wirelessly transmit bodily motion to a MIDI-capable device or computer in order to produce or alter sound, creating music that is immediately integrated with and inseparable from the dance.

To begin we briefly consider the history of movement mapping and dance notation. Moving into more recent history, we then present the technology employed (Arduino).

An accelerometer measures the motion. The x/y/z components are scaled and inserted into a MIDI message, which is then transmitted to a receiver and can be interpreted by any MIDI device. The motions can be mapped to parameters such as filters, pitch, etc., allowing the dancer to affect any sound that can be created electronically.

Several short vignettes will be used to demonstrate the device, followed by a three-minute piece showing the techniques working together as a whole.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

The Dancer From The Dance:  Mapping Motion With Sound Via Radio Transmission The Dancer From The Dance: Mapping Motion With Sound Via Radio Transmission Presentation Transcript

  • Tricia Postle – Artistic Director, Majlis Art Garden { }Leif Bloomquist – Programmer & Composer { }New Adventures In Sound ArtTransX Transmission Art Symposium – Toronto, ON, CanadaMay 19th, 2013The Dancer From The Dance:Mapping Motion With SoundVia Radio Transmission
  • The Dancer From The DanceO chestnut-tree, great-rooted blossomer,Are you the leaf, the blossom or the bole?O body swayed to music, O brightening glance,How can we know the dancer from the dance?William Butler Yeats(from The Tower, 1928)Source:
  • The Dancer From The Dance1. Introduction2. The Past: Movement Mapping and Dance Notation3. The Present: Arduinos, XBees and the MotionMIDI device4. The Future: What’s next?5. Other work in this area6. Vignettes7. Performance
  • Introduction• How to add visual interest to a live electronic musicperformance?• lights, lasers, strobes• projected visuals• cool outfits• etc• What about dance?• How to integrate the dancers with the performance?Source: www.thissongissick.comDeadmau5
  • The Past: Movement Mapping and Dance NotationBeauchamp-Feuillet Notation, c. 1680
  • Movement Mapping and Dance NotationFriederich Zorn’s systemfrom his “Grammatik derTanzkunst” (1887)
  • Movement Mapping and Dance NotationLabanotation/Kinetographyfirst developed in the 1920sSource: Wikipedia
  • 8Flexibility within the piece + constraint within the form8flamenco: llamada kathak: hastakskokie public libraryMovement Mapping and Dance Notation
  • Movement Mapping and Dance NotationMotion capture... optical, inertial, mechanical, magnetic
  • The Present: Introducing Arduino• Arduino is an open-source electronics prototyping platformbased on flexible, easy-to-use hardware and software. Itsintended for artists, designers, hobbyists, and anyoneinterested in creating interactive objects or environments.• Named for Arduin of Ivrea, King of Italy (1002-1014)Source:
  • Arduino Continued• Based on Atmel Microprocessors and the Processing language(a simplified version of C++)• Designs are “Open” – free to use, modify, and create derivatives (hardwareand software)• Strong community focus• Countless variations and sizes• Some (i.e. Arduino Lilypad) can be sewn into clothing!• Countless add-ons, “shields”, kits, sensors, actuators, otherdevices…like radios! – ZigBee (XBee), 802.11 (Wifi), Bluetooth…• SparkFun Electronics { }• AdaFruit Industries { }• Creatron (right here in Toronto!) { }• Many many more
  • The MotionMIDI Prototype• Version 1 presented at Toronto Mini-Maker Faire in May 2011• Version 2 presented at KwartzLab Makerspace in October 2012AccelerometerArduino UnoTransmitterXBeeReceiverXBeeUSB9V Battery
  • The MotionMIDI SystemXBee RadioReceiverFTDI Serial-to-USB AdaptorDigital AudioWorkstationFLStudio, Ableton, Cubase, Reason etc.VirtualCOM PortSerial to MIDIAdaptor/Driver) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) )Accelerometer(ADXL335)ArduinoUnoXBee RadioTransmitterAnalog Voltages(x, y, z)MIDI Messages@ 38400 baudWireless DataMIDI Messages@ 38400 baudUSBMIDI YokeMIDI Messages@ 38400 baudMovement andGravityOr MIDI toSynths,Keyboards, etc.
  • Some Vector Math• Acceleration forces in 3D can berepresented as 3 components:x, y, and z.• This includes gravity!• This gives you the direction andmagnitude of the acceleration.• Can determine orientation of thedevice (i.e. smartphones)
  • MIDI AssignmentsSource:•
  • This is the *entire* sketch (program)!#define CENTER 371 // Same for x, y, and z#define RANGE 100#define MAX (CENTER+RANGE)#define MIN (CENTER-RANGE)#define LED 13void setup(){// Direct connect or XBee. Use 31250 for Raw MIDISerial.begin(38400);pinMode(LED, OUTPUT);}void loop(){int x, y, z;byte x1, y1, z1;digitalWrite(LED, HIGH); // set the LED onx = analogRead(0); // read analog input pin 0y = analogRead(1); // read analog input pin 1z = analogRead(2); // read analog input pin 2x1 = AccelToCC(x);y1 = AccelToCC(y);z1 = AccelToCC(z);sendMIDI(0xB0,20,x1); // Channel 1, CC#20sendMIDI(0xB0,21,y1); // Channel 1, CC#21sendMIDI(0xB0,22,z1); // Channel 1, CC#22digitalWrite(LED, LOW); // set the LED offdelay(100); // wait 100ms for next reading}// Send a MIDI messagevoid sendMIDI(byte cmd, byte data1, byte data2){Serial.write(cmd);Serial.write(data1);Serial.write(data2);}// Map an input between MIN and MAX to 0 to 127byte AccelToCC(int in){// Boundif (in > MAX) in=MAX;if (in < MIN) in=MIN;// Change zero-offsetin = in-MIN;// Scale between 0.0 and 1.0float temp = (float)in/((float)MAX-(float)MIN);return temp*127;}{ }[MIT License]
  • Future Plans• Look into commercial and hobbyist sensors that could be employed (magnetic,inertial).• Use the sensor inputs to generate notes, directly or as seeds to a fractalgeneration algorithm.• Work on more pieces, involve and get feedback from the dance community(i.e. Coexisdance)• Small production run?• Explore and have fun!
  • AcknowledgementsSpecial thanks to:• Seth Hardy (Site3 coLaboratory) for XBee radio programming help.• Gauri Vanarase for previous dance history input.
  • Related work• Imogen Heap and “The Gloves”{, }• Loretta Faveri and SonicWear { }Commercialization of OCADU project similar to ours – in discussions aboutcollaboration.• Kinectar { } Microsoft XBOX Kinect with MIDI devices.• The LEAP Motion { } Handsfree 3D input device• Many music-related apps in development (coming July 2013)• Many more!
  • Vignette #1: sumi-e• Using the Image-Line (FL Studio) “Harmor” generator VST with a synthesizedGong preset.• X-Axis is mapped to the instrument pitch
  • Vignette #2: sweet synth strings• Using the FL Studio “Plucked!” virtual strings synthesizer.• Y-Axis CC is mapped to arpeggiation rate• X-Axis CC is mapped to tonal color
  • Vignette #3: oontz(The world’s shortest electronic body music [EBM] piece)• Force magnitude CC is mapped to the cutoff frequency for the low-pass filteron the main synth lineSource:
  • Any questions?
  • Performance
  • Biography – Tricia PostleTricia Postle is the artistic director of Majlis Art Garden, amultidisciplinary seasonal space in Queen West presentingpoetry, music, dance and storytelling.This summer Majlis will host a salon-style evening on theintersection of music and technology. For furtherinformation please visit .Tricia is a lyric mezzo and poet, and has been known toplay the hurdy-gurdy, psaltry, and qanun. She has recentlystarted setting contemporary poetry as art song. Sheholds a BA in Medieval Studies and Music from theUniversity of Toronto.
  • Biography – Leif BloomquistLeif Bloomquist has been creating computerized soundssince the days of the Commodore 64. Trained in clarinetand percussion, he now composes using sequencingsoftware and homebuilt hardware.His music can be heard in environments such as gothicnightclubs, CBC Radio 3, ambient festivals, and churches.He has released five albums to date through his SchemaFactor and Interweaver projects. For further informationplease visit .When not creating experimental music, Leif is a seniorengineer at MDA, an aerospace company best known fortheir work on the Canadarm. He holds a BASc in SystemsDesign Engineering from the University of Waterloo.