Three continents networked music and dance performance: CZ-ES-BR-KR

1. - three continents networked music and dance performance -
Prague(CZ)-Barcelona(ES)-Salvador(BR)-Daejeon(KR)
Aug. 21st, 2013
APAN e-Culture WG

2. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !2
Program
◇ Wednesday 21 Aug, 2013, at KAIST in Daejeon, Republic of Korea (UTC/GMT +9)
• 17:30~17:40 – Welcome & Introduction of the Cyber Performance(CP)
• 17:40~17:50 – CP Stage 1 : MY(16:40)-KR(17:40)
• (17:50~17:55) – Preparation for next stage
• 17:55~18:10 – CP Stage 2 : BR(05:55)-ES(10:55)-CZ(10:55)-KR(17:55)
• 18:10~18:20 – Thanks & Photo
• (18:20~18:30) – (Move to banquet-place)
※ Kuala Lumpur: UTC/GMT +8, Daejeon: UTC/GMT +9 (MY and KR: no DST on Wed 21 Aug, 2013)
• Wed 21 Aug, 2013 is applied by Barcelona: UTC/GMT +2, Prague: UTC/GMT +2, and Salvador: UTC/
GMT –3 (Standard time zone: ES: +1, CZ: +1, and BR: -3)

3. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !3
Introduction
• dQ13 Dancing beyond TIME at the 36th APAN meeting in Daejeon Korea, three continents
networked music and dance performance, joints artists and engineers located across the world
together in real time. The event begins simultaneously on Wednesday August 21, 2013 at
08:55 UTC/GMT. That translates to: 05:55 Salvador time, 10:55 DST Barcelona, 10:55 DST
Prague and 17:55 Daejeon time.
• This is organized and produced by the KAIST Graduate School of Culture Technology and
Korea Institute of Science and Technology Information(KISTI) in collaboration with the
Konic Thtr and i2CAT Foundation of Barcelona(ES), the Ivani Santana/GP Technological
Poetics and Rede Nacional de Ensino e Pesquisa(RNP) of Salvador(BR), Academy of
Performing Arts in Prague (HAMU), CESNET association of Prague(CZ) and eCulture WG
of APAN.
• The global audience can watch the show at http://live.kisti.re.kr (ID: apan36 PW: kreonet)
and also participate live via their mobile smartphones (or other network-connected device).

4. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !4
Concept
• During the APAN meeting, there have been cyber performances planed and presented by
eCulture WG. The members of the performance team are trying to find the best solution to
overcome obstacles in performing arts over networks, such as a main-sub relation between
stages and/or actors due to network delay including latency. To solve this problem, skeleton
data using a kinect sensor and visual art generated by MaxMSP program will be used between
sites symmetrically in order to interact with dancers via avatar. The delay time can be shorter
in this way as compared with video transmission and each site can serve as a main stage. Of
course, live videos will be transmitted between each site at the same time.
dQ12 Dancing across Oceans at Ciang May in the 34th APAN meeting

5. CZ
ES BR
KR
Chamber Music
4K Gateway
Dancer
Dancer
Dancer
4K Gateway
I2CAT Max6
Synapse
Max6
Synapse
Max6
Synapse
UFBA
UFBAI2CAT
network
audio
video
dQ13 Dancing beyond TIME
CZ-KR-ES-BR
eCulture WGFrom KRFrom BR From KRFrom ES
From ES From BR
From local
Local view From CZ
①
②
②
③ ③
④④
④ ④
⑤ ⑤

6. ES
BR
Dancer Dancer
network (HPDMnet)
video
Arthron’s diagram ~ KR-ES-BR
Arthron – ARTICULADOR / ENCODER encKR! ES & BR
encBR!KR encES!KR
decBR!KR decES!KR
encKR!ES & BR
decKR!BR decKR!ES
Arthron – ENCODER
encBR!KR
Arthron – DECODER
decKR!BR
Arthron – ENCODER
encES!KR
Arthron – DECODER
decKR!ES
KR
… …
…
UFBA i2CAT
Stage
Arthron DECODER
Arthron DECODER
decES!KR
decBR!KR

7. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !7
Process
• Three sites of KR, ES and BR consist of symmetrical configurations getting a source music from CZ.
Performers at each location except CZ can interact immediately with avatars from other site, although there is
a short delay in relation to Tx-Rx. The source music from CZ to KR is sent with the synchronized skeleton
data of the kinect to ES and BR simultaneously. By doing this, all sites can share just one music. Of course,
the exchange of skeleton data excluding audio between ES and BR happen together. By these symmetrical
structure, it can be possible to have audience and to manipulate the visualization at each location. (There is a
delay between the number ④ and ⑤ on the screen in KR.)
① A chamber music is played in CZ. The video and audio are transmitted to KR through 4k gateway
machines.
② In KR, a dance is performed to the music from CZ in front of a kinect sensor. Then the skeleton data of
dancer and direct sound of the chamber music from CZ are sent to ES and BR simultaneously.
③-④ In ES & BR, an avatar created by the skeleton data from KR is shown on the screen at each location.
Both dancers each interact with the avatar to the music from CZ via KR in front of a kinect sensor. Each
skeleton data of dancers in ES and BR is exchanged and sent to KR at the same time. All dancers can see
avatars each other almost instantly.
⑤ Each video excluding audio from ES and BR is sent to KR. Then all situations can be shown on one screen
in KR.
※ In some cases, an entire scene of KR stage can be transmitted to all other sites.

8. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !8
4K video details
Martinu Hall, HAMU, CZ
C500
4K camera
2x 3G-SDI
bayer pattern
4x HD-SDI
debayered 4K Gateway
2x balanced audio (XLR)
4K 4K
10GE
AJA Ki
Pro Quad
4K Gateway
10GE 4K
4x HD-SDI
4K
4x DVI
4x AJA HDP2
Sony SRX
2x unbalanced audio (cinch)

9. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !9
Backward channel 4K / HD
Martinu Hall, HAMU
4x HD-SDI
4K Gateway
HD 4K 10GE
downscaling
4K Gateway
10GE
4K
4x HD-SDI
4K
4x HD-SDI
AJA A/D & Embedder
JVC GY-HMQ10
2x balanced audio (XLR)
8K SAGE wall, IIM, CTU
4K
Depending on availability of the
second 4K camera, the backward
channel will be 4K or HD only

10. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !10
Equipments
Endpoint IP Hardware Software
CZ x1
• 4K Gateway TX x1
• 4K Gateway RX x1 (in KR)
• Video Camera x1
• Microphone(+ Audio Mixer) x1
n/a
KR x4
• Computer(Win7, MaxMSP, Synapse) x1
• Kinect x1
• Computer(Ubuntu, DeckLink card, Arthron) x3
• Speaker(L/R)(+Audio Mixer) x1
• Video Projector x4
Max6
Synapse
Arthron
ES x2
• Computer(Win7, MaxMSP, Synapse) x1
• Kinect x1
• Computer(Ubuntu, DeckLink card, Arthron) x2
• Video Camera x1
• Speaker(L/R)(+Audio Mixer) x1
• Video Projector x2
Max6
Synapse
Arthron
BR x2
• Computer(Win7, MaxMSP, Synapse) x1
• Kinect x1
• Computer(Ubuntu, DeckLink card, Arthron x2
• Video Camera x1
• Speaker(L/R)(+Audio Mixer) x1
• Video Projector x2
Max6
Synapse
Arthron
• One more IP and laptop for Skype communication may be needed.
• One more IP and computer(Win7, MaxMSP) for separated avatar-soundof skelecton data may be needed.
• In KR, one video camera and video mixer for transmission to ES, BR and CZ may be needed.

11. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !11
Software : Arthron
ARTHRON was developed to assist the execution of artistic performances that make use of multimedia
representations and real time sharing of real and virtual spaces. In this context, Arthron was used in the
domain of research and development in Art and Technology, as well as in artistic presentations like
Versus (2005), (In)TOQue (2008) and e-Pormundos Afeto (2009-1011) events. ARTHRON’s main
functionality is the simple and direct interface, designed for the manipulation of different sources and
streams of media (audio and video) simultaneously. This way the user can remotely add, remove and
configure the presentation format as well as schedule the streams of media in time and space during an
event.
ARTHRON is composed of six main components: Articulator, Encoder, Decoder, Reflector, VideoServer
and MapManager. The Articulator is responsible for the remote management of the other components,
concentrating a great part of the ARTHRON functionalities such as stream scheduling (manual or
automatic, via the creations of different scenarios), network monitoring and measurement, remote
configuration of other modules, access control, automatic generation of a web page for online publication
of low definition videos, addition of several video effects, chat for communication between the
components and much more. The Encoder is responsible for the capture and encoding (when necessary)
of the media source, which can be external (DV or HDV camera, DVD, etc) or internal (a local file). The
Decoder’s main functionality is to decode and display the media stream in a specific device (monitor,
projector, etc). The Reflector is responsible for the replication and redistribution of a media stream over
the network. The VideoServer is able to transcode the media streams that will be published online, being
able to transcode to the flv, ogg and h264 formats. The MapManager controls and displays the interactive
map of the ARTHRON components. All streams can be generated in geographically distributed locations.

12. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !12
Fig. A : Arthron’s Articulator main screen Fig. B : Elements of the e-Pormundos Afeto (2009)
event, on the stage of the Dragão do Mar Theater
In Figure B we can see a general overview of the objects in scene during the show. The dancer located inside the Dragão do
Mar Theater, in Fortaleza/Brazil, can be seen in (1). The central screen that receives the video stream of the dancer from
Barcelona/Spain can be seen in (3). On the left screen(2) is shown the video stream generated by the robot Galateia, from the
NatalNet laboratory. On the right screen (4) is the video sent to the internet and several people could watch online, inside the
GTMDA website. In (5) we can see the musician located in Fortaleza.

13. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !13
System module where the user can visualize the states (scenarios) and configure events
- Automatically generated web page of the transmission of a specific Decoder
“Versus” (2005) e_ePormundos Afeto (2009 - 2011)(In)TOQue (2008)
By Ivani Santana By Ivani Santana & Konic Thtr

14. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !14
◇ Hardware
• PC Intel Core 2 Duo CPU E8500 @ 3.16GHz or higher
• DeckLink card for video capture
• Videocamera with SDI/HDMI/Analog output configured to 1080/50i (preferably) and connected to the
DeckLink card
• NIC-1G
◇ Software
• Ubuntu 10.04.4 LTS 32 bits (http://releases.ubuntu.com/lucid/ubuntu-10.04.4-desktop-i386.iso); other
distributions may also work but it has been not tested.
• Decklink card driver
• i2CAT-VA binaries
• Dependencies: inside i2CAT-VA/dependencies/ there are two script files to get and install the needed
dependencies: aptget.sh and install.sh. Some specific version of the libraries will be installed, e.g. x264
0.125.x, libavcodec 53 61.100 or libavformat 53 32.100. Run sh aptget.sh && sh install.sh.
◇ Others
• Open ports from 5000 to 5020 in the PC
i2CAT-AV minimal hardware and software requirements

15. The venue of CP in KR (Lobby)
The position of a big screen
The position of projectors
The position of stage
Audience
Over 10m

16. 16m
6.5m
Stage
Audience
Big screen
The venue of CP in KR (Lobby)

17. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !17
16m
6.5m
9m
3mStage
4K
Control Desk
15,000 Ansi(Avatar)
ES KR BR
Screen
4,000 Ansi
Kinect
Lighting Lighting
DancerGround Floor
2nd Floor
10m

18. Kinect
Synapse
Max
Udpsend 10.0.1.87 8000
Udpsend 10.0.1.165 8000
Udpsend 127.0.0.1 8000
Udpreceive 8000
Udpreceive 8001
Udpreceive 8003
Kinect
Synapse
Max
Udpsend 10.0.1.87 8001
Udpsend 10.0.1.39 8001
Udpsend 127.0.0.1 8001
Udpreceive 8001
Udpreceive 8000
Udpreceive 8003
Kinect
Synapse
Max
Udpsend 10.0.1.165 8003
Udpsend 10.0.1.39 8003
Udpsend 127.0.0.1 8003 Udpreceive 8003
Udpreceive 8001
Udpreceive 8000
10.0.1.87
10.0.1.39 10.0.1.165
ES BR
KRCZ
4K Gateway
sound
Sound TX (from CZ)
(+ BGM sound)
Avatar data TX
CZ Sound RX
(+ BGM sound)
Avatar RXAvatar RX
Avatar RX
Sound RX
Avatar data TXAvatar data TX
CZ Sound RX
(+ BGM sound)
Jit.net.send @ip 10.0.1.39 @port 7003
Jit.net.send @ip 10.0.1.165 @port 7003
Jit.net.recv @port 7003Jit.net.recv @port 7003
MAX Networking

19. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !19
Max Audio Interface
(with 8 channels)
Audio Mixer
CH 1|2 ! SPAvatar Sound
CH 3|4 ! BR Avatar Sound
CH 5|6 ! KR Avatar Sound
CH 7|8 ! CZ & BGM Sound
Max Audio Mixing

20. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !20
Kinect
4m
①
②
③④
⑤
⑥
1m
FM-Synth Sound Noise-Tremolo Sound Sine-Vibrato Sound
Sensing 1 / 2
Note : nearer-slower, farther-faster
Sensing 3 / 4
Frequency : lower-lower, higher-higher
Sensing 5 / 6
Volume : nearer-louder, farther-quieter
Sensing 1 : Distance BTW left-hand & neck
Sensing 2 : Distance BTW right-hand & neck
Sensing 3 : max/min Height of left-hand
Sensing 4 : max/min Height of right-hand
Sensing 5 : Distance BTW left-hand & Kinect
Sensing 6 : Distance BTW right-hand & Kinect
Kinect & Max Sound Design
Sensing 1 / 2
Tremolo : nearer-slower, farther-faster
Sensing 3 / 4
Frequency : lower-lower, higher-higher
Sensing 5 / 6
Volume : nearer-louder, farther-quieter
Sensing 1 / 2
Vibrato : nearer-slower, farther-faster
Sensing 3 / 4
Frequency : lower-lower, higher-higher
Sensing 5 / 6
Volume : nearer-louder, farther-quieter

21. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !21
Artist
THE UNLIMITED TRIO is bielorussian-czech original experimental
instrumental musical project that has the main purpose not to limit
itself in area of styles or genres. The band consists of Nikita KREIN
(Acoustic guitar), Ilia CHERNOKLINOV (Viola) and Aleksander
YASINSKI (Accordion), a very unique instrumental mix so to speak.
- Obsession
- Another Side
- Darkness of Mine
Victoria Macarte
Konic Theatre
Haesoo Kim
Meta Dance Project
Isaura Tupiniquim
GP Poética Tecnológica UFBA

22. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !22
Dance
Music and Dance
0’ 1’ 2’ 3’ 4’ 5’ 6’ 7’ 8’ 9’ 10’ 11’ 12’ 13’
CZ(Live) F.O.
Unlimited: Obsession Unlimited: Another Side
F.O. F.I.
Unlimited: Darkness of Mine
Elec-sound
(Interactive)
F.I. F.O. F.I. F.O.
BGM(file play)
Generated by body movementGenerated by body movement
Glass & Bead Glass & Bead
BR
KR
ES
Plot
Lure
Mystery
Encounter
LongingIntro Dream Separation
Video
minute
Chamber Avatar Chamber
Avatar+Video
Avatar All
• Cyber World, now the tools of the digital age give us to easily communicate with
people on the other side of the world in new ways. New contacts create a new culture,
new arts create new paradigm. Here, there are gestures from three locations meeting
virtually and they are suddenly transferred to another space. Three points disperse and
flock dreamily. Harbor your futuristic dream in the wide world while taking a
cyberwalk with them.
Live Music: „The Unlimited“ trio (guitar, viola, accordion)
14’ 15’

23. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !23
APAN 36TH
VENUE
Connections

24. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !24
Southern Light
Sao Paulo, BR
NetherLight,
Amsterdam, NL
Starlight
Chicago, US
AMPATH/
Miami (FL), US
KRLight
Seattle PoP, US
KRLight
Daejeon PoP, KR
AmLight East
I2cat,
Barcelona, ES
KAIST Creative Learning BLD.
(APAN Venue)
Ivani Santana/
GP Technological
Poetics
KAIST
10G
10G
10G
Network for dQ2013 Stage 2
CWAVE/NLR
Bahia PoP/RNP
Salvador, BRIHAC/University of Bahia
(UBFA)
Kòniclab(Fabra i Coats)
10G
Academy of Performing Arts
in Prague (HAMU)
CESNET, CZ
ACE
KISTI, Daejeon
NREN IP Routed
Network
Lambda and Layer 2
Network
203.254.175.1
10G
10G
HPDMnet circuit
VLAN 2712
VLAN 464
VLAN Translation

26. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !26
Network/
Organization
End-sites IP address
Korea KAIST KAIST Creative Learning BLD 203.254.175.10 ~ 19
203.254.175.60 ~ 69
203.254.175.10 4K gateway
APAN 36th Venue
Prof, Goo’s Lab in KAIST GSCT
KEONET/KISTI - 203.254.175.13 (Daejeon)
KRLight/KISTI - 203.254.175.14
Brazil IHAC I v a n i S a n t a n a / G P
Technological Poetics
203.254.175.20-29
RNP
Southern Light
-
Spain i2cat Castelldefels Lab
Kòniclab Fabra i Coats 203.254.175.30-39
Czech
Republic
CESNET/HAMU - 203.254.175.11 4K gateway
US StarLight -
AMPATH
End-sites

27. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !27
Organization Country Address
KAIST
Korea Advanced Institute of
Science and Technology
Korea 291 Daehak-ro , Yuseong-gu, Daejeon 305-701, Korea
KISTI
Korea Institute of Science and
Technology Information
Korea 245 Daehak-ro, Yuseong-gu, Daejeon. 305-806, Korea
i2Cat
i2CAT Foundation - Audiovisual
Unit
Spain
C/ Gran Capità 2-4 Nexus 1 Building, 2ª planta, Office
203, 08034 Barcelona, Spain
Kòniclab Fabra i Coats c. de Sant Adrià,20 08030 Barcelona
IHAC
Institute of Humanities, Arts
and Science (IHAC),
University of Bahia
Brazil
Rua Barão de Jeremoabo, PAF IV - Ondina CEP
40170-115
Salvador, Bahia, Brazil
RNP
Rede Nacional de Ensino e
Pesquisa
Brazil
Rua Lauro Müller, 116
Room 1103
Botafogo
22290-906 Rio de Janeiro, RJ
CESNET
Czech Research and
Educational Network
Czech
Rep.
Zikova 4, Prague 6, 16000, Czech Rep.
HAMU
Academy of Performing Arts in
Prague
Czech
Rep.
Malostranské nám. 13, Praha 1, 11800, Czech Rep.
StarLight iCAIR/Northwestern University USA 750 N Lake Shore Dr. #600 Chicago IL 60611
Organizations

28. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !28
Participants : Network Engineers
<KR : KREONET/KRLight, KISTI>
Buseung Cho, bscho@kisti.re.kr
Minki Noh, mknoh@kisti.re.kr
<ES : i2cat>
Gerard Castillo Lasheras, gerard.castillo@i2cat.net
David Cuenca Gil, david.cuenca@i2cat.net
<CZ : CESNET, HAMU>
Jiri Navratil, jiri@cesnet.cz
Sven Ubik, ubik@cesnet.cz
Jaroslav Hrb, jaroslav.hrb@hamu.cz
<BR : RNP>
Michael Stanton, michael@rnp.br
Gustavo Dias, gustavo.dias@rnp.br
Leandro Ciuffo, leandro.ciuffo@rnp.br
Iara Machado, iara.machado@rnp.br
Alex Moura, alex@rnp.br
Fausto Vetter
Ítalo Valcy, italo@pop-ba.rnp.br
<US : StarLight (HPDMnet)>
Joe J Mambretti, j-mambretti@northwestern.edu
Jim Hao Chen, jim-chen@northwestern.edu
Participants | Organizers
Faridah, faridahn@gmail.com / Chair of APAN e-Culture WG, MY
Dae Young KIM, dykim@cnu.kr / Chair of APAN, CNU, KR
Artur Serra, artur.serra@i2cat.net / i2CAT, ES
Andrew Howard, andrew.howard@anu.edu.au / AU
Jeffrey Campbell, jeffrey.campbell@culture360.org / AU

29. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !29
Participants | Artists
<KR : KAIST>
Boncheol Goo, mgtech@kaist.ac.kr
Sung-il PARK, hakseng@kaist.ac.kr
Haesoo Kim, Dancer
<ES: Kòniclab. Fabra i Coats>
Rosa Sánchez, konic@koniclab.info
Alain Baumann, alain@koniclab.info
Victoria Macarte, Dancer
<BR : Institute of Humanities, Arts and Science (IHAC), University of Bahia>
Ivani Santana, Choreographer, ivanisantana.mapad2@gmail.com
Isaura Tupiniquim, Dancer, isauratupiniquim@gmail.com
Caetano Travassos, Videoartist, caetanobr@gmail.com
Jackson Espírito Santo e Pedro Lacerda, technology
Superintendência de Tecnologia da Informação – STI
<CZ : Academy of Performing Arts in Prague (HAMU), CEPOSE>
Ondřej Urban, ondrej.urban@hamu.cz
Martin Pinkas, martin.pinkas@hamu.cz
Aleš Dvořák, ales.dvorak@hamu.cz
Michal Růžička, michal.ruzicka@cepose.cz
The Unlimited trio :
Nikita Krein nikitakrein@gmail.com
Ilya Chernoklinov
Alexander Yasinsky

30. dQ13 Dancing beyond TIME Aug. 21st 2013 @ KAIST, Republic of Korea, e-Culture WG, 36th APAN meeting !30
Photos

31. Thank You