1.
A Dance Performance Environment
in which Performers Dance with
Multiple Robotic Balls
Shuhei Tsuchida†1, Tsutomu Terada†1†2, Masahiko Tsukamoto†1
†1 Kobe University
†2 JST PRESTO

2.
Demo Movie (ACE2015 in Malaysia) 2

3.
Tod Machover: Death and the Powers
https://www.youtube.com/watch?v=mPp9juefl2Y
elevenplay:Dance with Drones
https://www.youtube.com/watch?v=HQLORg5COiU
In recent years, as robotics technology progresses,
various robots have been developed to dance with humans.
Background 3

4.
Tod Machover: Death and the Powers
https://www.youtube.com/watch?v=mPp9juefl2Y
elevenplay:Dance with Drones
https://www.youtube.com/watch?v=HQLORg5COiU
In recent years, as robotics technology progresses,
various robots have been developed to dance with humans.
Background 3

5.
Tod Machover: Death and the Powers
https://www.youtube.com/watch?v=mPp9juefl2Y
elevenplay:Dance with Drones
https://www.youtube.com/watch?v=HQLORg5COiU
In recent years, as robotics technology progresses,
various robots have been developed to dance with humans.
Background 3

6.
Tod Machover: Death and the Powers
https://www.youtube.com/watch?v=mPp9juefl2Y
elevenplay:Dance with Drones
https://www.youtube.com/watch?v=HQLORg5COiU
In recent years, as robotics technology progresses,
various robots have been developed to dance with humans.
Background 3
However, it is difficult for the performer
to create the performance using the mobile robots.

7.
①Image
Engineer
Performer
System
③Creating
Result
Problem 7
②Input
＆
Output

8.
①Image
Engineer
Performer
System
③Creating
Result
Problem 7
Image
②Input
＆
Output

9.
①Image
Engineer
Performer
System
③Creating
Result
②Input
＆
Output
Problem 7
Image Creating Result

10.
①Image
Engineer
Performer
System
③Creating
Result
②Input
＆
Output
Problem 7
Image Creating Result
The performer cannot interactively
create a performance using the system.

11.
We construct a mechanism by which a performer
can interactively create a performance
while he/she considers the correspondence
between his/her motion and
the mobile robots' movement and light.
Purpose 8

12.
General Method
Proposed System (1/2) 9
①Image
Engineer
Performer
System
③Creating
Result
②Input
＆
Output
Image Creating Result

13.
Proposed System (1/2) 9
General Method
①Image
Engineer
Performer
System
③Creating
Result
②Input
＆
Output
Image Creating Result

14.
Proposed Method
Proposed System (2/2) 10
①Input
＆
Output
System
Performer
Image Creating Result

15.
・A performer with no knowledge of programming
can create patterns of the movement and
light of robotic balls to certain music and choreography
by mouse operation.
・A performer can easily confirm
the choreography of dancing with
robotic balls on a simulator.
Simulator
Motion Input
System Design 11

16.
Size︓ diameter 74 [mm]
Weight︓ 168 [g]
Maximum speed︓ 2 [m/s]
Communication distance︓ max 30 [m]
Hardware (1/2) 12

17.
Hardware (2/2) 13
LiPo battery
Driving part
of Sphero
Infrared LED

18.
ABS rein Plastic
Hardware (2/2) 13
LiPo battery
Driving part
of Sphero
Infrared LED

19.
1. Creating movements and lights 2. Confirming on the simulator
3. Saving the pattern 4. Confirming with robotic balls
PC
Mobile phone
Moving pattern
Light pattern
System Configuration 14

20.
1. Creating movements and lights 2. Confirming on the simulator
3. Saving the pattern 4. Confirming with robotic balls
PC
Mobile phone
Moving pattern
Light pattern
System Configuration 14
Application for Creating
the Movement and the Light of Robotic Balls
Control System for Robotic Balls

21.
1. Creating movements and lights 2. Confirming on the simulator
3. Saving the pattern 4. Confirming with robotic balls
PC
Mobile phone
Moving pattern
Light pattern
System Configuration 14
Application for Creating
the Movement and the Light of Robotic Balls
Control System for Robotic Balls

22.
Application for Creating
the Movement and the Light of Robotic Balls 15

23.
1. Creating movements and lights 2. Confirming on the simulator
3. Saving the pattern 4. Confirming with robotic balls
PC
Mobile phone
Moving pattern
Light pattern
System Configuration 14
Application for Creating
the Movement and the Light of Robotic Balls
Control System for Robotic Balls

24.
The user saves the pattern of the moving and light of
robotic balls to the mobile phone and PC for control.
moving
pattern
light
pattern
Control System for Robotic Balls (1/2) 16

25.
moving
pattern
light
pattern
infrared
Control System for Robotic Balls (1/2) 17
The user performs fitting the moving and the light of robotic
balls that synchronize a music.

26.
・The system cannot detect the position when the dancer was
between the camera and the robotic ball.
・The speed of robotic balls is limited.
・It cannot move to draw a curve.
System Restriction 18

27.
Survey items
・Performance creation time
・System usability
Participant
Males 4，Average age 22.3
・Performer who had been dancing for more than four years.
(4/4)
・Performer who had some experience in performance creation
using robotic balls. (1/4)
19
Evaluation Experiment (1/3)

28.
Evaluation Experiment (2/3) 20
Experiment Procedure
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create
choreography based on a song about 1 minutes,
10 seconds long at 120 BPM.
(3) The participant confirms the choreography
on the application.
(4) The participant carries out a performance
using robotic balls.

29.
Experiment Procedure
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create
choreography based on a song about 1 minutes,
10 seconds long at 120 BPM.
(3) The participant confirms the choreography
on the application.
(4) The participant carries out a performance
using robotic balls.
Evaluation Experiment (2/3) 20
20-minute explanation
①

30.
Evaluation Experiment (2/3) 20
Experiment Procedure
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create
choreography based on a song about 1 minutes,
10 seconds long at 120 BPM.
(3) The participant confirms the choreography
on the application.
(4) The participant carries out a performance
using robotic balls.
Creating a choreography
②
01m10s

31.
Experiment Procedure
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create
choreography based on a song about 1 minutes,
10 seconds long at 120 BPM.
(3) The participant confirms the choreography
on the application.
(4) The participant carries out a performance
using robotic balls.
Evaluation Experiment (2/3) 20
Confirming a choreography
on the simulator
③

32.
Experiment Procedure
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create
choreography based on a song about 1 minutes,
10 seconds long at 120 BPM.
(3) The participant confirms the choreography
on the application.
(4) The participant carries out a performance
using robotic balls.
Evaluation Experiment (2/3) 20
Kinect
Projetor
Images

33.
Evaluation Experiment (2/3) 20
Experiment Procedure
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create
choreography based on a song about 1 minutes,
10 seconds long at 120 BPM.
(3) The participant confirms the choreography
on the application.
(4) The participant carries out a performance
using robotic balls.
Performance
④

34.
Experiment Procedure
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create
choreography based on a song about 1 minute,
10 seconds long at 120 BPM.
(3) The participant confirms the choreography
on the application.
(4) The participant carries out a performance
using robotic balls.
Evaluation Experiment (2/3) 20
Recording Time
for creating

35.
・One B5 paper and a pen were also used
when creating the choreography.
・The experimenter received the question about
how to operate the application during the experiment.
・The time taken for the experimental procedures 2 and 3 was
noted and a 10-minute break was scheduled every hour.
At the end of the experiment
・Participants provided feedback on the application.
Evaluation Experiment (3/3) 21

36.
Results (1/3) 22

37.
Results of questionnaire
・The application was that its movements were
immediately clear at a glance.
・It was hard to determine the sense of distance
with the robotic balls while confirming the choreography
via simulator display.
・The setting to take color gradient between keyframes
made it hard to operate when participant was
deciding on the lighting
Results (2/3) 23

38.
Participant Performance creation time
1 2hour, 38minutes
2 3hour, 29minutes
3 2hour, 17minutes
4※ 1hour, 4minutes
※Participant 4 has
the experience using
the system
Time taken by participants for performance creation.
Results (3/3) 24

39.
Participant Performance creation time
1 2hour, 38minutes
2 3hour, 29minutes
3 2hour, 17minutes
4※ 1hour, 4minutes
※Participant 4 has
the experience using
the system
Time taken by participants for performance creation.
Results (3/3) 25
It is then easier to imagine the expression one wants to convey,
and the ease of coming up with ways to achieve the expression
contributed greatly to reducing the time.
Performer who has rich experiences
using the system created the performance

40.
Performer System
Consideration 26
Image

41.
Consideration 26
Performer System
Image
Libraries

42.
Automatic
Generation
Consideration 26
Performer System
Image
Libraries

43.
Automatic
Generation
Consideration 26
Various expression libraries will not only reduce the creation time
but also widen the range of expression the performer creates.
Performer System
Image
Libraries

44.
We investigated how participants took
both their own body expression and that of the robot
into consideration to create the performance.
→Those who frequently used the movement should be
in the library program.
Libraries
Enclose
Link a dancer
Emphasis
Gather
Into the space
….
Preview 1
Preview 2
Preview 3
Preview 4
…
System Image
Evaluation Experiment 27

45.
Participant 1 Participant 2 Participant 3 Participant 4
Evaluation Experiment 28
Memo of the participants

46.
Participant 1 Participant 2 Participant 3 Participant 4
Evaluation experiment 28
Memo of the participants

47.
Participant 1 Participant 2 Participant 3 Participant 4
Evaluation experiment 28
The participants commented on how they considered
the movement and the light of the robot each eight beats.
Memo of the participants

48.
Ways to show what the performer wants to express: participant 1.
Results (1/2) 29

49.
Ways to show what the performer wants to express: participant 1.
Results (1/2) 29
We extract and list the ways to show what the performer wants
to express on the basis of responses to how the expression
was shown.

50.
Results (2/2) 30
・The extracted expression
Enter a space(4 persons)，
Link a dancer(4) ，Emphasize(4) ，
Gather(3) ，Make a space(3) ，
Enclose，Spread，
Rotate the entire formation，
Move a border．
・The extacted movement
Slide，Line up in sequence，
Fix a point，One by one action.

51.
Results (2/2) 30
・The extracted expression
Enter a space(4 persons)，
Link a dancer(4) ，Emphasize(4) ，
Gather(3) ，Make a space(3) ，
Enclose，Spread，
Rotate the entire formation，
Move a border．
・The extacted movement
Slide，Line up in sequence，
Fix a point，One by one action.

52.
・The extracted expression
Enter a space(4 persons)，
Link a dancer(4) ，Emphasize(4) ，
Gather(3) ，Make a space(3) ，
Enclose，Spread，
Rotate the entire formation，
Move a border．
・The extacted movement
Slide，Line up in sequence，
Fix a point，One by one action.
Results (2/2) 30

53.
Results (2/2) 30
Our challenge for the future is to create a list of what we wish to
express and turn the corresponding robot movement and lighting
into a library that can be implemented in such a way that the
movements can be previewed on the application.
・The extracted expression
Enter a space(4 persons)，
Link a dancer(4) ，Emphasize(4) ，
Gather(3) ，Make a space(3) ，
Enclose，Spread，
Rotate the entire formation，
Move a border．
・The extacted movement
Slide，Line up in sequence，
Fix a point，One by one action.

54.
・We constructed a system by which a performer can interactively
create patterns of the movement and light of robotic balls.
・To evaluate the effectiveness of the system,
we had four performers use the system to create and
demonstrate performances.
・We proposed the method to speed up the creation of performance
and to widen the range of expression the performer creates.
・We confirmed that the system performed reliably
in real environment.
Conclusion 31

55.
Future Work 31

56.
Appendix 57

57.
58