A System for Practicing
Formations in Dance
Performance Supported by
Self-Propelled Screen
Shuhei Tsuchida
Tsutomu Terada
Masahiko Tsukamoto
Kobe University, Japan
1

Background
A Formation in a group dance is important.
2

3

Background
To perform well
・Keep an appropriate distance
・Move at the same time
4

5
Related Works
• Support the learning of choreography
Real-time mocap dance recognition
for an interactive dancing game
(L．Deng, 2011)
• Support the training using a robot
Partner Ball-room Dance Robot-PBDR-
（K. Kosuge, 2008）
We can not confirm the location of the formation

6
Research Purpose
Ｃ
Ｃ
Ｃ
Dancers usually practice using a mirror.
Reason“check the formation and choreography”
This is fine when there are more than one person.
if someone cannot participate in the practice, it is difficult for
the rest of the members to gain a sense of the proper formation.
We propose a practice support system for performing
the formation smoothly even if there is no dance partner.
But...
Therefore...

7
Methods of Practicing Formations
Normal
Dancers look at a mirror while practicing.
Three possible approaches
u Dancing by watching reference videos that was previously created
u Learning the distance and the absence of dancers by projecting them
onto a wall
u Feeling as if dancers are present using a self-propelled robot
that can move quickly like a person

u Dancing by watching reference videos that was previously created
u Learning the distance and the absence of dancers by projecting them
onto a wall
u Feeling as if dancers are present using a self-propelled robot
that can move quickly like a person
8
Methods of Practicing Formations
Normal
Dancers look at a mirror while practicing.
Three possible approaches

9
Methods of Practicing Formations
Normal
Dancers look at a mirror while practicing.
u Dancing by watching reference videos that was previously created
This can reduce the gap of the moving distance of a dancer
u Learning the distance and the absence of dancers by projecting them
onto a wall
A dancer will have an illusion that the other dancer is actually there
u Feeling as if dancers are present using a self-propelled robot
that can move quickly like a person
By moving the robot just like the absent dancer,
it can provide a presence of him or her.
Three possible approaches

Hypothesis
u When using a self-propelled robot
The dancer will feel difficult to
dance because he or she will be
worried about bumping into
the robot.
But, the dancer’s path will be closed
to the path when dancing in a group,
because the dancer will think of the
robot as a marker.
u When using a projected video
The dancer will be easy to dance
because there is no risk of
collision between the robot and
the dancer.
But, the path that the dancers
moves will be different from
when dancing in a group
because there is a gap between
the video and reality.

Evaluate questionnaire
Evaluate trajectories of movements
Hypothesis
u When using a self-propelled robot
The dancer will feel difficult to
dance because he or she will be
worried about bumping into
the robot.
But, the dancer’s path will be closed
to the path when dancing in a group,
because the dancer will think of the
robot as a marker.
u When using a projected video
The dancer will be easy to dance
because there is no risk of
collision between the robot and
the dancer.
But, the path that the dancers
moves will be different from
when dancing in a group
because there is a gap between
the video and reality.

12
Preliminary Experiment
for practicing formation(1/5)
u Subjects dance with a dancer
u We investigate the following
three methods for practicing
formations and compare them
to find out which method felt
the closet to dancing with an
actual dancer.
Dancing with an actual dancer

13
Experiment1:
Dancing alone
Experiment2:
Dancing with
a self-propelled robot
Experiment3:
Dancing with
a projected video
Preliminary Experiment
for practicing formation(2/5)

14
Experiment1:
Dancing alone
Experiment2:
Dancing with
a self-propelled robot
Experiment3:
Dancing with
a projected video
Preliminary Experiment
for practicing formation(2/5)

Preliminary Experiment
for practicing formation(2/5)
15
Experiment1:
Dancing alone
Experiment2:
Dancing with
a self-propelled robot
Experiment3:
Dancing with
a projected video

16
Experiment1:
Dancing alone
Experiment2:
Dancing with
a self-propelled robot
Experiment3:
Dancing with
a projected video
Preliminary Experiment
for practicing formation(2/5)

17
Experiment1:
Dancing alone
Experiment2:
Dancing with
a self-propelled robot
Experiment3:
Dancing with
a projected video
Preliminary Experiment
for practicing formation(2/5)

18
Dancing alone a self-propelled robot a projected video
Preliminary Experiment
for practicing formation(3/5)
Dancing with an actual dancer

19
u The subjects were nine dancers who had experience in
dancing for more than three years.
The subjects learned the choreography for approximately
12 seconds that consisted of three times eight beats.
u This choreography contained three elements that were
considered to be greatly influenced by the presence of the
others.
1.Intersection
2.Approaching
3.Parallel translation
Preliminary Experiment
for practicing formation(4/5)

20
u We saved the location information obtained from
a depth camera.
u All subjects danced the choreography three times each for
experiment 1,2,3 and dancing with a dancer.
u All subjects evaluated five question items in five stages.
Preliminary Experiment
for practicing formation(4/5)

21
Results and Consideration(1/5)
uResults from a questionnaire
Question 1 Question 2 Question 3 Question 4 Question 5
Ave. Var. Ave. Var. Ave. Var. Ave. Var. Ave. Var.
Alone 2.0 1.6 1.7 0.44 2.1 1.2 3.8 1.7 1.4 0.25
self-propelled
robot
3.0 1.1 2.7 2.2 2.6 1.8 2.0 1.6 2.6 1.1
Projected
video
2.0 0.67 3.2 0.84 3.4 1.1 4.1 0.77 2.9 0.32
Low evaluation
High evaluation
There was a significant difference

22
Results and Consideration(1/5)
uResults from a questionnaire
Question 1 Question 2 Question 3 Question 4 Question 5
Ave. Var. Ave. Var. Ave. Var. Ave. Var. Ave. Var.
Alone 2.0 1.6 1.7 0.44 2.1 1.2 3.8 1.7 1.4 0.25
self-propelled
robot
3.0 1.1 2.7 2.2 2.6 1.8 2.0 1.6 2.6 1.1
Projected
video
2.0 0.67 3.2 0.84 3.4 1.1 4.1 0.77 2.9 0.32
Low evaluation
High evaluation
Dancing with the projected video was
the closet to dancing with a dancer.

23
uThe average of the distance between method of
dancing with a dancer in each method
Dancing with a self-propelled robot was the closet to
the trajectory information of dancing with a dancer.
The distance is long
The distance is short
The subjects number 1 2 3 4 5 6 7 8 9
Alone 207 185 353 415 363 320 570 337 281
A self-propelled
robot
202 203 469 360 278 196 481 257 317
A projected video 254 218 532 552 306 256 280 237 469
Results and Consideration(2/5)

24
Dancing with a projected video
・The sense is close to the sense
dancing with an actual dancer
・The movement is not close to
the movement in a group
Dancing with a self-propelled robot
・The sense is not close to
the sense dancing with
an actual dancer
・The movement is close to
the movement in a group
・A dancer is easy to dance
・The movement is not close
to the movement in a group
Hypothesis
Hypothesis
・A dancer is difficult to dance
・The movement is close to
the movement in a group
Results
Results
Results and Consideration(3/5)

25
Dancing with a projected video
・The sense is close to the sense
dancing with an actual dancer
・The movement is not close to
the movement in a group
Dancing with a self-propelled robot
・The sense is not close to
the sense dancing with
an actual dancer
・The movement is close to
the movement in a group
・A dancer is easy to dance
・The movement is not close
to the movement in a group
Hypothesis
Hypothesis
・A dancer is difficult to
nce
・The movement is close to
the movement in a group
Results
Results
Results and Consideration(3/5)

26
Dancing with a projected video
・The sense is close to the
sense
dancing with an actual dancer
・The movement is not close
to
the movement in a group
Dancing with a self-propelled robot
・The sense is not close to
the sense dancing with
an actual dancer
・The movement is close to
the movement in a group
・A dancer is easy to dance
・The movement is not close
to the movement in a group
Hypothesis
Hypothesis
・A dancer is difficult to
nce
・The movement is close to
the movement in a group
Results
Results
Results and Consideration(3/5)

27
Demerit: The trajectory information was not close to
the trajectory information dancing with an actual dancer
We combine a self-propelled robot which
trajectory information is right and
a projected video which the sense is right
Dancing with a projected video
Merit: The sense was close to the sense dancing with an actual dancer
Dancing with a self-propelled robot
Merit: The trajectory information was close to the trajectory
information dancing with an actual dancer
Demerit: The sense was not close to the sense dancing with an actual dancer
Results and Consideration(3/5)

Dancing with
A projected video＋a self-propelled robot fixed a screen
28
Methods combined a self-propelled robot which trajectory
information is right and a projected video which the sense is right
The proposed method
Results and Consideration(5/5)

Automation of
a self-propelled screen
We controlled OMNIKIT2010 by hand with a wireless-controller.…

Automation of
a self-propelled screen
We controlled OMNIKIT2010 by hand with a wireless-controller.…
I aim to make the robot moves
automatically and more like an actual dancer.

31
System Configuration
Screen
Main PC
Projector
Self-propelled Robot
Small PC
Socket
Transmission
Speaker
Depth Data
Image Signal,
Signal
Depth Camera
Sixaxis motion sensor
Bluetooth Transmission

32
System Appearance

33
System Appearance

Experiment(1/5)
u Dancing with a self-propelled robot
u Dancing with a projected video
Dancing with
A projected video＋a self-propelled robot fixed a screen
Proposed methods is obtained the preliminary experiment
Compare
35
Can we use the merit of both dancing with a robot and with a video?

36
1.Intersection
2.Approaching
3.Parallel translation
u In this Experiment…
The complex and long choreography
Experiment(2/5)

Simple choreography
Simple choreography
37
Intersection
Approaching
Parallel translation
Simple choreography
Experiment(3/5)

Simple choreography
Simple choreography
38
Intersection
Approaching
Parallel
translation
Simple choreography Factors to perform well
u Keep an appropriate distance
（Front, Back）
u Move at same time
u Keep an appropriate distance
(Side）
Experiment(4/5)

39
u Subjects are nine dancers have danced for more than
three years.
u All subjects danced the choreography three times
each methods.
u We saved the location information obtained from
a depth camera.
u All subjects evaluated the following two question
items in five stages.
Experiment(5/5)

Results and Consideration(1/6)
Dancing with
an actual dancer
Dancing with
a self-propelled robot
Dancing with
an projected video
Dancing with a self-propelled
robot fixed a screen
+ a projected video
Approaching
40
-800 -700 -600 -500 -400 -300 -200 -100 0
1
2
3
4

Results and Consideration(1/6)
Dancing with
an actual dancer
Dancing with
a self-propelled robot
Dancing with
an projected video
Dancing with a self-propelled
robot fixed a screen
+ a projected video
Approaching
41
-800 -700 -600 -500 -400 -300 -200 -100 0
1
2
3
4

Results and Consideration(1/6)
Dancing with
an actual dancer
Dancing with
a self-propelled robot
Dancing with
an projected video
Dancing with a self-propelled
robot fixed a screen
+ a projected video
Approaching
42
-800 -700 -600 -500 -400 -300 -200 -100 0
1
2
3
4
It can not be said that we obtained the merit of robot

uResults of the questionnaire of approaching
＊Question 1： Was it close to the sense of dancing with a dancer?
＊Question 2：Was it easy to learn the sense of distance?
43
Robot Video
Robot
+Video
+Screen
Average
Question 1 2.0 2.9 2.9
Question 2 2.2 2.8 2.7
Variance
Question 1 1.0 1.5 2.5
Question 2 1.8 1.4 1.8
Results and Consideration(2/6)

uResults of the questionnaire of approaching
＊Question 1： Was it close to the sense of dancing with a dancer?
＊Question 2：Was it easy to learn the sense of distance?
44
Robot Video
Robot
+Video
+Screen
Average
Question 1 2.0 2.9 2.9
Question 2 2.2 2.8 2.7
Variance
Question 1 1.0 1.5 2.5
Question 2 1.8 1.4 1.8
Results and Consideration(2/6)

uResults of the questionnaire of approaching
＊Question 1： Was it close to the sense of dancing with a dancer?
＊Question 2：Was it easy to learn the sense of distance?
45
Robot Video
Robot
+Video
+Screen
Average
Question 1 2.0 2.9 2.9
Question 2 2.2 2.8 2.7
Variance
Question 1 1.0 1.5 2.5
Question 2 1.8 1.4 1.8
Results and Consideration(2/6)
It can be said that we obtained the merit of a video.

46
Results and Consideration(3/6)
The self-propelled screen was able to obtain
the advantages of the projected video.
However, it was not able to obtain
the advantages of the self-propelled robot.
Conclusion

Possible causes
uThe self-propelled screen gave subjects
a greater sense of presence than necessary.
The self-propelled screen made it easier to do irregular motions
by shaking the screen.
uTheir fear of collisions was stronger than
that with the other methods.
There was a possibility of injury because the outer frame
of the screen was made of aluminum.
uReliability with the self-propelled screen was low.
Because subjects did not practice with it repeatedly.
47
Results and Consideration(4/6)

Possible improvement methods
uThe self-propelled screen should move more accurately.
uWe need to reduce the risk of the collision by attaching
a buffer material to the screen to prevent injuries.
uPracticing with the self-propelled screen repeatedly
48
Results and Consideration(5/6)

We have not achieved the purpose
uIt is difficult for
both a dancer which can not move at the same time
and a dancer which can not keep appropriate distance
to improve their sense by this system .
49
Results and Consideration(6/6)

50
uThe self–propelled screen should move more accurately
uWe investigate the effectiveness of the self-propelled screen
uWe develop a system for practicing more complex
formations using more than two self-propelled screen
uWe evaluated which method was close to dancing with
an actual dancer for each method of practicing formations
uWe developed the self-propelled screen based on
the results of the experiment
Conclusion
Future Work
uWe investigated whether or not the self-propelled screen
obtained the advantages of two methods.