Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Interactive Dance Choreography Assistance presentation for ACE entertainment 2017

Interactive Dance Choreography Assistance Victor de Boer Josien Jansen Ana-Liza Tjon-A-Pauw Frank Nack

  • Be the first to comment

Interactive Dance Choreography Assistance presentation for ACE entertainment 2017

  1. 1. Just Dance 4
  2. 2. Ok, once more
  3. 3. Variation from (Victoria Zhou) 1. Step forward towards corner 2, into croisé derrière à terre, arms demi- seconde 2. Plié in 5th change direction to corner 1 3. Relevé derrière, arms in 4th en avant palms down 4. Passé and change direction to corner 2 place foot on pointe in 5th, arms move to a low kissing gesture 5. Retiré and back to 5th on pointe with the front foot, back foot and front foot arms move to demi-seconde 6. Repeat to the right, and repeat all. 7. Posé coupé effacé towards corner 1, arms in 4th en avant 8. Posé coupé arms in 4th palms down 9. Step and posé retiré croisé, place back foot in 5th on pointe, port de bras with the left arm from 5th en haut to 5th en avant, end with the arms forward in a low line, cross the wrists.
  4. 4. One more time
  5. 5. What representation for dance? for humans and machines Sci-FiDancingGIFbyColinRaff
  6. 6. While for music we have good (machine readable) representations, we lack these for dance.
  7. 7. Why do we need knowledge representation for dance? Three reasons
  8. 8. Archival and retrieval
  9. 9. Analysis: Digital humanities
  10. 10. Supporting creativity
  11. 11. Towards a choreography assistant tool Sensing Representation + Reasoning Presentation generation • Motion detection • Floor sensors • Move recognition • Dance movement representation • Dance choreography representation • Use of background knowledge • Pattern detection • Choreography generation • Visual presentation • 3-D animation • Auditory presentation Sensing data Choreography variation Presentation Choreography
  12. 12. Existing representations and tools
  13. 13. Labanotation
  14. 14. LabanXML and Laban Editor LED Labanotation editor http://donhe.topcities.com/pubs/led.htmlNakamura & Hachimura (2006)
  15. 15. Benesh
  16. 16. Dance Forms
  17. 17. Cecchetti system 7 elementary movements: [plie (bend), etandre (stretch), releve (rise), sauter (jump), glisse (glide), tourne (turn), elancer (dart)] Positions: 1st, 2nd, 3rd,.. (left right croise) Facing position (1…8) Position in space Direction of movement (de cote, dessous, dessus, en avant, en arriere, devant, derriere) Combinations (100+) pas-de-chat, pas-de- bourre, piroutte Ballet languages/systems Based on interview with Marije Koning
  18. 18. XML Dance Grammar Balakrishnan Ramadoss and Kannan Rajkumar. Modeling the Dance Video Semantics using Regular Tree Automata Fundamenta Informaticae 86 (2008) 175–189 175 IOS Press
  19. 19. Interactive Dance Choreography Assistance Victor de Boer Josien Jansen Ana-Liza Tjon-A-Pauw Frank Nack
  20. 20. Sensing Representation + Reasoning Presentation generation • Motion detection • Floor sensors • Move recognition • Dance movement representation • Dance choreography representation • Use of background knowledge • Pattern detection • Choreography generation • Visual presentation • 3-D animation • Auditory presentation Sensing data Choreography variation Presentation Choreography
  21. 21. To what extent can choreographers be supported by semi-automatic dance analysis and the generation of new creative elements in choreographies?
  22. 22. Method Questionnaire: How do choreographers work (with technologies) Tool: Proof of concept digital choreo assistant Evaluation: Test application to and different strategies
  23. 23. Questionnaire 54 Dutch choreographers Online questionnaire
  24. 24. Personal choreography archiving 0 2 4 6 8 10 12 14 16 18 written dance notation digital dance notation videotaping other memory, without problems memory, forget things
  25. 25. Preferred Notations 0 5 10 15 20 25 30 35
  26. 26. Notations Laban & Benesh 0 5 10 15 20 25 30 Never heard of it Cannot work with it Other Know Laban Can write both
  27. 27. Interest in support in the creative process Originality, Creativity and Emotion are most important aspects One very negative sub-group > Afraid to lose humanity One positive towards creative assistance Two sub-groups:
  28. 28. Tool Requirements based on MoSCoW method • A dancer must be able to add their dance style to the tool • A dancer must be able to add their existing choreography to the tool • The tool must be able to give new suggestions for variations of the choreography • The suggestions must be based on different strategies • The dancer must be able to see the whole choreography at any moment in time (written) • The communication of the tool are written dance terms • The tool must be “easy to use”, which means getting suggestions may take no longer than 2 minutes • The tool does have simplified body movements (legs, feet, arms, hands and head)
  29. 29. Proof-of-concept mobile app 3 different dance styles Ballet (including 78 steps) Modern dance (including 57 steps) Street dance (including 31 steps) Dancepiration – a tool for choreography assistance
  30. 30. 4 rule-based strategies for creating variations on existing choreographies 1. Random step replaced by random other step 2. Random step replaced by ontology-based other step 3. Random steps replaced by multiple strategies 4. Specific step replaced by ontology-based steps
  31. 31. Ontology-based variation for the 3 dance styles . El Raheb, et al. BalOnSe: Ballet Ontology for Annotating and Searching Video performances. In Proceedings of the 3rd International Symposium on Movement and Computing (p. 5). ACM, 2016
  32. 32. Evaluation Evaluation
  33. 33. 6 choreography students Random-based versus Ontology-based Each dance style is tested 3 times with both strategies per person Rate original choreography and each variation (10pt scale) Rate on 5pt Likert scale: Correctness, Creativity, Helpfulness, Meaningfulness
  34. 34. Results Respondents are positive about the tool …prefer to choose a specific step to change themselves … consider creativity in this tool very high (avg 4.2/5) Correctness is important to improve, it influences other factors the most
  35. 35. Ontology-based variant outperforms random variations Score OriginalRandomOntology-BasedDifference Sig Average grade 6.17 5.50 6.35 +0.85 ** Correctness 2.89 3.37 +0.48 * Creativity 3.19 3.37 +0.18 Helpfulness 2.59 3.00 +0.41 Meaningfulness 2.70 2.96 +0.26
  36. 36. Style matters *=statisticallysignificantatα=0.10(t-test/anova) **=statisticallysignificantatα=0.05(t-test/anova) Element Style Random Ontology-Based Difference Correctness Ballet 2.89 2.56 -0.33 Streetdance 2.78 3.56 +0.78 * Modern 3.00 4.00 +1.00 ** Creativity Ballet 3.44 3.56 +0.12 Streetdance 2.78 3.11 +0.33 Modern 3.11 3.44 +0.33 Helpfulness Ballet 2.67 2.67 0.00 Streetdance 2.44 2.89 +0.45 Modern 2.89 3.44 +0.55 Meaningfulness Ballet 2.89 2.78 -0.11 Streetdance 2.33 2.67 +0.34 Modern 2.89 3.44 +0.55
  37. 37. Sensing Representation + Reasoning Presentation generation • Motion detection • Floor sensors • Move recognition • Dance movement representation • Dance choreography representation • Use of background knowledge • Pattern detection • Choreography generation • Visual presentation • 3-D animation • Auditory presentation Sensing data Choreography variation Presentation Choreography
  38. 38. Which presentation methods are considered most effective for an interactive dance choreography assistant tool?
  39. 39. Experiment: Comparing 4 choreography presentation methods
  40. 40. 1: Textual descriptions
  41. 41. 2: 2D animations https://www.stykz.net/
  42. 42. 3: 3D animations DanceForms 2 (http://charactermotion.com/products/danceforms/
  43. 43. 4: auditory instructions
  44. 44. Experiment• 7 choreographers • 2 new(!) styles – Hip-hop and dancehall • Simple choreography + pre-generated variations • Large projection screen in practice space • 4 presentation variations (random)
  45. 45. Overall assessment Dancehall vs Hip-hop 0 1 2 3 4 5 6 7 8 9 10 Textual 2D animations 3D animations Auditory Score 0 1 2 3 4 5 6 7 8 9 10 Textual 2D animations 3D animations Auditory Score
  46. 46. 0 1 2 3 4 5 6 7 8 9 10 Overall assessment Stimulation of creativity Understandability (Un-)disruptiveness Textual 2D an. 3D an. Auditory 3D animations are the best
  47. 47. A significant sub-group of choreographers is interested in and enthusiastic about automatic choreography support Needs to be able to understand ‘dance language’ Knowledge representation matters Style matters Presentation styles matter -> 3D + dance language
  48. 48. Sensing Representation + Reasoning Presentation generation • Motion detection • Floor sensors • Move recognition • Dance movement representation • Dance choreography representation • Use of background knowledge • Pattern detection • Choreography generation • Visual presentation • 3-D animation • Auditory presentation Sensing data Choreography variation Presentation Choreography Next level: Representation and Reasoning Multi-tiered semantic model Low-level image features Atomic movements (Labanotation?) Compound movements (100+ movements) Emotional content, Socio-cultural layers etc. Machine Learning for classification and pattern detection Generative module (automatic choreographer)
  49. 49. Thank you v.de.boer@vu.nl @victordeboer http://victordeboer.com
  50. 50. Sensing • Motion capture – Marker-based – Marker-less • Joint rotations, limb positions etc. – unintuitive • Backup: Video annotation img: news.stanford.edu

×