Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

idalab seminar #18 - Valerio Velardo

54 views

Published on

In a time when creators of video games are trying to make gaming an ever more immersive and ever more real experience, AI is beginning to have an impact in the field. Everyone who’s ever played a video game knows: Music plays an important part in conveying the atmosphere of altering game states and thus needs to change dynamically as the gamer makes choices and variations of the storyline unfold. However, as storylines get ever more complex and the number of possible choices practically explodes, the task of implementing changes in music into a game’s engine, while at the same time ensuring smooth transitions, eventually becomes too complex to handle manually. The task clearly calls for some form of automatisation – but does AI have the ability to create effective video game music?

Using AI to generate adaptive video game music is not entirely new. A wealth of research has been carried out both in academia and the gaming industry. Deep Adaptive Music (DAM), an innovative solution developed by Melodrive, extends adaptive music by more intensely leveraging AI. DAM allows for the generation of music in realtime by an AI system running directly inside an interactive experience, adapting to emotional states within the game on a granular level. Responding to both the user’s interaction and to the changing game states, DAM complements the idea of co-creation between the AI and a human composer or player. In my talk, I will first give an overview on past research in the field, then dive into the mechanics behind DAM and eventually consider AI’s potentiality in creative tasks.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

idalab seminar #18 - Valerio Velardo

  1. 1. Valerio Velardo Agency for Data Science Machine learning & AI Mathematical modelling Data strategy Generating music in realtime with Artificial Intelligence – What if music could change automatically with the emotional state in a video game? idalab seminar #18 | April 4th 2019
  2. 2. Can AI Create Effective Video Game Music?
  3. 3. I am Valerio Velardo, PhD Cofounder & CEO @ Melodrive valerio@melodrive.com Hello!
  4. 4. Overview 1. A brief history of video game music 2. Deep adaptive music (DAM) 3. DAM and Melodrive 4. DAM applications 5. Takeaways
  5. 5. 1. A brief history of video game music From bleeps and bloops to full orchestral music
  6. 6. Films vs. video games
  7. 7. 2 types of musics Linear ◍ begin, middle and end ◍ predetermined ◍ finite Non-linear ◍ dynamic form ◍ non-predetermined ◍ infinite
  8. 8. Adaptive music ◍ Responds to and interacts with ◌ user input ◌ visual experience
  9. 9. Adaptive music ◍ Responds to and interacts with ◌ user input ◌ visual experience ◍ Adapts to the emotional setting of an interactive experience
  10. 10. History of adaptive music ◍ Bleeps and bloops era
  11. 11. Space Invaders (1978)
  12. 12. History of adaptive music ◍ Bleeps and bloops era ◍ 8-/16-bit era
  13. 13. Monkey Island (1991) iMuse system: Dynamic instrumentation/phrase changes (9’25’’)
  14. 14. History of adaptive music ◍ Bleeps and bloops era ◍ 8-/16-bit era ◍ High-fidelity era
  15. 15. Skyrim (2011)
  16. 16. Adaptive music techniques ◍ Vertical layering
  17. 17. Adaptive music techniques ◍ Vertical layering ◍ Horizontal resequencing
  18. 18. Adaptive music techniques ◍ Vertical layering ◍ Horizontal resequencing ◍ Audio manipulation
  19. 19. Issues with human-composed adaptive music ◍ Constrained and finite ◍ Visuals cannot be tightly scored ◍ Abrupt transitions ◍ Laborious composition and integration
  20. 20. Issues with human-composed adaptive music ◍ Constrained and finite ◍ Visuals cannot be tightly scored ◍ Abrupt transitions ◍ Laborious composition and integration ◍ Non-linear music can’t be efficiently handled by humans
  21. 21. Issues with human-composed adaptive music Missed opportunity for enhancing immersion
  22. 22. Non-linear music
  23. 23. 2. Deep adaptive music (DAM) Using AI to bring video game music to the next level
  24. 24. ◍ Realtime procedural generation Definition: Deep adaptive music
  25. 25. ◍ Realtime procedural generation ◍ Collaboration between human and AI Definition: Deep adaptive music
  26. 26. ◍ Realtime procedural generation ◍ Collaboration between human and AI ◍ Adapts on a granular level Definition: Deep adaptive music
  27. 27. ◍ Realtime procedural generation ◍ Collaboration between human and AI ◍ Adapts on a granular level ◍ Infinite Definition: Deep adaptive music
  28. 28. ◍ Realtime procedural generation ◍ Collaboration between human and AI ◍ Adapts on a granular level ◍ Infinite ◍ Smooth transitions between emotional states Definition: Deep adaptive music
  29. 29. ◍ Realtime procedural generation ◍ Collaboration between human and AI ◍ Adapts on a granular level ◍ Infinite ◍ Smooth transitions between emotional states ◍ Tailor-made to the user Definition: Deep adaptive music
  30. 30. Generative music in games: Spore (2008) Music evolves with your character
  31. 31. Generative music in games: Mini Metro(2015) Music generation sparking from gameplay
  32. 32. ◍ ML + statical-based techniques ◍ Symbolic AI ◍ Population based algorithms ◍ Self-similarity based systems AI techniques used in DAM
  33. 33. 3. Melodrive and DAM
  34. 34. What does the Melodrive engine do? Composition Performance Production Emotional API Music Game params User input
  35. 35. Melodrive: Oceania
  36. 36. How does the engine work? ◍ Reproducing the compositional process
  37. 37. How does the engine work? ◍ Reproducing the compositional process ◍ Hybrid AI approach ◌ Break down compositional task into several sub-tasks ◌ Use AI technique that works best
  38. 38. How does the engine work? ◍ Reproducing the compositional process ◍ Hybrid AI approach ◌ Break down compositional task into several sub-tasks ◌ Use AI technique that works best ◍ Mapping emotions onto musical parameters
  39. 39. Challenges building a DAM system ◍ No ground truth ◍ Subjective assessment ◍ Ill defined problem
  40. 40. 4. DAM applications Changing how music is made and experienced in video games
  41. 41. Redefining the role of the composer Infinite realtime music Core music ideas Rules DAM System
  42. 42. 5. Takeaways
  43. 43. Takeaways ◍ Human-composed video game music is limited
  44. 44. Takeaways ◍ Human-composed video game music is limited ◍ AI has been used in game music for along time
  45. 45. Takeaways ◍ Human-composed video game music is limited ◍ AI has been used in game music for along time ◍ DAM uses AI to create realtime music tailored to the player
  46. 46. Takeaways ◍ Human-composed video game music is limited ◍ AI has been used in game music for along time ◍ DAM uses AI to create realtime music tailored to the player ◍ DAM augments the composer
  47. 47. Vision A world where humans and AI collaborate to create superhuman music
  48. 48. https://medium.com/the-sound-of-ai @musikalkemist valerio_velardo valerio@melodrive.com

×