Cecile P-L, profile picture
Uploaded byCecile P-L
835 views

Ph.D. Defense: Expressive Sound Synthesis for Animation

Ce document présente une thèse de doctorat sur la synthèse sonore expressif pour l'animation, abordant des méthodes comme la synthèse sonore basée sur la physique, le modélage de contact, et la synthèse basée sur des exemples. Il discute des défis et des perspectives d'un modèle hybride pour la génération de son interactif dans des environnements virtuels. Les contributions incluent une analyse automatique des enregistrements préexistants et des approches pour la synthèse sonore flexible adaptée à l'animation guidée par la physique.

Technology◦Business◦
Related topics:
Virtual Reality•

Embed presentation

Downloaded 28 times
t Expressive Sound Synthesis For Animation Cécile Picard-Limpens University of Nice/Sophia-Antipolis École Doctorale STIC REVES INRIA Sophia-Antipolis, France Advisors: George Drettakis, INRIA Sophia Antipolis (Reves) François Faure, INRIA Rhône-Alpes (Evasion) Nicolas Tsingos, DOLBY Laboratories, CA, USA Defense for Ph.D. in Computer Science C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 1
t Outline 1 Sound and Virtuality 2 Physics-Based Sound Synthesis Contact Modeling Resonator Modeling 3 Example-Based Synthesis Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 2
t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) Traditional Approach Pre-Recordings Triggered + : Easy to implement – : Repetitive audio, discrepancies, lack of flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) Physical Sound Simulation + : Physical approach, easy parametrization, Low memory usage – : Preprocess computation, Interface between physics and sound system C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
t Controlling the Sound Simulation Sound and Virtuality Challenges General Background Motivation Physics-Based Synthesis Example-Based Synthesis Perspectives on Sound Coherent With Visuals a Hybrid Model Conclusion and Unpredictable character of sounds Discussion Real-time sound synthesis Parametrization and Expressiveness Control and interactivity Authoring C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 5
t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation Perspectives on a Hybrid Model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 7
t Sound from Contacts Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Dichotomy Contacts Resonator Modeling Impacts A Robust and Multi-Scale Modal Analysis Continuous contacts Example-Based Synthesis Two Schemes for Contact Force Modelling Perspectives on a Hybrid Model Feed-forward scheme Conclusion and [van den Doel et al. 01] Discussion Additive synthesis Direct computation of contact forces [Avanzini et al. 02] Bristle model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 8
t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model HOW Can We Solve Them? Conclusion and Discussion By extracting Excitation profiles from visual textures with Adaptive resolution [Picard et al., VRIPHYS 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
t Method for Impact Sounds Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 10
t Method for Continuous Contact Sounds Sound and Virtuality Extraction of Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 11
t Synthesis of Excitation Profiles Sound and Virtuality For the Audio Force Modelling Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Technique Resonator Modeling A Robust and Multi-Scale Modal Analysis Extraction from the visual texture image Example-Based Re-sampling along the trajectory Synthesis of the contact interaction (60Hz vs 44kHz) Perspectives on a Hybrid Model Conclusion and Based on the Complexity of the Histogram Discussion Simple texture image: Gradient of the image intensity Complex texture image: Isocurves of constant brightness (isophotes) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 12
t Complex Textures Sound and Virtuality Coding the Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Isophotes = Large amount of data Contacts Resonator Modeling How Can We Lighten the Info? A Robust and Multi-Scale Modal Analysis Example-Based By Coding the Excitation Profiles Synthesis = Main Features + Noise Part Perspectives on a Hybrid Model Conclusion and Discussion = + Noise Part: Statistical approximation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 13
t Real-Time Audio Management Sound and Virtuality A Flexible Audio Pipeline Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling Simulations Driven by Ageia’s PhysX (now NVIDIA) A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 14
t Audio Texture Synthesis Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based A Sound in Coherence with Visuals Synthesis Perspectives on a Hybrid Model Flexible Resolution Conclusion and Discussion Adapted to Procedural Generation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 15
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 16
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Generating Sounds Based on Physics Simulation Resonator Modeling A Robust and Multi-Scale Modal In computer musics Analysis [Iovino et al. 97, Cook 02] Example-Based Synthesis In computer graphics Perspectives on [Van Den Doel 01, O Brien et al. 02] a Hybrid Model Conclusion and Discussion Improvements for Interactive Sound Rendering Modal parameter tracking [Maxwell et al. 07] Frequency content sparsity [Bonneel et al. 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 17
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture 1 Get a Sounding Object and its Geometry Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis 2 Construct the FEM (ex: Tetrahedral Mesh) Perspectives on a Hybrid Model 3 Apply Newton Second Law to DOF Conclusion and Discussion ¨ ˙ M d + C d + Kd = f (1) 4 Eigendecomposition ⇒ Modal Parameters M = LL−T ; L−1 KL−T = V ΛV T (2) where V = matrix of eigenvectors Λ = diagonal matrix of eigenvalues C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 18
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts In Real-time: Resonator Modeling A Robust and Modal synthesis Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion 1 s(t) = ai sin(wi t)e −di t (3) n Control for vibration models C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 19
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion HOW Can We Solve Them? By proposing A robust and multi-scale modal analysis which is Coherent with the physics simulation [Picard et al., DAFx 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
t Our Deformation Model Sound and Virtuality Physics-Based Synthesis Inspired from Work by Nesme et al. Contact Modeling Audio Texture [Nesme et al. 06] Synthesis For Complex Contacts Resonator Modeling A Robust and Technique Multi-Scale Modal Analysis Merged voxels used as Hexahedral Finite Elements Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Implementation with the Sofa Framework Validation of the Model Tests on a metal cube C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 21
t Robustness Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Robust Even for Non-Manifold Geometries Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Material: Aluminium C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 22
t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 23
t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood: Different FE resolutions Synthesis Contact Modeling Audio Texture 3x3x3 4x4x4 8x8x8 9x9x9 Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Frequency Content = f (Hexahedral FE Resolution) Conclusion and Discussion Higher resolution models Frequency centroid shift Convergence of the frequency content C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 24
t Comparison with Classical Approach Sound and Virtuality Physics-Based Sounding Bowl - Material: Aluminium Synthesis Contact Modeling Audio Texture Classical Approach Our Approach Synthesis For Complex Contacts (816 modes) (75 modes) Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 25
t A Robust and Multi-Scale Modal Analysis Sound and Virtuality A Solution for Sound Synthesis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Realistic Synthesis Perspectives on a Hybrid Model Adapted to Non-Manifold Geometries Conclusion and Discussion Resources Flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 26
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Flexible Sound Synthesis Contact Modeling Retargetting Example Sounds Resonator Modeling Perspectives on a Hybrid Model 3 Example-Based Synthesis Conclusion and Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 27
t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Concatenative Synthesis Synthesis Retargetting Example [Roads 91, Schwarz 06] Sounds Perspectives on Sound Textures Based on Physics a Hybrid Model [Cook 99] Conclusion and Discussion [Dobashi et al. 03, Zheng et al. 09] Dobashi et al. 03 Authoring and Interactive Control [Cook 02] Cook 99 C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 28
t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion HOW Can We Solve Them? By Retargetting example sounds To physics-driven animation [Picard et al., AES 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
t Our Approach Sound and Virtuality Physics-Based Synthesis SINUSOIDAL AUDIO RECORDING Example-Based TRANSIENT Synthesis OBJECT GEOMETRY VIRTUAL ENVIRONMENT Flexible Sound 1 DICTIONARY OF AUDIO GRAINS Synthesis Impulsive / Continuous Retargetting Example BUILD COLLISION Sounds STRUCTURES Perspectives on 2 CORRELATION PATTERNS a Hybrid Model DEFINE PROCEDURES PREPROCESSING INTERACTIVE Conclusion and RETARGETTING RIGID-BODY Discussion TO ANIMATION SIMULATION AUDIO RENDERER VIDEO RENDERER ANIMATION WITH AUDIO Amplitude Time Our Contributions C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 30
t Preprocess: A Generic Analysis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Impulsive and Continuous Contacts Retargetting Example Sounds Spectral Modeling Synthesis (SMS) [Serra 97] Perspectives on a Hybrid Model Conclusion and Automatic Extraction of Audio Grains Discussion Dictionary: Impulsive/Continuous Generation of Correlation Patterns between original recordings and audio grains C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 31
t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity Flexible Audio Shading Approach Additional, User-defined Resynthesis Schemes Spectral domain adaptation/modification C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
t Resynthesis of the Original Recordings Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound 94 recordings (14.6Mb) Synthesis ≈ 5000 grains + 94 Correlation Patterns (20% Gain) Retargetting Example Sounds Perspectives on Breaking Glass a Hybrid Model Conclusion and Shooting Gun Discussion Rolling Additional Material: http://www-sop.inria.fr/members/Cecile.Picard/ "‘Supplemental AES"’ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 33
t Flexible Audio Shading Approach Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Easy Implementation of Time-Scaling Flexible Sound Synthesis Retargetting Example Faster Rolling Sounds Perspectives on Slower Breaking a Hybrid Model Conclusion and Discussion Synthesis of An Infinity Similar Audio Events by varying the audio content Rythmic pattern from Breaking Stone New material content: stone and gun Rythmic pattern from Breaking Glass New material content: ceramic C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 34
t Interactive Physics-Driven Animations Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Simulations Driven by Sofa Framework Synthesis Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 35
t Retargetting Example Sounds Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Retargetting Example Variety Sounds Perspectives on a Hybrid Model Adapted to Scenarios Conclusion and Discussion Small Memory Footprint Real-Time Rendering An attractive solution for industrial applications (Eden Games, an ATARI game studio) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 36
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Motivation Resonator Modeling A Hybrid Model for Fracture Events Conclusion and 3 Example-Based Synthesis Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 37
t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize Previous Work Modeling nonlinearities [O Brien et al. 01, Chadwick et al. 09] [Cook 02] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] Challenges Event depends on the material involved Differents phases emerge from fracture event C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
t Parametrization of Our Hybrid Model Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Selection Criteria Perspectives on Hybrid model applied when nonlinearity occurs a Hybrid Model Motivation A Hybrid Model for Fracture Events Techniques Conclusion and Discussion FM synthesis Audio grains FM synthesis Parametrization Smooth transition with vibration model Coherence inside the hybrid model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 40
t Discussion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Prospective model A Hybrid Model for Fracture Events Conclusion and Possible problem: report from the physics engine Discussion Simplicity of the tools allows real-time rendering C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 41
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 42
t Synthesis of Sounds for Animation Sound and Virtuality Difficulties Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Audio-Visual Coherence Discussion Contributions Extensions and Applications Extremely Dynamic Character Precision of Synthesis Large Variety of Objects C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 43
t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio Flexibility of Sound Design Audio grains and correlation patterns Dynamic retargetting to events Extended sound synthesis capabilities C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
t Contributions Sound and Virtuality Perspectives Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion A Prospective Hybrid Model Contributions for Complex Physical Phenomena Extensions and Applications Focus on Nonlinearity Combination of physically based and example-based methods Application Case: Fracture Events C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 45
t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 46
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene Example-Based Technique Clustering of similar grains Statistical analysis of correlation patterns Physics engine design C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Hybrid Model for Fracture Events Contributions Extensions and Fracture sound simulation framework Applications Tracking of relevant physical data C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 48
t Conclusion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model New Physically Based Algorithms Conclusion and Discussion for Sound Rendering Contributions Extensions and Flexibility of Sound Modeling Applications Ideas on an Adequate Hybrid Sound Model Additional info: http://www-sop.inria.fr/members/Cecile.Picard/ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 49
t Acknowledgements Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model George Drettakis, François Faure, Conclusion and and Nicolas Tsingos Discussion Contributions REVES Team Extensions and Applications Marie-Paule Cani and the Evasion Team Paul G. Kry at the McGill University, Montréal Eden Games, an ATARI game studio, Lyon C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 50

More Related Content

PDF
Adria Recasens, DeepMind – Multi-modal self-supervised learning from videos
byCodiax
 
PPT
Multimodal Analysis for Bridging Semantic Gap with Biologically Inspired Algo...
bytechkrish
 
PDF
igarss.pdf
bygrssieee
 
PDF
2001 Tf Lcmv Signal Enhancement Using Beamforming And Nonstationarity With Ap...
byluca
 
PPT
Graphics
bysitifadzilahmatnoor
 
PDF
IRJET- A Survey on Sound Recognition
byIRJET Journal
 
PDF
APPRAISAL AND ANALOGY OF MODIFIED DE-NOISING AND LOCAL ADAPTIVE WAVELET IMAGE...
byInternational Journal of Technical Research & Application
 
PDF
Lossy Compression Using Stationary Wavelet Transform and Vector Quantization
byOmar Ghazi
 
Adria Recasens, DeepMind – Multi-modal self-supervised learning from videos
byCodiax
 
Multimodal Analysis for Bridging Semantic Gap with Biologically Inspired Algo...
bytechkrish
 
igarss.pdf
bygrssieee
 
2001 Tf Lcmv Signal Enhancement Using Beamforming And Nonstationarity With Ap...
byluca
 
Graphics
bysitifadzilahmatnoor
 
IRJET- A Survey on Sound Recognition
byIRJET Journal
 
APPRAISAL AND ANALOGY OF MODIFIED DE-NOISING AND LOCAL ADAPTIVE WAVELET IMAGE...
byInternational Journal of Technical Research & Application
 
Lossy Compression Using Stationary Wavelet Transform and Vector Quantization
byOmar Ghazi
 

What's hot

PDF
Audio Steganography Coding Using the Discreet Wavelet Transforms
byCSCJournals
 
PPTX
Spatio-temporal control of light in complex media
bySébastien Popoff
 
PDF
A Brief Introduction of Anomalous Sound Detection: Recent Studies and Future...
byYuma Koizumi
 
PDF
Analysis of PEAQ Model using Wavelet Decomposition Techniques
byidescitation
 
PDF
Introduction to compressive sensing
byAhmed Nasser Agag
 
PDF
Performance Analysis of Digital Watermarking Of Video in the Spatial Domain
bypaperpublications3
 
PDF
Brunelli 2008: template matching techniques in computer vision
byzukun
 
PDF
Miller - Remote Sensing and Imaging Physics - Spring Review 2012
byThe Air Force Office of Scientific Research
 
PDF
On the scattering of light : various models and methods used in computer grap...
byToru Tamaki
 
Audio Steganography Coding Using the Discreet Wavelet Transforms
byCSCJournals
 
Spatio-temporal control of light in complex media
bySébastien Popoff
 
A Brief Introduction of Anomalous Sound Detection: Recent Studies and Future...
byYuma Koizumi
 
Analysis of PEAQ Model using Wavelet Decomposition Techniques
byidescitation
 
Introduction to compressive sensing
byAhmed Nasser Agag
 
Performance Analysis of Digital Watermarking Of Video in the Spatial Domain
bypaperpublications3
 
Brunelli 2008: template matching techniques in computer vision
byzukun
 
Miller - Remote Sensing and Imaging Physics - Spring Review 2012
byThe Air Force Office of Scientific Research
 
On the scattering of light : various models and methods used in computer grap...
byToru Tamaki
 

Similar to Ph.D. Defense: Expressive Sound Synthesis for Animation

PDF
Adaptive Music and Interactive Audio
byAdam Sporka
 
PDF
Foundations In Sound Design For Linear Media A Multidisciplinary Approach Mic...
bygijonyenorwg
 
PDF
Designing Sound For Animation Robin Beauchamp
byarnalpsi
 
PPT
Lecture6 audio
byMr SMAK
 
PPT
Lecture6 audio
byMr SMAK
 
PPT
Towards User-friendly Audio Creation
byJean Vanderdonckt
 
PDF
Computer sound design synthesis techniques and programming 2nd Edition Eduard...
byjurcyszibaia
 
PPT
Multimedia tools (sound)
bydhruv patel
 
PPT
Sound
byJay Patel
 
PDF
Echoes, Whispers, and Footsteps from the Conflux of Sonic Interaction Design ...
byEoin Brazil
 
PDF
Interactive Physically-Based Sound Design of 3D Model using Material Optimiza...
byyamo_o
 
PPTX
Project - Sound Model Similarity Search
bySudarshan Bala
 
PDF
Automated Sound Design 1st Edition Eric Lyon.
bysubriyahagi
 
PDF
A Generalized Wave Field Synthesis Theory (Ph.D. Thesis Booklet)
byLori Moore
 
PDF
AudioSIG28Nov2011
byNevin Eronde
 
PDF
Experimental Media Voodoo™
bySkyRonDotOrg
 
PDF
Computer Sound Design Synthesis Techniques And Programming 2nd Edition Eduard...
bygxfqisu378
 
PDF
Multimedia elements
byNiteshwar Bhardwaj
 
DOCX
Mit 8116
byJosh Bwiira
 
PDF
Survey On Speech Synthesis
byCSCJournals
 
Adaptive Music and Interactive Audio
byAdam Sporka
 
Foundations In Sound Design For Linear Media A Multidisciplinary Approach Mic...
bygijonyenorwg
 
Designing Sound For Animation Robin Beauchamp
byarnalpsi
 
Lecture6 audio
byMr SMAK
 
Lecture6 audio
byMr SMAK
 
Towards User-friendly Audio Creation
byJean Vanderdonckt
 
Computer sound design synthesis techniques and programming 2nd Edition Eduard...
byjurcyszibaia
 
Multimedia tools (sound)
bydhruv patel
 
Sound
byJay Patel
 
Echoes, Whispers, and Footsteps from the Conflux of Sonic Interaction Design ...
byEoin Brazil
 
Interactive Physically-Based Sound Design of 3D Model using Material Optimiza...
byyamo_o
 
Project - Sound Model Similarity Search
bySudarshan Bala
 
Automated Sound Design 1st Edition Eric Lyon.
bysubriyahagi
 
A Generalized Wave Field Synthesis Theory (Ph.D. Thesis Booklet)
byLori Moore
 
AudioSIG28Nov2011
byNevin Eronde
 
Experimental Media Voodoo™
bySkyRonDotOrg
 
Computer Sound Design Synthesis Techniques And Programming 2nd Edition Eduard...
bygxfqisu378
 
Multimedia elements
byNiteshwar Bhardwaj
 
Mit 8116
byJosh Bwiira
 
Survey On Speech Synthesis
byCSCJournals
 

Recently uploaded

PDF
The Emergence of Empathic XR: AWE Asia 2026 keynote
byMark Billinghurst
 
PDF
Mount File Systems using UUID and Label - RHCSA (RH134).pdf
byLinuxCert Guru
 
PDF
From DeFi POC to Production MVP - A 12-16 Week Blueprint.pdf
byniahiggins21
 
PDF
Track Phone Location by Number (2026)_ What’s Actually Possible for Free (and...
byEmily Woods
 
PDF
CI CD Observability, Metrics and DORA - Shifting Left and Cleaning Up! - Febr...
byPeter Souter
 
PPTX
WEEK 1-introduction of industrial arts grade 7.pptx
byrosierufinomaliongan
 
PDF
Poročilo odbora CIS (CH08873) za leto 2025 na letni skupščini IEEE Slovenija ...
byUniversity of Maribor
 
PPTX
Retrieval Augmented Generation- The Synergistic Power of Prompt Engineering
bySemantic SEO BD
 
PDF
TravelTech Paris 2025 | The EU Digital Identity Wallet and it’s impact on Tra...
byapidays
 
PDF
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
PPTX
Beyond the Algorithm: Designing Human-Centric Public Service with AI
byAlan Dix
 
PDF
UiPath Automation Developer Associate Training Series 2026 - Session 2
byDianaGray10
 
PDF
Advanced SELinux Management - RHCSA (RH134).pdf
byLinuxCert Guru
 
PDF
Track Phone Location via Number (2026)_ What Actually Works, What’s a Scam, a...
byEmily Woods
 
PDF
Startup Formation Collapses While Acquisition Activity Hits New High!
byMemoori
 
PDF
The Manifesto for AI-enabled Governance !
byYannis Charalabidis
 
PPTX
Automation Without Apprentices: How AI Challenges the Open Source Way
byIcinga
 
PDF
Chapter 4 Network Security in computer security
byGetnet Tigabie Askale -(GM)
 
PDF
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
PDF
Why Many Smart Device Platforms Fail to Scale?
byPromeraki Developments
 
The Emergence of Empathic XR: AWE Asia 2026 keynote
byMark Billinghurst
 
Mount File Systems using UUID and Label - RHCSA (RH134).pdf
byLinuxCert Guru
 
From DeFi POC to Production MVP - A 12-16 Week Blueprint.pdf
byniahiggins21
 
Track Phone Location by Number (2026)_ What’s Actually Possible for Free (and...
byEmily Woods
 
CI CD Observability, Metrics and DORA - Shifting Left and Cleaning Up! - Febr...
byPeter Souter
 
WEEK 1-introduction of industrial arts grade 7.pptx
byrosierufinomaliongan
 
Poročilo odbora CIS (CH08873) za leto 2025 na letni skupščini IEEE Slovenija ...
byUniversity of Maribor
 
Retrieval Augmented Generation- The Synergistic Power of Prompt Engineering
bySemantic SEO BD
 
TravelTech Paris 2025 | The EU Digital Identity Wallet and it’s impact on Tra...
byapidays
 
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
Beyond the Algorithm: Designing Human-Centric Public Service with AI
byAlan Dix
 
UiPath Automation Developer Associate Training Series 2026 - Session 2
byDianaGray10
 
Advanced SELinux Management - RHCSA (RH134).pdf
byLinuxCert Guru
 
Track Phone Location via Number (2026)_ What Actually Works, What’s a Scam, a...
byEmily Woods
 
Startup Formation Collapses While Acquisition Activity Hits New High!
byMemoori
 
The Manifesto for AI-enabled Governance !
byYannis Charalabidis
 
Automation Without Apprentices: How AI Challenges the Open Source Way
byIcinga
 
Chapter 4 Network Security in computer security
byGetnet Tigabie Askale -(GM)
 
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
Why Many Smart Device Platforms Fail to Scale?
byPromeraki Developments
 

Ph.D. Defense: Expressive Sound Synthesis for Animation

  • 1.
    t Expressive Sound Synthesis For Animation Cécile Picard-Limpens University of Nice/Sophia-Antipolis École Doctorale STIC REVES INRIA Sophia-Antipolis, France Advisors: George Drettakis, INRIA Sophia Antipolis (Reves) François Faure, INRIA Rhône-Alpes (Evasion) Nicolas Tsingos, DOLBY Laboratories, CA, USA Defense for Ph.D. in Computer Science C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 1
  • 2.
    t Outline 1 Sound and Virtuality 2 Physics-Based Sound Synthesis Contact Modeling Resonator Modeling 3 Example-Based Synthesis Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 2
  • 3.
    t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
  • 4.
    t Sound Rendering Sound and Virtuality General Background for Virtual Reality and Games Motivation Physics-Based Synthesis Example-Based Interactive Audio Rendering Synthesis Perspectives on a Hybrid Model Conclusion and Discussion (R. Vantielcke - WipeoutHD on Playstation 3) Traditional Approach Pre-Recordings Triggered + : Easy to implement – : Repetitive audio, discrepancies, lack of flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 3
  • 5.
    t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
  • 6.
    t From Playback of Samples Sound and Virtuality General Background to Synthesis Motivation Physics-Based Synthesis Digital Sound Synthesis Example-Based Synthesis Source modeling ← Perspectives on Sound propagation, Sound reception a Hybrid Model Conclusion and Techniques Discussion Rigid body simulation Finite Element Method (FEM) (ArtiSynth) Physical Sound Simulation + : Physical approach, easy parametrization, Low memory usage – : Preprocess computation, Interface between physics and sound system C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 4
  • 7.
    t Controlling the Sound Simulation Sound and Virtuality Challenges General Background Motivation Physics-Based Synthesis Example-Based Synthesis Perspectives on Sound Coherent With Visuals a Hybrid Model Conclusion and Unpredictable character of sounds Discussion Real-time sound synthesis Parametrization and Expressiveness Control and interactivity Authoring C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 5
  • 8.
    t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
  • 9.
    t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
  • 10.
    t Our Contribution Sound and Virtuality Three Research Axes General Background Motivation Physics-Based Synthesis Example-Based Synthesis Physics-Based Sound synthesis Perspectives on a Hybrid Model Contact modeling Conclusion and Resonator modeling Discussion Example-Based Sound Synthesis Automatic analysis of pre-recordings Flexible synthesis for physics-driven animation Perspectives on a Hybrid Model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 6
  • 11.
    t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 7
  • 12.
    t Sound from Contacts Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Dichotomy Contacts Resonator Modeling Impacts A Robust and Multi-Scale Modal Analysis Continuous contacts Example-Based Synthesis Two Schemes for Contact Force Modelling Perspectives on a Hybrid Model Feed-forward scheme Conclusion and [van den Doel et al. 01] Discussion Additive synthesis Direct computation of contact forces [Avanzini et al. 02] Bristle model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 8
  • 13.
    t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
  • 14.
    t Contact Modeling Sound and Virtuality Physics-Based Synthesis Contact Modeling What Are The Current Limitations Audio Texture Synthesis For Complex for Continuous Contacts? Contacts Resonator Modeling Rate for physics engine report A Robust and Multi-Scale Modal Analysis No geometric details when using visual textures Example-Based Synthesis Authoring and control are challenging Perspectives on a Hybrid Model HOW Can We Solve Them? Conclusion and Discussion By extracting Excitation profiles from visual textures with Adaptive resolution [Picard et al., VRIPHYS 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 9
  • 15.
    t Method for Impact Sounds Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 10
  • 16.
    t Method for Continuous Contact Sounds Sound and Virtuality Extraction of Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 11
  • 17.
    t Synthesis of Excitation Profiles Sound and Virtuality For the Audio Force Modelling Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Technique Resonator Modeling A Robust and Multi-Scale Modal Analysis Extraction from the visual texture image Example-Based Re-sampling along the trajectory Synthesis of the contact interaction (60Hz vs 44kHz) Perspectives on a Hybrid Model Conclusion and Based on the Complexity of the Histogram Discussion Simple texture image: Gradient of the image intensity Complex texture image: Isocurves of constant brightness (isophotes) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 12
  • 18.
    t Complex Textures Sound and Virtuality Coding the Excitation Profiles Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Isophotes = Large amount of data Contacts Resonator Modeling How Can We Lighten the Info? A Robust and Multi-Scale Modal Analysis Example-Based By Coding the Excitation Profiles Synthesis = Main Features + Noise Part Perspectives on a Hybrid Model Conclusion and Discussion = + Noise Part: Statistical approximation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 13
  • 19.
    t Real-Time Audio Management Sound and Virtuality A Flexible Audio Pipeline Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling Simulations Driven by Ageia’s PhysX (now NVIDIA) A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 14
  • 20.
    t Audio Texture Synthesis Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based A Sound in Coherence with Visuals Synthesis Perspectives on a Hybrid Model Flexible Resolution Conclusion and Discussion Adapted to Procedural Generation C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 15
  • 21.
    t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Contact Modeling Audio Texture Synthesis For Complex Contacts 2 Physics-Based Sound Synthesis Resonator Modeling Contact Modeling A Robust and Multi-Scale Modal Analysis Resonator Modeling Example-Based Synthesis 3 Example-Based Synthesis Perspectives on Flexible Sound Synthesis a Hybrid Model Conclusion and 4 Perspectives on a Hybrid Model Discussion Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 16
  • 22.
    t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Generating Sounds Based on Physics Simulation Resonator Modeling A Robust and Multi-Scale Modal In computer musics Analysis [Iovino et al. 97, Cook 02] Example-Based Synthesis In computer graphics Perspectives on [Van Den Doel 01, O Brien et al. 02] a Hybrid Model Conclusion and Discussion Improvements for Interactive Sound Rendering Modal parameter tracking [Maxwell et al. 07] Frequency content sparsity [Bonneel et al. 08] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 17
  • 23.
    t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture 1 Get a Sounding Object and its Geometry Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis 2 Construct the FEM (ex: Tetrahedral Mesh) Perspectives on a Hybrid Model 3 Apply Newton Second Law to DOF Conclusion and Discussion ¨ ˙ M d + C d + Kd = f (1) 4 Eigendecomposition ⇒ Modal Parameters M = LL−T ; L−1 KL−T = V ΛV T (2) where V = matrix of eigenvectors Λ = diagonal matrix of eigenvalues C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 18
  • 24.
    t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts In Real-time: Resonator Modeling A Robust and Modal synthesis Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion 1 s(t) = ai sin(wi t)e −di t (3) n Control for vibration models C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 19
  • 25.
    t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
  • 26.
    t Vibration Models Sound and Virtuality Modal Analysis Physics-Based Synthesis Contact Modeling Audio Texture What Are Synthesis For Complex Contacts The Current Limitations? Resonator Modeling A Robust and Multi-Scale Modal Meshing is difficult Analysis Example-Based No real control on the FEM resolution Synthesis No clear interface between physics and audio Perspectives on a Hybrid Model Conclusion and Discussion HOW Can We Solve Them? By proposing A robust and multi-scale modal analysis which is Coherent with the physics simulation [Picard et al., DAFx 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 20
  • 27.
    t Our Deformation Model Sound and Virtuality Physics-Based Synthesis Inspired from Work by Nesme et al. Contact Modeling Audio Texture [Nesme et al. 06] Synthesis For Complex Contacts Resonator Modeling A Robust and Technique Multi-Scale Modal Analysis Merged voxels used as Hexahedral Finite Elements Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Implementation with the Sofa Framework Validation of the Model Tests on a metal cube C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 21
  • 28.
    t Robustness Sound and Virtuality Physics-Based Synthesis Contact Modeling Audio Texture Robust Even for Non-Manifold Geometries Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Material: Aluminium C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 22
  • 29.
    t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 23
  • 30.
    t Multi-Scale for Efficient Memory Usage Sound and Virtuality Physics-Based A Squirrel in Pine Wood: Different FE resolutions Synthesis Contact Modeling Audio Texture 3x3x3 4x4x4 8x8x8 9x9x9 Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Frequency Content = f (Hexahedral FE Resolution) Conclusion and Discussion Higher resolution models Frequency centroid shift Convergence of the frequency content C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 24
  • 31.
    t Comparison with Classical Approach Sound and Virtuality Physics-Based Sounding Bowl - Material: Aluminium Synthesis Contact Modeling Audio Texture Classical Approach Our Approach Synthesis For Complex Contacts (816 modes) (75 modes) Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 25
  • 32.
    t A Robust and Multi-Scale Modal Analysis Sound and Virtuality A Solution for Sound Synthesis Physics-Based Synthesis Contact Modeling Audio Texture Synthesis For Complex Contacts Resonator Modeling A Robust and Multi-Scale Modal Analysis Example-Based Realistic Synthesis Perspectives on a Hybrid Model Adapted to Non-Manifold Geometries Conclusion and Discussion Resources Flexibility C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 26
  • 33.
    t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Flexible Sound Synthesis Contact Modeling Retargetting Example Sounds Resonator Modeling Perspectives on a Hybrid Model 3 Example-Based Synthesis Conclusion and Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 27
  • 34.
    t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Concatenative Synthesis Synthesis Retargetting Example [Roads 91, Schwarz 06] Sounds Perspectives on Sound Textures Based on Physics a Hybrid Model [Cook 99] Conclusion and Discussion [Dobashi et al. 03, Zheng et al. 09] Dobashi et al. 03 Authoring and Interactive Control [Cook 02] Cook 99 C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 28
  • 35.
    t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
  • 36.
    t Implementation of Signal-Based Models Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis What Are Flexible Sound Synthesis The Current Limitations? Retargetting Example Sounds Processing is not generic Perspectives on a Hybrid Model Parametrizing is difficult Conclusion and Discussion HOW Can We Solve Them? By Retargetting example sounds To physics-driven animation [Picard et al., AES 09] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 29
  • 37.
    t Our Approach Sound and Virtuality Physics-Based Synthesis SINUSOIDAL AUDIO RECORDING Example-Based TRANSIENT Synthesis OBJECT GEOMETRY VIRTUAL ENVIRONMENT Flexible Sound 1 DICTIONARY OF AUDIO GRAINS Synthesis Impulsive / Continuous Retargetting Example BUILD COLLISION Sounds STRUCTURES Perspectives on 2 CORRELATION PATTERNS a Hybrid Model DEFINE PROCEDURES PREPROCESSING INTERACTIVE Conclusion and RETARGETTING RIGID-BODY Discussion TO ANIMATION SIMULATION AUDIO RENDERER VIDEO RENDERER ANIMATION WITH AUDIO Amplitude Time Our Contributions C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 30
  • 38.
    t Preprocess: A Generic Analysis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Impulsive and Continuous Contacts Retargetting Example Sounds Spectral Modeling Synthesis (SMS) [Serra 97] Perspectives on a Hybrid Model Conclusion and Automatic Extraction of Audio Grains Discussion Dictionary: Impulsive/Continuous Generation of Correlation Patterns between original recordings and audio grains C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 31
  • 39.
    t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
  • 40.
    t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
  • 41.
    t On-Line: Flexible Sound Synthesis Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Resynthesis of the Original Recordings Flexible Sound Synthesis Candidate grains: max. correlation amplitude Retargetting Example Sounds Perspectives on Interactive Physics-Driven Animations a Hybrid Model Physics Info for Retargetting Conclusion and Discussion Contact type: impulsive or continuous? Penetration force and relative velocity Flexible Audio Shading Approach Additional, User-defined Resynthesis Schemes Spectral domain adaptation/modification C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 32
  • 42.
    t Resynthesis of the Original Recordings Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound 94 recordings (14.6Mb) Synthesis ≈ 5000 grains + 94 Correlation Patterns (20% Gain) Retargetting Example Sounds Perspectives on Breaking Glass a Hybrid Model Conclusion and Shooting Gun Discussion Rolling Additional Material: http://www-sop.inria.fr/members/Cecile.Picard/ "‘Supplemental AES"’ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 33
  • 43.
    t Flexible Audio Shading Approach Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Easy Implementation of Time-Scaling Flexible Sound Synthesis Retargetting Example Faster Rolling Sounds Perspectives on Slower Breaking a Hybrid Model Conclusion and Discussion Synthesis of An Infinity Similar Audio Events by varying the audio content Rythmic pattern from Breaking Stone New material content: stone and gun Rythmic pattern from Breaking Glass New material content: ceramic C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 34
  • 44.
    t Interactive Physics-Driven Animations Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Flexible Sound Simulations Driven by Sofa Framework Synthesis Retargetting Example Sounds Perspectives on a Hybrid Model Conclusion and Discussion C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 35
  • 45.
    t Retargetting Example Sounds Sound and Virtuality A Solution for Interactive Simulations Physics-Based Synthesis Example-Based Synthesis Flexible Sound Synthesis Retargetting Example Variety Sounds Perspectives on a Hybrid Model Adapted to Scenarios Conclusion and Discussion Small Memory Footprint Real-Time Rendering An attractive solution for industrial applications (Eden Games, an ATARI game studio) C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 36
  • 46.
    t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Motivation Resonator Modeling A Hybrid Model for Fracture Events Conclusion and 3 Example-Based Synthesis Discussion Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 37
  • 47.
    t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
  • 48.
    t Sound Modeling Sound and Virtuality When Nonlinearity Occurs Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Problems of Single Models A Hybrid Model for Fracture Events Vibration models assume linearity Conclusion and Discussion Example-based sounds are hard to parametrize Previous Work Modeling nonlinearities [O Brien et al. 01, Chadwick et al. 09] [Cook 02] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 38
  • 49.
    t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
  • 50.
    t Fracture Events Sound and Virtuality Physics-Based Synthesis Example-Based Background Synthesis Frequently occur in virtual environments Perspectives on a Hybrid Model Motivation Visual rendering A Hybrid Model for Fracture Events [O Brien et al. 99, 02] Conclusion and [Parker and O Brien. 09] Discussion Sound rendering: Little research [Warren et al. 84] [Rath et al. 03] Challenges Event depends on the material involved Differents phases emerge from fracture event C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 39
  • 51.
    t Parametrization of Our Hybrid Model Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Selection Criteria Perspectives on Hybrid model applied when nonlinearity occurs a Hybrid Model Motivation A Hybrid Model for Fracture Events Techniques Conclusion and Discussion FM synthesis Audio grains FM synthesis Parametrization Smooth transition with vibration model Coherence inside the hybrid model C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 40
  • 52.
    t Discussion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Motivation Prospective model A Hybrid Model for Fracture Events Conclusion and Possible problem: report from the physics engine Discussion Simplicity of the tools allows real-time rendering C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 41
  • 53.
    t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 42
  • 54.
    t Synthesis of Sounds for Animation Sound and Virtuality Difficulties Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Audio-Visual Coherence Discussion Contributions Extensions and Applications Extremely Dynamic Character Precision of Synthesis Large Variety of Objects C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 43
  • 55.
    t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
  • 56.
    t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
  • 57.
    t Contributions Sound and Virtuality An Overview Physics-Based Synthesis Example-Based Complex Contact Modeling Synthesis Perspectives on 2D visual textures used as roughness maps a Hybrid Model Audible and position-dependent variations Conclusion and Detail-layer mechanisms Discussion Contributions Extensions and Applications Improved Modal Analysis for Resonator Modeling Complex non-manifold geometries can be handled Multi-scale resolution Coherence between simulation and audio Flexibility of Sound Design Audio grains and correlation patterns Dynamic retargetting to events Extended sound synthesis capabilities C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 44
  • 58.
    t Contributions Sound and Virtuality Perspectives Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion A Prospective Hybrid Model Contributions for Complex Physical Phenomena Extensions and Applications Focus on Nonlinearity Combination of physically based and example-based methods Application Case: Fracture Events C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 45
  • 59.
    t Overview Sound and Virtuality Physics-Based Synthesis 1 Sound and Virtuality Example-Based Synthesis 2 Physics-Based Sound Synthesis Perspectives on a Hybrid Model Contact Modeling Conclusion and Resonator Modeling Discussion Contributions 3 Example-Based Synthesis Extensions and Applications Flexible Sound Synthesis 4 Perspectives on a Hybrid Model Motivation and Application 5 Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 46
  • 60.
    t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Extensions and Applications C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
  • 61.
    t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
  • 62.
    t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Complex Contact Modeling Example-Based Synthesis Two interacting textures Perspectives on Surface-based interactions a Hybrid Model Adequate perceptual experiments Conclusion and Discussion Contributions Improved Modal Analysis for Resonator Modeling Extensions and Applications Recent work from [Nesme et al. Siggraph 09] Investigations with GPU for in-line computation Complete integration in a virtual scene Example-Based Technique Clustering of similar grains Statistical analysis of correlation patterns Physics engine design C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 47
  • 63.
    t Promising Directions for Future Work Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model Conclusion and Discussion Hybrid Model for Fracture Events Contributions Extensions and Fracture sound simulation framework Applications Tracking of relevant physical data C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 48
  • 64.
    t Conclusion Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model New Physically Based Algorithms Conclusion and Discussion for Sound Rendering Contributions Extensions and Flexibility of Sound Modeling Applications Ideas on an Adequate Hybrid Sound Model Additional info: http://www-sop.inria.fr/members/Cecile.Picard/ C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 49
  • 65.
    t Acknowledgements Sound and Virtuality Physics-Based Synthesis Example-Based Synthesis Perspectives on a Hybrid Model George Drettakis, François Faure, Conclusion and and Nicolas Tsingos Discussion Contributions REVES Team Extensions and Applications Marie-Paule Cani and the Evasion Team Paul G. Kry at the McGill University, Montréal Eden Games, an ATARI game studio, Lyon C. Picard-Limpens December 4, 2009 Expressive Sound Synthesis For Animation 50