Cs665 writeup

363 views

Published on

  • Be the first to comment

  • Be the first to like this

Cs665 writeup

  1. 1. Cornell UniversityCS 6650: Computational Motion Semester Project Rigid Body Sound Synthesis Authors: Supervisor: Charles Moyes Dr. Doug L. James Shentong Wang May 16, 2011
  2. 2. Contents1 Abstract 22 Math Library 3 2.1 Dense Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Armadillo Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.3 Sparse Matrix Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.4 SVDLIBC C Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Sound Synthesis 5 3.1 Finite Element Model Matrix Formation . . . . . . . . . . . . . . . . . . . . 5 3.2 Material Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.3 Modal Synthesis Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.4 Audio Playback Using SDL . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Rigid Body Dynamics 8 4.1 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2 Resting Contact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2.1 Analytical Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2.2 Projected Relaxed Gauss-Seidel Solver . . . . . . . . . . . . . . . . . 8 4.3 Collision Detection and Response . . . . . . . . . . . . . . . . . . . . . . . . 9 4.3.1 Collision Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.3.2 Newton’s Law of Restitution . . . . . . . . . . . . . . . . . . . . . . . 9 4.3.3 Tangential Coulomb Friction . . . . . . . . . . . . . . . . . . . . . . . 9 4.4 Open Dynamics Engine (ODE) Integration . . . . . . . . . . . . . . . . . . . 105 Graphics Engine 11 5.1 Test GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5.1.1 Keyboard Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5.2 OpenGL Renderer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5.2.1 Keyboard Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Future Plans 157 Conclusion 16 1
  3. 3. Chapter 1AbstractThe basic idea of the project is to implement a rigid body sound synthesis engine in C++using OpenGL and a rigid body dynamics simulation. Modal synthesis in the frequency-domain will be used to generate rigid body contact sounds based on the contact forces andpositions applied to objects in the simulation. The Armadillo linear algebra library will beused to perform routine matrix decompositions such as Cholesky factorizations and singularvalue decompositions (SVDs). The SDL library will be used for audio output and graphicalrendering. Another goal is to build an interesting simulation to show off the applications ofthe engine. Figure 1.1: Proof-of-concept image from [?]. 2
  4. 4. Chapter 2Math LibraryThe project required linear algebra routines for forming the mass and stiffness matrices,along with solving the eigenvalue problem associated with modal analysis. For this project,we implemented our own dense and sparse matrix classes, along with using the Armadillolinear algebra library (found at http://arma.sourceforge.net/) and the SVDLIBC sparsesingular value decomposition library (found at http://tedlab.mit.edu/~dr/SVDLIBC/).We have implemented both a sparse and a dense matrix solver. We found that the densesolver works best with smaller tetrahedral meshes.2.1 Dense SolverThe dense solver was implemented first using the Armadillo library to have a metric fortesting the sparse matrix routines. Meanwhile, the sparse solver was developed concurrently.2.2 Armadillo IntegrationArmadillo was used for the dense solver for computing the Cholesky decomposition of themass matrix. For the sparse matrix solver, we generate a diagonal “lumped” mass ma-trix, avoiding the need to perform an explicit Cholesky decomposition. In this case, theCholesky decomposition is trivially the square root of the diagonal values of the matrix.The Eigen linear algebra library (http://eigen.tuxfamily.org/) was also investigatedfor this project, but we found that the Armadillo API was easier to use with our code.2.3 Sparse Matrix LibraryOur sparse matrix library initially handled both static and dynamic matrices. We used astorage format very similar to the Harwell-Boeing format. Column indices separate an arrayof non-zero values into partitions, where each group resides in a different column. Withineach column, each non-zero value in the sparse matrix has a row index associated with it.Very thorough test batteries were written to ensure the correctness of this code in order toprevent tedious debugging further down the road.2.4 SVDLIBC C LibraryAt first, we tried developing our own sparse matrix eigenvalue decomposition code, butafter consulting with Charles Van Loan and David Bindel, we decided that this endeavorwould be a separate project in itself. The SVDLIBC library was used for performingsparse eigenvalue decompositions. It uses the Lanczos algorithm for computing the singular 3
  5. 5. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Projectvalue decomposition, and it handles both dense and sparse matrices. Code was writtento generate sparse matrices in the Harwell-Boeing binary format for use with this library.Several Fortran libraries were investigated, including PROPACK, TRLan, and LAPACK.We found that SVDLIBC was the easiest to use, so we went with it instead. 4
  6. 6. Chapter 3Sound Synthesis3.1 Finite Element Model Matrix FormationWe used [?] as a reference for constructing the matrices for tetrahedral finite element models.We discretize our mesh into tetrahedron in order to perform modal analysis. The mesh ver-tices and tetrahedron indices were loaded directly from binary files generated using ChangxiZheng’s IsoStuffer utility [?]. We used linear basis functions as described in the paper.We wrote code to assemble the mass and stiffness matrices as suggested. The element massmatrix M was computed as: ∂ 2κ ρ vol m[ij]ab = = (1 + δij )δab (3.1) ∂ p[i]a p[j]b ˙ ˙ 20 and the “lumped” mass matrix was formed as follows: ρ vol mlumped = [ij]ab δij δab (3.2) 4 The stiffness matrix K was formed using: 3 ∂f[i]a vol k[ij]ab = =− (λβia βjb + µβib βja + µ βik βjk δab ) (3.3) ∂p[j]b p=prest 2 k=1 where β represents the Barycentric coordinates within a tetrahedral element (defining alinear shape function for that element), and the volume of a tetrahedral was computed usingthe triple scalar product: 1 vol = [(m[2] − m[1] ) × (m[3] − m[1] )] · (m[4] − m[1] ) (3.4) 6 where m[ i] is the position in “material” (or object) coordinates of element node i. The damping matrix C is defined as a linear combination of the stiffness and massmatrices (so-called Rayleigh damping): C = α1 K + α2 M (3.5)3.2 Material ParametersThe so-called Lam´ parameters λ and µ define the properties of the material in its elastic estress tensor (assuming the material is isotropic). Our simulation code allows one to tunethese parameters, along with the material density and its Rayleigh damping α coefficients.Example material parameters for various objects can be found in the following table: 5
  7. 7. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project Material λ (Pa) µ (Pa) α1 α2 ρ (Kg/m3 ) Ceramic 3.99 × 109 2.05 × 109 1 × 10−6 10 2700 Plastic 2.49 × 1010 1.28 × 1010 1 × 10−6 50 2700 Aluminum 4.98 × 1010 2.56 × 1010 1 × 10−7 0 2700 Wood 5.00 × 108 1.00 × 108 8 × 10−6 50 750 Metal 4.99 × 1010 2.56 × 1010 1 × 10−7 0 2700 Table 3.1: Example Material Parameters3.3 Modal Synthesis CalculationsModal synthesis is performed by computing the oscillation modes of the finite element mesh.A brief overview of the modal analysis calculations involved follows. Starting with a non-linear mass/spring/damping force system: ˙ ¨ K(d) + C(d, d) + M(d) = f (3.6) The system is linearized (with a loss in accuracy when modelling thin shells withoutmodal coupling) into the following form: ˙ ¨ Kd + Cd + Md = f (3.7) By letting M = LLT (Cholesky factorization) and L−1 KL−T = VΛV−T (Singular ValueDecomposition): ˙ ¨ Λz + (α1 Λ + α2 I)z + z = g (3.8)where g = VT L−1 f . The Rayleigh damping assumption allows the system to be diagonalized into a set ofsecond-order ordinary differential equations: λi zi + (α1 λi + α2 )zi + zi = gi ˙ ¨ (3.9) The analytical solution for each mode was computed to determine its frequency (|Im(ωi )|)and decay rate (Re(ωi )): + − ± −(α1 λi + α2 ) ± (α1 λi + α2 )2 − 4λi zi = c1 etωi + c2 etωi ⇒ ωi = (3.10) 2 Finally, given the c constants for mode reponses to projected impulses: 2∆tgi c1 = + − (3.11) ωi − ωi 2∆tgi c2 = − + (3.12) ωi − ωiThese c constants can be used to find a time-varying oscillator equation for the audio wave-form: 2∆tgi tRe(ωi ) sin(t|Im(ωi )|) zi = e (3.13) |Im(ωi )| 6
  8. 8. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project Thus, an eigenvalue decomposition can be used to find the modal frequency and decayrates of the mass-spring-damper system. Contact forces from the rigid body simulation canbe projected onto the vibration modes (the columns of L−T V). As an optimization, onlythe modes with frequencies within the audible frequency range of the human ear (20 Hz to20 kHz) are used for sound synthesis computations.3.4 Audio Playback Using SDLThe SDL audio library was used for cross-platform audio playback. The summed contributionof each mode’s oscillator equation is used to fill up the audio buffer, which is invoked usinga mix audio() callback function. A queue of modes contributing to the resultant waveformare stored in the STL vector, modeQueue. A mutex lock sound lock protects this queue fromrace conditions. Moreover, a sampling rate of 44,100 Hz was used, and the audio data isstored internally using floating-point numbers until the final mixdown where it is discretizedinto 16-bit signed integers. Special care was taken to avoid clipping by scaling the resultantwaveform so that it lies within the dynamic range of the audio buffer. This approach hasthe weakness that an unappropriately chosen scaling value will still allow clipping to occur,trading off dynamic range for safety against clipping. A smarter approach could be todynamically scale the resultant waveform (like audio compressors in sound engineering).This resultant waveform represents the sound you hear. 7
  9. 9. Chapter 4Rigid Body DynamicsA rigid body dynamics simulation was implemented mostly using the results from [?]. Therigid body dynamics simulation component of this project provided support for Newtonianphysics simulation. The physics engine uses a projected, relaxed Gauss-Seidel solver toanalytically determine contact forces: b = J(ut + ∆tM−1 fext ) (4.1) A = JM−1 JT (4.2) λ = ncp(A, b) (4.3) ut+1 = ut + M−1 JT λ + ∆tM−1 fext (4.4) st+1 = st + ∆tSut+1 (4.5)4.1 Numerical IntegrationThe Euler method of numerical integration is used to timestep the simulation by computingupdated velocities and positions for each body. It is represented in matrix-form as discussedin the Erleben paper. We needed a stable integrator to solve the dynamics equations. Prof.James suggested using a reverse Euler method. This worked fine for our needs. The Time-Corrected Verlet method was also considered for use, along with the fourth-order Runge-Kutta method. xn+1 = xn + f (tn , vn )∆t (4.6) vn+1 = vn + g(tn , xn )∆t (4.7)4.2 Resting Contact4.2.1 Analytical MethodContact forces were computed analytically using a relaxed Gauss-Seidel solver. To improvesimulation stability, I also manually resolved penetration at each time step.4.2.2 Projected Relaxed Gauss-Seidel SolverThe projected Gauss-Seidel solver was implemented using the optimizations suggested in theErleben paper. i−1 n−1 − j=0 Li,j λk+1 − j j=i+1 Ui,j λk − bi j λk+1 i = (4.8) Di,i 8
  10. 10. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project The projection step enforces the friction cone and the linear complementarity problemconstraints: λk+1 = min(max(λloi , λk+1 , λhii ) i i (4.9) We added a relaxation term ω = 0.25 to the iteration step given in the paper.4.3 Collision Detection and Response4.3.1 Collision DetectionInitially, the FreeSOLID library (found at http://www.win.tue.nl/~gino/solid/) wasused for collision detection. However, the Gauss-Seidel solver often exhibited divergence,necessitating the use of Velocity Shock Propagation (VSP) as described in [?]. Unfortunately,the implementation of this algorithm requires knowledge of the penetration depth. For simplecases like ground-sphere and ground-box collisions, this could be determined without any pre-determined knowledge from the FreeSOLID library, but in cases like stacking (where VSP isespecially needed to maintain stability and improve convergence), the FreeSOLID library’slack of a penetration depth result prevented the development of a VSP implementation.Unfortunately, this left us with either the option of implementing our own collision detectionlibrary or using the Open Dynamics Engine (ODE) for our rigid body calculations. We optedfor the latter option.4.3.2 Newton’s Law of RestitutionNewton’s Law of Restitution is used to compute the collision response impulses: AB −(1 + e)v1 · n j= (4.10) n·n 1 MA + 1 MB + I−1 (rAP × n) × rAP + I−1 (rBP × n) × rBP · n A B It is represented in Matrix form as part of the b vector used in the rigid body systemof equations discussed in the Erleben paper. This is an elegant way to add bouncing as acollision response to the simulation.4.3.3 Tangential Coulomb FrictionCoulomb friction was also incorporate in the rigid body solver using extra constraints on λenforced during the projection step as discussed in the Erleben paper: Ff ≤ µFn (4.11) 9
  11. 11. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project4.4 Open Dynamics Engine (ODE) IntegrationThe Open Dynamics Engine was integrated using a wrapper for rigid objects (with tetra-hedral meshes for sound generation). The contact forces, normals, and positions are firstaccumulated in the nearCallback() function. This function creates contact joints as spec-ified in the ODE documentation. Then in the updatePhysicsState() function, the worldis time stepped, and the joint feedback functionality of ODE is used to retrieve the forcesstored in the callback function. The relative velocity and acceleration for the contactingbodies are computed, and if the relative velocity is non-zero, then the bodies are not inresting contact. This means that they are “bouncing,” and so a rigid body contact soundshould be triggered. To prevent artifacts from contact sounds that are triggered too quicklyin succession, a grace time of MIN SOUND REPEAT TIME (set to 0.25 seconds by default) isused. Following the procedure outlined in [?], the contact force is projected onto the modes bytransforming the world space contact position back to object space and finding the nearestvertex in the tetrahedral mesh. This vertex is the excited node, and so the force is appliedto that node in the f vector. From there, the g vector is computed as outlined in section§ 3.3. The mode frequencies and rates are then determined, and if the frequency lies withinthe audible hearing range, a new mode is added to the mode queue for the sound synthesiscode. An interesting curiousity in the Open Dynamics Engine is that its integrator is onlystable with constant time steps. However, one can update the error reduction parameter(ERP) and constraint force mixing (CFM) values as functions of the current time step size∆t, as outlined in [?]: ∆t · k ERP = (4.12) ∆t · k + c 1 CFM = (4.13) ∆t · k + c These updates allow ODE’s numerical integrator to remain stable, even with variably-sized time steps. 10
  12. 12. Chapter 5Graphics EngineMultiple user interfaces were developed for this project. Initially, a testing interface thatused SDL’s 2D drawing capabilities was used to ensure that the modal synthesis code wasworking properly. This test interface featured an on-screen oscilloscope and a visualizationof the excitation of each modal amplitude. Later, OpenGL 3D graphics and rigid bodydynamics were added to visualize the actual rigid body contact sounds being generated.5.1 Test GUIThe test GUI has several features, including the features to generate test modes, produce anA 440 Hz test tone, project test forces onto the tetrahedral mesh, and hear all of the modesat once. Moreover, some of the parameters can be changed on-the-fly without re-evaluatingthe system matrices. A screenshot of the test GUI follows. The red bars represent therelative amplitudes of the excited modes: Figure 5.1: Sound Synthesis GUI5.1.1 Keyboard ControlsA table outlining the basic keyboard controls of the test GUI follows:5.2 OpenGL RendererThe OpenGL renderer allows the user to spawn falling spheres and boxes to demo the rigidbody contact sound functionality of the sound synthesis engine. The 3D camera view can be 11
  13. 13. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project Functionality Key Exit [ESCAPE] Play 20 Highest Frequency Modes [Q] A 440 Hz Test Tone [W] Clear Mode Queue [E] Project Test Force [R] Play All Modes At Once [T] Increase Number of Modes [A] Decrease Number of Modes [S] Increase α1 [Z] Decrease α1 [X] Increase α2 [C] Decrease α2 [V] Increase ρ Scale Factor [B] Decrease ρ Scale Factor [N] Increase Lam´ Scale Factor e [M] Decrease Lam´ Scale Factor e [,] Increase Excited Node Index [.] Decrease Excited Node Index [/] Toggle Full-Screen [TAB] Table 5.1: Test GUI Keyboard Controlsadjusted using keyboard and mouse controls. Moreover, the parameters can be tuned usingthe keyboard controls, much like with the Test GUI. A screenshot of the OpenGL rendererdemonstrating a sample physics environment follows:5.2.1 Keyboard ControlsA table outlining the basic keyboard controls of the OpenGL 3D renderer follows. Note thatthe parameter tuning keyboard controls are identical to the Test GUI interface. Additionally,by dragging with the left mouse button, the viewing direction can be manipulated. 12
  14. 14. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project Figure 5.2: OpenGL 3D Graphics Engine 13
  15. 15. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project Functionality Key Exit [ESCAPE] Camera Forward [W] Camera Backward [S] Camera Left [A] Camera Right [D] Apply Test Force [SPACE] Create Rigid Box [H] Create Rigid Sphere [J] Clear Mode Queue [Q] Increase α1 [Z] Decrease α1 [X] Increase α2 [C] Decrease α2 [V] Increase ρ Scale Factor [B] Decrease ρ Scale Factor [N] Increase Lam´ Scale Factor e [M] Decrease Lam´ Scale Factor e [,] Increase Excited Node Index [.] Decrease Excited Node Index [/] Toggle Full-Screen [TAB] Table 5.2: OpenGL 3D Renderer Keyboard Controls 14
  16. 16. Chapter 6Future PlansSeveral future plans are in place for expanding this project: • Investigate and implement a far-field acoustic radiation model. [?] • Add support for non-linear mode coupling (as in harmonic shells). [?] • Parallelize the sound synthesis engine using GPGPU programming (CUDA or GLSL) or multi-threading. • Investigate and implement fracture sound synthesis. [?] • Implement recent advances in modal synthesis such as contact damping and modal re-distribution. [?] 15
  17. 17. Chapter 7ConclusionThe project was successful in that rigid body contact sounds were able to be generated usingthe model presented in [?]. A flexible user interface was developed, and the results are thatobjects within the virtual 3D environment are able to generate contact sounds accordingly.One can envision applications for real-time rigid body sound synthesis in videogames andanimated movies. The audio in videogames is often limited to playing back a pre-recorded setof sound samples when game events take place. The end result is that the same “glass shat-tering” sample is played regardless of the fracture pattern or the actual in-game shatteringbehavior. Furthermore, sound artists have to resort to manually recording and overdubbingthe audio tracks in films. These processes could be automated if a sufficiently realistic modelfor sound generation is used. 16
  18. 18. Bibliography [1] David Baraff. Fast contact force computation for nonpenetrating rigid bodies. In Pro- ceedings of the 21st annual conference on Computer graphics and interactive techniques, SIGGRAPH ’94, pages 23–34, New York, NY, USA, 1994. ACM. [2] Barringer. Tweaking ode for variable time steps, 2007. [3] Nicolas Bonneel, George Drettakis, Nicolas Tsingos, and Doug James. Fast modal sounds with scalable frequency-domain synthesis, 2008. [4] Jeffrey N. Chadwick, Steven S. An, and Doug L. James. Harmonic shells: a practical nonlinear sound model for near-rigid thin shells. ACM Transactions on Graphics, 28, 2009. [5] Kees Van Den Doel, Paul G. Kry, and Dinesh K. Pai. Foleyautomatic: Physically-based sound effects for interactive simulation and animation. In in Computer Graphics (ACM SIGGRAPH 01 Conference Proceedings, pages 537–544. ACM Press, 2001. [6] Kenny Erleben. Velocity-based shock propagation for multibody dynamics animation. ACM Trans. Graph., 26, June 2007. [7] Gene H. Golub and Charles F. Van Loan. Matrix Computations. The Johns Hopkins University Press, 3rd edition, 1996. [8] Doug L. James, Jernej Barbiˇ, and Dinesh K. Pai. Precomputed acoustic trans- c fer: Output-sensitive, accurate sound generation for geometrically complex vibration sources. ACM Transactions on Graphics (SIGGRAPH 2006), 25(3), August 2006. [9] Danny M. Kaufman. Staggered projections for frictional contact in multibody systems, 2008.[10] James F. O’Brien, Perry R. Cook, and Georg Essl. Synthesizing sounds from physically based motion. In Proceedings of ACM SIGGRAPH 2001, pages 529–536, August 2001.[11] James F. O’Brien and Jessica K. Hodgins. Graphical modeling and animation of brittle fracture. In Proceedings of the 26th annual conference on Computer graphics and in- teractive techniques, SIGGRAPH ’99, pages 137–146, New York, NY, USA, 1999. ACM Press/Addison-Wesley Publishing Co.[12] James F. O’Brien, Chen Shen, and Christine M. Gatchalian. Synthesizing sounds from rigid-body simulations. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, SCA ’02, pages 175–181, New York, NY, USA, 2002. ACM.[13] C´cile Picard, Christian Frisson, Fran¸ois Faure, George Drettakis, and Paul G. Kry. e c Advances in modal analysis using a robust and multiscale method. EURASIP J. Adv. Signal Process, 2010:7:1–7:12, February 2010. 17
  19. 19. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project[14] David E. Stewart. Rigid-body dynamics with friction and impact. SIAM Rev., 42:3–39, March 2000.[15] Changxi Zheng. Tetrahedra generation.[16] Changxi Zheng and Doug L. James. Toward high-quality modal contact sound, 2001.[17] Changxi Zheng and Doug L. James. Rigid-body fracture sound with precomputed soundbanks. ACM Trans. Graph., 29:69:1–69:13, July 2010. 18

×