1. Cornell University
CS 6650: Computational Motion Semester Project
Rigid Body Sound Synthesis
Authors:
Supervisor:
Charles Moyes
Dr. Doug L. James
Shentong Wang
May 16, 2011
3. Chapter 1
Abstract
The basic idea of the project is to implement a rigid body sound synthesis engine in C++
using OpenGL and a rigid body dynamics simulation. Modal synthesis in the frequency-
domain will be used to generate rigid body contact sounds based on the contact forces and
positions applied to objects in the simulation. The Armadillo linear algebra library will be
used to perform routine matrix decompositions such as Cholesky factorizations and singular
value decompositions (SVDs). The SDL library will be used for audio output and graphical
rendering. Another goal is to build an interesting simulation to show off the applications of
the engine.
Figure 1.1: Proof-of-concept image from [?].
2
4. Chapter 2
Math Library
The project required linear algebra routines for forming the mass and stiffness matrices,
along with solving the eigenvalue problem associated with modal analysis. For this project,
we implemented our own dense and sparse matrix classes, along with using the Armadillo
linear algebra library (found at http://arma.sourceforge.net/) and the SVDLIBC sparse
singular value decomposition library (found at http://tedlab.mit.edu/~dr/SVDLIBC/).
We have implemented both a sparse and a dense matrix solver. We found that the dense
solver works best with smaller tetrahedral meshes.
2.1 Dense Solver
The dense solver was implemented first using the Armadillo library to have a metric for
testing the sparse matrix routines. Meanwhile, the sparse solver was developed concurrently.
2.2 Armadillo Integration
Armadillo was used for the dense solver for computing the Cholesky decomposition of the
mass matrix. For the sparse matrix solver, we generate a diagonal “lumped” mass ma-
trix, avoiding the need to perform an explicit Cholesky decomposition. In this case, the
Cholesky decomposition is trivially the square root of the diagonal values of the matrix.
The Eigen linear algebra library (http://eigen.tuxfamily.org/) was also investigated
for this project, but we found that the Armadillo API was easier to use with our code.
2.3 Sparse Matrix Library
Our sparse matrix library initially handled both static and dynamic matrices. We used a
storage format very similar to the Harwell-Boeing format. Column indices separate an array
of non-zero values into partitions, where each group resides in a different column. Within
each column, each non-zero value in the sparse matrix has a row index associated with it.
Very thorough test batteries were written to ensure the correctness of this code in order to
prevent tedious debugging further down the road.
2.4 SVDLIBC C Library
At first, we tried developing our own sparse matrix eigenvalue decomposition code, but
after consulting with Charles Van Loan and David Bindel, we decided that this endeavor
would be a separate project in itself. The SVDLIBC library was used for performing
sparse eigenvalue decompositions. It uses the Lanczos algorithm for computing the singular
3
5. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
value decomposition, and it handles both dense and sparse matrices. Code was written
to generate sparse matrices in the Harwell-Boeing binary format for use with this library.
Several Fortran libraries were investigated, including PROPACK, TRLan, and LAPACK.
We found that SVDLIBC was the easiest to use, so we went with it instead.
4
6. Chapter 3
Sound Synthesis
3.1 Finite Element Model Matrix Formation
We used [?] as a reference for constructing the matrices for tetrahedral finite element models.
We discretize our mesh into tetrahedron in order to perform modal analysis. The mesh ver-
tices and tetrahedron indices were loaded directly from binary files generated using Changxi
Zheng’s IsoStuffer utility [?]. We used linear basis functions as described in the paper.
We wrote code to assemble the mass and stiffness matrices as suggested. The element mass
matrix M was computed as:
∂ 2κ ρ vol
m[ij]ab = = (1 + δij )δab (3.1)
∂ p[i]a p[j]b
˙ ˙ 20
and the “lumped” mass matrix was formed as follows:
ρ vol
mlumped =
[ij]ab δij δab (3.2)
4
The stiffness matrix K was formed using:
3
∂f[i]a vol
k[ij]ab = =− (λβia βjb + µβib βja + µ βik βjk δab ) (3.3)
∂p[j]b p=prest 2 k=1
where β represents the Barycentric coordinates within a tetrahedral element (defining a
linear shape function for that element), and the volume of a tetrahedral was computed using
the triple scalar product:
1
vol = [(m[2] − m[1] ) × (m[3] − m[1] )] · (m[4] − m[1] ) (3.4)
6
where m[ i] is the position in “material” (or object) coordinates of element node i.
The damping matrix C is defined as a linear combination of the stiffness and mass
matrices (so-called Rayleigh damping):
C = α1 K + α2 M (3.5)
3.2 Material Parameters
The so-called Lam´ parameters λ and µ define the properties of the material in its elastic
e
stress tensor (assuming the material is isotropic). Our simulation code allows one to tune
these parameters, along with the material density and its Rayleigh damping α coefficients.
Example material parameters for various objects can be found in the following table:
5
7. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Material λ (Pa) µ (Pa) α1 α2 ρ (Kg/m3 )
Ceramic 3.99 × 109 2.05 × 109 1 × 10−6 10 2700
Plastic 2.49 × 1010 1.28 × 1010 1 × 10−6 50 2700
Aluminum 4.98 × 1010 2.56 × 1010 1 × 10−7 0 2700
Wood 5.00 × 108 1.00 × 108 8 × 10−6 50 750
Metal 4.99 × 1010 2.56 × 1010 1 × 10−7 0 2700
Table 3.1: Example Material Parameters
3.3 Modal Synthesis Calculations
Modal synthesis is performed by computing the oscillation modes of the finite element mesh.
A brief overview of the modal analysis calculations involved follows. Starting with a non-
linear mass/spring/damping force system:
˙ ¨
K(d) + C(d, d) + M(d) = f (3.6)
The system is linearized (with a loss in accuracy when modelling thin shells without
modal coupling) into the following form:
˙ ¨
Kd + Cd + Md = f (3.7)
By letting M = LLT (Cholesky factorization) and L−1 KL−T = VΛV−T (Singular Value
Decomposition):
˙ ¨
Λz + (α1 Λ + α2 I)z + z = g (3.8)
where g = VT L−1 f .
The Rayleigh damping assumption allows the system to be diagonalized into a set of
second-order ordinary differential equations:
λi zi + (α1 λi + α2 )zi + zi = gi
˙ ¨ (3.9)
The analytical solution for each mode was computed to determine its frequency (|Im(ωi )|)
and decay rate (Re(ωi )):
+ −
± −(α1 λi + α2 ) ± (α1 λi + α2 )2 − 4λi
zi = c1 etωi + c2 etωi ⇒ ωi = (3.10)
2
Finally, given the c constants for mode reponses to projected impulses:
2∆tgi
c1 = + − (3.11)
ωi − ωi
2∆tgi
c2 = − + (3.12)
ωi − ωi
These c constants can be used to find a time-varying oscillator equation for the audio wave-
form:
2∆tgi tRe(ωi ) sin(t|Im(ωi )|)
zi = e (3.13)
|Im(ωi )|
6
8. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Thus, an eigenvalue decomposition can be used to find the modal frequency and decay
rates of the mass-spring-damper system. Contact forces from the rigid body simulation can
be projected onto the vibration modes (the columns of L−T V). As an optimization, only
the modes with frequencies within the audible frequency range of the human ear (20 Hz to
20 kHz) are used for sound synthesis computations.
3.4 Audio Playback Using SDL
The SDL audio library was used for cross-platform audio playback. The summed contribution
of each mode’s oscillator equation is used to fill up the audio buffer, which is invoked using
a mix audio() callback function. A queue of modes contributing to the resultant waveform
are stored in the STL vector, modeQueue. A mutex lock sound lock protects this queue from
race conditions. Moreover, a sampling rate of 44,100 Hz was used, and the audio data is
stored internally using floating-point numbers until the final mixdown where it is discretized
into 16-bit signed integers. Special care was taken to avoid clipping by scaling the resultant
waveform so that it lies within the dynamic range of the audio buffer. This approach has
the weakness that an unappropriately chosen scaling value will still allow clipping to occur,
trading off dynamic range for safety against clipping. A smarter approach could be to
dynamically scale the resultant waveform (like audio compressors in sound engineering).
This resultant waveform represents the sound you hear.
7
9. Chapter 4
Rigid Body Dynamics
A rigid body dynamics simulation was implemented mostly using the results from [?]. The
rigid body dynamics simulation component of this project provided support for Newtonian
physics simulation. The physics engine uses a projected, relaxed Gauss-Seidel solver to
analytically determine contact forces:
b = J(ut + ∆tM−1 fext ) (4.1)
A = JM−1 JT (4.2)
λ = ncp(A, b) (4.3)
ut+1 = ut + M−1 JT λ + ∆tM−1 fext (4.4)
st+1 = st + ∆tSut+1 (4.5)
4.1 Numerical Integration
The Euler method of numerical integration is used to timestep the simulation by computing
updated velocities and positions for each body. It is represented in matrix-form as discussed
in the Erleben paper. We needed a stable integrator to solve the dynamics equations. Prof.
James suggested using a reverse Euler method. This worked fine for our needs. The Time-
Corrected Verlet method was also considered for use, along with the fourth-order Runge-
Kutta method.
xn+1 = xn + f (tn , vn )∆t (4.6)
vn+1 = vn + g(tn , xn )∆t (4.7)
4.2 Resting Contact
4.2.1 Analytical Method
Contact forces were computed analytically using a relaxed Gauss-Seidel solver. To improve
simulation stability, I also manually resolved penetration at each time step.
4.2.2 Projected Relaxed Gauss-Seidel Solver
The projected Gauss-Seidel solver was implemented using the optimizations suggested in the
Erleben paper.
i−1 n−1
− j=0 Li,j λk+1 −
j j=i+1 Ui,j λk − bi
j
λk+1
i = (4.8)
Di,i
8
10. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
The projection step enforces the friction cone and the linear complementarity problem
constraints:
λk+1 = min(max(λloi , λk+1 , λhii )
i i (4.9)
We added a relaxation term ω = 0.25 to the iteration step given in the paper.
4.3 Collision Detection and Response
4.3.1 Collision Detection
Initially, the FreeSOLID library (found at http://www.win.tue.nl/~gino/solid/) was
used for collision detection. However, the Gauss-Seidel solver often exhibited divergence,
necessitating the use of Velocity Shock Propagation (VSP) as described in [?]. Unfortunately,
the implementation of this algorithm requires knowledge of the penetration depth. For simple
cases like ground-sphere and ground-box collisions, this could be determined without any pre-
determined knowledge from the FreeSOLID library, but in cases like stacking (where VSP is
especially needed to maintain stability and improve convergence), the FreeSOLID library’s
lack of a penetration depth result prevented the development of a VSP implementation.
Unfortunately, this left us with either the option of implementing our own collision detection
library or using the Open Dynamics Engine (ODE) for our rigid body calculations. We opted
for the latter option.
4.3.2 Newton’s Law of Restitution
Newton’s Law of Restitution is used to compute the collision response impulses:
AB
−(1 + e)v1 · n
j= (4.10)
n·n 1
MA
+ 1
MB
+ I−1 (rAP × n) × rAP + I−1 (rBP × n) × rBP · n
A B
It is represented in Matrix form as part of the b vector used in the rigid body system
of equations discussed in the Erleben paper. This is an elegant way to add bouncing as a
collision response to the simulation.
4.3.3 Tangential Coulomb Friction
Coulomb friction was also incorporate in the rigid body solver using extra constraints on λ
enforced during the projection step as discussed in the Erleben paper:
Ff ≤ µFn (4.11)
9
11. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
4.4 Open Dynamics Engine (ODE) Integration
The Open Dynamics Engine was integrated using a wrapper for rigid objects (with tetra-
hedral meshes for sound generation). The contact forces, normals, and positions are first
accumulated in the nearCallback() function. This function creates contact joints as spec-
ified in the ODE documentation. Then in the updatePhysicsState() function, the world
is time stepped, and the joint feedback functionality of ODE is used to retrieve the forces
stored in the callback function. The relative velocity and acceleration for the contacting
bodies are computed, and if the relative velocity is non-zero, then the bodies are not in
resting contact. This means that they are “bouncing,” and so a rigid body contact sound
should be triggered. To prevent artifacts from contact sounds that are triggered too quickly
in succession, a grace time of MIN SOUND REPEAT TIME (set to 0.25 seconds by default) is
used.
Following the procedure outlined in [?], the contact force is projected onto the modes by
transforming the world space contact position back to object space and finding the nearest
vertex in the tetrahedral mesh. This vertex is the excited node, and so the force is applied
to that node in the f vector. From there, the g vector is computed as outlined in section
§ 3.3. The mode frequencies and rates are then determined, and if the frequency lies within
the audible hearing range, a new mode is added to the mode queue for the sound synthesis
code.
An interesting curiousity in the Open Dynamics Engine is that its integrator is only
stable with constant time steps. However, one can update the error reduction parameter
(ERP) and constraint force mixing (CFM) values as functions of the current time step size
∆t, as outlined in [?]:
∆t · k
ERP = (4.12)
∆t · k + c
1
CFM = (4.13)
∆t · k + c
These updates allow ODE’s numerical integrator to remain stable, even with variably-
sized time steps.
10
12. Chapter 5
Graphics Engine
Multiple user interfaces were developed for this project. Initially, a testing interface that
used SDL’s 2D drawing capabilities was used to ensure that the modal synthesis code was
working properly. This test interface featured an on-screen oscilloscope and a visualization
of the excitation of each modal amplitude. Later, OpenGL 3D graphics and rigid body
dynamics were added to visualize the actual rigid body contact sounds being generated.
5.1 Test GUI
The test GUI has several features, including the features to generate test modes, produce an
A 440 Hz test tone, project test forces onto the tetrahedral mesh, and hear all of the modes
at once. Moreover, some of the parameters can be changed on-the-fly without re-evaluating
the system matrices. A screenshot of the test GUI follows. The red bars represent the
relative amplitudes of the excited modes:
Figure 5.1: Sound Synthesis GUI
5.1.1 Keyboard Controls
A table outlining the basic keyboard controls of the test GUI follows:
5.2 OpenGL Renderer
The OpenGL renderer allows the user to spawn falling spheres and boxes to demo the rigid
body contact sound functionality of the sound synthesis engine. The 3D camera view can be
11
13. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Functionality Key
Exit [ESCAPE]
Play 20 Highest Frequency Modes [Q]
A 440 Hz Test Tone [W]
Clear Mode Queue [E]
Project Test Force [R]
Play All Modes At Once [T]
Increase Number of Modes [A]
Decrease Number of Modes [S]
Increase α1 [Z]
Decrease α1 [X]
Increase α2 [C]
Decrease α2 [V]
Increase ρ Scale Factor [B]
Decrease ρ Scale Factor [N]
Increase Lam´ Scale Factor
e [M]
Decrease Lam´ Scale Factor
e [,]
Increase Excited Node Index [.]
Decrease Excited Node Index [/]
Toggle Full-Screen [TAB]
Table 5.1: Test GUI Keyboard Controls
adjusted using keyboard and mouse controls. Moreover, the parameters can be tuned using
the keyboard controls, much like with the Test GUI. A screenshot of the OpenGL renderer
demonstrating a sample physics environment follows:
5.2.1 Keyboard Controls
A table outlining the basic keyboard controls of the OpenGL 3D renderer follows. Note that
the parameter tuning keyboard controls are identical to the Test GUI interface. Additionally,
by dragging with the left mouse button, the viewing direction can be manipulated.
12
14. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Figure 5.2: OpenGL 3D Graphics Engine
13
15. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Functionality Key
Exit [ESCAPE]
Camera Forward [W]
Camera Backward [S]
Camera Left [A]
Camera Right [D]
Apply Test Force [SPACE]
Create Rigid Box [H]
Create Rigid Sphere [J]
Clear Mode Queue [Q]
Increase α1 [Z]
Decrease α1 [X]
Increase α2 [C]
Decrease α2 [V]
Increase ρ Scale Factor [B]
Decrease ρ Scale Factor [N]
Increase Lam´ Scale Factor
e [M]
Decrease Lam´ Scale Factor
e [,]
Increase Excited Node Index [.]
Decrease Excited Node Index [/]
Toggle Full-Screen [TAB]
Table 5.2: OpenGL 3D Renderer Keyboard Controls
14
16. Chapter 6
Future Plans
Several future plans are in place for expanding this project:
• Investigate and implement a far-field acoustic radiation model. [?]
• Add support for non-linear mode coupling (as in harmonic shells). [?]
• Parallelize the sound synthesis engine using GPGPU programming (CUDA or GLSL)
or multi-threading.
• Investigate and implement fracture sound synthesis. [?]
• Implement recent advances in modal synthesis such as contact damping and modal
re-distribution. [?]
15
17. Chapter 7
Conclusion
The project was successful in that rigid body contact sounds were able to be generated using
the model presented in [?]. A flexible user interface was developed, and the results are that
objects within the virtual 3D environment are able to generate contact sounds accordingly.
One can envision applications for real-time rigid body sound synthesis in videogames and
animated movies. The audio in videogames is often limited to playing back a pre-recorded set
of sound samples when game events take place. The end result is that the same “glass shat-
tering” sample is played regardless of the fracture pattern or the actual in-game shattering
behavior. Furthermore, sound artists have to resort to manually recording and overdubbing
the audio tracks in films. These processes could be automated if a sufficiently realistic model
for sound generation is used.
16
18. Bibliography
[1] David Baraff. Fast contact force computation for nonpenetrating rigid bodies. In Pro-
ceedings of the 21st annual conference on Computer graphics and interactive techniques,
SIGGRAPH ’94, pages 23–34, New York, NY, USA, 1994. ACM.
[2] Barringer. Tweaking ode for variable time steps, 2007.
[3] Nicolas Bonneel, George Drettakis, Nicolas Tsingos, and Doug James. Fast modal
sounds with scalable frequency-domain synthesis, 2008.
[4] Jeffrey N. Chadwick, Steven S. An, and Doug L. James. Harmonic shells: a practical
nonlinear sound model for near-rigid thin shells. ACM Transactions on Graphics, 28,
2009.
[5] Kees Van Den Doel, Paul G. Kry, and Dinesh K. Pai. Foleyautomatic: Physically-based
sound effects for interactive simulation and animation. In in Computer Graphics (ACM
SIGGRAPH 01 Conference Proceedings, pages 537–544. ACM Press, 2001.
[6] Kenny Erleben. Velocity-based shock propagation for multibody dynamics animation.
ACM Trans. Graph., 26, June 2007.
[7] Gene H. Golub and Charles F. Van Loan. Matrix Computations. The Johns Hopkins
University Press, 3rd edition, 1996.
[8] Doug L. James, Jernej Barbiˇ, and Dinesh K. Pai. Precomputed acoustic trans-
c
fer: Output-sensitive, accurate sound generation for geometrically complex vibration
sources. ACM Transactions on Graphics (SIGGRAPH 2006), 25(3), August 2006.
[9] Danny M. Kaufman. Staggered projections for frictional contact in multibody systems,
2008.
[10] James F. O’Brien, Perry R. Cook, and Georg Essl. Synthesizing sounds from physically
based motion. In Proceedings of ACM SIGGRAPH 2001, pages 529–536, August 2001.
[11] James F. O’Brien and Jessica K. Hodgins. Graphical modeling and animation of brittle
fracture. In Proceedings of the 26th annual conference on Computer graphics and in-
teractive techniques, SIGGRAPH ’99, pages 137–146, New York, NY, USA, 1999. ACM
Press/Addison-Wesley Publishing Co.
[12] James F. O’Brien, Chen Shen, and Christine M. Gatchalian. Synthesizing sounds from
rigid-body simulations. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics
symposium on Computer animation, SCA ’02, pages 175–181, New York, NY, USA,
2002. ACM.
[13] C´cile Picard, Christian Frisson, Fran¸ois Faure, George Drettakis, and Paul G. Kry.
e c
Advances in modal analysis using a robust and multiscale method. EURASIP J. Adv.
Signal Process, 2010:7:1–7:12, February 2010.
17
19. Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
[14] David E. Stewart. Rigid-body dynamics with friction and impact. SIAM Rev., 42:3–39,
March 2000.
[15] Changxi Zheng. Tetrahedra generation.
[16] Changxi Zheng and Doug L. James. Toward high-quality modal contact sound, 2001.
[17] Changxi Zheng and Doug L. James. Rigid-body fracture sound with precomputed
soundbanks. ACM Trans. Graph., 29:69:1–69:13, July 2010.
18
