Performance Assessment of Computing Systems: Computer Graphics, Virtual Environment and Animation Designing

1. FDP on
Performance Assessment of Computing Systems:
Computer Graphics, Virtual Reality, Animation Designing
Dr. Suma Dawn,
Dept. of CSE/IT, JIIT Noida

2. Performance Assessment in Computer
Graphics
 Visual Content – intensive
 Quality & Performance: visual impact of the
artefacts introduced by CG techniques
 Artefact visibility Vs. Global Quality [2]
◦ Global Quality Index: Mean- opinion score.
Global level of annoyance (due to artefacts &
distortion
◦ Local visibility of the artefacts: predicting their
spatial localization in the image- auto
corrections
 Objective Vs. Subjective Quality
Assessment: Metrics

3. Performance Assessment in Computer
Graphics
 Reference Vs. No Reference:
◦ Full reference
◦ Reduced reference
◦ No reference
 Image Artefacts Vs. Model Artefacts:
Graphics Pipeline (3D to 2D, rendering,
tone mapping)
 Black-box Metrics Vs. White-box Metrics:
Involve m/c learning techniques, and
attempts to model human visual system
processes.

4. IMAGE QUALITY
METRICS [1]
Physical Simulation of Light : Massive
amount of light particles in a scene –
Huge amount of computation ~
perceptually plausible solution of the
visual system.
Challenges:
How to allocate samples to improve
perceptual quality.
When to stop collecting samples.
• Metric: estimation of error bounds based
on approximation of the final image
• Fidelity metrics : difference between
reference and the test images
• Perceptual metrics: estimating error
bounds based on approximation of the final
image. [3]
Fast GPU methods [4]
Simulation of only direct light (ray casting)
Approximating an image in frequency domain
Texture based
Intermediate rendering results [5]
Consecutive animation frames [6]
Visual Model based
Rendering Metrics

5. IMAGE QUALITY
METRICS [1]
• Visual metrics
Visual difference predictors [7]
Spatio-temporal contrast sensitivity function (CSF) [6]
Colour processing & chromatic (CSF) [8]
Saliency models [9]
• Image decomposition
Wavelets
DCT transform
DOGs
Spatial – sensitivity
Multi-band decomposition
• Just-noticeable-difference
Visual Model based
Rendering Metrics

6. IMAGE QUALITY
METRICS [1]
• Perceptual difference metrics: eg., differences
in colour
Visual Model based
Rendering Metrics
Open source metrics

7. IMAGE QUALITY
METRICS [1]
• Prediction of differences based on a large data
set
Labeling of visible artefacts
Mean-opinion score
Localized distortions maps
• Local statistics (mean, variance, skewness,
kurtosis)
• Deblurring metrics.
Visual Model based
Rendering Metrics
Open source metrics
Data-driven metrics for
rendering

8. IMAGE QUALITY
METRICS [1]
• Assessment of medium – display / print –
luminance difference/ masking
• Visual difference predictor – PSNR, SIM
Visual Model based
Rendering Metrics
Open source metrics
Data-driven metrics for
rendering
High dynamics range
metrics for rendering

9. IMAGE QUALITY
METRICS [1]
• Transferring image representation from
physically accurate units – radiance/ luminance
to pixel values to be displayed on limited range
screen. [10]
• Similarity – histogram distribution
• Contrast-sensitive function
• Loss of visible contrast
• Amplification of invisible contrast
• Contrast reversal.
Visual Model based
Rendering Metrics
Open source metrics
Data-driven metrics for
rendering
High dynamics range
metrics for rendering
Tone-mapping metrics

10. IMAGE QUALITY
METRICS [1]
• Pleasing to the eye??
• Natural looking??
• Subjective assessment & Statistics:
Machine learning techniques to train
predictorVisual Model based
Rendering Metrics
Open source metrics
Data-driven metrics for
rendering
High dynamics range
metrics for rendering
Tone-mapping metrics
Aesthetics and
Naturalness

11. IMAGE QUALITY
METRICS [1]
Performance:
State-of-the-art fidelity metric-
Perceptual models
Texture statistics
Colour differences
Arithmetic differences
Subjective distortion mapsVisual Model based
Rendering Metrics
Open source metrics
Data-driven metrics for
rendering
High dynamics range
metrics for rendering
Tone-mapping metrics
Aesthetics and
Naturalness

12. 3D MODEL QUALITY
METRICS [11]
• Visual quality of 3D Model from different
viewpoints
Mesh Structural Distortion Measure [12]
Local
Global
Tensor-based Perceptual Distance Measure
Diheral Angle Mesh Error
• Visual Equivalence Predictor
• Optimization of textured 3D mesh
Spatio-Temporal Edge Differences
• Geometry-based
comparison

13. 3D MODEL QUALITY
METRICS [11]
Level of Detail
Sarnoff Visual Discrimination Model
Structural Similarity index
• Geometry-based
comparison
• Visual quality of 3D
Model from different
viewpoints

14. 3D MODEL QUALITY
METRICS [11]
Performance
Spearman Rank Order Correlation
Coefficient
Pearson Linear Correction Coefficient
Visual quality experts group-
Cumulative Gaussian Function
Classical Geometric distances
Hausdorff
RMS / mean / max
• Geometry-based
comparison
• Visual quality of 3D
Model from different
viewpoints

15. 3D MODEL
ANIMATION [13]
Artifacts – texture, Lighting & silhouette
(dominant)
 Rendering of local regions containing silhouette areas
from different viewpoints and compare.
 Psychophysical models of visual perception – contras
sensitivity function [14].
 Just Noticeable Difference: CSF + Masking effect [15]• Geometry-based
comparison
• Visual quality of 3D
Model from different
viewpoints

16. 3D MODEL
ANIMATION [13]
Geometry
Texture & other visual attributes
Nature of movement
Velocity
- Additional cognitive phenomena
- Realism
- Exaggeration
• Geometry-based
comparison
• Visual quality of 3D
Model from different
viewpoints

17. 3D MODEL
ANIMATION [13]
Weighted Average : Surface Reconstruction Buffer
• dedicated rasterizer unit ?? , which would efficiently
traverse the bounding rectangle of a surface and identify
the pixels the overlaps - Fragment Shader.
•Accurate accumulation and normalization of attributes cannot
be done in a single pass due to the lack of necessary
blending modes.
•Fragment Test , Depth test, Higher order filtering Vs.
Interpolation for rendering: Kernel size
•The attribute accumulation imposes a heavy burden on frame
buffer caches due to the overlap of splat kernels, and
current caches may not be optimal for the task:
Rasterization Setup
Multi-pass Shader: Deferred Shading [34]
Deferred Rendering [35]
Handling Outlier Vertices
• Geometry-based
comparison
• Visual quality of 3D
Model from different
viewpoints
• Rendering [31], [34]

18. 3D MODEL
ANIMATION [13]
Surfels
1. Interpolatory
2. Triangulation
3. Point Cloud - geometric coherence
4. Smoothness parameters
5.
• Geometry-based
comparison
• Visual quality of 3D
Model from different
viewpoints
• Rendering [31]
• Surface Reconstruction
[32], [33]

19. 3D MODEL
ANIMATION
• Hair Simulation
• Snow simulation
• Fur Simulation
• Cloth Simulation
• Fire Simulations
• Skin Simulation
• Water Simulation

20. 3D MODEL
ANIMATION
1. Degree of freedom
2. Non-Penetration Constraints [40], [41]
3. Friction [43]
4. Collision Detection and Handling [37], [42]
5. Deformation Models [38], [44]
6. Stabilisation: velocity, force, stress,
optimization [39], [41]
• Hair Simulation
• Fur Simulation
• Cloth Simulation
• Fire Simulations
• Water Simulation
[36]

21. Motion Capture & Doodles
 3D, 2D, Deformations
 Marker Based, Image Based
 Motion Imitations [25] – pose
recomputation
 Cursive Motion – Constraints & Timing
[24]
 Gesture Vocabulary
◦ Type of Motion
◦ Complexity
◦ Detailing
◦ Gestures – action lines

22. Motion Capture & Doodles
 Device, Mapping & Retargeting precisions
 Device, Mapping & Retargeting precisions
[26]

23. Facial Animation
• Real-time tracking [27]
• Geometry , Texture, Prior
Learning
• Acquisition system.
• Depth Adjustment [28]
• Precision
• Velocity & Occlusion [29]
• Photo-realistic synthesis
[30]

24. Collaborative Virtual Environments
 Standards:
◦ Rendering and Graphics
◦ Communications Middleware
 Performance [16]: delay
 display – the “on-screen” delay created by monitors, projectors, stereo
goggles, and other displaying equipment;
 graphics – the delay caused by the graphics card/drivers and other graphical
component of the system;
 simulation: the lag introduced as a result of processing/computing performed
by the core multi-user engine hardware and software;
 Interconnections: networking and communication delays (client-to-client
delay)
 Object Insertion Delay- how long it takes for a newly created object to
appear on every participant’s screen
 Framerate measures how quickly a user can navigate and interact with
objects from a graphical stand-point.
 Ownership Transfer Delay: only one person at a time can interact with a
shared entity in order to avoid event collision and unwanted conflicts [17].

25. Virtual Environments [47], [50]
• Game Engine dependencies
• Monitoring Intersections, triggers and strategies [51]
• Display technologies, sensory modalities & their Fidelity
[44]
• Presence [45]
• Distance [53] & Auditory Perception [46], [49]
• Peripheral Devices and technologies [51]: Immersive
Visualisation, HCI Technologies;
• Frame Rate, Texture Pixels, [52]

26. Distributed Multimedia Server [23]
 Rendering of Continuous Media
 Frame streaming and consumption
rates : Variable Bit Rate, Constant Bit
Rate, Buffer size
 Layout of blocks which compose the
various multimedia objects are laid out
across the disks in the system: Striping
versus Random Layout
 Stating , Fault Tolerance.

27. Improving Multimedia
Performance [18]
• Multimedia Extensions: Include specialized instructions in general-
purpose processors that are optimized for typical multimedia applications:
media instructions
• Enhance compiler to analyze the program for superword-level parallelism,
short groups of statements performing the same operation that could be
grouped and replaced by a single media instruction. (use isomorphism)
• High-level libraries – Designing primitives and abstractions
• Optimization of dominated by small, tight loops
• High memory & network Bandwidth

28. Benchmarks
Benchmarks used for performance
evaluation of computers should be
representative of applications that are run
on actual systems. Contemporary
computer applications include a variety of
applications, and different benchmarks are
appropriate for systems targeted for
different purposes.
 Multimedia Embedded Digital Signal
Processing
 MediaBench [20]
 EEMBC benchmarks [21]
 BDTI benchmarks [22]

29. Current Research …
CG VE
• Simulations: Healthcare, Medical,
Education
• Creativity, Aesthetics – art,
history, media, culture
• Behaviour & Sociology
• Big-Data, Machine Learning,
Deep Learning
• Crowd Sourcing

30. Centers for Research in
Animations & VE

31. REFERENCES
Thank you

32. QUESTIONS ??
Thank you