Perceptually Based Depth-Ordering Enhancement for Direct Volume Rendering
•Download as PPT, PDF•
0 likes•330 views
Slide deck for our new TVCG paper which was presented by Lin Zheng in IEEE Vis 2013.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Perceptually Based Depth-Ordering Enhancement for Direct Volume Rendering
Similar to Perceptually Based Depth-Ordering Enhancement for Direct Volume Rendering (20)
Recently uploaded
Recently uploaded (20)
Perceptually Based Depth-Ordering Enhancement for Direct Volume Rendering
- 1. Perceptually Based Depth-Ordering Enhancement for Direct Volume Rendering Lin Zheng, Yingcai Wu, and Kwan-Liu Ma VIDI Research Group, UC Davis
- 2. Introduction • Depth Perception: • visual ability to perceive the distance of 3D objects. • Depth cues Monocular Cues Occlusions Size Binocular Cues Shading Stereopsis Disparity
- 3. Introduction • In many visualizations, the depth ordering is ambiguous. • If there is no interaction: Neghip • static images on the magazine • posters • Possible approaches: • Perspective projection • Halos, shadows, warm/cool color
- 4. Related Work: Halos • Enhancing Depth-Perception with Flexible Volumetric Halos, Stefan Bruckner and M. Eduard Gröller, 2007 • Depth-Dependent Halos: Illustrative Rendering of Dense Line Data, MH Evert and etc., 2009
- 5. Related Work: Warm/Cool Color • Color Design for Illustrative Visualization, L. Wang, J. Giesen, K.T. McDonnell, P. Zolliker, and K. Mueller, 2008
- 6. Perception Models • Only change the inherent factors: Luminance, opacity • We introduce two major models for depth perception: • X-junction Model • Transmittance Anchoring Principle (TAP) • X-junction Model has limitation • TAP can be a complement
- 7. X-Junction Model: C-configuration • • • • • • • Which layer is in the front A or B? Luminance(s) > Luminance (r) > Luminance (p) > Luminance (q) The luminance decreasing in a “C” configuration.
- 8. X-Junction Model: A-configuration • Which layer is in the front • Luminance (r) = (q) • The Luminance decreasing order can be s>r=q>p • Or s>q=r>p • A-ambiguity
- 9. X-Junction Model: Z-configuration • Luminance s>r>q>p • The luminance decreasing in a “Z”-configuration • Still ambiguous? • + TAP model
- 10. Application of Perception Models • TAP: the highest contrast is perceived to be at the background • Applying X-junction Model and TAP Model. • Improve A-ambiguity to Z-configuration, then to C-configuration A-ambiguity Z-configuration C-configuration
- 11. Energy Function Design • Three terms : depth ordering transparency • Enhance the Perceived Depth Ordering • Keep the Perceived Transparency • Keep the Image Faithfulness image faithfulness
- 12. Energy Function Design • Perceived Depth Ordering:configuration of the junction area • Wrong C-configuration will not appear in semi-transparent structure • Four configurations (in DVR): • Wrong Z-configurations • A configuration (A-ambiguity) • Correct Z-configuration • Correct C-configuration
- 13. Energy Function Design • Perceived Transparency: Metelli’s episcotister model Luminance of transparent layers • Image Faithfulness: Information Entropy Conditional entropy
- 15. User Study • Design: • A between-subjects study (12 subjects) • 60 cases total: 30 enhanced and 30 original • Fisher’s exact test • Users were significantly more accurate in enhanced cases: P-value = 0.0016 task interface
- 16. Results: neghip • Although the difference is subtle, our user study shows that enhancement improves depth perception significantly enhanced initial
- 21. Discussion • + Easy to be embedded in current visualization system • + Luminance as the visual cue: • a primary visual cue in visual psychology • does not introduce additional overhead • - Limitations of perception models: • deal with two overlapping layers at a time • do not work for enclosing and separate structures • consistency problem with intertwined structures
- 22. Conclusion and Future Work • Investigated how to perceptually enhance depth ordering • Used perception models for quantitative measurement • Depth ordering (X-junction Model, TAP) • Image quality (Metelli episcotic Model) • Designed an optimization framework for enhancing depth perception • Conducted a user study showing the effectiveness of our approach • Future work: animation
- 23. Thank you! • This research has been sponsored in part by the US National Science Foundation (NSF) through grant CCF-0811422 and US Department of Energy (DOE) with award DESC0002289. QuickTime™ and a GIF decompressor are needed to see this picture.
Editor's Notes
- Thank you for introduction. Hi every one. I am Lin Zheng, come from vidi research lab. Today I am going to present work with Yingcai wu and Kwan-Liu Ma. This work is about how we employ perceptual theories to volume visualization enhancement.
- Depth perception is important in visualization. The definition of depth perception is the visual ability to perceive the distance of 3D objects. In the real world, we have two major groups of visual cues. One is the Binocular cues, which is not that easy to achieve on traditional 2D screen without 3D glasses. Here I am going to talk about the monocular cues. For the monocular cues, there are occlusions, relative size, motion parallax and shading. These cues are not all available in semi-transparent volume rendering. For example, the motion parallax and relative size.
- In some cases, Direct Volume Rendering fails to produce the correct depth ordering. For example, in the right image, simulated protein molecule dataset. Which part is in the front? The deep purple ball or the two light purple bubbles? The ground truth is that the deep purple ball is in the back. If we can rotate the volume, then it would be easy to figure out the depth ordering. But how about without interaction? With a static image, researchers developed other solutions to alleviate this problem. Perspective projection is a general choice. But it is not enough. Shadows, halos, warm/cool color are also applied to improve the depth perception.
- Here shows the example of adding halos to volume rendering. It is used to convey depth information for opaque surfaces or dense line data. In the top image, it shows a medical dataset which enhance the vessels by adding halos. The bottom two images shows a line dataset. Halos help to show the details. However, this benefit comes at the expense of occluding other structures and therefore it is inappropriate for showing inner semitransparent structures.
- These two images show how cool and warm color affect the perception of front/back ordering. In Image (a), the blue layer is in the front. In Image (b), the red layer is in the front. However, in some cases, the color should be preserved for specific meanings. Like the photographics volume which are used more often in recent medical and biological visualization, the data keeps the natural color of the structures which means the voxel color is predetermined. What we pursue, is to only change the inherent factors like transparency and the luminance.
- Fortunately, we find some perceptual models using only these depth cues in the psychological field and introduce them into volume visualization. They are the X-junction model and the transmittance anchoring principle. To optimize the depth-ordering, we will employ these two models to quantitatively evaluate the quality of depth ordering. In the following slides, I am going to introduce both models. You will see that there is a limitation with only x-junction model. That’s why we add TAP as a complement.
- Here are two layers, A and B. Which one is in the front? Keep answer to yourself and we continue. We can see that the luminance value decrease in clockwise order: s, r, p, and q, which construct a C configuration. Researchers found that with such luminance configuration, it would be easier for human beings to find the correct depth order. The ground truth is that Layer A is in the front. Is it consistent with your result? This theory is called X-junction Model in visual psychology.
- In this image, the luminance value of r and q is equal, the decreasing order can go either way which make the depth information ambiguous. We call it A-configuration or A-ambiguity.
- Z-configuration means the decreasing order of the junction area constructs a “Z” shape. In the illustration it is s, r, q, p. Based on X-junction model, the depth relationship is still ambiguous. That’s why we need the TAP model to step in.
- TAP agrees with X-junction model. Basically, it says Z-configuration is better then A-configuration. I will explain it. The TAP states that, the highest contrast is perceived to be at the background. Because the contrast between the white background and B is higher than the contrast between the white background and A, so in Z-configuration, human beings are able to determine B is in the back of A. So we want to improve from A-ambiguity to Z-configuration, and then to C-configuration.
- There are three terms in the energy function. The first term is for the quality of depth perception. We want to enhance it with the perception models as criteria. The second term is for perceived transparency. We want to keep it stable, not changing too much from the original perceived transparency. The third term is for the image faithfulness. We want to keep it as similar to the original one as possible.
- How we define the energy value of the perceived depth ordering? It depends on the configuration of the junction area. Because we consider about the semi-transparent structures, the wrong c-configuration will not appear. There are 4 configurations, the worst one is the wrong Z-configuration because observers may get wrong depth perception. A-ambiguity is better because at least it does not convey the wrong depth ordering. Correct Z-configuration is even better, but since Z-configuration is weaker than the correct C-configuration, so minimum energy would be assigned to correct C-configuration. The higher the luminance difference are between the junction areas, the stronger C-configuration it is.
- To achieve better depth perception, we changed the transparency perceived by users. However, if we only consider the depth ordering term, the result may turn out to be totally different from the original image. The color and the perceived transparency will be changed too much. So we should add constrains on the changes. For the perceived transparency term, we applied the Metelli’s episcotister model to calculate the perceived transparency. Image faithfulness is another energy term that we establish for optimizing image quality. Because merely preserving visual properties such as transparency is insufficient. Other information like color and shape can also be changed a little during optimization. When we change the luminance, the color may be affected. So we employ the conditional entropy method to measure how faithful the optimized image is to the original version.
- Here is the optimization framework, we first input the initial image with user defined transfer function, detect the x-junction areas. We consider three terms, depth perception, perceived transparency, image faithfulness. We quantify the different terms, adjust the luminance and opacity to obtain the optimal result. If we did not get the best results, we just repeat the loop by adjusting the luminance and the opacity.
- We conduct a between-subjects study by having each subject tested only once per image for either the enhanced or the original result. This is designed to avoid the learning effect. either the enhanced version or the original version was randomly shown. The P-value is significant here, which means people were able to perceive the real depth ordering.
- We are changing the image very little. But it has a large effect on the accuracy of depth perception. This is the whole view, we can see the zoom in view in the following slides.
- Here are some additional results. The difference is expected to be subtle, only the luminance of the overlapping areas are changed. With two images side by side, it can be confused, and that’s why subjects in the user study can see only enhanced or initial version of one data set.
- In the initial image, it is hard to see the real depth order. But in the enhanced version, we can see the small bubbles are in the front.
- This is a turbulent vortex flow dataset. The transparent bubble is in the front of the green balls. In the initial image, this spatial relationship is ambiguous. The perceived depth ordering become more clear.
- This is a brain tumor data set. In the first image, the vessel looks like going through the tumor. Actually it is in front of the tumor. We can see the depth perception has been improved in the second image.
- After seeing the results, let’s discuss the benefits and limitations. Since it only change the luminance and opacity value, it would be easy to be embedded in the current visualization system. We use the luminance as the visual cue because luminance is a primary visual cue in visual psychology. Not like other cues, it does not introduce additional overhead. There also some limitation of these perception models. First is that, they can only deal with two overlapping layers at a time. We can work with the semitransparent layers pair by pair. Second is that, they do not work for enclosing and separate structures because there is not x-junction. The third is a consistency problem. For intertwined structures, one structure might be part in the foreground and part in the background. Our approach based on the individual x-junctions, so it enhances the depth perception locally. It may cause the consistency problem.
- As a conclusion, we have three major contributions. First, we investigate a new problem of how to perceptually enhance the depth ordering of semitransparent structures by adjusting only transparency and luminance. Second, we introduce perception models for quantitative measurement of perceived depth ordering. And we also introduce a new measure to assess the image quality of the final results. Third, we design an optimization framework to do the depth perception enhancement with inherent visual cues. And try to preserve the original perceived transparency and image information. We also conduct a user study which show the effectiveness of our approach. In the future, we are going to apply it to animation by interpolating the parameters between perceptually optimized key frames.
- Questions?