Downloaded 17 times
Uploaded byMohamed Elshenawy
Fast texture synthesis using tree structured vector quantization
This document proposes a fast texture synthesis algorithm using tree-structured vector quantization (TSVQ) that can generate textures in 3 sentences or less that match an input texture efficiently. It models textures based on local spatial neighborhoods and uses an exhaustive search-based algorithm. By representing texture patches in a tree structure, it achieves a speedup of two orders of magnitude compared to previous methods, synthesizing textures in 12 seconds versus over 30 minutes for other algorithms. The method produces results that look like the input texture while being efficient and extensible to applications like image editing and temporal texture synthesis.
More Related Content
Similar to Fast texture synthesis using tree structured vector quantization
Fast texture synthesis using tree structured vector quantization
- 1. FAST TEXTURE SYNTHESISUSING TREE- STRUCTURED VECTOR QUANTIZATION By : Mohamed Mahmoud El-shenawy
- 2. INTRODUCTION Texture Synthesis Input Result
- 3. DESIRABLE PROPERTIES • Resultlooks like the input • Efficient • General • Easy to use • Extensible
- 4. TEXTURE • Texture analysisand synthesis is very important for computer graphics, vision, and image processing • Texture can describe a wide variety of surface characteristics such as terrain , plants, minerals, fur and skin.
- 5. TEXTURE MODEL • Texturesare • local • stationary • Model textures by • local spatial neighborhoods
- 6. BASIC ALGORITHM • Basedon the assumptions of locality and stationarity, our algorithm uses a simple exhaustive search for texture synthesis.
- 7. BENEFIT • Better imagequality & faster computation 1 level 1 level 3 levels 5 5 11 11 5 5
- 8. RESULTS Random Oriented Regular Semi-regular
- 9. FAILURES • Non-planar structures •Global information
- 10. COMPARISON Input 12 secs ! Heeger 95 De Bonet 97 Efros 99 Our method 1941 secs
- 11. ACCELERATION • Computation bottleneck:neighborhood search Input
- 12. TIMING Time complexity :O(log N) instead of O(N) • 2 orders of magnitude speedup for non-trivial images Efros 99 Full searching TSVQ 1941 secs 503 secs 12 secs
- 13.
- 14. IMAGE EDITING BYTEXTURE REPLACEMENT
- 15. TEMPORAL TEXTURE Input Result Fire Smoke Waves
Editor's Notes
- #2 I am going to talk about some work done by , Li-Yi Wei, and Marc Levoyat Stanford University , on a new algorithm for fast texture synthesis using tree-structuredvector quantization.
- #3 Textures are images containing repeating patterns, with the possbility of random variations. An example is shown in the figure.Given (such) an input texture image, we would like to have an algorithm that could synthesize an output texture automatically. The result texture should look like the input, and the user should be able to specify the size of the result. In addition, the result texture should be tileable ( to avoid visible discontinuities for texture mapping or background images).*************************************************Say a few about the definition of textures. In particular, we are refering to textures as images containing some repeating patterns, not the general notion of "textures" in texture mapping where a texture can be an arbitrary image.We can define texture in general form as Texture is the surface quality of an object, and what gives a design the feeling of a surface. Another way to put this is how to make a design more realistic or give it depth and detail of any kind.
- #4 There are several desirable properties for a texture synthesis algorithm.First, the generated result should look like the input texture.The algorithm should also be easy to use , in the sense that it shouldn't require a user to tune 100 parameters to get the result right.The algorithm should also run reasonably fast. In particular, if it is fast enough for real time texture synthesis, we can avoid a lot of problems for texture memory and bandwidth.Finally, the algorithm should be flexible enough so that extensions to other applications can be easily done.
- #6 As we have seen in previous slides, textures are images with repeating structures. More specifically they have 2 important properties, called locaity and stationarity. A texture image is local because each pixel is visually related to a small set of neighborhood pixels, as denoted by the black squares shown in the figures. A texture is stationary in the sense that, if we slide the window around, the content of the windows will change, but appears to be visually similar.
- #7 Our texture generation process takes an example texture patch (left) and a random noise (middle) as input, and modifies this random noise to makeit look like the given example texture. The synthesized texture (right) can be of arbitrary size, and is perceived as very similar to the given example. Using ouralgorithm, textures can be generated within seconds, and the synthesized results are always tileable.We will now illustrate our algorithm with a simple example. An input texture image is given, and the user specify the size of the result image. We initialize the output as a white random noise. We then assign the values of the output texture in a scan line ordering, until all the output pixels are determined.To start with, let's try to determine the value of the first pixel, which is at the upper-left corner of the output texture. It's the target pixel for synthesis, and is indicated by a yellow boundary.Because of the locality of texture images, we use a small neighborhood to determine the pixel value.To make sure the result is tileable, neighborhoods which cross the boundaries will be constructed toroidally. For example, the pixel above the target pixel will come from the bottom of the noise. The other neighborhood pixels are built in a similar fashion.After we determine the output neighborhood, we compare it with all possible neighborhoods in the input, with the same size and shape. We choose the one which is most similar to the output neighborhood, and assigns the central pixel to the output. We use a simple L2 norm to determine the similarity between neighborhood templates.This same process is repeated for each output pixel in a scan line order. Note that for a pixel around the middle of the output, such as the one shown in the figure, the neighborhood contains only already assigned pixels. Actually, the noise is only used during the synthesis of the first few rows and columns of the output texture. It's ignored for most of the other pixels.
- #8 For example, using a single resolution algorithm with a 5x5 neighborhood, the result will not look good. We have to extend the neighborhood to 11x11 for the single resolution case. However, using still a 5x5 neighborhoods, we can have satisfiable result with a multiresolution algorithm. We use 3 pyramid levels in this case.
- #9 Here there are 4 typical examples of textures with different flavors here. All 4 textures are generated using 4 level Gaussian pyramids, with neighborhood size 9x9 at the top level.Even though the textures contain different scales and orientations of structures, the results look reasonable,even though we use the same neighborhoods for all cases.In addition, all the textures are tileable, so they can be used for texture mapping or background images without visual discontinuities.
- #10 Becausetextures here are characterized as local and stationary, the algorithm will fail in cases when the assumption fails. For example, since the algorithm use 2D neighborhoods only, it won't work for 3D structures, such as a group pumpkins.It also won't preserve global structures, such as the perspectivity of a building exterior. Only the windows are preserved, but the global arrangement is gone.
- #11 A comparison of our approach with several previous works is shown here.[Heeger 95] uses marginal histograms to synthesize textures. It works well for stochastic textures, but is less effective for structured ones.[Debonet 97] improved on this by considering cross scale dependencies of texture features. Though it preserves the grid structures better than [Heeger 95], visible blocking discontinuities are visible. This is because the lateral relationship between pixels at the same level is not considered.Recently, [Efros 99] proposes a simple algorithm based on Markov Random Fields. The algorithm is able to produce very good results, at the expense of long computation time.The synthesis quality of our results is comparable to Efros. However, exhaustive search is too slow for texture synthesis. In the rest of the talk, we demonstrate how we accelerate the synthesis process without noticeable quality degradation using Tree-structured VQ.
- #12 fortunately, this neighborhood searching process can be easily accelerated.Let's reconsider the synthesis algorithm, this time from the point of view of a nearest point searching problem.The input of the algorithm consists of a collection of input neighborhoods, collected from an input texture.for each output pixel, we use its neighborhood to find the best matched ones from the input. We use an exhaustive searching in the basic algorithm.Because we can consider those neighborhoods as points in a high dimensional space, we can use any nearest point searching algorithm in the literature for acceleration.
- #13 Because of tree traversal has time complexity O(Log N) instead of O(N), the synthesis process can be greatly accelerated. A comparison of image quality with TSVQ acceleration is shown in this slide. Both [Efros 99] and full searching take a lot of time to compute. However, using TSVQ as the acceleration, the same texture can be synthesized with much less computation. for non-trivially sized images, we can have at least 2 orders of magnitude acceleration over full searching and traditional MRF approaches.
- #14 In addition to pure texture synthesis, several applications are possible with out algorihtm.One example is constrained synthesis. For example, given a texture region with a hole inside, how can we fill in the hole so that it looks naturally blended with the existing textures?
- #15 or texture replacement for real world scenes. In this case, the black spot light on the roof is replaced by the shingles texture.
- #16 Several examples of temporal textures are shown below.(show the animations)talk about artifacts while showing them.Because we model textures as local and stationary, we are not able to handle situations, such as the semi-transparency of the fire, and the large scale wave motion.Still, the results preserve most of the textural motions.