0
Outline
                      Ph.D. Research Work
  Conclusion and Possible Future Directions




 Fast NMRF based texture...
Outline
                                                        Earlier Methods
                                Ph.D. Rese...
Outline
                                                            Earlier Methods
                                    Ph...
Outline
                                            Earlier Methods
                    Ph.D. Research Work
              ...
Outline
                                                                               Earlier Methods
                   ...
Outline
                                                                Earlier Methods
                                  ...
Outline
                                                          Earlier Methods
                               Ph.D. Res...
Outline
                                                              Earlier Methods
                                Ph.D...
Outline
                                                            Earlier Methods
                                Ph.D. ...
Outline
                                                                 Earlier Methods
                               Ph...
Outline
                                                                                    Earlier Methods
              ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                               Outline
                              ...
Order Estimation from Fourier Domain
                                  Outline
                                           ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                              Outline
                               ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                               Outline
                              ...
Order Estimation from Fourier Domain
                                        Outline
                                     ...
Order Estimation from Fourier Domain
                                                    Outline
                         ...
Order Estimation from Fourier Domain
                                              Outline
                               ...
Order Estimation from Fourier Domain
                                           Outline
                                  ...
Order Estimation from Fourier Domain
                                                                          Outline
   ...
Order Estimation from Fourier Domain
                                           Outline
                                  ...
Order Estimation from Fourier Domain
                                               Outline
                              ...
Order Estimation from Fourier Domain
                                                     Outline
                        ...
Order Estimation from Fourier Domain
                                                     Outline
                        ...
Order Estimation from Fourier Domain
                                                     Outline
                        ...
Order Estimation from Fourier Domain
                                     Outline
                                        ...
Order Estimation from Fourier Domain
                                          Outline
                                   ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                     Outline
                                        ...
Order Estimation from Fourier Domain
                                  Outline
                                           ...
Order Estimation from Fourier Domain
                                  Outline
                                           ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                            Outline
                                 ...
Order Estimation from Fourier Domain
                                                  Outline
                           ...
Order Estimation from Fourier Domain
                                            Outline
                                 ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                             Outline
                                ...
Order Estimation from Fourier Domain
                                  Outline
                                           ...
Order Estimation from Fourier Domain
                                  Outline
                                           ...
Order Estimation from Fourier Domain
                                  Outline
                                           ...
Order Estimation from Fourier Domain
                                              Outline
                               ...
Order Estimation from Fourier Domain
                                            Outline
                                 ...
Order Estimation from Fourier Domain
                                            Outline
                                 ...
Order Estimation from Fourier Domain
                                            Outline
                                 ...
Order Estimation from Fourier Domain
                                              Outline
                               ...
Order Estimation from Fourier Domain
                                     Outline
                                        ...
Order Estimation from Fourier Domain
                                            Outline
                                 ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                                Outline
                             ...
Order Estimation from Fourier Domain
                                  Outline
                                           ...
Computationally Efficient NMRF model based Texture Synthesis
Computationally Efficient NMRF model based Texture Synthesis
Computationally Efficient NMRF model based Texture Synthesis
Computationally Efficient NMRF model based Texture Synthesis
Computationally Efficient NMRF model based Texture Synthesis
Computationally Efficient NMRF model based Texture Synthesis
Computationally Efficient NMRF model based Texture Synthesis
Computationally Efficient NMRF model based Texture Synthesis
Upcoming SlideShare
Loading in...5
×

Computationally Efficient NMRF model based Texture Synthesis

658

Published on

"Texture" provides the perceptual information about the surface, nature etc. about the visual objects. Study in texture learning and synthesis with a mathematical model will hopefully provide us the mathematical nature of visual perceptiveness.

On the other hand, Markov Random Field, nonparametric density estimation and their applications in the real world problems, are becoming popular in both research and industrial fields. The reason for this popularity is because of the mathematical models have more robustness, flexibility and simplicity.

The research problems (given this background) are order estimation and large computational complexity. In my PhD thesis I have tried to solve these issues for the application in homogeneous texture synthesis.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
658
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Computationally Efficient NMRF model based Texture Synthesis"

  1. 1. Outline Ph.D. Research Work Conclusion and Possible Future Directions Fast NMRF based texture synthesis algorithms Arnab Sinha arnab@iitk.ac.in April 16, 2009 Thesis Supervisor: Dr. Sumana Gupta ACES-205, Dept. of EE, Indian Institute of Technology Kanpur, India Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  2. 2. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions 1 Outline Earlier Methods Research problems in NMRF-tex-syn algorithms 2 Ph.D. Research Work Order Estimation from Fourier Domain Reduction of Computational complexity Order Estimation : Revisited Inverse Texture Synthesis 3 Conclusion and Possible Future Directions Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  3. 3. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions The significance of texture synthesis • What defines texture ? • Locally varying intensities and/or color values • The local variations can be found perceptually similar within the total region • Texture Synthesis: Original D104 Texture Given a small texture exempler, synthesize an arbitrary sample of texture, so that the synthesized texture is visually similar to the original sample. Synthesized texture should look alike the original texture • Application of texture synthesis in - • Image segmentation, classification, synthesis, etc. • Content-based image retrieval • Development of high-level computer vision algorithms • Animation of real scenes • Perceptual analysis • Computationally fast and efficient handling of objects Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  4. 4. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Texture synthesis: Difficulty Figure: Spectrum of Natural Textures Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  5. 5. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Brief History of Models Texture Synthesis Algorithm Image Domain Model Mixed Domain Model Transformed Domain Model Non−Linear Models Linear Non−Linear Hidden Markov Tree Non−Linearity Introduced by Histogram Equalization Fan and Xia (2003) 2D−NCAR, Hard−limited Gaussian Process Chellappa and Kashyap (1985) Jacovitti et al. (1998) 2D−Wold Francos et al. (1993) Zhang et al. (1998) Wavelet + AR 2D− MA 1. Zhu et al. (1997) Eom (1998) Circular Harmonic Func 2. Zhu et al. (2000) + Hard−limited Gaussian Campasi and Scarano (2002) 3. Portilla and Simoncelli (2000) NNMRF Charalampidis (2006) Paget and Longstaff (1998) Mathematical Models Intuitive Models Heeger and Bergen (1995) Efros and Leung (1999) Wei and Levoy (2000) Pixel−based Ashikhmin (2001) We are working within this Framework Sampling Process Tonietto et al. (2005) Kwatra et al. (2003) Patch−based Patch−based sampling with wavelet transformation Wu et al. (2004) as a feature set for graph−cut algorithm Popular Methods Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  6. 6. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Description of N-MRF model • S is the lattice • Ys is the random variable at site s ∈ S • Concept of Neighborhood system: ℵs •s s = (i,j), site • r ∈ ℵs ⇔ s ∈ ℵr 1st order neighbors 2 2 • Circular neighborhood: ℵs = {r; s.t., |r − s| ≤ o } { } 2nd order neighbors • say, Xs = {Yr ; r ∈ ℵs } • Say, Y(s) = {Yr ; r s}, r, s ∈ S • Definition of MRF: P(Ys |Y(s) ) = P(Ys |{Yr ; r ∈ ℵs }) • parameteric model for P(Ys |Xs ) • semi-parameteric model for P(Ys |Xs ) • non-parameteric model for P(Ys |Xs ) Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  7. 7. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Description of N-MRF model: Kernel Density Estimation Definition of KDE, [Scott(1992)] N 1 • single dimensional: P(x) = N Kh (x − xi ) i=1 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  8. 8. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Description of N-MRF model: Kernel Density Estimation Definition of KDE, [Scott(1992)] N 1 • single dimensional: P(x) = N Kh (x − xi ) i=1 N d 1 • multi-dimensional: P(X) = Khj (X (j) − Xi (j)) i=1 j=1 N (X (j)−Xi (j))2 • where, in case of Gaussian kernel, Khj (X (j) − Xi (j)) = √1 } exp{− 2hj2 N 2πhj • and, hj = σj N −1/(d+4) Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  9. 9. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Texture Synthesis Algorithm Some definitions • Input texture field: {Ys }, where, s ∈ Sin • Output texture field: {Yq }, where, q ∈ Sout Kh (Ys −Yq )Kh (Xs −Xq ) s∈Sin Y X • Definition of LCPDF: P(Yq |Xq ) = Kh (Xs −Xq ) s∈Sin X Iterative Conditional Mode (ICM) algorithm • Evaluate P(Yq = y|Xq ), for y = 0, 1, . . . , 255 gray values. • Assign Yq = y, for which the above conditional probability is maximum Local Simulated Annealing • Define a Confidence field, Cq ; q ∈ Sout , and a matrix Φq = DIAG{Cr ; r ∈ ℵq } • KhX (Xs − Xq ; Φq ) = exp{−(Xs − Xq )T Φh,q (Xs − Xq )}, and Φh,q = Φq HX ≈ hΦq • Updation rule for the confidence field 1 • Cq = min{1, |r∈ℵ | r∈ℵ Cr + u × e} q q • where, u is a random number and e is a constant scale factor Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  10. 10. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Texture Synthesis Algorithm Approximate Independent Conditional Mode (ICM) algorithm • Ds,q = (Xs − Xq )T Φq (Xs − Xq ) • Define Sq = {r ∈ Sin } ⊂ Sin , s.t., ∀r ∈ Sq , Dr,q = constant. • Assign Yq = yr , where r is sampled from the set Sq randomly. S in S out C q q Input texture Output Texture Confidence Field Output Neighborhood Output Confidence Vector X q Vector Wq Matrix Input Neighborhood Vectors Similarity Measure t {X s} N−MRF : ( X q − X s) ( X q − X s) t WL alg : ( X q − X s) ( X q − X s) Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  11. 11. Outline Earlier Methods Ph.D. Research Work Research problems in NMRF-tex-syn algorithms Conclusion and Possible Future Directions Research Problems • Order estimation • Large Computational Complexity • Computational complexity ∝ d, the dimension of the neighborhood vector. • Computational complexity ∝ (M × N), the input image size • Computational complexity ∝ I, the number of iterations required to attain global convergence Original Texture Neighborhood vector dimension ’d’ 8000 7000 computational complexity of 6000 texture synthesis algorithm is proportional to ’d’ 5000 4000 3000 2000 1000 0 0 10 20 30 40 50 Model order ’o’ Order 4 Order 8 Order 14 Figure: Effect of order on the synthesis results and computational complexity Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  12. 12. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Order estimation from two fundamental frequencies Y c d o b a X Yj Xj Yi Xi Figure: Points a, b, c, d are the four corners of texton defined by the fundamental spatial period vectors [Xi Yi ] and [Xj Yj ]. The major diagonal o gives the order of causal circular neighborhood and o/2 gives the order of non-causal circular neighborhood. Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  13. 13. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Extraction of the parameters • Dimitri’s algorithm • estimate the two fundamental frequecies from the two-dimensional DFT of the texture sample. • computational complexity is of the order of image size. • Hays’s algorithm • it estimates the two fundamental spatial vectors from the correlation function • the algorithm is iterative • computationally expensive with respect to Dimitri’s algorithm Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  14. 14. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis D21 D20 order = 9 order = 23 order = 4 order = 18 D52 D35 order = 48 order = 8 order = 22 order = 21 Figure: Comparison of estimated order through Dimitrios and Hays’s methods with (NR) texture synthesis results Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  15. 15. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis A new neighborhood system Y proposed Non−causal neighborhood X Yj Xj Yi circular Non−causal neighborhood Xi Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  16. 16. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results: Proposed neighborhood system Circular Neighborhood Proposed Neighborhood Proposed Neighborhood Circular Neighborhood D104 D65 D95 D64 D67 D3 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  17. 17. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Two approaches Computational complexity affected by the neighbourhood dimension, d, and 1 the number of input pixels, N 2 Reduction methodologies Dimensionality reduction methodologies, e.g., Principal Component Analysis 1 (PCA) – to reduce the effect of d A data structure for fast search 2 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  18. 18. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis With Dimensionality reduction methods How the distance metric Ds,q looks after projection (Xq − Xs )T Φq (Xq − Xs ) (X q − X s )T Φ q (X q − X s ) ≈ [PrT Pr (Xq − Xs )]T Φq [PrT Pr (Xq − Xs )] = [Pr (Xq − Xs )]T Pr Φq PrT [Pr (Xq − Xs )] = (Zq − Zs )T Ψq (Zq − Zs ) = T • What is Ψq = Pr Φq Pr ? • Is it reducing the computational complexity ? Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  19. 19. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Simulated Annealing for Principal components Original Ds,q (Xq − Xs )T Φq (Xq − Xs ) = Ds,q where, Φq = DIAG{W1 , W2 , . . . , Wd } ˆ Proposed Ds,q ˆ ˆ (Zq − Zs )T Φq (Zq − Zs ) = Ds,q ˆ where, Φq = DIAG{W1 , W2 , . . . , Wk } WHY ? Because we need only • a steady increase in the value of confidence, and • the starting value has to be ”0” and ending value has to be ”1” Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  20. 20. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results Table: Comparison of dimensionality; Original dimension, |ℵs | = d; Reduced dimension, k (<< d), η is the ratio of computational complexities between earlier and proposed one Texture Type Texture order d k η NR D20 20 1516 60 21.9405 NR D3 30 2820 580 3.7926 NR D21 25 1960 56 29.2642 NR D22 20 1256 177 6.3042 NR D35 28 2452 287 6.6612 NR D36 22 1516 258 5.1024 ST D7 27 2288 511 3.6438 ST D13 24 1792 131 11.6006 NR+ST D18 32 3208 95 25.5666 NR+ST D4 29 2628 465 4.4755 NR+ST D5 29 2628 179 11.6262 IN/STR D15 23 1652 167 8.4897 IN/STR D42 26 2120 293 5.9699 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  21. 21. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results continued ... NNMRF Proposed Algorithm Proposed Algorithm NNMRF D20 D7 k = 511 d = 2288 d = 1516 k=60 D13 D21 k=131 d=1792 d=1960 k=56 D22 D42 k = 293 d=2120 k = 177 d=1256 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  22. 22. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis With Fast Kernel Density Estimation Assumption: The h parameters along all the directions are equal. • Let Rn = {ts : ||ts − tn || ≤ R} N 1 • P(tn ) = N s=1 KH (ts − tn ) Y 1 • P(tn ) = N s∈Rn KH (ts − tn ) • Let Nn = |{s Rn }| R=100 1 = KH (ts − tn ) Err(tn , R) N s Rn Nn K (R) ≤ To calculate KDE NH at this target point X we only need KH (R) ≤ Source data vector these two points Target data vector max(Err) • Rel err(R) = max(Probability) Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  23. 23. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Earlier FKDE algorithms • Improved Fast Gaussian Transform (IFGT) [Yang et al.(2003)Yang, Duraiswami, Gumerov, and Davis] • kd-tree based FKDE [Gray and Moore(2003)] • Reconstructionhistogram [Zhang et al.(2005)Zhang, Tang, and Kwok] Reconstructionhistogram • Clustering: {Clusti ; i = 1 . . . M} • Let ni as the number of source data vectors within i th cluster M 1 • P(tn ) = N i=1 KH (tn − Clusti )ni • KDE of tn given the source data points at cluster centroids with a weight factor ni /N • flexibility ? Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  24. 24. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Improved Fast Gaussian Transform • P(tn ) = KH (tn − Clusti )f (tn , Clusti ) ||tn −Clusti ||≤RIFGT 1 • P(tn ) = N KH (tn − ts ) ||tn −Clusti ||≤RIFGT s∈Clusti Source data clusters Target data vectors Due to this overlap we need to consider this source cluster R IFGT R To consider the source cluster In effect it can include some source cluster R the radius threshold has to be IFGT which was not needed at all The R IFGT can vary with the overlap size and cluster shape Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  25. 25. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis kd-tree-based FKDE Build up the kd-tree and Search according to the radius R. Y 6 σ X > σY Hyper−Sphere Y -6 - Partition Span in Y direction e 6 e ee R max_rec_err Hyper−Rectangle e ee e e e e e e e Eigen vectors e e e e e e e e Centroid e e - e ee Sub-spaces e e - e ? e X e Reconstruction X Error R rec_err,n - R tn > R+ R max_rec_err - ? Span in X direction Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  26. 26. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Why do we need another algorithm for FKDE Table: Why do we need another algorithm for FKDE ? C-FKDE KD-FKDE Advantage Clustering algorithm provides more Due to the hyper-plane boundary, one can use original radius R compact representation of the data space for strict error bound optimal RIFGT has to estimated Disadvantage kd-tree is not a good clustering for every tn , maximum RIFGT algorithm, therefore it does not can increase computational provide compact representation complexity of the data space Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  27. 27. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Principal Directive Divisive Partitioning (PDDP)[Boley(1998)] • project each source data point within the present space onto the first principal direction (eigen vector corresponding to the largest eigen value). • partition the present space into two sub-spaces with respect to the mean (or median) of the projected values. Hieararchical Boundaries Source data Tree structure of the nodes 6th IV I 1st VI 7th 2nd II III I III II VII 5th V VI VII 4th IV 8th V 8th 6th 7th 2nd 3rd 4th 5th 1st 3rd Leaf nodes Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  28. 28. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis FKDE based on PDDP If the present node is a leaf then evaluate KDE. 1 Source data vectors Projection of source data Target data vector (t n) Projection of terget point D rec_err 2nd Direction T is within left child D If R D rec_err T Return 0 Drec_err Else If D R process both children Process its children D 2nd Direction Dlb (Boundary for Partition) 1st Direction mr vn D For the right child lb If Dlb R (Which child to process) (Process child) Return 0 If Dlb R = return 0 if D R = process left child Else If R D rec_err Else if Drec_err R = return 0 Else process both children Return 0 Else process Else Process its children vn Projected target data vector mr Projected mean vector 1st Direction (a) Target point is outside (b) Target point is inside Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  29. 29. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis FKDE based on PDDP If the present node is a leaf then evaluate KDE. 1 Is there any need to go further for the children of the present node ? 2 Source data vectors Projection of source data Target data vector (t n) Projection of terget point D rec_err 2nd Direction T is within left child D If R D rec_err T Return 0 Drec_err Else If D R process both children Process its children D 2nd Direction Dlb (Boundary for Partition) 1st Direction mr vn D For the right child lb If Dlb R (Which child to process) (Process child) Return 0 If Dlb R = return 0 if D R = process left child Else If R D rec_err Else if Drec_err R = return 0 Else process both children Return 0 Else process Else Process its children vn Projected target data vector mr Projected mean vector 1st Direction (c) Target point is outside (d) Target point is inside Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  30. 30. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis FKDE based on PDDP If the present node is a leaf then evaluate KDE. 1 Is there any need to go further for the children of the present node ? 2 Which child node (left or right or both) of the present node to process further ? 3 Source data vectors Projection of source data Target data vector (t n) Projection of terget point D rec_err 2nd Direction T is within left child D If R D rec_err T Return 0 Drec_err Else If D R process both children Process its children D 2nd Direction Dlb (Boundary for Partition) 1st Direction mr vn D For the right child lb If Dlb R (Which child to process) (Process child) Return 0 If Dlb R = return 0 if D R = process left child Else If R D rec_err Else if Drec_err R = return 0 Else process both children Return 0 Else process Else Process its children vn Projected target data vector mr Projected mean vector 1st Direction (e) Target point is outside (f) Target point is inside Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  31. 31. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Comparison between the FKDE algorithms (i−1,j) 2 (i,j+1) 3 (i,j) 1 4 (i,j+1) {1} X RGB == 3 dimensions {1,2} X RGB == 6 dimensions {1,2,3} X RGB == 9 dimensions {1,2,3,4} X RGB == 12 dimensions (g) Image considered (h) Creation of the data for creating the data set space Figure: Data set creation for FKDE analysis Table: Time comparison Dimension 3 6 9 12 KDE: Time (sec) 981.78 1204.77 2529.22 2668.58 PDDP-FKDE: Time (sec) 11.3 23.39 52.55 65.79 KD-FKDE: Time (sec) 22.55 33.88 110.62 228.44 IFGT-FKDE: Time (sec) 399.79 425.45 209.33 384.08 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  32. 32. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Comparison between the FKDE algorithms Table: Comparative analysis of FKDE algorithms FKDE Dimension Maximum Maximum Relative Radius algorithms probability Error Error threshold 4.33531e − 05 0.0003e − 04 PDDP-FKDE 3 0.0007 14.5445 1.32977e − 10 0.0000e − 10 6 0.0000 28.1622 9.64769e − 18 0.0014e − 18 9 0.0001 47.5693 1.71367e − 23 0.0000e − 25 12 0.0000 59.9263 4.33531e − 05 0.0005e − 04 KD-FKDE 3 0.0011 14.5445 1.32977e − 10 0.0001e − 10 6 0.0000 28.1622 9.64769e − 18 0.0014e − 18 9 0.0001 47.5693 1.71367e − 23 0.0000e − 25 12 0.0000 59.9263 4.33531e − 05 0.2618e − 04 IFGT-FKDE 3 0.6040 18.1807 1.32977e − 10 0.5389e − 10 6 0.4053 35.2028 9.64769e − 18 0.4183e − 18 9 0.0434 59.4616 1.71367e − 23 0.132e − 25 12 0.0007703 74.9079 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  33. 33. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Computationally efficient Texture synthesis algorithm with FKDE algorithms Problems how to include the effect of Wq (the temperature field) within the PDDP-based tree 1 structure for the implementation of FKDE, and there are two joint densities corresponding to {Yq , Xq } and Xq ; therefore, it 2 requires two FKDE structure, which is not computationally efficient. Inclusion of Wq • Starting State: {Wq,i = 0} ⇒ P(Xq ; Wq ) = constant ⇒ P(Xq ) is uniform ⇒ Each Xs has equal effect upon Xq ⇒ Every Xs should be considered in the KDE ⇒ Rnew is very large Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  34. 34. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Computationally efficient Texture synthesis algorithm with FKDE algorithms Problems how to include the effect of Wq (the temperature field) within the PDDP-based tree 1 structure for the implementation of FKDE, and there are two joint densities corresponding to {Yq , Xq } and Xq ; therefore, it 2 requires two FKDE structure, which is not computationally efficient. Inclusion of Wq • Starting State: {Wq,i = 0} ⇒ P(Xq ; Wq ) = constant ⇒ P(Xq ) is uniform ⇒ Each Xs has equal effect upon Xq ⇒ Every Xs should be considered in the KDE ⇒ Rnew is very large • Ending State: {Wq,i = 1} ⇒ P(Xq ; Wq ) = P(Xq ) ⇒ Rnew = R Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  35. 35. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Computationally efficient Texture synthesis algorithm with FKDE algorithms Problems how to include the effect of Wq (the temperature field) within the PDDP-based tree 1 structure for the implementation of FKDE, and there are two joint densities corresponding to {Yq , Xq } and Xq ; therefore, it 2 requires two FKDE structure, which is not computationally efficient. Inclusion of Wq • Starting State: {Wq,i = 0} ⇒ P(Xq ; Wq ) = constant R = Rnew ⇒ P(Xq ) is uniform cq ⇒ Each Xs has equal effect upon Xq abs(vn − mr ) ≤ Rnew ⇒ Every Xs should be considered in the KDE R ⇒ Rnew is very large ⇒ abs(vn − mr ) ≤ cq • Ending State: ⇒ abs(vn − mr )cq ≤ R {Wq,i = 1} ⇒ P(Xq ; Wq ) = P(Xq ) ⇒ Rnew = R Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  36. 36. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results with comparisons Original NNMRF kd−tree IFGT Proposed D102 D49 D20 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  37. 37. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results with comparisons Original kd−tree NNMRF IFGT Proposed D53 D104 D4 D82 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  38. 38. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results with comparisons Original NNMRF IFGT kd−tree Proposed D110 D60 D93 D97 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  39. 39. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results with comparisons Table: Time taken in texture synthesis: input texture size 128 × 128 and output texture size 256 × 256 NNMRF C-FKDE KD-FKDE PDDP-FKDE hours 8 5 8 6 minutes 7 55 34 0 seconds 39 12 56 41 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  40. 40. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Maximum Log-Pseudo-likelihood LPL = log[P(Ys |Xs )] s∈Sin For 1st order neighborhood system For 2nd order neighborhood system Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  41. 41. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis How to estimate MLPL ? • parametric MRF model • non-parametric MRF model: what should be the kernel ? • Gaussian kernel: as used in [Paget and Longstaff(1998)] • Dirac-delta kernel • Some other solution Effect of kernel upon the MLPL estimate nd LPL is getting saturated before 2 order LPL is not saturating rather it is increasing 0 −10000 −50 −20000 −100 −30000 −150 −40000 LPL −200 LPL −250 −50000 D102: Near regular −300 D104: Near regular −60000 −350 D110: Stochastic −70000 −400 D60: Stochastic D93: Stochastic −450 −80000 0 1 2 3 4 0 5 10 15 20 25 30 35 40 Order Order Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  42. 42. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Why does the original LPL measure, not saturate ? Why ? Kh (Ys −Yq )Kh (Xs −Xq ) s∈S • original LCPDF: p(Yq |Xq ) = q∈S Kh (Xs −Xq ) • Changing terms with order: • hy = σy N −1/(d+4) : changes due to change in d and N, with order √ • In case of LCPDF the normalizing term becomes: 2πhy ; • Moreover, hy also affect the argument within the exponential term. • One can not neglect this term. Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  43. 43. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis A new definition for LCPDF δ(Ys − Yq )Kh (Xs − Xq ) q∈S p(Ys |Xs ) = Kh (Xs − Xq ) q∈S Two reasons in the support for this new definition • From the texture synthesis algorithm point of view • From a numerical point of view 0.018 0.016 Probability calculated with Gaussian kernel D104 Near Regular Texture 0.014 D110 Stochastic Texture 0.012 0.01 probaility = 0.003544 0.008 0.006 Probability = 0.0002963 0.004 0.002 0 0 50 100 150 200 250 300 Gray Levels Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  44. 44. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results: D104 D104 8 2 6 4 10 14 12 16 22 24 18 20 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  45. 45. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results: D9 D9 5 1 3 7 9 11 13 15 19 17 21 23 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  46. 46. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results for Near-regular textures Original Original NNMRF Our Synthesis Algorithm NNMRF Our Synthesis Algorithm D104 D20 o = 12 o = 18 D22 D34 o = 17 0 = 13 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  47. 47. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results for Stochastic textures Our Synthesis Algorithm Original NNMRF Original NNMRF Our Synthesis Algorithm D4 D9 O=9 O = 10 D93 D97 O = 16 O=9 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  48. 48. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Results for Some other textures Original NNMRF Our Synthesis Algorithm Original NNMRF Our Synthesis Algorithm D53 D55 O = 14 O = 17 D82 D80 O = 14 O = 12 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  49. 49. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Problem Definition Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  50. 50. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Problem Definition Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  51. 51. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Problem Definition Texture synthesis HOW ? Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  52. 52. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Motivation Applications of Inverse Texture Synthesis • Understanding of Textures • Content-based image/video retrieval • Perceptual Image/Video compression • Computer Vision Tasks • Perceptual understanding of textures within the image • Creation of animation – Collecting information from natural images/sequences • Perceptual Understanding of temporal texture – such as, dance sequence, walk sequence, music sequence etc. Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  53. 53. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Definition of Objective Functions According to N-MRF model • Distance between two LCPDF’s evaluated w.r.t. both input and output texture patches. Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  54. 54. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Definition of Objective Functions According to N-MRF model • Distance between two LCPDF’s evaluated w.r.t. both input and output texture patches. • What distance function ? Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  55. 55. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Definition of Objective Functions According to N-MRF model • Distance between two LCPDF’s evaluated w.r.t. both input and output texture patches. • What distance function ? • Computationally expensive Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  56. 56. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Definition of Objective Functions According to N-MRF model • Distance between two LCPDF’s evaluated w.r.t. both input and output texture patches. • What distance function ? • Computationally expensive According to N-MRF model: Intuitively • M×N • Size of the output patch 1 min{||Xs − Xq ||2 , where • |S | • Do the input neighborhood vectors s∈Sin in exist within output patch ? q ∈ Sout } Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  57. 57. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Problem with these two objective functions Scaled up versions of solutions A B approximately same Neighborhood Deformation/variation within quot;Aquot; difficult to find within quot;Bquot; Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  58. 58. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Three objectives • Say Sout = {s ∈ Sin , s.t., (si − i)2 ≤ M 2 and (sj − j)2 ≤ N 2 } {i,j,M,N} = S − Sout • Define S in • First objective finds neighborhood from input texture within the output texture • Second objective finds neighborhood from output texture within the input texture, excluding the part of Sout 1 min{||Xs − Xq ||2 ; q ∈ Sout } = F1 |Sin | s∈Sin 1 {i,j,M,N} min{||Xq − Xs ||2 ; s ∈ Sin = } F2 |Sout | q∈Sout = M×N F3 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  59. 59. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Multi-objective Framework    f1 (x)      f2 (x)   inequality constraints: gj (x) ≥ 0, j = 1, 2, ..., J       such that   minx  .    equality constraints: hk (x) = 0, k = 1, 2, ..., K .     .     solution space: xiL ≤xi ≤ xiU , i = 1, 2, ..., N fm (x) Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  60. 60. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Multi-objective Framework    f1 (x)      f2 (x)   inequality constraints: gj (x) ≥ 0, j = 1, 2, ..., J       such that   minx  .    equality constraints: hk (x) = 0, k = 1, 2, ..., K .     .     solution space: xiL ≤xi ≤ xiU , i = 1, 2, ..., N fm (x) Domination A vector x ∈ RN is said to dominate y ∈ RN if both the conditions stated below hold true: fi (x) ≤ fi (y), ∀i ∈ [1 . . . m] ∃ j ∈ [1 . . . m], such that, fj (x) fj (y) Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  61. 61. Order Estimation from Fourier Domain Outline Reduction of Computational complexity Ph.D. Research Work Order Estimation : Revisited Conclusion and Possible Future Directions Inverse Texture Synthesis Pareto-optimal Front Best in F 1 2nd objective function F 2 Worst in F 2 All are optimal solutions Best in F 2 Worst in F 1 1st objective function F1 Arnab Sinha arnab@iitk.ac.in Fast NMRF based texture synthesis algorithms
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×