Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Color direction patch-sparsity-based image inpainting using multidirection features

39 views

Published on

Final Year IEEE Projects for BE, B.Tech, ME, M.Tech,M.Sc, MCA & Diploma Students latest Java, .Net, Matlab, NS2, Android, Embedded,Mechanical, Robtics, VLSI, Power Electronics, IEEE projects are given absolutely complete working product and document providing with real time Software & Embedded training......

JAVA, .NET, NS2, MATLAB PROJECTS:

Networking, Network Security, Data Mining, Cloud Computing, Grid Computing, Web Services, Mobile Computing, Software Engineering, Image Processing, E-Commerce, Games App, Multimedia, etc.,

EMBEDDED SYSTEMS:

Embedded Systems,Micro Controllers, DSC & DSP, VLSI Design, Biometrics, RFID, Finger Print, Smart Cards, IRIS, Bar Code, Bluetooth, Zigbee, GPS, Voice Control, Remote System, Power Electronics, etc.,

ROBOTICS PROJECTS:

Mobile Robots, Service Robots, Industrial Robots, Defence Robots, Spy Robot, Artificial Robots, Automated Machine Control, Stair Climbing, Cleaning, Painting, Industry Security Robots, etc.,

MOBILE APPLICATION (ANDROID & J2ME):

Android Application, Web Services, Wireless Application, Bluetooth Application, WiFi Application, Mobile Security, Multimedia Projects, Multi Media, E-Commerce, Games Application, etc.,

MECHANICAL PROJECTS:

Auto Mobiles, Hydraulics, Robotics, Air Assisted Exhaust Breaking System, Automatic Trolley for Material Handling System in Industry, Hydraulics And Pneumatics, CAD/CAM/CAE Projects, Special Purpose Hydraulics And Pneumatics, CATIA, ANSYS, 3D Model Animations, etc.,

CONTACT US:

ECWAY TECHNOLOGIES
23/A, 2nd Floor, SKS Complex,
OPP. Bus Stand, Karur-639 001.
TamilNadu , India.Cell: +91 9894917187.
Website: www.ecwayprojects.com | www.ecwaytechnologies.com
Mail to: ecwaytechnologies@gmail.com.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Color direction patch-sparsity-based image inpainting using multidirection features

  1. 1. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com COLOR-DIRECTION PATCH-SPARSITY-BASED IMAGE INPAINTING USING MULTIDIRECTION FEATURES By A PROJECT REPORT Submitted to the Department of electronics &communication Engineering in the FACULTY OF ENGINEERING & TECHNOLOGY In partial fulfillment of the requirements for the award of the degree Of MASTER OF TECHNOLOGY IN ELECTRONICS &COMMUNICATION ENGINEERING APRIL 2016
  2. 2. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com CERTIFICATE Certified that this project report titled “Color-Direction Patch-Sparsity-Based Image Inpainting Using Multidirection Features” is the bonafide work of Mr. _____________Who carried out the research under my supervision Certified further, that to the best of my knowledge the work reported herein does not form part of any other project report or dissertation on the basis of which a degree or award was conferred on an earlier occasion on this or any other candidate. Signature of the Guide Signature of the H.O.D Name Name
  3. 3. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com DECLARATION I hereby declare that the project work entitled “Color-Direction Patch-Sparsity-Based Image Inpainting Using Multidirection Features” Submitted to BHARATHIDASAN UNIVERSITY in partial fulfillment of the requirement for the award of the Degree of MASTER OF APPLIED ELECTRONICS is a record of original work done by me the guidance of Prof.A.Vinayagam M.Sc., M.Phil., M.E., to the best of my knowledge, the work reported here is not a part of any other thesis or work on the basis of which a degree or award was conferred on an earlier occasion to me or any other candidate. (Student Name) (Reg.No) Place: Date:
  4. 4. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com ACKNOWLEDGEMENT I am extremely glad to present my project “Color-Direction Patch-Sparsity-Based Image Inpainting Using Multidirection Features” which is a part of my curriculum of third semester Master of Science in Computer science. I take this opportunity to express my sincere gratitude to those who helped me in bringing out this project work. I would like to express my Director,Dr. K. ANANDAN, M.A.(Eco.), M.Ed., M.Phil.,(Edn.), PGDCA., CGT., M.A.(Psy.)of who had given me an opportunity to undertake this project. I am highly indebted to Co-OrdinatorProf. Muniappan Department of Physics and thank from my deep heart for her valuable comments I received through my project. I wish to express my deep sense of gratitude to my guide Prof. A.Vinayagam M.Sc., M.Phil., M.E., for her immense help and encouragement for successful completion of this project. I also express my sincere thanks to the all the staff members of Computer science for their kind advice. And last, but not the least, I express my deep gratitude to my parents and friends for their encouragement and support throughout the project.
  5. 5. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com ABSTRACT: This paper proposes a color-direction patchsparsity-based image inpainting method to better maintain structure coherence, texture clarity, and neighborhood consistence of the inpainted region of an image. The method uses super-wavelet transform to estimate the multi-direction features of a degraded image, and combines with color information to construct the weighted color- direction distance (WCDD) to measure the difference between two patches. Based on the WCDD, the color-direction structure sparsity is defined to obtain a more robust filling order and more suitable multiple candidate patches are searched. Then, the target patches are sparsely represented by the multiple candidate patches under neighborhood consistency constraints in both the color and the multi-direction spaces. Experimental results are presented to demonstrate the effectiveness of the proposed approach on tasks such as scratch removal, text removal, block removal, and object removal. The effects of super-wavelet transforms and direction features are also investigated.
  6. 6. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com INTRODUCTION: Image inpainting, also known as image completion or image disocclusion, was initially proposed as a mean for the restoration of old and scratched pictures or artworks. With recent developments of digital image manipulation, it has become an active research subject of computer vision and image processing. Image inpainting aims to repair missing region of a degraded image in a visually plausible way to prevent the inpainted region from easily observed by an ordinary unsuspecting person. Applications of image inpainting include many aspects such as old film restoration, art conservation, and digital restoration. Image inpainting approaches can be classified roughly into three categories: the diffusion- based, the exemplar-based, and the sparse-based approaches. The diffusion-based methods solve Partial Differential Equations (PDE) or similar diffusion systems so as to diffuse image information from source regions into missing regions. Bertalmio et al. filled the missing areas by diffusing the information along the isophote direction. The Navier-Stokes equation in fluid dynamics was then introduced into the image inpainting task. Total variation (TV) model has been used to recover the damaged areas . The curvature-driven diffusion (CDD) equation was proposed to amend the TV model which cannot hold the connectivity principle. Zhang et al. Improved the TV model using P-Harmonic energy minimization. Other approaches utilize the neighborhood pixels to interpolate the missing pixels. Telea adopted the neighboring known pixels to fill missing pixels from outside to inside. A unified framework for interpolation based on elliptic partial differential equations was presented in Takeda et al. made connections with the field of nonparametric statistics and expanded kernel regression ideas for image reconstruction. Researchers also investigated the means to obtain a more suitable model for image inpainting task in order to obtain better recovery results Recently, the diffusion-based algorithms have achieved significant results for filling non-textured and reversely smaller damaged area. However, they perform poorly on small structured or textured missing regions due to the lack of semantic texture/structure synthesis, even worse for large missing regions.The image sparse representation has been introduced to image inpainting problems. In this approach, the image is represented by the sparse combination of an over- complete dictionary (e.g., wavelet, Curvelet, Contourlet, DCT, etc.) and the missing pixels are estimated by adaptively updating
  7. 7. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com This sparse representation. For example, the damaged image is separated into the cartoon and the texture layers, and represented sparsely by the DCT and Curvelet transforms, respectively Fadili et al. introduced an expectation maximization (EM) algorithm for image inpainting based on the idea of EM mechanism in Bayesian framework, where sparsity promoting prior penalty is imposed on the reconstructed coefficients. Since the bases of these transforms are fixed, generalization of the algorithms is limited. To address the limitation, dictionary learning methods are proposed to enhance the generalization. Roth proposed a framework for learning generic, expressive image priors and applied it to image denoising and image inpainting. Mairal developed a K-SVD method to train an over-complete dictionary which was used to represent an image sparsely. Furthermore, they proposed an online learning method to construct the overcomplete dictionary. Then Zhou et al. presented a nonparametric Bayesian dictionary learning for incomplete images. Wohlberg proposed to represent the missing regions by a sparse linear combination of example blocks extracted from the image being inpainted or external training image set. A matrix completion method was adopted for the image completion problem Moreover, some researchers combined the image interpolation and sparsity to reconstruct the incomplete image. For example, a patch-based nonlocal image interpolation algorithm was presented in Ram et al. proposed an image processing scheme based on reordering of its patches. These approaches effectively fill in the missing region with relatively composite textures and structures, especially in the application of missing block completion. However, they may either fail to recover the structure and texture or introduce the smooth effect when filling the missing structure and texture regions or the large missing regions, similar to problems encountered by the diffusion-based approaches. The exemplar-based inpainting approaches propagate image information from source regions into missing regions at patch level. These methods perform more effectively for larger holes, and can be roughly categorized into two groups: the matchingbased and the MRF (Markov Random Field)-based. Patch selection and patch inpainting are two main procedures in the matching-based inpainting algorithms. In patch selection, a patch with the highest priority value is selected to be the target patch and will be filled. The priority function is designed to ensure the patches in the structure region are filled in first so that the missing area can be repaired with better structure coherence. Criminisi et al. proposed a priority function defined by the inner product of
  8. 8. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com isophote direction and normal direction of the missing region boundary. However, this priority function is sensitive to noise and not robust to the orientation on the boundary of missing regions. Many studies have been conducted to obtain a more robust filling order. Wu and Ruan adopted a cross-isophote diffusion item to determine the filling order. Xu and Sun proposed the structure sparsity to compute the priority function. Hesabi et al. applied structure sparsity and modified confidence term to compute the priority. Florinabel et al. developed a priority function using the DCT coefficients of patch. The structure tensors were used to define the priority function . Zhang and Lin proposed a novel priority scheme based upon color distribution. The aforementioned priority patch selection schemes work well when the damaged image is smooth or with simple structure. When the missing areas are composed of complex structures as well as textures, however, these priority schemes do not clearly differentiate the structure patch and the texture patch. As a result, the structure coherence will be lost in the inpainted image. In the patch inpainting procedure, the target patch is restored by the search candidate patches under a certain match criterion in the known region. Many methods adopt the most similar patch search via the sum of squared distance (SSD) to recover the target patch. However, this approach easily causes the block effect and the seam effect. Several methods have been developed to reduce these two effects. Some try to construct a better match criterion to search a more suitable patch. For example, Florinabel et al. added the edge information into the match criterion to find the most similar patch. Structure tensors were also used to find the template matching These modified match criteria can alleviate, but not completely remove the block effect and the seam effect. Others attempt to infer the missing information of target patch. Wong and Orchard applied nonlocal image information from multiple patches within the image to fill the missing regions. Shen et al. considered the problem of image inpainting from the viewpoint of sequential incomplete signal recovery and adopted a sparse representation over a redundant dictionary to restore each degraded image patch. The image inpainting problem was also considered as a matrix completion problem, and the weighted sparse non-negative matrix factorization method was applied to complete the target patch Multiple candidate patches were used to sparsely represent the target patch under a local consistence constraint Guillemot et al. adopted a neighbor embedding scheme to estimate the unknown pixels. In the target patch is repaired by a 2D non-harmonic analysis approach. Additionally, Ogawa and Haseyama applied Fourier transform magnitude estimation scheme to
  9. 9. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com infer the missing information of target patch. Although the schemes using sparse linear combination of multiple candidate patches can effectively alleviate the block effect and the seam effect and retain the structure and texture clarity for images with simple structures and textures the repaired results still have a tendency to be smooth or blurred. Moreover, the structure and texture clarity will be lost for the images with complex structures and textures. This is mainly due to the fact that the constraint equations for the sparse representation of multiple candidate patches only consider the local consistency in the color space. The MRF-based methods are generally realized by optimizing discrete Markov Random Fields, as in Priority-BP and Shift-map Komodakis adopted priority-based message scheduling to optimize the energy of a discrete MRF, and applied dynamic label pruning to reduce the computational complexity. A variational model was used to combine three energy function and tried to approximate the minimum of the proposed energy function Pritch allowed all possible offsets and optimized the energy function using graph cuts However, the computation complexity is very high due to large numbers of offsets. To reduce the computation time, He only adopted a few dominant offsets, which was achieved by matching similar patches in the image, to optimize the energy function by using multi- label graph cuts. Further, in several inpainting with different parameter settings are performed on a low-resolution image. The inpainted images are then fused together to get a unique low- resolution image. Then a single-image super-resolution algorithm is applied to recover the high frequencies. Liu and Caselles adopted a feature representation computed at the original image resolution to compensate the loss of information at low resolution levels. This paper focuses on the matching-based inpainting approaches using patch sparsity. A color- direction patch sparsity based image inpainting algorithm using multi-direction features (MDF) is proposed to maintain the structure coherence, texture clarity, and neighborhood consistency. The patch sparsity can be reflected by structure sparsity and patch sparse representation. Inspired by the image inpainting algorithms using sparse transforms, super-wavelet transform is adopted to extract the MDF of an image, which will be combined with the color information to construct a weighted color-direction distance (WCDD) to measure the similarity between two patches. Then the color-direction structure sparsity (CDSS) function, which aims to maintain structure coherence, is designed by computing the WCDD between a patch located at fill-front and its neighboring known patches. In addition, multiple candidate patches are searched
  10. 10. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com based on WCDD to provide a better subspace for patch sparse representation. Furthermore, an optimization equation with local patch consistency constraints in both the color and the multi- direction spaces is constructed to obtain sparse linear combination coefficients of candidate patches. This process aims to maintain inpainted regions more consistently with their neighbors and to improve the structure and texture clarity The main contributions of this paper include: 1) Introduce the MDF to the inpainting algorithm; 2) Apply WCDD to search suitable candidate patches; 3) Apply CDSS to obtain robust filling order; and 4) Incorporate the color and multi-direction constraints into the optimization criterion to obtain high-quality inpainting results. Common notations used in this paper are listed in Table I. The remainder of this paper is organized as follows. Section II presents the means to extract multi-direction features of an image. Detailed description of the proposed exemplar-based completion method is given in Section III. Effectiveness of the proposed image inpainting algorithm verified by experimental results is shown in Section IV, followed by the conclusions.
  11. 11. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com CONCLUSION: In This PaperWe Have PresentedAnImage Inpainting Algorithm Which Utilizes Color And Multi- Direction Features In The Inpainting Procedure To Maintain Structure Coherence And Neighborhood Consistency For Block, Scratch, Text, And Object Removal. In The Proposed Inpainting Method, We Introduce Multi-Direction Features Into The Image Inpainting Algorithm, And Apply A New Distance Measure WCDD To Search Candidate Patches And A New Sparsity Function CDSS To Obtain Robust Filling Order And Maintain Structure Coherence. Moreover, The Color And Multi-Direction Constraints Are Incorporated Into The Optimization Criterion To Obtain Sharp Inpainting Results. Experimental Results Have Demonstrated That The Proposed Method Improves The Structure Coherence Over The Existing Inpainting Techniques For The Degraded Images With Relatively Regular Directional Structure Features. However,For The DegradedImages With Many Irregular Directional Features,The ProposedMay Fail To Maintain The Neighborhood Consistence. In Addition, We Have Empirically Investigated The Effect Of Weight Coefficients Via Different Super-Wavelet Transforms And Different Direction Numbers. Improved Performance Of The Proposed Approach Is Mainly Attributed To The Extracted Multi-Direction Features. Inaccurate Multi-Direction Features Inferred By The Preliminary Inpainted Image May Result In Relatively Inferior Inpainting Results. Hence, We Are Currently Investigating The Means To Accurately Extract The Multi-Direction Features. ACKNOWLEDGMENT Many Thanks Go To The Authors Of [46] For The Supplied Repaired Results And Many Images With Degraded Masks Which Help Enrich The Present Work. The Authors Also Thank The Anonymous Reviewers For Their Helpful Comments.
  12. 12. OUR OFFICES @CHENNAI/ TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE / BANGALORE / HYDRABAD CELL: +91 9894917187 | 875487 1111 / 2111 / 3111 / 4111 / 5111 / 6111 ECWAY TECHNOLOGIES IEEE SOFTWARE | EMBEDDED | MECHANICAL | ROBOTICS PROJECTS DEVELOPMENT Visit: www.ecwaytechnologies.com | www.ecwayprojects.com Mail to: ecwaytechnologies@gmail.com REFERENCES: [1] H. Y. Zhang, B. Wu, Q. C. Peng, and Y. D. Wu, “Digital image inpainting based on p-harmonic energy minimization,” Chin. J. Electron., vol. 3, no. 3, pp. 525–530, 2007. [2] A. Telea, “An image inpainting technique based on the fast marching method,” J. Graph. Tools, vol. 9, no. 1, pp. 23–34, 2004. [3] J. Weickert and M. Welk, “Tensor field interpolation with PDEs,” in Visualization and Processing of Tensor Fields. Berlin, Germany: Springer-Verlag, 2006, pp. 315–325. [4] H. Takeda, S. Farsiu, and P. Milanfar, “Kernel regression for image processing and reconstruction,” IEEE Trans. Image Process., vol. 16, no. 2, pp. 349–366, Feb. 2007. [5] J. Miyoun, X. Bresson, T. F. Chan, and L. A. Vese, “Nonlocal Mumford–Shah regularizers for color image restoration,” IEEE Trans. Image Process., vol. 20, no. 6, pp. 1583–1598, Jun. 2011. [6] Y.-W. Wen, R. H. Chan, and A. M. Yip, “A primal–dual method for total-variation-based wavelet domain inpainting,” IEEE Trans. Image Process., vol. 21, no. 1, pp. 106–114, Jan. 2012. [7] M. Mainberger et al., “Optimising spatial and tonal data for homogeneous diffusion inpainting,” in Scale Space and Variational Methods in Computer Vision. Berlin, Germany: Springer-Verlag, 2012, pp. 26–37.

×