Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Artyom Makovetskii - An Efficient Algorithm for Total Variation Denoising

307 views

Published on

An Efficient Algorithm for Total Variation Denoising

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Artyom Makovetskii - An Efficient Algorithm for Total Variation Denoising

  1. 1. Artyom Makovetskii Department of Mathematics, Chelyabinsk State University, Russian Federation Sergei Voronin Department of Mathematics, Chelyabinsk State University, Russian Federation Vitaly Kober Division of Applied Physics, CICESE, Ensenada, Mexico E-mails: artemmac@csu.ru, voron@csu.ru, vkober@cicese.mx An efficient algorithm for total variation denoising AIST'2016
  2. 2. Introduction L๐ž๐ญ ๐‘ข0 be an observed signal that is distorted by additive noise ๐‘›: ๐’– ๐ŸŽ = ๐’— + ๐’ Initial function ๐‘ฃ
  3. 3. Noisy function ๐‘ข0 One of the most known techniques for denosing of noisy signals was proposed by Rudin and Osher. This is a total variation (TV) regularization problem..
  4. 4. Let ๐‘ฑ ๐’– be the following functional : ๐‘ฑ ๐’– = โˆฅ ๐’– โˆ’ ๐’– ๐ŸŽ โˆฅ ๐‘ณ ๐Ÿ ๐Ÿ + ๐€ ๐‘ป๐‘ฝ ๐’– , where โˆฅ ๐’– โˆ’ ๐’– ๐ŸŽ โˆฅ ๐‘ณ ๐Ÿ ๐Ÿ is called a fidelity term and ๐€๐‘ป๐‘ฝ(๐’–) is called a regularization term. Consider the following variational problem: ๐’–โˆ— = ๐’‚๐’“๐’ˆ ๐’Ž๐’Š๐’ ๐‘ฑ ๐’– , ๐’– โˆˆ ๐‘ฉ๐‘ฝ ๐œด where ๐’–โˆ— is extremal function for ๐‘ฑ ๐’– .
  5. 5. Here we consider a one-dimensional TV (1D TV) regularization problem in framework of Strong and Chan. They considered the behavior of explicit solutions of the 1D TV problem when the parameter ๐€ in TV problem is sufficiently small. In this paper, explicit solutions of TV problem for any parameter ๐€ are analyzed. Strong, D. M. and Chan, T. F, โ€œ Edge-preserving and scale-dependent properties of total variation regularization, โ€ Inverse Problems 19, 165โ€“187 (2003).
  6. 6. How to describe the exact solutions of 1D TV regularization problem Continuous case TV( ๐’–) = | ๐’– ๐’™ | ๐’…๐’™ โˆฅ ๐’– โˆ’ ๐’– ๐ŸŽ โˆฅ ๐‘ณ ๐Ÿ ๐Ÿ = (๐’– โˆ’ ๐’– ๐ŸŽ) ๐Ÿ ๐’…๐’™ Discrete case TV( ๐’–) = ๐’Š=๐Ÿ ๐’โˆ’๐Ÿ |๐’–๐’Š+๐Ÿ โˆ’ ๐’–๐’Š| โˆฅ ๐’– โˆ’ ๐’– ๐ŸŽ โˆฅ ๐‘ณ ๐Ÿ ๐Ÿ = ๐’Š=๐Ÿ ๐’ (๐’–๐’Š โˆ’ ๐’–๐’Š ๐ŸŽ ) ๐Ÿ where ๐’– = (๐’– ๐Ÿ, โ€ฆ , ๐’– ๐’) ๐œด ๐œด
  7. 7. From BV class to ๐‘ฉ๐‘ฝ ๐ŸŽ class Let the function ๐’– belongs to the ๐๐• class and ๐’– has a discontinuity in the point ๐’™ ๐’…: ๐‘ชยฑ = ๐’๐’Š๐’Ž ๐’™โ†’๐’™ ๐’…ยฑ๐ŸŽ ๐’–(๐’™). If the value ๐ฎ(๐’™ ๐’…) is such that ๐’– ๐’™ ๐’… โˆ‰ [๐‘ชโˆ’; ๐‘ช+] , then by replacing ๐’–(๐’™ ๐’…) by either ๐‘ชโˆ’ or ๐‘ช+ we obtain a function with the same value of the fidelity term and a smaller value of the regularization term.
  8. 8. The solution ๐’–โˆ— in the BV space to be found in the space ๐‘ฉ๐‘ฝ ๐ŸŽ of bounded variation functions with the condition ๐’– ๐’™ ๐’… โˆˆ [๐‘ชโˆ’; ๐‘ช+] for each discontinuity point ๐’™ ๐’…. Thus, the solution in the class BV is reduced to the solution of the problem in the class ๐‘ฉ๐‘ฝ ๐ŸŽ.
  9. 9. Piecewise constant functions of continuous argument Let ๐’‚ = ๐’“ ๐ŸŽ < ๐’“ ๐Ÿ < โ‹ฏ < ๐’“ ๐’ = ๐’ƒ be a partition of the segment [๐’‚; ๐’ƒ], โˆ†๐’Š be a segment [๐’“๐’Šโˆ’๐Ÿ; ๐’“๐’Š], ๐’Š = ๐Ÿ, โ€ฆ , ๐’. Let the partition ๐œน = {โˆ†๐’Š} be a set of such segments. The function ๐’– is called piecewise constant, i.e. ๐’– โˆˆ ๐‘ท๐‘ช(๐œด), if ๐’– ๐’™ = ๐’–๐’Š when ๐’™ โˆˆ โˆ†๐’Š. Let ๐‘ท๐‘ช(๐œน) be a class of piecewise constant function on the segment ๐œด = [๐’‚; ๐’ƒ] with the partition ๐œน.
  10. 10. Suppose that ๐’– ๐ŸŽ โˆˆ ๐‘ท๐‘ช(๐œน), then there is a function ๐’– โˆˆ ๐‘ท๐‘ช(๐œน) such that ๐‘ฑ ๐’– โ‰ค ๐‘ฑ ๐’– for any ๐’– โˆˆ ๐‘ฉ๐‘ฝ ๐ŸŽ. Theorem 1
  11. 11. FORMULATION OF 1D TV REGULARIZATION AS FINITE- DIMENSIONAL PROBLEM By using Theorem 1, the TV problem in the space ๐‘ฉ๐‘ฝ ๐ŸŽ can be reduced to the problem in the space ๐‘ท๐‘ช(๐œน). Let a function ๐’– belongs to ๐‘ท๐‘ช(๐œน). It means that ๐’– ๐’™ = ๐’–๐’Š , ๐’™ โˆˆ โˆ†๐’Š, ๐’Š = ๐Ÿ, โ€ฆ , ๐’ , i.e. ๐’– ๐’™ = {๐’– ๐Ÿ, โ€ฆ , ๐’– ๐’}. Therefore, the functional space ๐‘ท๐‘ช(๐œน) can be identified with the space โ„ ๐’ . For functions from ๐‘ท๐‘ช(๐œน), ๐‘ฑ ๐’– takes the following form: ๐‘ฑ ๐’– = ๐’Š=๐Ÿ ๐’ (๐’–๐’Š โˆ’ ๐’– ๐ŸŽ ๐’Š ) ๐Ÿ |โˆ†๐’Š| + ๐›Œ ๐’Š=๐Ÿ ๐’โˆ’๐Ÿ | ๐’–๐’Š+๐Ÿ โˆ’ ๐’–๐’Š|.
  12. 12. Suppose that the function ๐’– ๐’™ = {๐’– ๐Ÿ, โ€ฆ , ๐’– ๐’} satisfies the following condition: ๐’–๐’Š โ‰  ๐’–๐’Š+๐Ÿ, ๐’Š = ๐Ÿ, โ€ฆ , ๐’ โˆ’ ๐Ÿ. Let a vector ๐œบ(๐’– ) = ๐œบ ๐Ÿ, โ€ฆ , ๐œบ ๐’โˆ’๐Ÿ for the function ๐’– ๐’™ = {๐’– ๐Ÿ, โ€ฆ , ๐’– ๐’} be defined as follows: ๐œบ๐’Š = ๐Ÿ, ๐’–๐’Š < ๐’–๐’Š+๐Ÿ โˆ’๐Ÿ, ๐’–๐’Š > ๐’–๐’Š+๐Ÿ . Then total variation of the function ๐’– ๐’™ can be expressed with the vector ๐œบ ๐’– , ๐‘ป๐‘ฝ ๐’– = ๐’Š=๐Ÿ ๐’โˆ’๐Ÿ | ๐’–๐’Š+๐Ÿ โˆ’ ๐’–๐’Š| = ๐’Š=๐Ÿ ๐’โˆ’๐Ÿ ๐œบ๐’Š( ๐’–๐’Š+๐Ÿ โˆ’ ๐’–๐’Š).
  13. 13. One can observe that each component ๐’–๐’Š of the vector ๐’– ๐’™ is included in ๐‘ป๐‘ฝ ๐’– with the coefficient ๐’†๐’Š = + ๐Ÿ( ๐’†๐’Š = โˆ’๐Ÿ), if ๐’–๐’Š is boundary maximum (boundary minimum); ๐’†๐’Š = +๐Ÿ ๐’†๐’Š = โˆ’๐Ÿ , if ๐’–๐’Š is local maximum (local minimum) and ๐’†๐’Š = ๐ŸŽ, if ๐’–๐’Š is a โ€œstepโ€ region of the function ๐’– ๐’™ , ๐‘ป๐‘ฝ ๐’– = ๐’Š=๐Ÿ ๐’ ๐’†๐’Š ๐’–๐’Š, where the vector ๐’†(๐’–) = ๐’† ๐Ÿ , โ€ฆ , ๐’† ๐’ . Let ๐’† ๐ŸŽ be a vector of the initial function ๐’– ๐ŸŽ, ๐’–โˆ— be a solution, ๐’†โˆ— be a vector of the function ๐’–โˆ—.
  14. 14. Strong-Chan formulas for small ๐€ Coefficient ๐’Œ๐’Š = ยฑ ๐Ÿ ๐Ÿ if the segment is boundary maximum or boundary minimum, ๐’Œ๐’Š = โˆ’๐Ÿ if the segment is local minimum, ๐’Œ๐’Š = ๐ŸŽ if the segment is the step, ๐’Œ๐’Š = ๐Ÿ if the segment is local maximum. ๐’‰_๐’๐’†๐’˜ ๐’Š = ๐’‰ ๐’Š โˆ’ ๐’Œ๐’Š ๐€ โˆ†๐’Š
  15. 15. Theorem 2. If ๐’† ๐ŸŽ ๐’Š โ‰ฅ ๐ŸŽ (๐’† ๐ŸŽ ๐’Š โ‰ค ๐ŸŽ), then ๐’†โˆ— ๐’Š โ‰ฅ ๐ŸŽ (๐’†โˆ— ๐’Š โ‰ค ๐ŸŽ), ๐’Š = ๐Ÿ, โ€ฆ , ๐’.
  16. 16. REDUCTION OF 1D TV PROBLEM BY ๐€ Let ๐œน ๐ŸŽ be a partition of the segment [๐’‚; ๐’ƒ] generated by the function ๐’– ๐ŸŽ. Let a function ๐’– ๐Ÿ be a solution of the following variational problem: ๐’‚๐’“๐’ˆ ๐’Ž๐’Š๐’ ๐’Š=๐Ÿ ๐’ (๐’–๐’Š โˆ’ ๐’– ๐ŸŽ ๐’Š ) ๐Ÿ |โˆ†๐’Š| + ๐€ ๐Ÿ ๐’Š=๐Ÿ ๐’โˆ’๐Ÿ | ๐’–๐’Š+๐Ÿ โˆ’ ๐’–๐’Š|, ๐ฎ โˆˆ ๐๐‚(๐œน ๐ŸŽ) (1) where ๐’ is the number of segments in the partition ๐œน ๐ŸŽ and ๐€ ๐Ÿ is the value corresponding to the first meeting of the two neighboring segments. Let ๐œน ๐Ÿ be a partition of the segment [๐’‚; ๐’ƒ] generated by the function ๐’– ๐Ÿ.
  17. 17. Let a function ๐’– ๐œบ be a solution of the following problem: ๐’‚๐’“๐’ˆ ๐’Ž๐’Š๐’ ๐’Š=๐Ÿ ๐’Œ (๐’–๐’Š โˆ’ ๐’– ๐Ÿ ๐’Š ) ๐Ÿ |โˆ†๐’Š| + ๐€ ๐œบ ๐’Š=๐Ÿ ๐’Œโˆ’๐Ÿ | ๐’–๐’Š+๐Ÿ โˆ’ ๐’–๐’Š|. ๐ฎ โˆˆ ๐๐‚(๐œน ๐Ÿ) (2) where ๐’Œ is the number of segments in the partition ๐œน ๐Ÿ and ๐€ ๐œบ is sufficiently small.
  18. 18. Consider the following variational problem: ๐’‚๐’“๐’ˆ ๐’Ž๐’Š๐’ ๐’Š=๐Ÿ ๐’ (๐’–๐’Š โˆ’ ๐’– ๐ŸŽ ๐’Š ) ๐Ÿ |โˆ†๐’Š| + (๐€ ๐Ÿ + ๐€ ๐œบ) ๐’Š=๐Ÿ ๐’โˆ’๐Ÿ | ๐’–๐’Š+๐Ÿ โˆ’ ๐’–๐’Š|. ๐ฎ โˆˆ ๐๐‚(๐œน ๐ŸŽ) (3) Theorem 3. The solution ๐’– ๐œบ of the variational problem in (2) is also a solution of (2).
  19. 19. EXPLICIT SOLUTIONS OF 1D TV REGULARIZATION PROBLEM From Theorems 1-3 follows that the exact solution of the variational problem can be described by the following way. The parameter ๐€ can be considered as time. When time ๐€ is small the solution of (1) can be can be obtained by the Strong and Chan formulas. Let ๐€ ๐Ÿ be the moment of time corresponding to the first meeting of the two neighboring segments. If value of parameter ๐€ less than ๐€ ๐Ÿ we have description of the extremal function.
  20. 20. If value of parameter ๐€ more than ๐€ ๐Ÿ do following. Instead the initial variational problem consider variational problem with substitution of obtained in moment ๐€ ๐Ÿ function ๐’– ๐Ÿ instead ๐’– ๐ŸŽ and with new value of parameter ๐€ โ‰” ๐€ โˆ’ ๐€ ๐Ÿ . We obtain new variational problem with smaller number of segments and smaller value of parameter ๐€.
  21. 21. ๐œ† = 4
  22. 22. ๐œ† = 50
  23. 23. ๐œ† = 1000
  24. 24. ๐œ† = 5000
  25. 25. Conclusion In this paper we considered exact solutions of the 1D TV problem. We proved the existence and uniqueness of the solutions and described their properties. Recently, was proposed explicit solutions of the 1DTV problem as well as a direct fast algorithm for the case of discrete functions. The algorithm is very fast and has complexity of ๐‘ถ(๐’) for typical discrete functions. In contrast, the proposed approach to finding exact solutions has a clear geometrical meaning.

ร—