Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Обзор методов восстановления старого видео

1,501 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Обзор методов восстановления старого видео

  1. 1. Восстановление сжатого видео Моисейцев Алексей Video Group CS MSU Graphics & Media Lab
  2. 2. Only for Maxus  Содержание  Введение  Простые модели  Adaptive Fuzzy Post-Filtering  DCT Re-application  Классификация  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация  Заключение 2 CS MSU Graphics & Media Lab (Video Group)
  3. 3. Only for Maxus  Введение 3 CS MSU Graphics & Media Lab (Video Group)
  4. 4. Only for Maxus  Введение 4 CS MSU Graphics & Media Lab (Video Group)
  5. 5. Only for Maxus  Содержание  Введение  Простые модели  Adaptive Fuzzy Post-Filtering  DCT Re-application  Классификация  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация  Заключение 6 CS MSU Graphics & Media Lab (Video Group)
  6. 6. Only for Maxus  Adaptive Fuzzy Post-Filtering  j x j  (x c , x j ) N ~ xc  1  j  (x c , x j ) N 1 / 2 2  (a , b )  e (a b ) 2  2  сила фильтра, зависит от типа блока ~ x c  новое значение пикселя Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004. 7 CS MSU Graphics & Media Lab (Video Group)
  7. 7. Only for Maxus  Adaptive Fuzzy Post-Filtering Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004. 8 CS MSU Graphics & Media Lab (Video Group)
  8. 8. Only for Maxus  Adaptive Fuzzy Post-Filtering PSNR 30.61 dB PSNR 30.93 dB Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004. 9 CS MSU Graphics & Media Lab (Video Group)
  9. 9. Only for Maxus  Adaptive Fuzzy Post-Filtering  Преимущества  Быстрая работа  Простота в реализации  Недостатки  Просто маскировка артефактов  Замыливание изображения 10 CS MSU Graphics & Media Lab (Video Group)
  10. 10. Only for Maxus  Содержание  Введение  Простые модели  Adaptive Fuzzy Post-Filtering  DCT Re-application  Классификация  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация  Заключение 11 CS MSU Graphics & Media Lab (Video Group)
  11. 11. Only for Maxus  DCT Re-application 1. Сдвинуть изображение по вертикали и горизонтали на (i,j) 2. Сжать изображение 3. Вернуть изображение обратно, т.е. сдвинуть на (-i,-j) 4. Повторить для всех сдвигов из [-3, 4] x [-3, 4] 5. Усреднить результат Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002. 12 CS MSU Graphics & Media Lab (Video Group)
  12. 12. Only for Maxus  DCT Re-application Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria 13 Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, CS MSU Graphics & Media Lab (Video Group) Richardson. 2002.
  13. 13. Only for Maxus  DCT Re-application Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria 14 Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, CS MSU Graphics & Media Lab (Video Group) Richardson. 2002.
  14. 14. Only for Maxus  DCT Re-application y   i , j D ( i ,  j )  QD i , j  x  i, j x - сжатое изображени е y - восстановл енное изображени е D - оператор сдвига Q - оператор кодирования и декодирования  i, j Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria 15 Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, CS MSU Graphics & Media Lab (Video Group) Richardson. 2002.
  15. 15. Only for Maxus  DCT Re-application Пример работы Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002. 16 CS MSU Graphics & Media Lab (Video Group)
  16. 16. Only for Maxus  DCT Re-application Пример работы Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002. 17 CS MSU Graphics & Media Lab (Video Group)
  17. 17. Only for Maxus  DCT Re-application Пример работы Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002. 18 CS MSU Graphics & Media Lab (Video Group)
  18. 18. Only for Maxus  DCT Re-application  Преимущества  Заметное уменьшение артефактов  Прост в реализации  Недостатки  Желательно знать параметры сжатия  Ориентирован на изображения  Невысокая скорость работы (~58 умножений на пиксель) 19 CS MSU Graphics & Media Lab (Video Group)
  19. 19. Only for Maxus  Содержание  Введение  Простые модели  Adaptive Fuzzy Post-Filtering  DCT Re-application  Классификация  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация  Заключение 20 CS MSU Graphics & Media Lab (Video Group)
  20. 20. Only for Maxus  Support Vector Regression  Support Vector Machine Compression Artifact Reduction Using Support Vector Regression. 21 CS MSU Graphics & Media Lab (Video Group) Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
  21. 21. Only for Maxus  Support Vector Regression y  f (x) - моделируемая функция xi , yi  - тренировочные данные y   (w), x  b - вид модели Минимизация w, w  C  i  i*  l 1 2 i 1 при условии  yi  w,  ( xi )  b    i   допустимое отклонение для всех элементов   w,  ( xi )  b  yi     i* i , i*  отклонение для каждого элемента  ,  *  0  i i Compression Artifact Reduction Using Support Vector Regression. 22 CS MSU Graphics & Media Lab (Video Group) Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
  22. 22. Only for Maxus  Support Vector Regression Compression Artifact Reduction Using Support Vector Regression. 23 CS MSU Graphics & Media Lab (Video Group) Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
  23. 23. Only for Maxus  Support Vector Regression Compression Artifact Reduction Using Support Vector Regression. 24 CS MSU Graphics & Media Lab (Video Group) Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
  24. 24. Only for Maxus  Support Vector Regression Compression Artifact Reduction Using Support Vector Regression. 25 CS MSU Graphics & Media Lab (Video Group) Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006
  25. 25. Only for Maxus  Support Vector Regression  Преимущества  Визуальное улучшение на некоторых типах изображений  Недостатки  Зависимость от обучающей выборки  Сложно прогнозируемое поведение 26 CS MSU Graphics & Media Lab (Video Group)
  26. 26. Only for Maxus  Содержание  Введение  Простые модели  Adaptive Fuzzy Post-Filtering  DCT Re-application  Классификация  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация  Заключение 27 CS MSU Graphics & Media Lab (Video Group)
  27. 27. Only for Maxus  MMRCVQ (Modified Mean-Removed Classified Vector Quantization) Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004) 28 CS MSU Graphics & Media Lab (Video Group)
  28. 28. Only for Maxus  MMRCVQ Построение кодирующих последовательностей  Выбор блоков 4x4 T '  x'i : i  1,, N   сжатые данные T  xi : i  1,, N   исходные данные  Нормировка z ' i  x' i  x ' i , zi  xi  xi  Кластеризация Алгоритм Linde-Buzo-Gray Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004) 29 CS MSU Graphics & Media Lab (Video Group)
  29. 29. Only for Maxus  MMRCVQ Linde-Buzo-Gray Выбор начального разбиения Удвоение числа Оценка K-means кластеров дисперсии Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004) 30 CS MSU Graphics & Media Lab (Video Group)
  30. 30. Only for Maxus  MMRCVQ Linde-Buzo-Gray 31 CS MSU Graphics & Media Lab (Video Group)
  31. 31. Only for Maxus  MMRCVQ Linde-Buzo-Gray 32 CS MSU Graphics & Media Lab (Video Group)
  32. 32. Only for Maxus  MMRCVQ  Выбор кодирующих последовательностей C '  y'i : i  1,..., M   Разбиение несжатых данных Rk  xi | d ( x'i , yk )  d ( x'i , yl ): i  l k  1...M , i  1...N  Выбор декодирующих последовательностей x x j Ri j yi  Ri Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004) 33 CS MSU Graphics & Media Lab (Video Group)
  33. 33. Only for Maxus  MMRCVQ Восстановление - Классификация Граница блока Определение Однородный Текстура направления границы Отсечение Поиск плохих Замена блока результатов + Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004) 34 CS MSU Graphics & Media Lab (Video Group)
  34. 34. Only for Maxus  MMRCVQ Результаты Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004) 35 CS MSU Graphics & Media Lab (Video Group)
  35. 35. Only for Maxus  MMRCVQ Результаты Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004) 36 CS MSU Graphics & Media Lab (Video Group)
  36. 36. Only for Maxus  MMRCVQ  Преимущества  Хорошее качество  Приемлемая скорость восстановления  Недостатки  Только устранение блочности 37 CS MSU Graphics & Media Lab (Video Group)
  37. 37. Only for Maxus  Содержание  Введение  Простые модели  Adaptive Fuzzy Post-Filtering  DCT Re-application  Классификация  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация  Заключение 38 CS MSU Graphics & Media Lab (Video Group)
  38. 38. Only for Maxus  Regularized Iterative Restoration  f1   g1   n1  f  g  n  f  исходный кадр  2  2  2         g  наблюдаемы й кадр f   , g   , n   fk   gk   nk  n  шум         D  оператор ллинейно деградации        fN    gN    nN    g  Df  n J( f )  f  g  E ( f ) 2 Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 39 CS MSU Graphics & Media Lab (Video Group)
  39. 39. Only for Maxus  Regularized Iterative Restoration Пространственная регуляризация f (i, j )  f (i  1, j ) if i  8l for l  1,2,..., width/8 - 1 QHB f (i, j )     0 otherwise  f (i, j )  f (i  1, j ) if i  8l for l  1,2,..., width/8 - 1 Q HB f (i, j )    0 otherwise QB  QHB f  QVB f 2 2 2 2 QB  QHB f  QVB f  Es ( f )  QHB f l 2  QVB f l 2    Q HB fl 2  QVB f l 2  Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 40 CS MSU Graphics & Media Lab (Video Group)
  40. 40. Only for Maxus  Regularized Iterative Restoration Временная регуляризация    f (i, j )  f i  m  width height 2 MFD ( f k , f l )  l k (i , j ) , j  n(i, j ) (i , j ) i 1 j 1 m (i , j )  , n(i, j ) (i , j ) - вектор движения пикселя (i, j ) Et ( f l )   MFD ( f k , f l ) k l MC 2 Et ( f l )  f l  f l Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 41 CS MSU Graphics & Media Lab (Video Group)
  41. 41. Only for Maxus  Regularized Iterative Restoration Решение  Цель J ( f )  f  g  s Es ( f )  t Et ( f ) 2  Решение   f i k 1  f i k   g  I  S1 QB QB  S2 QB QB  t  f i  f i MC  f i k T T   Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 42 CS MSU Graphics & Media Lab (Video Group)
  42. 42. Only for Maxus  Regularized Iterative Restoration Многокадровое восстановление ~  f  f1T , f 2T ,..., f LT T ~  L 2 ~ ~ J ( f )    f l  g l   s Es ( f )  t Et ( f )  l 1  ~ L l 1  Es ( f )   1 QHB f l 2  QVB f l 2    Q 2 HB fl 2  QVB f l 2  L ~ Et ( f )   f l  f l MC 2 l 1 Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 43 CS MSU Graphics & Media Lab (Video Group)
  43. 43. Only for Maxus  Regularized Iterative Restoration Многокадровое восстановление ~ ~ ~2 ~2 ~2 ~ ~ MC 2 J ( f )  f  g  S1 QB f  S2 QB f  t f  f   ~ ~ MC   f f ~ k 1 ~ k f ~   ~k  f    g  I  S1 QB QB  S 2 QB QB f  t  T T  ~    f ~ ~k     f  f  Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 44 CS MSU Graphics & Media Lab (Video Group)
  44. 44. Only for Maxus  Regularized Iterative Restoration Примеры работы # of case 1 spatial only 2 16x16 ME 3 4x4 ME 4 dense 4x4 ME bidirectional 4x4 5 ME dense bidirectional 6 4x4 ME Compressed Video Enhancement using Regularized Iterative Restoration 45 CS MSU Graphics & Media Lab (Video Group) Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
  45. 45. Only for Maxus  Regularized Iterative Restoration Примеры работы dense 4x4 ME Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 46 CS MSU Graphics & Media Lab (Video Group)
  46. 46. Only for Maxus  Regularized Iterative Restoration Примеры работы dense bidirectional 4x4 ME Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999 47 CS MSU Graphics & Media Lab (Video Group)
  47. 47. Only for Maxus  Regularized Iterative Restoration Оптимизация 2 J ( f )  f  g  s QB f  t f  f MC 2 - цель минимизации  J ( f )  f  g  s  QBi , j f  t f  f MC 2  i, j  идея: добавить вспомогательную переменную, не изменяющую минимум J  t   inf bt 2  b  b  ' t  bt   - дополнительная переменная 2t J * ( f , b)  f  g  s  bi , j QBi , j f 2     b  2 t f  f MC i, j - новая цель минимизации A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003. 48 CS MSU Graphics & Media Lab (Video Group)
  48. 48. Only for Maxus  Regularized Iterative Restoration Оптимизация Алгоритм: Initializa tion : f 0  g repeat {  b n 1  min J * f n , b n  with f n fixed :  b n 1   QB f n f n   f n 1  min J * f n , b n 1  with b fixed } Until convergence Минимизация может быть выполнена методом градиентного спуска A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003. 49 CS MSU Graphics & Media Lab (Video Group)
  49. 49. Only for Maxus  Regularized Iterative Restoration Оптимизация i, j  2 i, j  j  2   j i, j  J * ( f n , b n1 )   f i ,nj  f i ,nj1  s  bin, 1 QB i , j f n1  bin, 1  t  f i ,nj1  f i ,MC j 2 s bin, 1 n 1 fi, j  1 1  t xi , j  n j 1  t 1  2sbin, 1  t   fi,nj  fi,nj 1  j t 1  s bin, 1  t  s t bin, 1  f i ,mc  j j f i ,mc 1 1  t 1  2sbin, 1  t  j 1  t 1  2sbin, 1  t  j  j j A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003. 50 CS MSU Graphics & Media Lab (Video Group)
  50. 50. Only for Maxus  Regularized Iterative Restoration Пример работы A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and 51 CS MSU Graphics & Media Lab (Video Group) Weisi Lin. 2003.
  51. 51. Only for Maxus  Regularized Iterative Restoration Пример работы A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi 52 CS MSU Graphics & Media Lab (Video Group) Lin. 2003.
  52. 52. Only for Maxus  Regularized Iterative Restoration Пример работы A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003. 53 CS MSU Graphics & Media Lab (Video Group)
  53. 53. Only for Maxus  Regularized Iterative Restoration  Преимущества  Высокое качество восстановления  Недостатки  Очень медленная работа  Сложен в реализации 54 CS MSU Graphics & Media Lab (Video Group)
  54. 54. Only for Maxus  Заключение  Рассмотрены методы  Adaptive Fuzzy Post-Filtering  DCT Re-application  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация 55 CS MSU Graphics & Media Lab (Video Group)
  55. 55. Only for Maxus  Литература • A Coding Artifacts Removal Algorithm Based On Spatial And Temporal Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin. 2003.  Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751 (2004)  Compressed Video Enhancement using Regularized Iterative Restoration Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999  Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006  Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.  Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria Nosratinia. Department of Electrical Engineering, University of Texas at Dallas, Richardson. 2002.  A Tutorial on Support Vector Regression. Alex J. Smola† and Bernhard Sch¨olkop. 2003  Application Of The Motion Vector Constraint To The Regularized Enhancement Of Compressed Video. C. Andrew Segall And Aggelos K. Katsaggelos. 2001 56 CS MSU Graphics & Media Lab (Video Group)
  56. 56. Only for Maxus  Вопросы ? 57 CS MSU Graphics & Media Lab (Video Group)

×