Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Super Resolution для сжатого видео

2,680 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Super Resolution для сжатого видео

  1. 1. Super Resolution для сжатого видео Моисейцев Алексей Video Group CS MSU Graphics & Media Lab
  2. 2. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 2 CS MSU Graphics & Media Lab (Video Group)
  3. 3. Only for Maxus  Введение 3 CS MSU Graphics & Media Lab (Video Group)
  4. 4. Only for Maxus  Введение Lossless 2Mbps 1Mbps 256Kbps Bi-cubic source Super-res Super-Resolving Compressed Video with Large Artifacts, Wen-Yi Zhao, ICPR 4 CS MSU Graphics & Media Lab (Video Group) 2004
  5. 5. Only for Maxus  Введение Алгоритмы SR:  Iterative Backprojection (IBP)  Projection Onto Convex Sets (POCS)  Probabilistic Methods  Maximum a posteriori (MAP)  Model-based approach (MBSR)  Example-based 6 CS MSU Graphics & Media Lab (Video Group)
  6. 6. Only for Maxus  Введение IBP POCS C2 C1 C f=Pg P1g P2g C3 P3g g 7 CS MSU Graphics & Media Lab (Video Group)
  7. 7. Only for Maxus  Введение 8 CS MSU Graphics & Media Lab (Video Group) Super resolution: an overview, C Papathanassiou and M Petrou, Geoscience and Remote Sensing Symposium, 2005
  8. 8. Only for Maxus  Введение Типичная скорость работы: 0.15-0.4 fps 9 CS MSU Graphics & Media Lab (Video Group)
  9. 9. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 10 CS MSU Graphics & Media Lab (Video Group)
  10. 10. Only for Maxus  Example-based SR  В SR только по LR-кадрам существует теоретический предел качества  Иногда есть доступ и к отдельным кадрам в высоком разрешении 11 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  11. 11. Only for Maxus  Example-based SR Registration Для компенсации движения используется алгоритм Lucas-Kanade: 12 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  12. 12. Only for Maxus  Example-based SR Registration В 99% случаев ошибка не превышает 0.15 LR-пикселя и 0.45 HR-пикселя 13 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  13. 13. Only for Maxus  Example-based SR Построение пар HR-LR  Используются LR-блоки 8x8  Рассматриваются повороты блоков  Ключ — 10 AC-коэффициентов LR- блока и 68 граничных пикселей HR-блока 14 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  14. 14. Only for Maxus  Example-based SR Восстановление 15 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  15. 15. Only for Maxus  Example-based SR Восстановление 16 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  16. 16. Only for Maxus  Example-based SR Результат source, frame 0 compressed frame 20 SR, frame 20  MJPEG, Q=50  Обучение на первом кадре  Восстановление двадцатого кадра 17 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  17. 17. Only for Maxus  Example-based SR Результат compressed frame 20 SR, frame 20 Spatial example-based SR, frame 20 18 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  18. 18. Only for Maxus  Key-frame based SR Идея: использовать HR ключевые кадры 19 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  19. 19. Only for Maxus  Key-frame based SR Восстановление 20 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  20. 20. Only for Maxus  Key-frame based SR Результаты 21 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  21. 21. Only for Maxus  Key-frame based SR Результаты 22 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  22. 22. Only for Maxus  Key-frame based SR Результаты 23 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  23. 23. Only for Maxus  Example-based SR  Слабая обоснованность метода  Требуется наличие специфичного видеопотока  Зависимость от обучающей выборки 24 CS MSU Graphics & Media Lab (Video Group)
  24. 24. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 25 CS MSU Graphics & Media Lab (Video Group)
  25. 25. Only for Maxus  SRME Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 26 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  26. 26. Only for Maxus  SRME Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 27 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  27. 27. Only for Maxus  SRME — апостериорная вероятность — интересующее решение Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 28 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  28. 28. Only for Maxus  SRME Задание вероятностей (1) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 29 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  29. 29. Only for Maxus  SRME Задание вероятностей (2) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 30 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  30. 30. Only for Maxus  SRME Задание вероятностей (3) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 31 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  31. 31. Only for Maxus  SRME Решение (1) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 32 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  32. 32. Only for Maxus  SRME Решение (2) Две итерации:  Компенсация движения  Построение улучшенного кадра Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 33 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  33. 33. Only for Maxus  SRME source Y-PSNRSimultaneous motion estimation and resolution enhancement of = 20.44dB compressed low resolution video. 34 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  34. 34. Only for Maxus  SRME bilinear Y-PSNRSimultaneous motion estimation and resolution enhancement of = 21.98dB compressed low resolution video. 35 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  35. 35. Only for Maxus  SRME proposed SR Y-PSNRSimultaneous motion estimation and resolution enhancement of = 25.64dB compressed low resolution video. 36 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  36. 36. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 37 CS MSU Graphics & Media Lab (Video Group)
  37. 37. Only for Maxus  HMRF SR Standard decompression Smoothed image Sharpened image Restoration of Compressed Video using Temporal Information, 38 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  38. 38. Only for Maxus  HMRF SR Restoration of Compressed Video using Temporal Information, 39 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  39. 39. Only for Maxus  HMRF SR Шум квантования (1) Restoration of Compressed Video using Temporal Information, 40 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  40. 40. Only for Maxus  HMRF SR Шум квантования (2) Restoration of Compressed Video using Temporal Information, 41 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  41. 41. Only for Maxus  HMRF SR Шум компенсации (1) Restoration of Compressed Video using Temporal Information, 42 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  42. 42. Only for Maxus  HMRF SR Шум компенсации (2) — общее выражение — независимость шума Restoration of Compressed Video using Temporal Information, 43 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  43. 43. Only for Maxus  HMRF SR Условие гладкости Restoration of Compressed Video using Temporal Information, 44 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  44. 44. Only for Maxus  HMRF SR — гладкость — квантование — компенсация Restoration of Compressed Video using Temporal Information, 45 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  45. 45. Only for Maxus  HMRF SR Решение Градиент: Restoration of Compressed Video using Temporal Information, 46 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  46. 46. Only for Maxus  HMRF SR Решение Упрощение: Градиентный спуск: Restoration of Compressed Video using Temporal Information, 47 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  47. 47. Only for Maxus  HMRF SR Результаты Restoration of Compressed Video using Temporal Information, 48 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  48. 48. Only for Maxus  HMRF SR Результаты Restoration of Compressed Video using Temporal Information, 49 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  49. 49. Only for Maxus  HMRF SR Результаты Restoration of Compressed Video using Temporal Information, 50 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  50. 50. Only for Maxus  Заключение  Рассмотрены Example-based и MAP методы  Многие методы используют информацию из потока или требуют дополнительные данные для работы  Прямые реализации алгоритмов сложны, но тем не менее многие из них возможно распараллелить (например, с использованием CUDA) 51 CS MSU Graphics & Media Lab (Video Group)
  51. 51. Only for Maxus  Литература  Simultaneous motion estimation and resolution enhancement of compressed low resolution video. Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000  Restoration of Compressed Video using Temporal Information, Mark A. Robertson and Robert L. Stevenson, SPIE 2001  DCT Quantization Noise in Compressed Images, Mark A. Robertson and Robert L. Stevenson, 2004  Super-Resolving Compressed Video with Large Artifacts, Wen-Yi Zhao, ICPR 2004  Z. Lin and H-Y. Shum. Fundamental limits of reconstruction-based superresolution algorithms under local translation. PAMI, 2004  Resolution enhancement of low quality videos using a high-resolution frame, Tuan Phama, Lucas van Vlieta, Klamer Schutte, SPIE 2006  T.Q. Pham, M. Bezuijen, L.J. van Vliet, K. Schutte, and C.L. Luengo Hendriks. Performance of optimal registration estimators, SPIE 2005  Super resolution: an overview, C Papathanassiou and M Petrou, Geoscience and Remote Sensing Symposium, 2005  Super Resolution of Video Using Key Frames, Fernanda Brandi, Ricardo de Queiroz, Debargha Mukherjee, ISCAS 2008 52 CS MSU Graphics & Media Lab (Video Group)
  52. 52. Only for Maxus  Вопросы ? 53 CS MSU Graphics & Media Lab (Video Group)

×