Super Resolution для сжатого видео

2,425 views
2,270 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,425
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Super Resolution для сжатого видео

  1. 1. Super Resolution для сжатого видео Моисейцев Алексей Video Group CS MSU Graphics & Media Lab
  2. 2. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 2 CS MSU Graphics & Media Lab (Video Group)
  3. 3. Only for Maxus  Введение 3 CS MSU Graphics & Media Lab (Video Group)
  4. 4. Only for Maxus  Введение Lossless 2Mbps 1Mbps 256Kbps Bi-cubic source Super-res Super-Resolving Compressed Video with Large Artifacts, Wen-Yi Zhao, ICPR 4 CS MSU Graphics & Media Lab (Video Group) 2004
  5. 5. Only for Maxus  Введение Алгоритмы SR:  Iterative Backprojection (IBP)  Projection Onto Convex Sets (POCS)  Probabilistic Methods  Maximum a posteriori (MAP)  Model-based approach (MBSR)  Example-based 6 CS MSU Graphics & Media Lab (Video Group)
  6. 6. Only for Maxus  Введение IBP POCS C2 C1 C f=Pg P1g P2g C3 P3g g 7 CS MSU Graphics & Media Lab (Video Group)
  7. 7. Only for Maxus  Введение 8 CS MSU Graphics & Media Lab (Video Group) Super resolution: an overview, C Papathanassiou and M Petrou, Geoscience and Remote Sensing Symposium, 2005
  8. 8. Only for Maxus  Введение Типичная скорость работы: 0.15-0.4 fps 9 CS MSU Graphics & Media Lab (Video Group)
  9. 9. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 10 CS MSU Graphics & Media Lab (Video Group)
  10. 10. Only for Maxus  Example-based SR  В SR только по LR-кадрам существует теоретический предел качества  Иногда есть доступ и к отдельным кадрам в высоком разрешении 11 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  11. 11. Only for Maxus  Example-based SR Registration Для компенсации движения используется алгоритм Lucas-Kanade: 12 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  12. 12. Only for Maxus  Example-based SR Registration В 99% случаев ошибка не превышает 0.15 LR-пикселя и 0.45 HR-пикселя 13 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  13. 13. Only for Maxus  Example-based SR Построение пар HR-LR  Используются LR-блоки 8x8  Рассматриваются повороты блоков  Ключ — 10 AC-коэффициентов LR- блока и 68 граничных пикселей HR-блока 14 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  14. 14. Only for Maxus  Example-based SR Восстановление 15 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  15. 15. Only for Maxus  Example-based SR Восстановление 16 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  16. 16. Only for Maxus  Example-based SR Результат source, frame 0 compressed frame 20 SR, frame 20  MJPEG, Q=50  Обучение на первом кадре  Восстановление двадцатого кадра 17 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  17. 17. Only for Maxus  Example-based SR Результат compressed frame 20 SR, frame 20 Spatial example-based SR, frame 20 18 CS MSU Graphics & Media Lab (Video Group) Resolution enhancement of lowKlamer Schutte,using a high-resolution frame, Tuan Phama, Lucas van Vlieta, quality videos SPIE 2006
  18. 18. Only for Maxus  Key-frame based SR Идея: использовать HR ключевые кадры 19 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  19. 19. Only for Maxus  Key-frame based SR Восстановление 20 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  20. 20. Only for Maxus  Key-frame based SR Результаты 21 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  21. 21. Only for Maxus  Key-frame based SR Результаты 22 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  22. 22. Only for Maxus  Key-frame based SR Результаты 23 CS MSU Graphics & Media Lab (Video Group) Super Resolution ofMukherjee, ISCASFrames, Fernanda Brandi, Ricardo de Queiroz, Debargha Video Using Key 2008
  23. 23. Only for Maxus  Example-based SR  Слабая обоснованность метода  Требуется наличие специфичного видеопотока  Зависимость от обучающей выборки 24 CS MSU Graphics & Media Lab (Video Group)
  24. 24. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 25 CS MSU Graphics & Media Lab (Video Group)
  25. 25. Only for Maxus  SRME Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 26 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  26. 26. Only for Maxus  SRME Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 27 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  27. 27. Only for Maxus  SRME — апостериорная вероятность — интересующее решение Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 28 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  28. 28. Only for Maxus  SRME Задание вероятностей (1) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 29 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  29. 29. Only for Maxus  SRME Задание вероятностей (2) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 30 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  30. 30. Only for Maxus  SRME Задание вероятностей (3) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 31 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  31. 31. Only for Maxus  SRME Решение (1) Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 32 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  32. 32. Only for Maxus  SRME Решение (2) Две итерации:  Компенсация движения  Построение улучшенного кадра Simultaneous motion estimation and resolution enhancement of compressed low resolution video. 33 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  33. 33. Only for Maxus  SRME source Y-PSNRSimultaneous motion estimation and resolution enhancement of = 20.44dB compressed low resolution video. 34 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  34. 34. Only for Maxus  SRME bilinear Y-PSNRSimultaneous motion estimation and resolution enhancement of = 21.98dB compressed low resolution video. 35 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  35. 35. Only for Maxus  SRME proposed SR Y-PSNRSimultaneous motion estimation and resolution enhancement of = 25.64dB compressed low resolution video. 36 CS MSU Graphics & Media Lab (Video Group) Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000
  36. 36. Only for Maxus  Содержание  Введение  Example-based SR  SRME  HMRF SR 37 CS MSU Graphics & Media Lab (Video Group)
  37. 37. Only for Maxus  HMRF SR Standard decompression Smoothed image Sharpened image Restoration of Compressed Video using Temporal Information, 38 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  38. 38. Only for Maxus  HMRF SR Restoration of Compressed Video using Temporal Information, 39 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  39. 39. Only for Maxus  HMRF SR Шум квантования (1) Restoration of Compressed Video using Temporal Information, 40 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  40. 40. Only for Maxus  HMRF SR Шум квантования (2) Restoration of Compressed Video using Temporal Information, 41 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  41. 41. Only for Maxus  HMRF SR Шум компенсации (1) Restoration of Compressed Video using Temporal Information, 42 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  42. 42. Only for Maxus  HMRF SR Шум компенсации (2) — общее выражение — независимость шума Restoration of Compressed Video using Temporal Information, 43 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  43. 43. Only for Maxus  HMRF SR Условие гладкости Restoration of Compressed Video using Temporal Information, 44 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  44. 44. Only for Maxus  HMRF SR — гладкость — квантование — компенсация Restoration of Compressed Video using Temporal Information, 45 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  45. 45. Only for Maxus  HMRF SR Решение Градиент: Restoration of Compressed Video using Temporal Information, 46 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  46. 46. Only for Maxus  HMRF SR Решение Упрощение: Градиентный спуск: Restoration of Compressed Video using Temporal Information, 47 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  47. 47. Only for Maxus  HMRF SR Результаты Restoration of Compressed Video using Temporal Information, 48 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  48. 48. Only for Maxus  HMRF SR Результаты Restoration of Compressed Video using Temporal Information, 49 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  49. 49. Only for Maxus  HMRF SR Результаты Restoration of Compressed Video using Temporal Information, 50 CS MSU Graphics & Media Lab (Video Group) Mark A. Robertson and Robert L. Stevenson, SPIE 2001
  50. 50. Only for Maxus  Заключение  Рассмотрены Example-based и MAP методы  Многие методы используют информацию из потока или требуют дополнительные данные для работы  Прямые реализации алгоритмов сложны, но тем не менее многие из них возможно распараллелить (например, с использованием CUDA) 51 CS MSU Graphics & Media Lab (Video Group)
  51. 51. Only for Maxus  Литература  Simultaneous motion estimation and resolution enhancement of compressed low resolution video. Javier Mateos, A. K. Katsaggelos, Rafael Molina, ICIP, 2000  Restoration of Compressed Video using Temporal Information, Mark A. Robertson and Robert L. Stevenson, SPIE 2001  DCT Quantization Noise in Compressed Images, Mark A. Robertson and Robert L. Stevenson, 2004  Super-Resolving Compressed Video with Large Artifacts, Wen-Yi Zhao, ICPR 2004  Z. Lin and H-Y. Shum. Fundamental limits of reconstruction-based superresolution algorithms under local translation. PAMI, 2004  Resolution enhancement of low quality videos using a high-resolution frame, Tuan Phama, Lucas van Vlieta, Klamer Schutte, SPIE 2006  T.Q. Pham, M. Bezuijen, L.J. van Vliet, K. Schutte, and C.L. Luengo Hendriks. Performance of optimal registration estimators, SPIE 2005  Super resolution: an overview, C Papathanassiou and M Petrou, Geoscience and Remote Sensing Symposium, 2005  Super Resolution of Video Using Key Frames, Fernanda Brandi, Ricardo de Queiroz, Debargha Mukherjee, ISCAS 2008 52 CS MSU Graphics & Media Lab (Video Group)
  52. 52. Only for Maxus  Вопросы ? 53 CS MSU Graphics & Media Lab (Video Group)

×