De-Interlacing Techniques<br />Ramesh Prasad<br />
Introduction<br />De-interlacing is the process of converting interlaced video, such as common analog television signals o...
Introduction<br />Progressive Frame<br />Odd Field<br />Even Field<br />
Introduction<br />Interlacing is a form of “Spatio-temporal” sub sampling<br />De-interlacing is the reverse operation, ai...
Introduction<br />
Techniques for De-Interlacing<br />Intra field de-interlacing<br />Inter field de-interlacing<br />Spatio-Temporal Interpo...
Intra field de-interlacing - Overview<br />Based on Spatial filtering<br />Exploits the correlation between vertically nei...
Intra field de-interlacing – Scan line duplication (Bob)<br />The missing line is generated by duplicating the line direct...
Intra field de-interlacing – Scan line duplication (Bob)<br />Jagged Effect<br />Flicker<br />Flicker<br />
Intra field de-interlacing – Scan line interpolation<br />The missing line is generated by taking the average of the lines...
Intra field de-interlacing – Edge Line Average (ELA)<br />Interpolation is done in the direction of edge<br />Three pixels...
Intra field de-interlacing – Edge Line Average (ELA)<br />
Inter field de-interlacing - Overview<br />Based on Temporal filtering<br />Exploits the correlation between temporally ne...
Inter field de-interlacing – Field Insertion (Weave)<br /><ul><li>The two fields are “woven” into one frame
It is best solution in case of still images as all vertical frequencies are preserved
If there is motion between the even and odd fields being woven; artifact called serration or mouse teeth occurs because th...
Inter field de-interlacing – Field Insertion (Weave)<br />Difference between Bob and Weave<br />Bob<br />Weave<br />
Inter field de-interlacing – Field Insertion (Weave)<br />Mouse Teeth Artifact<br />
Inter field de-interlacing – Temporal Averaging<br />Types -<br />Mean – <br />Display a half-picture that is created as t...
Inter field de-interlacing – Temporal Averaging<br />Ghosting Artifact<br />
Spatio-Temporal Interpolation - Overview<br />Based on Spatio-Temporal filtering<br />Exploits the correlation between bot...
Spatio-Temporal Interpolation<br />
Motion adaptive de-interlacing - Overview<br />It combines the advantages of both intra-field and inter-field interlacing<...
Motion adaptive de-interlacing<br />
Motion adaptive de-interlacing<br />The motion detection block detects the motion between the two fields<br />The output o...
Motion adaptive de-interlacing<br />Instead of a switch, a fade mechanism could be used which combines the intra and inter...
Fmot is the result of interpolation for moving image parts
Motion detector determines the mix factor “alpha” with “alpha” = 0 for very high motion and “alpha” = 1 in absence of moti...
Motion adaptive de-interlacing – Motion Detection<br />
Motion adaptive de-interlacing – Motion Detection<br />4-Field motion detection<br />To improve the accuracy of motion det...
Motion adaptive de-interlacing – Motion Detection<br />
Motion adaptive de-interlacing – Motion Detection<br />
Motion adaptive de-interlacing – Implicit Adaptation (Median Filtering)<br />Median Filtering adapts implicitly to motion<...
Motion adaptive de-interlacing – Implicit Adaptation (Median Filtering)<br />
Motion compensated de-interlacing - Overview<br />It’s the most advanced technique for interpolation<br />Motion compensat...
Motion compensated de-interlacing<br />
Motion compensated de-interlacing<br />Motion estimation is performed on the fields of same parity (backward and forward f...
Motion compensated de-interlacing<br />Block shifting to match the sampling grid<br />Pixel value in the current interpola...
Upcoming SlideShare
Loading in …5
×

De Interlacing Techniques

5,877 views

Published on

This presentation discusses the various techniques used for video de-interlacing.

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,877
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
151
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

De Interlacing Techniques

  1. 1. De-Interlacing Techniques<br />Ramesh Prasad<br />
  2. 2. Introduction<br />De-interlacing is the process of converting interlaced video, such as common analog television signals or 1080i format HDTV signals, into a non-interlaced form.<br />Interlaced video frame consists of two sub-fields taken in sequence, each sequentially scanned at odd and even lines of the image sensor; analog television employed this technique because it allowed for less transmission bandwidth and matched the properties of CRT screens.<br />However, most current displays are inherently progressive, so the two fields need to be combined to a single frame, which leads to various visual defects which the de-interlacing process should try to avoid.<br />
  3. 3. Introduction<br />Progressive Frame<br />Odd Field<br />Even Field<br />
  4. 4. Introduction<br />Interlacing is a form of “Spatio-temporal” sub sampling<br />De-interlacing is the reverse operation, aiming at removal of sub-sampling artifacts<br />For stationary pictures (no camera or object motion or intensity changes) the de-interlacing is trivial as alternating even and odd fields completely describe the captured scene<br />However, for pictures with camera/object motion and intensity changes (which always is the case with real life scenario); de-interlacing becomes non-trivial and entails processing, as it requires interpolation of picture data that was never transmitted or even captured.<br />A good de-interlacing algorithm should try to avoid interlacing artifacts as much as possible and not sacrifice image quality in the process.<br />
  5. 5. Introduction<br />
  6. 6. Techniques for De-Interlacing<br />Intra field de-interlacing<br />Inter field de-interlacing<br />Spatio-Temporal Interpolation<br />Motion adaptive de-interlacing<br />Motion compensated de-interlacing<br />
  7. 7. Intra field de-interlacing - Overview<br />Based on Spatial filtering<br />Exploits the correlation between vertically neighboring samples in a field when interpolating intermediate pixels<br />They have all pass temporal frequency response which guarantees the absence of motion artifacts<br />Low computational cost<br />Loss of vertical details and artifacts when the object exists in only one parity field<br />
  8. 8. Intra field de-interlacing – Scan line duplication (Bob)<br />The missing line is generated by duplicating the line directly above it<br />Cons-<br />Jagged effect will occur in the oblique line <br />flicker <br />Blur<br />
  9. 9. Intra field de-interlacing – Scan line duplication (Bob)<br />Jagged Effect<br />Flicker<br />Flicker<br />
  10. 10. Intra field de-interlacing – Scan line interpolation<br />The missing line is generated by taking the average of the lines vertically above and below it<br />Less blur, flicker and jagging than scan line duplication method<br />
  11. 11. Intra field de-interlacing – Edge Line Average (ELA)<br />Interpolation is done in the direction of edge<br />Three pixels in the previous scan line and the next scan line are referenced to determine the obvious edge in the image<br />This method can eliminate the blurring effect of the bilinear interpolation and gives sharp/straight edges.<br />
  12. 12. Intra field de-interlacing – Edge Line Average (ELA)<br />
  13. 13. Inter field de-interlacing - Overview<br />Based on Temporal filtering<br />Exploits the correlation between temporally neighboring samples between fields when interpolating intermediate pixels<br />They have all pass spatial frequency response which guarantees the absence of artifacts for stationary pictures<br />Low computational cost but require additional field memory<br />Artifacts in areas where motion occurs<br />
  14. 14. Inter field de-interlacing – Field Insertion (Weave)<br /><ul><li>The two fields are “woven” into one frame
  15. 15. It is best solution in case of still images as all vertical frequencies are preserved
  16. 16. If there is motion between the even and odd fields being woven; artifact called serration or mouse teeth occurs because the moving objects are not shown at the same position for even and odd lines of the single output frame</li></ul>Odd Field<br />Even Field<br />
  17. 17. Inter field de-interlacing – Field Insertion (Weave)<br />Difference between Bob and Weave<br />Bob<br />Weave<br />
  18. 18. Inter field de-interlacing – Field Insertion (Weave)<br />Mouse Teeth Artifact<br />
  19. 19. Inter field de-interlacing – Temporal Averaging<br />Types -<br />Mean – <br />Display a half-picture that is created as the average of the two original half-pictures<br />The Mean algorithm simply pairs the original lines, and averages each pair into one output line. Line 1 of output is the mean of lines 1 and 2 in input, line 2 of output is the mean of lines 3 and 4 in input, and so on.<br />Blend – <br />Each line of the picture is created as the average of a line from the odd and a line from the even half-pictures. <br />This ignores the fact that they are supposed to be displayed at different times <br />The first line of output is copied from the first line of input. For any other output line N, the line is the mean of input lines N and N-1. That is, the second line of output is the mean of lines 1 and 2 in input, the third line is the mean of lines 2 and 3, ... and finally, the last line of output is the mean of the last two lines in input. The sliding averaging procedure preserves the original vertical resolution.<br />Pros – Mouse Teeth artifact is avoided<br />Cons – Artifact called ghosting is introduced<br />
  20. 20. Inter field de-interlacing – Temporal Averaging<br />Ghosting Artifact<br />
  21. 21. Spatio-Temporal Interpolation - Overview<br />Based on Spatio-Temporal filtering<br />Exploits the correlation between both spatially and temporally neighboring samples when interpolating intermediate pixels<br />The filter is usually designed such that the contribution from the neighboring fields is limited to the higher vertical frequencies<br />Hence, motion artifacts are absent for objects without vertical details that move horizontally<br />The vertical detail from the previous field is being combined with the temporally shifted current field, so some motion artifacts may occur.<br />
  22. 22. Spatio-Temporal Interpolation<br />
  23. 23. Motion adaptive de-interlacing - Overview<br />It combines the advantages of both intra-field and inter-field interlacing<br />It detects the motion first, and then adopts intra field de-interlacing in motion areas and inter-field de-interlacing in static areas<br />High resolution and flicker free picture can be realized in motion and static area<br />It relies on accurate motion detection; erroneous detection cause artifacts<br />
  24. 24. Motion adaptive de-interlacing<br />
  25. 25. Motion adaptive de-interlacing<br />The motion detection block detects the motion between the two fields<br />The output of the motion detection block controls the selector switch for the intra or inter interpolated values<br />For moving image parts, intra interpolated values are passed and for static image parts inter interpolated values are passed<br />Finally an interleaving switch interleaves the current field and the interpolated sample to form the progressive frame<br />The intra and inter interpolation mechanism could be any of the techniques described earlier<br />
  26. 26. Motion adaptive de-interlacing<br />Instead of a switch, a fade mechanism could be used which combines the intra and inter interpolated samples<br /><ul><li>Fst is the result of interpolation for static image parts
  27. 27. Fmot is the result of interpolation for moving image parts
  28. 28. Motion detector determines the mix factor “alpha” with “alpha” = 0 for very high motion and “alpha” = 1 in absence of motion</li></li></ul><li>Motion adaptive de-interlacing – Motion Detection<br />Same-parity-field detection<br />For interlaced video, the vertical position of even field and odd field is slightly different<br />Same-parity-field detection detects the motion area by the even-field to even-field or odd-field to odd-field difference<br />
  29. 29. Motion adaptive de-interlacing – Motion Detection<br />
  30. 30. Motion adaptive de-interlacing – Motion Detection<br />4-Field motion detection<br />To improve the accuracy of motion detection, 4 fields rather than 2 fields are used<br />The fields are labeled backward, current, forward and forward-forward<br />The differences are computed for fields of same parity; backward-forward and current-forward-forward<br />
  31. 31. Motion adaptive de-interlacing – Motion Detection<br />
  32. 32. Motion adaptive de-interlacing – Motion Detection<br />
  33. 33. Motion adaptive de-interlacing – Implicit Adaptation (Median Filtering)<br />Median Filtering adapts implicitly to motion<br />The simplest version is a 3-tap spatio-temporal filter which uses the immediate vertical up, down and temporal neighbor in the previous field<br />The underlying assumption is that in case of stationarity, the temporal neighbor is likely to have a value between that of the vertical neighbors of the current field <br />In case of motion, intra-field interpolation is likely to result, since the correlation between the samples in the current field is likely to be highest<br />Hence Median filtering automatically realizes the intra/inter switch<br />Pros: Superior properties at vertical edges and low hardware cost<br />Cons: Distortion of vertical details and introduction of alias<br />
  34. 34. Motion adaptive de-interlacing – Implicit Adaptation (Median Filtering)<br />
  35. 35. Motion compensated de-interlacing - Overview<br />It’s the most advanced technique for interpolation<br />Motion compensated method try to interpolate in the direction with highest correlation; with motion vector available, this is the interpolation along the motion trajectory<br />It allows the moving sequences to be converted virtually into stationary ones and use methods which work well on stationary sequences like line insertion etc.<br />Motion compensated methods needs to be combined with scene change detection, otherwise it will attempt to find motion between two completely different scenes<br />Usually a combination of Edge directed spatial interpolation and motion compensated temporal interpolation is used in practice<br />It outperforms other techniques in terms of quality of the de-interlaced frame<br />It’s the most complex and processing intensive technique as it involves motion estimation<br />
  36. 36. Motion compensated de-interlacing<br />
  37. 37. Motion compensated de-interlacing<br />Motion estimation is performed on the fields of same parity (backward and forward fields) as they have the same sampling grid<br />Half pel/quarter pel accuracy can be used for motion estimation<br />As the backward and forward fields are of different parity than the current field, the blocks need to be shifted to the sampling grid of the current field<br />To obtain the position of the block in the current interpolated field from the previous field, the motion vectors are halved and the blocks of backward, current and forward field moved simultaneously to the sampling grid of the current interpolated field<br />
  38. 38. Motion compensated de-interlacing<br />Block shifting to match the sampling grid<br />Pixel value in the current interpolated field<br />
  39. 39. Motion compensated de-interlacing<br />To handle scene change detection, the pixel values obtained by motion compensated interpolation are compared with the spatial edge directed interpolated pixel values<br />If the difference is beyond the threshold value, its assumed that interpolation based on motion estimation is incorrect<br />In such case, directional interpolation is used instead of motion compensated interpolation<br />The motion compensated/edge directed interpolated pixels are finally combined to get the progressive frame<br />
  40. 40. References<br />Altera White Paper - High-Definition Video Deinterlacing Using FPGAs<br />Deinterlacing - An Overview<br />GERARD DE HAAN, ERWIN B. BELLERS<br />Deinterlacing using directional interpolation and motion compensation<br />Ohjae Kwon, KwanghoonSohn, Chulhee Lee<br />Motion Adaptive Interpolation with Horizontal Motion Detection for Deinterlacing<br />Shyh-Feng Lin, Yu-Ling Chang, and Liang-Gee Chen<br />
  41. 41. Thank You<br />

×