SlideShare a Scribd company logo
1 of 31
Download to read offline
Comparison between Blur Transfer and
Blur Re-Generation in Depth Image
Based Rendering
Norishige Fukushima†, Naoki Kodera†, Yutaka Ishibashi†,
Masayuki Tanimoto‡
July 2-4, 2014 Budapest, Hungary 3DTV-CON 2014
†Graduate School of Engineering, Nagoya Institute of Technology,
Japan
‡Nagoya Industrial Science Research Institute,
Japan
Outline
 Background
 Related Works
 View Synthesis Methods
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Experimental Results
 Conclusion and Future Works
Free Viewpoint
Image
Depth Image
Based Rendering
Original Image Depth Map
Background (1/3)
Background (2/3)
Input view
and
depth map
3D
warping
Hole
filling
Free viewpoint
image
Background (2/3)
Input view
and
depth map
3D
warping
Hole
filling
Free viewpoint
image
Background (3/3)
Blurred region
FB
Background (3/3)
Blurred region
FB
3D warp
to left
Split
Background (3/3)
Blurred region
FB
3D warp
to left
Split
Filled
Background (3/3)
Blurred region
FB
3D warp
to left
Split
Filled
Boundary regions are degraded.
View synthesis method
3 types of blur treatment
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Improved blur transfer method
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Improved blur transfer method (proposed method)
View synthesis method
3 types of blur treatment
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Improved blur transfer method
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Improved blur transfer method (proposed method)
Blur erasing type
3D warp
Erase
Artifact
Blur erasing type
3D warp
Erase
Filled
Blurs are broken.
Artifact
View synthesis method
3 types of blur treatment
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Improved blur transfer method
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Improved blur transfer method (proposed method)
Blur re-generation type
 Generating blurred region using alpha matting
 Splitting input image into three images
→ foreground, background and alpha mask
 Alpha blending three images after DIBR
Input image
Split
Foreground
Alpha mask
Background
Blur re-generation type
Fore
Alpha
Back
3D warp blendMatting
Blur re-generation
View synthesis method
3 types of blur treatment
 Blur erasing type
 Blur re-generation type
 Blur transfer type
 Improved blur transfer method (proposed method)
Blur transfer type
Dilation
Color image
Depth map
Blur transfer type
Dilation
Color image
Depth map
Foreground depth value can
cover almost fuzzy/blurred region.
Blur transfer type
Blur keep
3D warping
Blur transfer type
3D warping
interpolating
Blur keep
Canny and
Gaussian
Filtering
Dilation
DIBR
Blur transfer type
Improved blur transfer method
(proposed method)
Adding a simple process
Improved blur
transfer method
(proposed method)
Improved blur transfer method
(proposed method)
 Generating mask by canny filter
 Smoothing masked region by Gaussian filter
Generate mask
Smooth
Masked region
Comparison
 Blur erasing type
 × Blurs are broken. ◎ Fastest
 Blur re-generation type
 ○ Blurs are reconstructed. ×Slowest
 Blur transfer type
 ○ Blurs are kept. ○ Faster
For better boundary treatment, we compare
blur re-generation type with blur transfer type.
Experimental Results (1/6)
 Evaluating PSNR of these methods
 Basic type [1]
 Blur re-generation type [2]
 Blur transfer type [3]
 Improved blur transfer method (proposed method)
: TeddyH
(950×750)
[1]: Y. Mori et al., Signal Process. Image Commu., vol. 24, no. 1, pp. 65-72, Jan. 2009.
[2]: N. Kodera et al., IEEE VCIP 2013.
[3]: X. Xu et al., IEEE ICASSP 2012.
: Bowling1
(671×555)
: Reindeer
(671×555)
Input depth map is ground truth
Experimental Results (2/6)
PSNR
Basic
Re-
generation
Transfer Proposed
average 35.62 38.85 37.93 38.54
TeddyH 32.74 35.03 34.97 35.13
Reindeer 34.37 38.14 36.75 38.15
Bowling1 34.42 39.72 35.46 35.93
. . . . .
. . . . .
. . . . .
. . . .
(dB)
#Average: using 30 data set
Experimental Results (3/6)
Basic
TeddyH: 950 x 750
Re-generation Proposed
Proposed method can soften contour artifacts,
look like re-generation method.
Experimental Results (4/6)
Reindeer: 671 x 555
Re-generationProposed
 In proposed method, blurs around object
boundary blend background color.
 Proposed method dilates depth value to the
unnecessary region.
Experimental Results (5/6)
Bowling1: 671 x 555
Re-generationProposed
 Proposed method interpolates using
foreground or mixed color.
 Depth maps are not dilated enough.
Experimental Results (6/6)
Computational cost
 Proposed method ≒ Basic method (15ms) + 4ms
 2 dilations for input depth map < 1ms
 Canny filter for depth map on the synthesized image < 3ms
 Gaussian filter with a small kernel < 1ms
 Blur re-generation type > 100ms
 Matting ≒ 55ms
 3 times DIBR ≒ 45ms
 Blending < 2ms
Conclusions
 Comparing 3 types of DIBR method in blur treatment.
 Proposed method improves subjective quality at object boundaries,
and objectively has 0.61dB improvement.
 This is the second best (0.31dB lower than the blur re-generation).
 Proposed method reaches the state of the arts of blur regeneration
method, and computational cost is about x5 effective.
 Proposed (19 ms) vs Blur re-generation (>100 ms)
Future Work
 Investigating effect of proposed method using estimated
depth maps.
 Considering effect of coding distortions.

More Related Content

What's hot

Digital image processing - Image Enhancement (MATERIAL)
Digital image processing  - Image Enhancement (MATERIAL)Digital image processing  - Image Enhancement (MATERIAL)
Digital image processing - Image Enhancement (MATERIAL)Mathankumar S
 
Point processing
Point processingPoint processing
Point processingpanupriyaa7
 
Image Enhancement in Spatial Domain
Image Enhancement in Spatial DomainImage Enhancement in Spatial Domain
Image Enhancement in Spatial DomainDEEPASHRI HK
 
SPATIAL FILTERING IN IMAGE PROCESSING
SPATIAL FILTERING IN IMAGE PROCESSINGSPATIAL FILTERING IN IMAGE PROCESSING
SPATIAL FILTERING IN IMAGE PROCESSINGmuthu181188
 
Smoothing Filters in Spatial Domain
Smoothing Filters in Spatial DomainSmoothing Filters in Spatial Domain
Smoothing Filters in Spatial DomainMadhu Bala
 
Spatial filtering using image processing
Spatial filtering using image processingSpatial filtering using image processing
Spatial filtering using image processingAnuj Arora
 
Image Restoration And Reconstruction
Image Restoration And ReconstructionImage Restoration And Reconstruction
Image Restoration And ReconstructionAmnaakhaan
 
Digital image processing
Digital image processingDigital image processing
Digital image processingABIRAMI M
 
Digital image processing img smoothning
Digital image processing img smoothningDigital image processing img smoothning
Digital image processing img smoothningVinay Gupta
 
Simultaneous Smoothing and Sharpening of Color Images
Simultaneous Smoothing and Sharpening of Color ImagesSimultaneous Smoothing and Sharpening of Color Images
Simultaneous Smoothing and Sharpening of Color ImagesCristina Pérez Benito
 
Spatial enhancement
Spatial enhancement Spatial enhancement
Spatial enhancement abinarkt
 
Chapter 6 Image Processing: Image Enhancement
Chapter 6 Image Processing: Image EnhancementChapter 6 Image Processing: Image Enhancement
Chapter 6 Image Processing: Image EnhancementVarun Ojha
 
Sharpening using frequency Domain Filter
Sharpening using frequency Domain FilterSharpening using frequency Domain Filter
Sharpening using frequency Domain Filterarulraj121
 
Image filtering in Digital image processing
Image filtering in Digital image processingImage filtering in Digital image processing
Image filtering in Digital image processingAbinaya B
 
Recovering high dynamic range radiance maps from photographs
Recovering high dynamic range radiance maps from photographsRecovering high dynamic range radiance maps from photographs
Recovering high dynamic range radiance maps from photographsPrashanth Kannan
 
Image Processing: Spatial filters
Image Processing: Spatial filtersImage Processing: Spatial filters
Image Processing: Spatial filtersA B Shinde
 

What's hot (20)

Sharpening spatial filters
Sharpening spatial filtersSharpening spatial filters
Sharpening spatial filters
 
Digital image processing - Image Enhancement (MATERIAL)
Digital image processing  - Image Enhancement (MATERIAL)Digital image processing  - Image Enhancement (MATERIAL)
Digital image processing - Image Enhancement (MATERIAL)
 
Point processing
Point processingPoint processing
Point processing
 
Image Enhancement in Spatial Domain
Image Enhancement in Spatial DomainImage Enhancement in Spatial Domain
Image Enhancement in Spatial Domain
 
Ppt ---image processing
Ppt ---image processingPpt ---image processing
Ppt ---image processing
 
SPATIAL FILTERING IN IMAGE PROCESSING
SPATIAL FILTERING IN IMAGE PROCESSINGSPATIAL FILTERING IN IMAGE PROCESSING
SPATIAL FILTERING IN IMAGE PROCESSING
 
Smoothing Filters in Spatial Domain
Smoothing Filters in Spatial DomainSmoothing Filters in Spatial Domain
Smoothing Filters in Spatial Domain
 
Spatial filtering using image processing
Spatial filtering using image processingSpatial filtering using image processing
Spatial filtering using image processing
 
Image Restoration And Reconstruction
Image Restoration And ReconstructionImage Restoration And Reconstruction
Image Restoration And Reconstruction
 
Digital image processing
Digital image processingDigital image processing
Digital image processing
 
Digital image processing img smoothning
Digital image processing img smoothningDigital image processing img smoothning
Digital image processing img smoothning
 
Spatial filtering
Spatial filteringSpatial filtering
Spatial filtering
 
Simultaneous Smoothing and Sharpening of Color Images
Simultaneous Smoothing and Sharpening of Color ImagesSimultaneous Smoothing and Sharpening of Color Images
Simultaneous Smoothing and Sharpening of Color Images
 
Spatial enhancement
Spatial enhancement Spatial enhancement
Spatial enhancement
 
Chapter 6 Image Processing: Image Enhancement
Chapter 6 Image Processing: Image EnhancementChapter 6 Image Processing: Image Enhancement
Chapter 6 Image Processing: Image Enhancement
 
Sharpening using frequency Domain Filter
Sharpening using frequency Domain FilterSharpening using frequency Domain Filter
Sharpening using frequency Domain Filter
 
Lecture 7
Lecture 7Lecture 7
Lecture 7
 
Image filtering in Digital image processing
Image filtering in Digital image processingImage filtering in Digital image processing
Image filtering in Digital image processing
 
Recovering high dynamic range radiance maps from photographs
Recovering high dynamic range radiance maps from photographsRecovering high dynamic range radiance maps from photographs
Recovering high dynamic range radiance maps from photographs
 
Image Processing: Spatial filters
Image Processing: Spatial filtersImage Processing: Spatial filters
Image Processing: Spatial filters
 

Viewers also liked

複数台のKinectV2の使い方
複数台のKinectV2の使い方複数台のKinectV2の使い方
複数台のKinectV2の使い方Norishige Fukushima
 
Keynote at 23rd International Display Workshop
Keynote at 23rd International Display WorkshopKeynote at 23rd International Display Workshop
Keynote at 23rd International Display WorkshopChristian Sandor
 
OpenCVの拡張ユーティリティ関数群
OpenCVの拡張ユーティリティ関数群OpenCVの拡張ユーティリティ関数群
OpenCVの拡張ユーティリティ関数群Norishige Fukushima
 
コンピューテーショナルフォトグラフィ
コンピューテーショナルフォトグラフィコンピューテーショナルフォトグラフィ
コンピューテーショナルフォトグラフィNorishige Fukushima
 
コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎Norishige Fukushima
 
Popcntによるハミング距離計算
Popcntによるハミング距離計算Popcntによるハミング距離計算
Popcntによるハミング距離計算Norishige Fukushima
 
ガイデットフィルタとその周辺
ガイデットフィルタとその周辺ガイデットフィルタとその周辺
ガイデットフィルタとその周辺Norishige Fukushima
 
組み込み関数(intrinsic)によるSIMD入門
組み込み関数(intrinsic)によるSIMD入門組み込み関数(intrinsic)によるSIMD入門
組み込み関数(intrinsic)によるSIMD入門Norishige Fukushima
 
OpenCVをAndroidで動かしてみた
OpenCVをAndroidで動かしてみたOpenCVをAndroidで動かしてみた
OpenCVをAndroidで動かしてみた徹 上野山
 
マルチコアを用いた画像処理
マルチコアを用いた画像処理マルチコアを用いた画像処理
マルチコアを用いた画像処理Norishige Fukushima
 
画像処理ライブラリ OpenCV で 出来ること・出来ないこと
画像処理ライブラリ OpenCV で 出来ること・出来ないこと画像処理ライブラリ OpenCV で 出来ること・出来ないこと
画像処理ライブラリ OpenCV で 出来ること・出来ないことNorishige Fukushima
 

Viewers also liked (13)

複数台のKinectV2の使い方
複数台のKinectV2の使い方複数台のKinectV2の使い方
複数台のKinectV2の使い方
 
Keynote at 23rd International Display Workshop
Keynote at 23rd International Display WorkshopKeynote at 23rd International Display Workshop
Keynote at 23rd International Display Workshop
 
Libjpeg turboの使い方
Libjpeg turboの使い方Libjpeg turboの使い方
Libjpeg turboの使い方
 
OpenCVの拡張ユーティリティ関数群
OpenCVの拡張ユーティリティ関数群OpenCVの拡張ユーティリティ関数群
OpenCVの拡張ユーティリティ関数群
 
WebP入門
WebP入門WebP入門
WebP入門
 
コンピューテーショナルフォトグラフィ
コンピューテーショナルフォトグラフィコンピューテーショナルフォトグラフィ
コンピューテーショナルフォトグラフィ
 
コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎
 
Popcntによるハミング距離計算
Popcntによるハミング距離計算Popcntによるハミング距離計算
Popcntによるハミング距離計算
 
ガイデットフィルタとその周辺
ガイデットフィルタとその周辺ガイデットフィルタとその周辺
ガイデットフィルタとその周辺
 
組み込み関数(intrinsic)によるSIMD入門
組み込み関数(intrinsic)によるSIMD入門組み込み関数(intrinsic)によるSIMD入門
組み込み関数(intrinsic)によるSIMD入門
 
OpenCVをAndroidで動かしてみた
OpenCVをAndroidで動かしてみたOpenCVをAndroidで動かしてみた
OpenCVをAndroidで動かしてみた
 
マルチコアを用いた画像処理
マルチコアを用いた画像処理マルチコアを用いた画像処理
マルチコアを用いた画像処理
 
画像処理ライブラリ OpenCV で 出来ること・出来ないこと
画像処理ライブラリ OpenCV で 出来ること・出来ないこと画像処理ライブラリ OpenCV で 出来ること・出来ないこと
画像処理ライブラリ OpenCV で 出来ること・出来ないこと
 

Similar to Comparison between Blur Transfer and Blur Re-Generation in Depth Image Based Rendering

IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...ijistjournal
 
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...ijistjournal
 
G'MIG water color filter tutorial
G'MIG water color filter tutorial G'MIG water color filter tutorial
G'MIG water color filter tutorial Arto Huotari
 
DSP presentation_latest
DSP presentation_latestDSP presentation_latest
DSP presentation_latestHaowei Jiang
 
BilateralFiltering
BilateralFilteringBilateralFiltering
BilateralFilteringJacob Logas
 
Advanced Lighting Techniques Dan Baker (Meltdown 2005)
Advanced Lighting Techniques   Dan Baker (Meltdown 2005)Advanced Lighting Techniques   Dan Baker (Meltdown 2005)
Advanced Lighting Techniques Dan Baker (Meltdown 2005)mobius.cn
 
ICPR2014-Poster
ICPR2014-PosterICPR2014-Poster
ICPR2014-PosterBo Dong
 
study Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded Imagesstudy Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded ImagesChiamin Hsu
 
IMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAIN
IMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAINIMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAIN
IMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAINijma
 
Digital image processing and interpretation
Digital image processing and interpretationDigital image processing and interpretation
Digital image processing and interpretationP.K. Mani
 
Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...
Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...
Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...CSCJournals
 
Image Denoising Using Earth Mover's Distance and Local Histograms
Image Denoising Using Earth Mover's Distance and Local HistogramsImage Denoising Using Earth Mover's Distance and Local Histograms
Image Denoising Using Earth Mover's Distance and Local HistogramsCSCJournals
 
Parallel Processing for Digital Image Enhancement
Parallel Processing for Digital Image EnhancementParallel Processing for Digital Image Enhancement
Parallel Processing for Digital Image EnhancementNora Youssef
 
V.KARTHIKEYAN PUBLISHED ARTICLE
V.KARTHIKEYAN PUBLISHED ARTICLEV.KARTHIKEYAN PUBLISHED ARTICLE
V.KARTHIKEYAN PUBLISHED ARTICLEKARTHIKEYAN V
 
SINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAIN
SINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAINSINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAIN
SINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAINijma
 
Image Restoration (Digital Image Processing)
Image Restoration (Digital Image Processing)Image Restoration (Digital Image Processing)
Image Restoration (Digital Image Processing)Kalyan Acharjya
 
How i made my magazine cover background
How i made my magazine cover backgroundHow i made my magazine cover background
How i made my magazine cover backgroundkorous
 

Similar to Comparison between Blur Transfer and Blur Re-Generation in Depth Image Based Rendering (20)

IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
 
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
IMPROVEMENT OF BM3D ALGORITHM AND EMPLOYMENT TO SATELLITE AND CFA IMAGES DENO...
 
Defocus magnification
Defocus magnificationDefocus magnification
Defocus magnification
 
G'MIG water color filter tutorial
G'MIG water color filter tutorial G'MIG water color filter tutorial
G'MIG water color filter tutorial
 
channel_mzhazbay.pdf
channel_mzhazbay.pdfchannel_mzhazbay.pdf
channel_mzhazbay.pdf
 
DSP presentation_latest
DSP presentation_latestDSP presentation_latest
DSP presentation_latest
 
BilateralFiltering
BilateralFilteringBilateralFiltering
BilateralFiltering
 
Advanced Lighting Techniques Dan Baker (Meltdown 2005)
Advanced Lighting Techniques   Dan Baker (Meltdown 2005)Advanced Lighting Techniques   Dan Baker (Meltdown 2005)
Advanced Lighting Techniques Dan Baker (Meltdown 2005)
 
ICPR2014-Poster
ICPR2014-PosterICPR2014-Poster
ICPR2014-Poster
 
study Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded Imagesstudy Diffusion Curves: A Vector Representation for Smooth-Shaded Images
study Diffusion Curves: A Vector Representation for Smooth-Shaded Images
 
IMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAIN
IMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAINIMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAIN
IMAGE DENOISING BY MEDIAN FILTER IN WAVELET DOMAIN
 
Digital image processing and interpretation
Digital image processing and interpretationDigital image processing and interpretation
Digital image processing and interpretation
 
Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...
Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...
Performance Evaluation of 2D Adaptive Bilateral Filter For Removal of Noise F...
 
Image Denoising Using Earth Mover's Distance and Local Histograms
Image Denoising Using Earth Mover's Distance and Local HistogramsImage Denoising Using Earth Mover's Distance and Local Histograms
Image Denoising Using Earth Mover's Distance and Local Histograms
 
Parallel Processing for Digital Image Enhancement
Parallel Processing for Digital Image EnhancementParallel Processing for Digital Image Enhancement
Parallel Processing for Digital Image Enhancement
 
V.KARTHIKEYAN PUBLISHED ARTICLE
V.KARTHIKEYAN PUBLISHED ARTICLEV.KARTHIKEYAN PUBLISHED ARTICLE
V.KARTHIKEYAN PUBLISHED ARTICLE
 
SINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAIN
SINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAINSINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAIN
SINGLE IMAGE SUPER RESOLUTION IN SPATIAL AND WAVELET DOMAIN
 
Image Restoration (Digital Image Processing)
Image Restoration (Digital Image Processing)Image Restoration (Digital Image Processing)
Image Restoration (Digital Image Processing)
 
How i made my magazine cover background
How i made my magazine cover backgroundHow i made my magazine cover background
How i made my magazine cover background
 
Flat Map
Flat MapFlat Map
Flat Map
 

More from Norishige Fukushima

計算スケジューリングの効果~もし,Halideがなかったら?~
計算スケジューリングの効果~もし,Halideがなかったら?~計算スケジューリングの効果~もし,Halideがなかったら?~
計算スケジューリングの効果~もし,Halideがなかったら?~Norishige Fukushima
 
多チャンネルバイラテラルフィルタの高速化
多チャンネルバイラテラルフィルタの高速化多チャンネルバイラテラルフィルタの高速化
多チャンネルバイラテラルフィルタの高速化Norishige Fukushima
 
計算機アーキテクチャを考慮した高能率画像処理プログラミング
計算機アーキテクチャを考慮した高能率画像処理プログラミング計算機アーキテクチャを考慮した高能率画像処理プログラミング
計算機アーキテクチャを考慮した高能率画像処理プログラミングNorishige Fukushima
 
3次元計測とフィルタリング
3次元計測とフィルタリング3次元計測とフィルタリング
3次元計測とフィルタリングNorishige Fukushima
 
デプスセンサとその応用
デプスセンサとその応用デプスセンサとその応用
デプスセンサとその応用Norishige Fukushima
 

More from Norishige Fukushima (6)

画像処理の高性能計算
画像処理の高性能計算画像処理の高性能計算
画像処理の高性能計算
 
計算スケジューリングの効果~もし,Halideがなかったら?~
計算スケジューリングの効果~もし,Halideがなかったら?~計算スケジューリングの効果~もし,Halideがなかったら?~
計算スケジューリングの効果~もし,Halideがなかったら?~
 
多チャンネルバイラテラルフィルタの高速化
多チャンネルバイラテラルフィルタの高速化多チャンネルバイラテラルフィルタの高速化
多チャンネルバイラテラルフィルタの高速化
 
計算機アーキテクチャを考慮した高能率画像処理プログラミング
計算機アーキテクチャを考慮した高能率画像処理プログラミング計算機アーキテクチャを考慮した高能率画像処理プログラミング
計算機アーキテクチャを考慮した高能率画像処理プログラミング
 
3次元計測とフィルタリング
3次元計測とフィルタリング3次元計測とフィルタリング
3次元計測とフィルタリング
 
デプスセンサとその応用
デプスセンサとその応用デプスセンサとその応用
デプスセンサとその応用
 

Recently uploaded

Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 

Recently uploaded (20)

Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 

Comparison between Blur Transfer and Blur Re-Generation in Depth Image Based Rendering

  • 1. Comparison between Blur Transfer and Blur Re-Generation in Depth Image Based Rendering Norishige Fukushima†, Naoki Kodera†, Yutaka Ishibashi†, Masayuki Tanimoto‡ July 2-4, 2014 Budapest, Hungary 3DTV-CON 2014 †Graduate School of Engineering, Nagoya Institute of Technology, Japan ‡Nagoya Industrial Science Research Institute, Japan
  • 2. Outline  Background  Related Works  View Synthesis Methods  Blur erasing type  Blur re-generation type  Blur transfer type  Experimental Results  Conclusion and Future Works
  • 3. Free Viewpoint Image Depth Image Based Rendering Original Image Depth Map Background (1/3)
  • 4. Background (2/3) Input view and depth map 3D warping Hole filling Free viewpoint image
  • 5. Background (2/3) Input view and depth map 3D warping Hole filling Free viewpoint image
  • 8. Background (3/3) Blurred region FB 3D warp to left Split Filled
  • 9. Background (3/3) Blurred region FB 3D warp to left Split Filled Boundary regions are degraded.
  • 10. View synthesis method 3 types of blur treatment  Blur erasing type  Blur re-generation type  Blur transfer type  Improved blur transfer method  Blur erasing type  Blur re-generation type  Blur transfer type  Improved blur transfer method (proposed method)
  • 11. View synthesis method 3 types of blur treatment  Blur erasing type  Blur re-generation type  Blur transfer type  Improved blur transfer method  Blur erasing type  Blur re-generation type  Blur transfer type  Improved blur transfer method (proposed method)
  • 12. Blur erasing type 3D warp Erase Artifact
  • 13. Blur erasing type 3D warp Erase Filled Blurs are broken. Artifact
  • 14. View synthesis method 3 types of blur treatment  Blur erasing type  Blur re-generation type  Blur transfer type  Improved blur transfer method  Blur erasing type  Blur re-generation type  Blur transfer type  Improved blur transfer method (proposed method)
  • 15. Blur re-generation type  Generating blurred region using alpha matting  Splitting input image into three images → foreground, background and alpha mask  Alpha blending three images after DIBR Input image Split Foreground Alpha mask Background
  • 16. Blur re-generation type Fore Alpha Back 3D warp blendMatting Blur re-generation
  • 17. View synthesis method 3 types of blur treatment  Blur erasing type  Blur re-generation type  Blur transfer type  Improved blur transfer method (proposed method)
  • 19. Blur transfer type Dilation Color image Depth map Foreground depth value can cover almost fuzzy/blurred region.
  • 20. Blur transfer type Blur keep 3D warping
  • 21. Blur transfer type 3D warping interpolating Blur keep
  • 22. Canny and Gaussian Filtering Dilation DIBR Blur transfer type Improved blur transfer method (proposed method) Adding a simple process Improved blur transfer method (proposed method)
  • 23. Improved blur transfer method (proposed method)  Generating mask by canny filter  Smoothing masked region by Gaussian filter Generate mask Smooth Masked region
  • 24. Comparison  Blur erasing type  × Blurs are broken. ◎ Fastest  Blur re-generation type  ○ Blurs are reconstructed. ×Slowest  Blur transfer type  ○ Blurs are kept. ○ Faster For better boundary treatment, we compare blur re-generation type with blur transfer type.
  • 25. Experimental Results (1/6)  Evaluating PSNR of these methods  Basic type [1]  Blur re-generation type [2]  Blur transfer type [3]  Improved blur transfer method (proposed method) : TeddyH (950×750) [1]: Y. Mori et al., Signal Process. Image Commu., vol. 24, no. 1, pp. 65-72, Jan. 2009. [2]: N. Kodera et al., IEEE VCIP 2013. [3]: X. Xu et al., IEEE ICASSP 2012. : Bowling1 (671×555) : Reindeer (671×555) Input depth map is ground truth
  • 26. Experimental Results (2/6) PSNR Basic Re- generation Transfer Proposed average 35.62 38.85 37.93 38.54 TeddyH 32.74 35.03 34.97 35.13 Reindeer 34.37 38.14 36.75 38.15 Bowling1 34.42 39.72 35.46 35.93 . . . . . . . . . . . . . . . . . . . (dB) #Average: using 30 data set
  • 27. Experimental Results (3/6) Basic TeddyH: 950 x 750 Re-generation Proposed Proposed method can soften contour artifacts, look like re-generation method.
  • 28. Experimental Results (4/6) Reindeer: 671 x 555 Re-generationProposed  In proposed method, blurs around object boundary blend background color.  Proposed method dilates depth value to the unnecessary region.
  • 29. Experimental Results (5/6) Bowling1: 671 x 555 Re-generationProposed  Proposed method interpolates using foreground or mixed color.  Depth maps are not dilated enough.
  • 30. Experimental Results (6/6) Computational cost  Proposed method ≒ Basic method (15ms) + 4ms  2 dilations for input depth map < 1ms  Canny filter for depth map on the synthesized image < 3ms  Gaussian filter with a small kernel < 1ms  Blur re-generation type > 100ms  Matting ≒ 55ms  3 times DIBR ≒ 45ms  Blending < 2ms
  • 31. Conclusions  Comparing 3 types of DIBR method in blur treatment.  Proposed method improves subjective quality at object boundaries, and objectively has 0.61dB improvement.  This is the second best (0.31dB lower than the blur re-generation).  Proposed method reaches the state of the arts of blur regeneration method, and computational cost is about x5 effective.  Proposed (19 ms) vs Blur re-generation (>100 ms) Future Work  Investigating effect of proposed method using estimated depth maps.  Considering effect of coding distortions.

Editor's Notes

  1. Today I introduce Comparison between Blur Transfer and Blur Re-Generation in Depth Image Based Rendering. This is a rendering method for view synthesis.
  2. The outline of our presentation is as follows; At first , we introduce background and Related Works Then, we review 3 view synthesis method. Blur erasing type Blur re-generation type Blur transfer type Next, Experimental Results are shown, And finally, we conclude our presentation.
  3. I bileifly introduce a ftee viewpoint rendering method of depth image based rendering DIBR. The DIBR use RGB images and its associated depth maps.
  4. For the free viewpoint image rendering, the pair of the input view and the depth map are warped to the desired viewpoint. Next, oposit side of view is also warped, and then, still remaining holes are filled.
  5. In this presentation, we improve the rendering result of this part. This is the zoomed image.
  6. In this region, fore ground image and background image are mixed.
  7. warping in this condition, the mixed color and the foreground object are split.
  8. and after hole filling, the remained mixed color looks artifact.
  9. Thus the boundary regions are degraded.
  10. To solve the boundary region problem, we introduce 3 type of method. 1st one is Blur erasing type 2nd one is Blur re-generation type 3rd one is Blur transfer type. In this presentation, we compare the three type of the methods. And we make a little improvement for the last type.
  11. The fast type is blur erasing type.
  12. In this type, mixed color is erased by some method, for example erosion or thresholding…
  13. The method can remove mixed color in the background side. But the shape of blur is broken. The radius of blur become small.
  14. The 2nd type is blur regeneration type.
  15. The type use alpha matting processing. With alpha mating, an input natural image is split into 3 part of images, foreground, background and alpha mask. The foreground and background image has mixed color in there images and the mixed information is represented by the alpha mask.
  16. For DIBR processing with alpha matting, we warp these images individually. And then, we alpha-blend the warped fore and background images with warped alpha map. The method can suppress background mixed color, and also keep the shape of blur. Thus the method has the highest image quality. The drawback of this type is computational cost. The method requires 3 times DIBR and a additional processing of matting.
  17. The last type is blur transfer type.
  18. With this type, the depth map is dilated to cover the mixed region before warping.
  19. Alter the warping blur is kept and mixed color is stiched to the foreground object.
  20. Alter the warping, we perform holefilling and then we can obtain free viewpoint images.
  21. The flow of the type is dilation for the depth maps and then process DIBR. We add simple process to improve the quality of this type. The additional processes are fundamental processing of Canny and Gaussian filtering/
  22. In the additional process, we first perform Canny filtering to detect discontinuity regions, and then filter the region by Gaussing an filtering. The assumption of the blur transfer type is background color is simple or flat, and shape of blur is not changed after the warping. To keep the assumption, the smoothing of the object boundary is effective.
  23. To review there types the characteristics of the types are as shown. Then we compare them.
  24. We use Middlebury stereo dataset for our evaluation. The data set has ground truth depth map and multi view images. We render the center viewpoint between the left and right images by using the 3 types of DIBR, and then compare with the captured RGB image by using PSNR. We use 30 pairs in the dataset.
  25. This is a table of PSNR. Left side of basic is blur erasing type And next side of re-generation is blur regeneration with matting method And transfer means with depth map dilation and proposed has additional processing of canny and Gaussian filtering for the blur transfer type. The results show that blur regeneration type has the highest PSNR and the proposed method is the 2nd best. But the difference between two method is about 0.3 dB
  26. This is a rendering result. The blur regeneration and proposed method looked same and the contor artifacts are soften.
  27. This is a bad case of proposed and blur transferred type.
  28. The computational cost of the type are shown. The basic method and blur transfer, proposed method have realtime performance. But blur regeneration method does not has realtime performance, even if the method uses the fastest matting method.