Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

1

Share

Download to read offline

SLFC: Scalable Light Field Coding

Download to read offline

Light field imaging enables some post-processing capabilities like refocusing, changing view perspective, and depth estimation. As light field images are represented by multiple views they contain a huge amount of data that makes compression inevitable. Although there are some proposals to efficiently compress light field images, their main focus is on encoding efficiency. However, some important functionalities such as viewpoint and quality scalabilities, random access, and uniform quality distribution have not been addressed adequately. In this paper, an efficient light field image compression method based on a deep neural network is proposed, which classifies multiple views into various layers. In each layer, the target view is synthesized from the available views of previously encoded/decoded layers using a deep neural network. This synthesized view is then used as a virtual reference for the target view inter-coding. In this way, random access to an arbitrary view is provided. Moreover, uniform quality distribution among multiple views is addressed. In higher bitrates where random access to an arbitrary view is more crucial, the required bitrate to access the requested view is minimized.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

SLFC: Scalable Light Field Coding

  1. 1. All rights reserved. ©2020 All rights reserved. ©2020 SLFC: Scalable Light Field Coding Hadi Amirpour, Christian Timmerer, and Mohammad Ghanbari 1
  2. 2. All rights reserved. ©2020 ● Introduction ● Light field compression ● SLFC ● Experimental results ● Conclusion All rights reserved. ©2020 2
  3. 3. All rights reserved. ©2020 Introduction 3
  4. 4. All rights reserved. ©2020 Light field photography ● conventional camera can only record 2D light intensity on an imaging sensor ● light field camera will capture a 4D light field; that is, the intensity of light in a scene, and also the direction that the light rays are traveling in space 4
  5. 5. All rights reserved. ©2020 Light field photography 5
  6. 6. All rights reserved. ©2020 Light field photography u x v y 6
  7. 7. All rights reserved. ©2020 Introduction http://lightfield-forum.com/what-is-the-lightfield/ 7
  8. 8. All rights reserved. ©2020 Introduction ● refocusing ● changing viewport ● depth estimation ● 3D modeling ● synthesizing new views Post-processing task 8
  9. 9. All rights reserved. ©2020 Introduction Refocusing Lytro Light Field Camera: from scientific research to a $50-million business 9
  10. 10. All rights reserved. ©2020 Introduction Changing viewport 10
  11. 11. All rights reserved. ©2020 Introduction Changing viewport 11
  12. 12. All rights reserved. ©2020 ● Light field compression and its challenges All rights reserved. ©2020 12
  13. 13. All rights reserved. ©2020 Light field compression 13
  14. 14. All rights reserved. ©2020 Light field compression 14
  15. 15. All rights reserved. ©2020 Light field compression 15
  16. 16. All rights reserved. ©2020 Light field compression 16
  17. 17. All rights reserved. ©2020 Light field compression Encoder Decoder Bitstream 17
  18. 18. All rights reserved. ©2020 Light field compression Encoder Decoder Bitstream Dependency between views 18
  19. 19. All rights reserved. ©2020 Light field compression challenges ● encoding efficiency ● viewport scalability ● quality scalability ● random access ● uniform quality distribution 19
  20. 20. All rights reserved. ©2020 Viewport scalability To increase compatibility with: ● display ● capturing device ● network condition ● processing power ● storage capacity 20
  21. 21. All rights reserved. ©2020 Viewport scalability To increase compatibility with: ● display ● capturing device ● network condition ● processing power ● storage capacity 21
  22. 22. All rights reserved. ©2020 Quality scalability To increase compatibility with: ● network condition ● processing power ● storage capacity 22
  23. 23. All rights reserved. ©2020 Random access Navigating between views can increase: ● decoding delay ● bandwidth requirement ● processing power 23
  24. 24. All rights reserved. ©2020 Random access Navigating between views can increase: ● decoding delay ● bandwidth requirement ● processing power 24
  25. 25. All rights reserved. ©2020 Quality distribution It is undesirable to face different quality levels when navigating between viewports 25
  26. 26. All rights reserved. ©2020 Quality distribution It is undesirable to face different quality levels when navigating between viewports 26
  27. 27. All rights reserved. ©2020 ● SLFC: Scalable Light Field Coding All rights reserved. ©2020 27
  28. 28. All rights reserved. ©2020 SLFC ● light field image views are divided into seven layers ○ red view belong to that layer ○ gray views belong to the previous layers ○ black views belong to the next layers L1 L2 L3 L4 L5 L6 L7 28
  29. 29. All rights reserved. ©2020 SLFC ● The central view is intra-coded and can be accessed independently. ● Views in the second layer, are encoded independent of each other but using the central view as their reference image. ● We see encoding of the remaining views as view/frame interpolation problem. L1 L2 L3 L4 L5 L6 L7 29
  30. 30. All rights reserved. ©2020 SLFC Residual with target view Residual with target view Residual with target view 30
  31. 31. All rights reserved. ©2020 SLFC SepConv: Video Frame Interpolation via Adaptive Separable Convolution 31
  32. 32. All rights reserved. ©2020 SLFC ● In each layer (3 to 7), two views from the previously encoded layers are used to synthesize their intermediate view. ● When the intermediate view is synthesized, it is added as a virtual reference frame to the reference list in the video encoder. ● To encode each intermediate view four references are used for inter-coding: (i) the most central view, (ii, iii) two views that are used for interpolation, (v) the synthesized view. 32
  33. 33. All rights reserved. ©2020 SLFC 33
  34. 34. All rights reserved. ©2020 Encoding efficiency Rate distortion curves for Table light field images. Anchor: JPEG Pleno anchor WaSP: Hierarchical warping, merging, and sparse prediction for light field image compression MuLE: A 4D DCT-Based lenslet light field codec PSB: Pseudo sequence based 2-D hierarchical coding structure for light-field image compression 34
  35. 35. All rights reserved. ©2020 Viewport scalability Bitrate allocated to each layer at bpp4 = 0.75 The number of views in each layer 35
  36. 36. All rights reserved. ©2020 Quality scalability SLFC provides two quality levels : ● quality 1: When the most central view and corner views are available, SLFC can synthesize all remaining views by utilizing SepConv. ● quality 2: SLFC first synthesize a target view, then utilize the synthesized view for inter-coding at the cost of additional bitrate. 36
  37. 37. All rights reserved. ©2020 Random access Ap for all views of greek light field images at bpp4 = 0.75 37
  38. 38. All rights reserved. ©2020 Uniform quality distribution PSNR heatmap plot for greek light field image 38
  39. 39. All rights reserved. ©2020 Conclusion ● SLFC utilizes a DNN to synthesize intermediate views. ● Synthesized views are used as virtual references for inter-coding. ● SLFC provides: ○ High encoding efficiency ○ Viewport scalability ○ Quality scalability ○ Random access ○ Uniform quality distribution 39
  40. 40. All rights reserved. ©2020 Thanks for your attention 40 www.athena.itec.aau.at
  • danielhollister

    Aug. 29, 2021

Light field imaging enables some post-processing capabilities like refocusing, changing view perspective, and depth estimation. As light field images are represented by multiple views they contain a huge amount of data that makes compression inevitable. Although there are some proposals to efficiently compress light field images, their main focus is on encoding efficiency. However, some important functionalities such as viewpoint and quality scalabilities, random access, and uniform quality distribution have not been addressed adequately. In this paper, an efficient light field image compression method based on a deep neural network is proposed, which classifies multiple views into various layers. In each layer, the target view is synthesized from the available views of previously encoded/decoded layers using a deep neural network. This synthesized view is then used as a virtual reference for the target view inter-coding. In this way, random access to an arbitrary view is provided. Moreover, uniform quality distribution among multiple views is addressed. In higher bitrates where random access to an arbitrary view is more crucial, the required bitrate to access the requested view is minimized.

Views

Total views

564

On Slideshare

0

From embeds

0

Number of embeds

443

Actions

Downloads

1

Shares

0

Comments

0

Likes

1

×