1. Digital holography allows recording of 3D scene information as complex-valued data, unlike standard 2D images. Reconstructions from digital holograms can extract depth information. 2. Algorithms are needed to process volumes of reconstructions to extract 3D information like depth maps and segmented objects. This includes techniques like focus detection, depth-from-focus, and creating extended focused images. 3. Applications include background segmentation, depth segmentation to separate occluded objects, and synthetic scene creation by superimposing additional objects.

1. Digital Holography Conor Mc Elhinney Deptartment of Computer Science, National University of Ireland, Maynooth. 21 st Nov 2007

2. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

3. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

4. Using digital holography we can record a scene in a complex valued data structure which retains some of the scene's 3D information. A standard image obtained with a camera records a 2D focused image of the scene from one perspective. Why digital holography? However reconstructing a digital hologram returns a 2D image of the scene at a specific depth (300mm from the camera) from an individual perspective (along the optical axis). Algorithms and processing techniques need to be developed to extract the 3D information from digital holograms by processing multiple (volumes of) reconstructions. Image Processing Depth Map Reconstructions Why do we need image processing?

5. Why not 2D Image Processsing? Standard 2D image processing techniques can be applied to individual digital holographic reconstructions with varying success. 2D 3D 2D Image Processing Reconstructions Digital Holographic Image Processing However, we are interested in developing the field of digital holographic image processing (DHIP) where we use volumes of reconstructions to extract 3D information from digital holograms. Using this information we can develop techniques which are more accurate than standard 2D approaches.

6. Photography Holography Recording with photography and holography Object Beam Photo Film Holo Film Laser Sun Lens Reference Beam

7. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

8. Viewing of recorded images from photography PhotoFilm Viewer Viewer Laser Holo Film Photography Holography Sun

9. Viewing of recorded images from holography Photography Holography

10. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

11. Recording with digital holography Digital Holography Object Beam Laser CCD Recorded Image Reference Beam

12. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

13. Reconstructing with digital holography Discrete Fresnel Transform Digital Hologram Digital Reconstruction Distance d

14. Numerical focusing of digital holograms Holograms can be numerically reconstructed at an arbitrary depth away from the camera.

15. Discrete Fresnel Transform Digital Hologram Digital Reconstruction Distance d Reconstructing with a subset of pixels

16. Reconstructing with a subset of pixels If you take a window of pixels from a hologram plane, the reconstruction will still be of the full scene but a reduced quality Hologram reconstruction Hologram plane Simulated Image Captured using a camera

17. Reconstructing different perspectives

18. Reconstructing different perspectives A hologram encodes multiple perspectives and these can be reconstructed by selectively choosing a subwindow from the hologram plane. Hologram reconstruction Hologram plane

19. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

20. Digital Hologram recording Object wavefront

21. Digital Hologram recording Object wavefront Amplitude Phase

22. Digital Hologram recording Object wavefront Reference Beam

23. Digital Hologram recording Object wavefront Reference Beam Interferogram + =

24. Digital Hologram recording Object wavefront Reference Beam Interferogram + =

25. Digital Hologram recording Object wavefront Reference Beam Interferogram A camera records intensity + =

26. Digital Hologram recording Object wavefront Reference Beam Interferogram Recorded Recorded Recorded + = + =

27. Digital Hologram recording Object wavefront Reference Beam Interferogram + = Recorded Recorded Recorded + + = + =

28. Digital Hologram recording Object wavefront Reference Beam Interferogram + = Recorded Recorded Recorded Intensity Only Intensity Only + + = + =

29. Digital Hologram recording CCD Object wavefront Reference Beam Interferogram + = Recorded Recorded Recorded + + = + = + = Intensity and Phase Information

30. Digital Hologram recording CCD Object wavefront Reference Beam Interferogram + = Recorded Recorded Recorded + + = + = + = Objects Amplitude Objects Phase

31. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

32. Focus Metric applied to digital holograms We employed variance calculated on a block of pixels as our focus metric. We split the 40 hologram reconstructions into 4 quadrants, each of size 512 x 512. These blocks were then processed using variance and the depth with the maximum variance was taken as the estimated depth. We are now advancing this to autofocus a digital holographic reconstruction. depth variance

33. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

34. What is Depth-From-Focus? Depth-From-Focus is an image processing technique which is used to determine the depth of a scene or a region within a scene through processing images taken at different focal depths. Why is this applicable to digital holography? Digital Holograms can be numerically reconstructed at an arbitrary depth. These numerical reconstructions are each at a different focal plane, which make them a good input to a Depth-From-Focus algorithm. What do we get from Depth-From-Focus? We can then create depth maps of the scene, segment the scene and create extended focused images of the scene.

35. What is Depth-From-Focus? Depth-From-Focus is an image processing technique which is used to determine the depth of a scene or a region within a scene through processing images taken at different focal depths. Why is this applicable to digital holography? Digital Holograms can be numerically reconstructed at an arbitrary depth. These numerical reconstructions are each at a different focal plane, which make them a good input to a Depth-From-Focus algorithm. What do we get from Depth-From-Focus? We can then create depth maps of the scene, segment the scene and create extended focused images of the scene.

36. What is Depth-From-Focus? Depth-From-Focus is an image processing technique which is used to determine the depth of a scene or a region within a scene through processing images taken at different focal depths. Why is this applicable to digital holography? Digital Holograms can be numerically reconstructed at an arbitrary depth. These numerical reconstructions are each at a different focal plane, which make them a good input to a Depth-From-Focus algorithm. What do we get from Depth-From-Focus? We can then create depth maps of the scene, segment the scene and create extended focused images of the scene.

37. What is Depth-From-Focus?

38. Multiple perspective shape extraction

39. Depth Map from perspective 1 L1

40. Depth Map from perspective 2 L2

41. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

42. What is an Extended Focused Image? This means that reconstructions can contain large blurry regions. Using our depth maps and the volume of reconstructions used to create them we can create an extended focused image. A disadvantage of holographic reconstructions is the limited depth of field. For a reconstruction at depth d only object points that are located at distance d from the camera are in focus. Why do we want to create an extended focused image? Depth Map Volume of Reconstructions = + Extended Focused Image

43. Extended Focused Image How do we create an Extended Focused Image?

44. Extended Focused Image How do we create an Extended Focused Image?

45. Extended Focused Image Reconstruction at the front of the scene Reconstruction at the back of the scene Extended Focused Image

46. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

47. What is Object segmentation? For tasks such as object recognition, it is advantageous to segment a scene into object and background before attempting recognition. Object segmentation is the partitioning of a scene into object and background. Why do we want to perform object segmentation? 1 2 1 2 Threshold Line Reconstruction Depth (mm) Variance

48. Segmentation Examples Numerical Reconstruction Segmentation Mask Segmented Reconstruction

49. Segmentation Examples Numerical Reconstruction Segmentation Mask Segmented Reconstruction

50. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

51. What is Depth segmentation? Again using the example of object recognition, if a scene is complex (containing multiple occluding objects), using depth segmentation we can partition the scene into independent objects for analysis. Depth segmentation is the partitioning of a scene into individual objects after the background has been segmented. Why do we want to perform depth segmentation? 1 2 1 2 1 2 Reconstruction Depth Map Depth Maps Histogram

52. Segmenting reconstructions We now have a segmentation image where the value of each pixel corresponds to the object it belongs to. We can use this to segment a reconstruction into its different objects. Depth Segmentation 1 2 Segmentation Image Reconstruction of Segmented object 1 Reconstruction of Segmented object 2

53. Occluding Objects Through the use of depth information we have a strong criteria for determining if a region in the scene is an independent object or belongs to an earlier identified object. Advantage of segmentation based on depth information 1 2 Segmentation Image Reconstruction of Segmented object 1 Reconstruction of Segmented object 2

54. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline

55. Superposition Optical Axis 360mm CCD Simulated experimental set-up for the superposed hologram, with an second object superposed a distance of 90mm from the original object. Optical Axis 270mm 360mm CCD Simulated original setup for an object placed at 360mm away from the CCD. Reference Wave Object Wave Reference Wave Object Wave

56. Show recons of new hologram We will now show the reconstructions of both objects from the new superposed hologram which shows similar reconstruction quality to the originals even in the presence of a second object. (a) (b) (a) is a reconstruction of the first object. (b) is a reconstruction of the second object.

57. Keeping the same perspective We want to reconstruct both objects from the same perspective. a x a x ' d d' Near Object Plane Hologram Plane Far Object Plane Optical Axis    ’

58. Parallax (a) (b) Original reconstruction Original reconstruction In this example we have reconstructed the two objects in our superposed digital hologram using a window from the top of the hologram plane. We have kept the perspective focused on the near object. As can be seen from the figure below, the near object, (a), has remained unchanged in position, while the far object, (b), has moved.

59. Parallax (a) (b) New Perspective New Perspective In this example we have reconstructed the two objects in our superposed digital hologram using a window from the top of the hologram plane. We have kept the perspective focused on the near object. As can be seen from the figure below, the near object, (a), has remained unchanged in position, while the far object, (b), has moved.

60. Introduction Optical imaging Digital Holography Recording Reconstructing What is in a DH? Focus Detection Depth-from-focus Applications Extended Focused Image Background Segmentation Depth Segmentation Synthetic scene creation Other interesting work Outline