This document outlines a project to convert 2D images to 3D models. It discusses the problem definition, history of previous related work, objectives, challenges, aspects of conversion, project phases, supporting tools, and references. The goal is to revive images and allow navigation inside converted 3D models of buildings, humans, cars, and other objects found in images. Challenges include ambiguity, determining camera viewpoint, recognizing objects, recovering occlusions and surface layout from a single image. Conversion aspects include medical, buildings, humans, and general modeling. Project phases involve region classification, labeling, superpixels, constellations, and creating the final 3D model through cuts and folds.

1. IMAGE REVIVING Faculty of computer and information science Ain shams university

2. Supervisors:- - Prof.Dr. Sayed El- Harbete. - Dr. Ahmed Gawish. Assistants:- - Amr Gamgoum. Project Team:- - Sara Ezzat Lamey. - Christina Saweres Yousef. - Marim Tewfik Rabee. - Mina Mazooz Fares. - Mina Esam Kamel. 2

4. History.

5. Objectives.

6. Challenges

7. Aspects In Converting from 2D into 3D Model.

8. Project Phases.

9. Supporting Tools.

10. References.AGENDA 3

11. 4 How to convert 2D image to 3D model in order to simulate the real world like (visualizing, navigating and simulating buildings and museums), That will be help games developers, campus mapping , help engineers in design and modify buildings structure and helping governments to track roads for reaching to the perfect design. ProblemDefinition

12. 5 There are many 3D models of buildings available for purchase online, but if you want to have a custom 3D of a particular house or building, you are going to either have to hire a 3D modeler to make it, or make it yourself !!!! Our program will solve that ….  Problem Definition

14. History.

15. Objectives.

16. Challenges

17. Aspects In Converting from 2D into 3D Model.

18. Project Phases.

19. Supporting Tools.

20. References.AGENDA 6

21. 7 History - There are many web sites and programs can convert the 2D image into 3D model like: 1 - http://3dsee.net/Main.aspx It created by Dr. David McKinnon from Queensland University of Technology, has recently launched this site that turns your sets of 2D images into realistic 3D bump maps (beta version and provide a gray scale bump map) 2- http://make3d.stanford.edu was created by Ashutosh Saxena, Prof. Andrew Y. Ng, and other team members of the Stanford 3D Reconstruction Group.

22. 8 History(cont’) 3- 3D pop-up model based on the geometrysoftware that construct 3D model out of a single outdoor image. The system labels each region of an outdoor image as ground, vertical, or sky. Is based on Geometric Context from a Single Imageresearches. it is a software provided by Carnegie Melon University at (10/07/05).

24. History.

25. Objectives.

26. Challenges

27. Aspects In Converting from 2D into 3D Model.

28. Project Phases.

29. Supporting Tools.

30. References.AGENDA 9

31. 10 Objectives To revive the image and navigate inside it with converting it to 3D model , our goal is to do that on building aspect, and if the image contain humans or cars our program will do 3D models to them also.

33. History.

34. Objectives.

35. Challenges

36. Aspects In Converting from 2D into 3D Model.

37. Project Phases.

38. Supporting Tools.

39. References.AGENDA 11

40. 12 Dealing with the inherent ambiguity of the image, One image by itself, simply does not contain enough information to recover 3D spatial layout. knowing CPOV Camera point of view and Putting Objects in Perspective. Challenges

42. 15 Challenges (cont’) First, if the image contain humans or cars we will remove them from the image so the challenge is:- how to fill the empty places with the similar degree of the image background color. Recovering surface layout from the 2D single image . Humans have an amazing ability to instantly grasp the overall 3D structure of a scene—ground orientation, relative positions of major landmarks, etc.—even from a single image How can we represent the 3D space of the scene in away that facilitates tasks such as navigation and recognition?

44. History.

45. Objectives.

46. Challenges

47. Aspects In Converting from 2D into 3D Model.

48. Project Phases.

49. Supporting Tools.

50. References.AGENDA 16

51. 17 Medical Aspect. Converting from 2D logo to 3D model. Building Aspect. Converting from 2D human image to 3D model. General 3D modeling to 2D images of (Animals, Cars, Bicycles,..). Aspects in Converting 2D into 3D Model

53. History.

54. Objectives.

55. Challenges

56. Aspects In Converting from 2D into 3D Model.

57. Project Phases.

58. Supporting Tools.

59. References.AGENDA 18

60. 19 Project Phases Determining the geometric class of an image region Across ( color, texture, location in the image, shape, and projective geometry cues). Color : is valuable in identifying the material of a surface (sky is usually blue or white and the ground is often green (grass) or brown (dirt). We represent color using two color spaces: RGB : allows the “blueness” or “greenness” of a region to be easily extracted. HSV : allows perceptual color attributes such as “hue” and “grayness” to be measured. Texture : provides additional information about the material of a surface (sharpness ,smoothing,….).

61. 20 Location : in the image also provides strong cues for distinguishing between ground (tends to be low in the image), vertical structures, and sky (tends to be high in the image). Shape : region shape helps distinguish vertical regions (often roughly convex) from ground and sky regions (often non-convex and large). Projective geometry cues : 3D Geometry features help determine the 3D orientation of surfaces. Knowledge of the vanishing line of a plane completely specifies its 3D orientation relative to the viewer. Project Phases(cont’)

62. 21 Labeling the Image We gradually build our structural knowledge of the image, from pixels to super pixels to constellations of super pixels, Once we have formed multiple sets of constellations, we estimate the constellation label likelihoods and the likelihood that each constellation is homogeneously labeled from which we infer the most likely geometric labels of the super pixels. Obtaining Super pixels :small, nearly-uniform regions in the image ,first step is to form super pixels from those raw pixel intensities . Forming Constellations : we group super pixels that are likely to share a common geometric label into “constellations”. Geometric Classification : For each constellation, we estimate the (label likelihood) whether all super pixels in the Constellationshave the same label. Project Phases(cont’)

63. 22 Project Phases(cont’) Input Image Vertical Labels Support Sky Center Left Right

64. 23 Project Phases(cont’) Super pixels Input Image Multi Segmentations

65. 24 Determining Camera Parameters To obtain true 3D world coordinates, we would need to know the camera parameters. We can, however, create a reasonable scaled model by estimating the horizon line (giving the angle of the camera with respect to the ground plane). Project Phases(cont’)

66. 25 Project Phases(cont’) Original Image Object representation due to the 2D image Objects representation due to 3D Computer Vision

67. 26 Creating the 3D Model (Finally) Cutting and Folding We construct a simple 3D model by making “cuts” and “folds” in the image based on the geometric labels (texture mapping). Project Phases(cont’)

69. History.

70. Objectives.

71. Challenges

72. Aspects In Converting from 2D into 3D Model.

73. Project Phases.

74. Supporting Tools.

75. References.AGENDA 27

76. 28 Microsoft Visual Studio 2008. MATLAB. OpenGL. Supporting Tools

77. 29 http://www.cs.cmu.edu/~efros/ImageInterpretation/,Geometrically Coherent Image Interpretation, Graduate Student Researcher: Derek Hoiem . Seeing the World Behind the Image: Spatial Layout for 3D Scene Understanding ,Derek Hoiem PhD Thesis, Robotics Institute, Carnegie Mellon University, August 2007 . Closing the Loop on Scene Interpretation ,Derek Hoiem, Alexei A. Efros, Martial Hebert ,in CVPR 2008 , See 3Dreconstruction compared to Photo Pop-up and Make3D. Recovering Occlusion Boundaries from a Single Image Derek Hoiem, Andrew Stein, Alexei A. Efros, Martial Hebert , in ICCV 2007. Putting Objects in Perspective ,Derek Hoiem, Alexei A. Efros, Martial HebertIn CVPR 2006,Best Paper Award . Geometric Context from a Single Image , Derek Hoiem, Alexei A. Efros, Martial Hebert ,In ICCV 2005. Automatic Photo Pop-up , Derek Hoiem, Alexei A. Efros, Martial Hebert. References

78. 30 THANK YOU