Build Your Own 3D Scanner: Introduction

36,622 views
36,309 views

Published on

Build Your Own 3D Scanner:
Introduction

http://mesh.brown.edu/byo3d/

SIGGRAPH 2009 Courses
Douglas Lanman and Gabriel Taubin

This course provides a beginner with the necessary mathematics, software, and practical details to leverage projector-camera systems in their own 3D scanning projects. An example-driven approach is used throughout; each new concept is illustrated using a practical scanner implemented with off-the-shelf parts. The course concludes by detailing how these new approaches are used in rapid prototyping, entertainment, cultural heritage, and web-based applications.

0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
36,622
On SlideShare
0
From Embeds
0
Number of Embeds
15,373
Actions
Shares
0
Downloads
0
Comments
0
Likes
18
Embeds 0
No embeds

No notes for slide
  • Welcome to the “Build Your Own 3D Scanner” course. Over the last decade digital photography has entered the mainstream, with inexpensive, miniaturized cameras routinely included in consumer electronics. Whether for mobile phones, teleconferencing, or even gaming consoles, cheap cameras have become ubiquitous. At the moment, digital projection is poised to make a similar impact. Even now, a variety of vendors are offering small form factor, low-cost digital projectors for embedded applications. As a result, active imaging is a topic of renewed interest in the computer graphics community. In particular, low-cost homemade 3D scanners are now within reach of students and hobbyists with a modest budget. In this course we will provide the technical and practical details necessary to build your own 3D scanners using relatively inexpensive consumer electronics. For the students in the audience, some of the key topics we'll cover along the way include: 3D triangulation, camera/projector calibration, mesh processing, and visualization methods. For the more practically-inclined attendees, this course will provide all the software and step-by-step instructions required to build your own versions of the various 3D scanners we describe.
  • This is a two session course. In the first session, we'll describe one of the most widely-used 3D scanners: the “swept-plane” scanner. In this design, a hand-held laser pointer is modified to project a single stripe of light. We will describe how a 3D model can be reconstructed by processing a video sequence during which the stripe is manually swept across the scene. We will also detail a more recent design in which the swept-plane is created by waving a stick in front of a single point light source. Both of these methods require a detailed understanding of 3D triangulation and camera calibration. These topics will be covered in this session. After the break we'll describe one more scanner composed of a single digital camera and projector pair. Various structured illumination patterns will be described that allow 3D models to be acquired quickly and without moving parts. In the later half of the second session, we'll cover the algorithms needed to post-process the raw point clouds produced by the various scanners. We'll show how high-quality 3D meshes can be extracted and used in your own interactive applications.
  • Now, before the coffee break, we’ll describe everything you‘ll need to build your first 3D scanner using a digital camera, halogen lamp, and a wooden stick (or laser pointer).
  • Image sources: http://www.cgarchitect.com/vb/14131-model-free-3d-scan-torolf.html http://1.bp.blogspot.com/_liDP9r7h7UY/SdMVezgMzhI/AAAAAAAAAGU/fxk93AbGNVA/s1600-h/Face+Mocap.jpg http://www.lysator.liu.se/~eru/research/ http://www.cs.ubc.ca/~stpopa/ http://gl.ict.usc.edu/Research/SpatialRelighting/
  • Image sources: http://images.motortrend.com/features/scenes/112_0606_cars_16z+disney_pixar_film_cars+clay_models.jpg http://gl.ict.usc.edu/Research/LS5/ http://ecommons2.library.cornell.edu/web_archive/explore.cornell.edu/scene7354.html?scene=The%203D%20Body%20Scanner&stop=3D%20-%20Sizing%20Research&view=allViews http://assets.gearlive.com/playfeed/blogimages/project-natal-sensor-xbox.jpg http://www.hardwaresphere.com/wp-content/uploads/2009/06/microsoft-project-natal-for-xbox-360.jpg
  • Image sources: http://graphics.stanford.edu/projects/mich/color-david/fd35bigh-cbal-e.jpg http://content.techrepublic.com.com/2346-3513_11-33075.html http://wwwreno.nrc-cnrc.gc.ca/eng/projects/iit/mona-lisa/photo9.html http://www.debevec.org/Parthenon/Images/
  • Image sources: http://dentalscanner.com/product-specs-specs.asp http://www.devicelink.com/mddi/archive/00/05/004.html http://www.informatik.umu.se/~jwworth/execsum34.gif
  • Image sources: http://www.inrim.it/ar2005/ar/va.html http://64.202.120.86/upload/image/articles/2007/darpa-urban-challenge-2007/stanley-2005.jpg
  • Image sources: http://en.wikipedia.org/wiki/File:3D_scanning_and_printing.jpg http://www.spaceflightnow.com/shuttle/sts125/090512fd2/sts125fd2inspect_400.jpg http://www.theamericansurveyor.com/PDF/TheAmericanSurveyor_Jenkins-ScanningDiscovery_November05.pdf
  • Image sources: http://sawdust.see-do.org/notreadyforprimetime/files/page25_blog_entry17-dscf2892.jpg http://www.worldwideflood.com/ark/noahs_cubit/cubit_references.htm http://www.renishaw.com/media/img/gen/448c3eeefbb9418e9bb0ad370e0e4a00.jpg http://www.rhk-tech.com/images/gallery_results_large/cd-rom.jpg
  • Image sources: http://upload.wikimedia.org/wikipedia/commons/6/6c/Binocular_disparity.png http://www.wv.inf.tu-dresden.de/Research/Robotics/gfx/Bumblebee/3_400x302.jpg http://farm2.static.flickr.com/1398/1484478905_e02295ffcf.jpg http://en.wikipedia.org/wiki/File:Home_plate_anim.gif
  • Image sources: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FAVARO1/dfdtutorial.html
  • Image sources: http://en.wikipedia.org/wiki/File:Ct-workstation-neck.jpg http://images.google.com/imgres?imgurl=http://www.aapm.org/meetings/07AM/images/multiplexing.jpg&imgrefurl=http://www.aapm.org/meetings/07AM/VirtualPressRoom/LayLanguage/IIMultiplexing.asp&usg=__TR21RhgobYmcXv8UFZaxgQSm2D0=&h=318&w=482&sz=46&hl=en&start=5&um=1&tbnid=VqSRRDZTI4PTMM:&tbnh=85&tbnw=129&prev=/images%3Fq%3Dcomputed%2Btomography%26hl%3Den%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-US:official%26sa%3DG%26um%3D1
  • Image sources: http://en.wikipedia.org/wiki/File:UltrasoundProbe2006a.jpg http://en.wikipedia.org/wiki/File:Embryo_at_14_weeks_profile.JPG http://upload.wikimedia.org/wikipedia/commons/3/39/3dultrasound.png http://en.wikipedia.org/wiki/File:OUPRIME1.png
  • Image sources: http://blue-c-ii.ethz.ch/?Technology:3D_video http://www1.cs.columbia.edu/CAVE/projects/shape_focus/shape_focus.php
  • Image sources: http://upload.wikimedia.org/wikipedia/commons/1/19/TOF_Kamera_3D_Gesicht.jpg http://www.kotaku.com.au/2009/06/microsoft-project-natal-can-support-multiple-players-see-fingers/ http://www.designworldonline.com/uploads/ImageGallery/ifm-3d-sensor-4.jpg
  • Image sources: http://en.wikipedia.org/wiki/File:LaserPrinciple.png http://graphics.stanford.edu/projects/mich/more-david/scanner-head-and-david-head-s.jpg
  • Image sources: http://community.middlebury.edu/~schar/papers/structlight/p1.html
  • Image sources: http://meshlab.sourceforge.net/images/screenshots/SnapMeshLab.align1.png
  • Image sources: http://www.cns.pl/getfile/fde3d859-c349-4ba1-89a1-f90e187c3bd3/NextEngine.aspx http://www.deskeng.com/articles/aaajwm.htm http://www.ittc.co.jp/hproduct/sgb/prm-plm.jpg http://www.qubic.com.au/images/prod/minolta910.jpg http://gadgets.kenxu.com/real-view-3d-scanner-turns-all-object-into-digital-version/ http://news.thomasnet.com/images/large/545/545444.jpg http://www.ittc.co.jp/hproduct/sgb/prm-plm.jpg NextEngine, Creaform3d EXAscan, ShapeGrabber, Konica Minolta Vivid 910fw Laser Scanner, Real-View 3D Scanner
  • Image sources: http://blog.makezine.com/archive/2006/10/how_to_build_your_own_3d.html http://www.make-digital.com/make/vol14/?pg=195 http://www.shapeways.com/blog/uploads/david-starter-kit.jpg http://www.shapeways.com/blog/archives/248-DAVID-3D-Scanner-Starter-Kit-Review.html#extended http://www.david-laserscanner.com/ http://www.youtube.com/watch?v=XSrW-wAWZe4 http://www.chromecow.com/MadScience/3DScanner/3DScan_02.htm Liquid scanner, various laser scanners
  • Image sources: http://developer.apple.com/documentation/QuickTime/InsideQT_QTVR/2Chap/2-QTVR-Authoring.html#//apple_ref/doc/uid/TP40000944-CH206-BAJGAGFA http://www.terrystoeger.com/site/tips/tutorials/qtvr/qtvr.html
  • Build Your Own 3D Scanner: Introduction

    1. 1. Build Your Own 3D Scanner: 3D Photography for Beginners SIGGRAPH 2009 Courses 5 August 2009, 8:30 am - 12:15 pm Douglas Lanman Brown University Gabriel Taubin Brown University
    2. 2. <ul><li>Session I (8:30 am - 10:15 am) </li></ul><ul><li>Introduction </li></ul><ul><li>The Mathematics of 3D Triangulation </li></ul><ul><li>3D Scanning with Swept-Planes </li></ul><ul><li>Camera and Swept-Plane Light Source Calibration </li></ul><ul><li>Reconstruction and Visualization using Point Clouds </li></ul><ul><li>Session II (10:30 am - 12:15 pm) </li></ul><ul><li>Structured Lighting </li></ul><ul><li>Projector Calibration and Structured Light Reconstruction </li></ul><ul><li>Combining Point Clouds Recovered from Multiple Views </li></ul><ul><li>Surface Reconstruction from Point Clouds </li></ul><ul><li>Elementary Mesh Processing </li></ul><ul><li>Conclusion / Q & A </li></ul>Course Schedule
    3. 3. Session I Triangulation and Scanning with Swept-Planes http://mesh.brown.edu/byo3d
    4. 4. <ul><li>Session I </li></ul><ul><li>Introduction </li></ul><ul><li>The Mathematics of 3D Triangulation </li></ul><ul><li>3D Scanning with Swept-Planes </li></ul><ul><li>Camera and Swept-Plane Light Source Calibration </li></ul><ul><li>Reconstruction and Visualization using Point Clouds </li></ul>Course Schedule
    5. 5. Introduction to 3D Scanning 3D Scanning Geometry Dynamics (Motion, Deformation, etc.) Rendering (Illumination Model)
    6. 6. Applications of 3D Scanning: Entertainment and Consumer Applications Andreas Wenger et al . Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination . ACM SIGGRAPH, 2005 M. Waschbüsch et al . Scalable 3D Video of Dynamic Scenes . The Visual Computer , 2005. <ul><li>Import sculptures into a 3D modeling/rendering pipeline </li></ul><ul><li>Capture geometric (and photometric) properties for relighting </li></ul><ul><li>Fit clothes, track 3D interaction, free-viewpoint video (3D TV), etc. </li></ul>
    7. 7. Applications of 3D Scanning: Historical Preservation M. Levoy et al. The Digital Michelangelo Project: 3D Scanning of Large Statues. ACM SIGGRAPH, 2000 P. Debevec. Making “The Parthenon” . Intl. Sym. on Virtual Reality, Archaeology, and Cultural Heritage, 2005 L. Borgeat et al. Visualizing and Analyzing the Mona Lisa . IEEE Computer Graphics and Applications, 2007 <ul><li>Preserve/restore deteriorating works and unite dispersed collections </li></ul><ul><li>Facilitate academic study (tooling, lighting, pentimenti, revision history) </li></ul><ul><li>Replicate collections (souvenirs, retain repatriated works, etc.) </li></ul>
    8. 8. Applications of 3D Scanning: Medical Imaging and Surgical Planning <ul><li>Medical imaging (X-ray, CT, MRI, etc.) and surgical planning </li></ul><ul><li>Measuring dimensions (dental impressions and hip replacement surgery) </li></ul><ul><li>Tele-surgery (augmented virtual reality, video see-through, etc.) </li></ul>
    9. 9. Applications of 3D Scanning: Robotics (Interaction and Navigation) <ul><li>Motion planning (manipulation, gripping, pushing/pulling, etc.) </li></ul><ul><li>Simultaneous localization and mapping (SLAM) </li></ul><ul><li>Autonomous navigation (DARPA Grand/Urban Challenge) </li></ul>
    10. 10. Applications of 3D Scanning: Inspection and Reverse Engineering <ul><li>Manufacturing and process control (tolerances and alignment) </li></ul><ul><li>Reverse engineering (repairing antiques and replicating designs) </li></ul><ul><li>Remote inspection (inaccessible or dangerous environments) </li></ul>
    11. 11. Taxonomy of 3D Scanning: Direct Contact Taxonomy adapted from Szymon Rusinkiewicz Contact Non-Contact Direct Measurements (rulers , calipers, pantographs, coordinate measuring machines (CMM), AFM)
    12. 12. Taxonomy of 3D Scanning: Stereo/Multi-view Photography Contact Non-Contact Active Passive Direct Measurements (rulers , calipers, pantographs, coordinate measuring machines (CMM), AFM) Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.)
    13. 13. Contact Non-Contact Active Passive Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Taxonomy of 3D Scanning: Shape-from-Silhouettes J. Starck and A. Hilton. Surface Capture for Performance-Based Animation . IEEE Computer Graphics and Applications , 2007
    14. 14. Contact Non-Contact Active Passive Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Taxonomy of 3D Scanning: Shape-from-Focus/Defocus M. Watanabe and S. Nayar. Rational filters for passive depth from defocus . Intl. J. of Comp. Vision , 27(3):203-225, 1998
    15. 15. Contact Non-Contact Active Passive Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Taxonomy of 3D Scanning: Computed Tomography (CT) Parallel/Fan-beam Projections 0 50 100 150 rotation angle (degrees) Density Function Transmissive Reflective Computed Tomography (CT) Transmissive Ultrasound
    16. 16. Contact Non-Contact Active Passive Transmissive Reflective Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Computed Tomography (CT) Transmissive Ultrasound Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Non-optical Methods (reflective ultrasound, radar, sonar, MRI) Taxonomy of 3D Scanning: Non-optical Active Methods
    17. 17. Contact Non-Contact Active Passive Transmissive Reflective Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Active Variants of Passive Methods (stereo/focus/defocus using projected patterns) Computed Tomography (CT) Transmissive Ultrasound Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Non-optical Methods (reflective ultrasound, radar, sonar, MRI) Taxonomy of 3D Scanning: Active Variants of Passive Methods M. Watanabe and S. Nayar. Rational Filters for Passive Depth from Defocus . Intl. J. of Comp. Vision , 27(3):203-225, 1998 M. Waschbüsch et al. Scalable 3D Video of Dynamic Scenes . The Visual Computer , pp. 629-638, 2005
    18. 18. Contact Non-Contact Active Passive Transmissive Reflective Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Active Variants of Passive Methods (stereo/focus/defocus using projected patterns) Time-of-Flight Computed Tomography (CT) Transmissive Ultrasound Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Non-optical Methods (reflective ultrasound, radar, sonar, MRI) Taxonomy of 3D Scanning: Time-of-Flight
    19. 19. Contact Non-Contact Active Passive Transmissive Reflective Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Active Variants of Passive Methods (stereo/focus/defocus using projected patterns) Time-of-Flight Triangulation (laser striping and structured lighting) Computed Tomography (CT) Transmissive Ultrasound Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Non-optical Methods (reflective ultrasound, radar, sonar, MRI) Taxonomy of 3D Scanning: Triangulation with Laser Striping M. Levoy et al. The Digital Michelangelo Project: 3D Scanning of Large Statues. Proc. ACM SIGGRAPH, 2000
    20. 20. Contact Non-Contact Active Passive Transmissive Reflective Shape-from-X (stereo/multi-view, silhouettes, focus/defocus, motion, texture, etc.) Active Variants of Passive Methods (stereo/focus/defocus using projected patterns) Time-of-Flight Triangulation (laser striping and structured lighting) Computed Tomography (CT) Transmissive Ultrasound Direct Measurements (rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM) Non-optical Methods (reflective ultrasound, radar, sonar, MRI) Taxonomy of 3D Scanning: Triangulation with Structured Lighting
    21. 21. Challenges of Optical 3D Scanning Godin et al. An Assessment of Laser Range Measurement on Marble Surfaces . Intl. Conf. Optical 3D Measurement Techniques, 2001 M. Levoy. Why is 3D scanning hard? 3DPVT, 2002 <ul><li>Must be simultaneously illuminated and imaged (occlusion problems) </li></ul><ul><li>Non-Lambertian BRDFs (transparency, reflections, subsurface scattering) </li></ul><ul><li>Acquisition time (dynamic scenes), large (or small) features, etc. </li></ul>
    22. 22. The 3D Scanning Pipeline Assign Texture (BRDF, etc.) Align 3D Point Clouds Reconstruct 3D Point Cloud(s) Data Capture Extract Isosurface (Polyhedral Mesh)
    23. 23. Commercial 3D Scanners <ul><li>Features, Limitations, and Benefits </li></ul><ul><li>Most commercial scanners use laser striping + turntables/fiducials </li></ul><ul><li>Cost varies (NextEngine ~$3,000 USD, others more expensive) </li></ul><ul><li>Complete pipeline (including registration and isosurface extraction) </li></ul>
    24. 24. Do-It-Yourself (DIY) 3D Scanners <ul><li>Features, Limitations, and Benefits </li></ul><ul><li>Most DIY scanners also use laser striping + turntables </li></ul><ul><li>Relatively inexpensive (DAVID laser scanner ~$550 USD for starter kit) </li></ul><ul><li>Incomplete pipeline (lacking registration and isosurface extraction) </li></ul><ul><li>Most (but not all) lack proper camera and light source calibration </li></ul>
    25. 25. Topics/Scanners in this Course 1) Scanning with Swept-Planes 3) Post-processing Pipeline: Registration and Isosurfaces 2) Structured Lighting using Projector-Camera Systems
    26. 26. When not to Scan? <ul><li>Scanning is (usually) unnecessary when output is another image! </li></ul><ul><li>Better to use image-based rendering (light fields, QTVR, etc.) </li></ul>Marc Levoy. Stanford Spherical Gantry . On-line, 2005
    27. 27. <ul><li>Session I </li></ul><ul><li>Introduction </li></ul><ul><li>The Mathematics of 3D Triangulation </li></ul><ul><li>3D Scanning with Swept-Planes </li></ul><ul><li>Camera and Swept-Plane Light Source Calibration </li></ul><ul><li>Reconstruction and Visualization using Point Clouds </li></ul>Course Schedule

    ×