Interactive Animation And Modeling By Drawing - Pedagogical Applications In Medicine

1,236 views

Published on

Medicine is a discipline where visualization is an essential component of learning. However, the three-dimensional, dynamic structure of the human body poses difficult teaching challenges. There is a need for truly interactive computer tools that will enable students to create and manipulate computer models, not just watch them. We propose di erent approaches with that goal in mind. We were first interested in interactive physically-based animation of anisotropic elastic materials. One possible application scenario is an anatomy course on heart physiology where students can build interactive samples of cardiac muscular tissue. To achieve this, our model exhibits two key features. The first one is a low computational cost that results in high frame rates; the second one is an intuitive system image that ensures easy control by the user. Next, we were interested in interaction in three dimensions using two-dimensional input, either for annotating existing models, or for creating new models; taking advantage of the fact that drawing practice is still considered a fundamental learning method by some anatomy teachers in the French medical school curriculum. Our 3D drawing system has a stroke representation that enables drawing redisplay when the viewpoint changes. Moreover, this representation can be mixed freely with existing polygonal surfaces for annotation purposes. In contrast, our modeling by drawing tool uses information from both stroke geometry and the drawn image, to allow three-dimensional modeling without explicit depth specification.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,236
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Interactive Animation And Modeling By Drawing - Pedagogical Applications In Medicine

  1. 1. Animation interactive et modélisation par le dessin Applications pédagogiques en médecine David Bourguignon Doctorat de l’INPG Spécialité : modèles et instruments en médecine et biologie Préparé au sein du laboratoire GRAVIR Sous la direction de Marie-Paule Cani M.C. Escher
  2. 2. Information Technology in Teaching <ul><li>Burgeoning field </li></ul>
  3. 3. Information Technology in Teaching <ul><li>Burgeoning field </li></ul><ul><li>Biology and medicine are visual disciplines </li></ul><ul><ul><li>Three-dimensional shapes </li></ul></ul>Frog tomography data
  4. 4. Information Technology in Teaching <ul><li>Burgeoning field </li></ul><ul><li>Biology and medicine are visual disciplines </li></ul><ul><ul><li>Three-dimensional shapes </li></ul></ul><ul><ul><li>Dynamic phenomena </li></ul></ul>Dynamic imaging left ventricle human heart
  5. 5. Information Technology in Teaching <ul><li>Burgeoning field </li></ul><ul><li>Biology and medicine are visual disciplines </li></ul><ul><ul><li>Three-dimensional shapes </li></ul></ul><ul><ul><li>Dynamic phenomena </li></ul></ul><ul><li>Problems for teaching using real organisms </li></ul><ul><ul><li>Practical availability </li></ul></ul>
  6. 6. Information Technology in Teaching <ul><li>Burgeoning field </li></ul><ul><li>Biology and medicine are visual disciplines </li></ul><ul><ul><li>Three-dimensional shapes </li></ul></ul><ul><ul><li>Dynamic phenomena </li></ul></ul><ul><li>Problems for teaching using real organisms </li></ul><ul><ul><li>Practical availability </li></ul></ul><ul><ul><li>Practical feasibility </li></ul></ul>
  7. 7. Information Technology in Teaching <ul><li>Current solutions </li></ul><ul><ul><li>Anatomical databases (images, 3D models) </li></ul></ul>http://www.netanatomy.com
  8. 8. Information Technology in Teaching <ul><li>Current solutions </li></ul><ul><ul><li>Anatomical databases (images, 3D models) </li></ul></ul><ul><ul><li>Multimedia documents </li></ul></ul>http://www.froguts.com
  9. 9. Information Technology in Teaching <ul><li>Current solutions </li></ul><ul><ul><li>Anatomical databases (images, 3D models) </li></ul></ul><ul><ul><li>Multimedia documents </li></ul></ul>http://www.froguts.com
  10. 10. Information Technology in Teaching <ul><li>Current solutions </li></ul><ul><ul><li>Anatomical databases (images, 3D models) </li></ul></ul><ul><ul><li>Multimedia documents </li></ul></ul>http://www.froguts.com
  11. 11. Information Technology in Teaching <ul><li>Current solutions </li></ul><ul><ul><li>Anatomical databases (images, 3D models) </li></ul></ul><ul><ul><li>Multimedia documents </li></ul></ul><ul><li>Drawbacks </li></ul><ul><ul><li>No editing tools available </li></ul></ul>
  12. 12. Information Technology in Teaching <ul><li>Current solutions </li></ul><ul><ul><li>Anatomical databases (images, 3D models) </li></ul></ul><ul><ul><li>Multimedia documents </li></ul></ul><ul><li>Drawbacks </li></ul><ul><ul><li>No editing tools available </li></ul></ul><ul><ul><li>Teacher and students in observer role </li></ul></ul>
  13. 13. Information Technology in Teaching <ul><li>Need for truly interactive teaching tools </li></ul><ul><ul><li>User-centered </li></ul></ul>
  14. 14. Information Technology in Teaching <ul><li>Need for truly interactive teaching tools </li></ul><ul><ul><li>User-centered </li></ul></ul><ul><ul><li>Understand shape and function of organs </li></ul></ul>
  15. 15. Information Technology in Teaching <ul><li>Need for truly interactive teaching tools </li></ul><ul><ul><li>User-centered </li></ul></ul><ul><ul><li>Understand shape and function of organs </li></ul></ul><ul><ul><li>Create, edit, animate models </li></ul></ul>
  16. 16. Information Technology in Teaching <ul><li>Need for truly interactive teaching tools </li></ul><ul><li>Two interdisciplinary collaborations </li></ul><ul><ul><li>MENRT research action “Beating heart” </li></ul></ul>
  17. 17. Information Technology in Teaching <ul><li>Need for truly interactive teaching tools </li></ul><ul><li>Two interdisciplinary collaborations </li></ul><ul><ul><li>MENRT research action “Beating heart” </li></ul></ul><ul><ul><li>Anatomy laboratory CHU Grenoble </li></ul></ul>
  18. 18. Information Technology in Teaching <ul><li>Need for truly interactive teaching tools </li></ul><ul><li>Two interdisciplinary collaborations </li></ul><ul><li>Two pedagogical scenarios </li></ul><ul><ul><li>Physiological anatomy course </li></ul></ul><ul><ul><ul><li>Build interactive samples </li></ul></ul></ul><ul><ul><ul><li>Experiment </li></ul></ul></ul>
  19. 19. Information Technology in Teaching <ul><li>Need for truly interactive teaching tools </li></ul><ul><li>Two interdisciplinary collaborations </li></ul><ul><li>Two pedagogical scenarios </li></ul><ul><ul><li>Physiological anatomy course </li></ul></ul><ul><ul><li>Structural anatomy course </li></ul></ul><ul><ul><ul><li>Draw or annotate </li></ul></ul></ul><ul><ul><ul><li>Create or edit </li></ul></ul></ul>
  20. 20. Our Contributions <ul><li>Part 1: Interactive physically based animation </li></ul><ul><ul><li>Animating anisotropic elastic materials [Bourguignon and Cani, EGCAS 2000] </li></ul></ul><ul><li>Part 2: Interaction in 3D using 2D input </li></ul><ul><ul><li>Drawing in 3D [Bourguignon et al., EG 2001] </li></ul></ul><ul><ul><li>Modeling by drawing </li></ul></ul>
  21. 21. Our Contributions <ul><li>Part 1: Interactive physically based animation </li></ul><ul><ul><li>Animating anisotropic elastic materials [Bourguignon and Cani, EGCAS 2000] </li></ul></ul><ul><li>Part 2: Interaction in 3D using 2D input </li></ul><ul><ul><li>Drawing in 3D [Bourguignon et al., EG 2001] </li></ul></ul><ul><ul><li>Modeling by drawing </li></ul></ul>
  22. 22. Motivation <ul><li>Manipulate interactive samples </li></ul>Part 1
  23. 23. Motivation <ul><li>Manipulate interactive samples </li></ul><ul><li>Biological materials </li></ul><ul><ul><li>Dynamics </li></ul></ul><ul><ul><li>Nonlinear elasticity </li></ul></ul><ul><ul><li>Anisotropy </li></ul></ul><ul><ul><li>Incompressibility </li></ul></ul>Part 1 Computer model of cardiac geometry and muscle fiber (McCulloch, UCSD)
  24. 24. Motivation <ul><li>Manipulate interactive samples </li></ul><ul><li>Biological materials </li></ul><ul><ul><li>Dynamics </li></ul></ul><ul><ul><li>Nonlinear elasticity </li></ul></ul><ul><ul><li>Anisotropy </li></ul></ul><ul><ul><li>Incompressibility </li></ul></ul>Human liver (Epidaure, INRIA) Part 1
  25. 25. Motivation <ul><li>Manipulate interactive samples </li></ul><ul><li>Biological materials </li></ul><ul><li>Intuitively and efficiently </li></ul>Part 1
  26. 26. Previous Work <ul><li>Continuous Models </li></ul><ul><ul><li>Large deformations [O’Brien and Hodgins, 1999] </li></ul></ul>Part 1
  27. 27. Previous Work <ul><li>Continuous Models </li></ul><ul><ul><li>Large deformations [O’Brien and Hodgins, 1999] </li></ul></ul><ul><ul><li>Multiresolution [Debunne et al., 2001] </li></ul></ul>Part 1
  28. 28. Previous Work <ul><li>Continuous Models </li></ul><ul><ul><li>Large deformations [O’Brien and Hodgins, 1999] </li></ul></ul><ul><ul><li>Multiresolution [Debunne et al., 2001] </li></ul></ul><ul><ul><li>Physical nonlinearities and transversal isotropy [Picinbono et al., 2001] </li></ul></ul>Part 1
  29. 29. Previous Work <ul><li>Continuous Models </li></ul><ul><ul><li>Large deformations [O’Brien and Hodgins, 1999] </li></ul></ul><ul><ul><li>Multiresolution [Debunne et al., 2001] </li></ul></ul><ul><ul><li>Physical nonlinearities and transversal isotropy [Picinbono et al., 2001] </li></ul></ul><ul><li>Problems </li></ul><ul><ul><li>Incompressibility </li></ul></ul>Part 1
  30. 30. Previous Work <ul><li>Continuous Models </li></ul><ul><ul><li>Large deformations [O’Brien and Hodgins, 1999] </li></ul></ul><ul><ul><li>Multiresolution [Debunne et al., 2001] </li></ul></ul><ul><ul><li>Physical nonlinearities and transversal isotropy [Picinbono et al., 2001] </li></ul></ul><ul><li>Problems </li></ul><ul><ul><li>Incompressibility </li></ul></ul><ul><ul><li>Parameters setting </li></ul></ul>Part 1
  31. 31. Previous Work <ul><li>Discrete Models </li></ul><ul><ul><li>Large deformations </li></ul></ul>Part 1
  32. 32. Previous Work <ul><li>Discrete Models </li></ul><ul><ul><li>Large deformations </li></ul></ul><ul><ul><li>Physical nonlinearities [Lee et al., 1995] </li></ul></ul>Part 1
  33. 33. Previous Work <ul><li>Discrete Models </li></ul><ul><ul><li>Large deformations </li></ul></ul><ul><ul><li>Physical nonlinearities [Lee et al., 1995] </li></ul></ul><ul><li>Problems </li></ul><ul><ul><li>No multiresolution [Debunne et al., 2001] </li></ul></ul>Part 1
  34. 34. Previous Work <ul><li>Discrete Models </li></ul><ul><ul><li>Large deformations </li></ul></ul><ul><ul><li>Physical nonlinearities [Lee et al., 1995] </li></ul></ul><ul><li>Problems </li></ul><ul><ul><li>No multiresolution [Debunne et al., 2001] </li></ul></ul><ul><ul><li>Anisotropy [Ng and Fiume, 1997] </li></ul></ul>Part 1
  35. 35. Previous Work <ul><li>Discrete Models </li></ul><ul><ul><li>Large deformations </li></ul></ul><ul><ul><li>Physical nonlinearities [Lee et al., 1995] </li></ul></ul><ul><li>Problems </li></ul><ul><ul><li>No multiresolution [Debunne et al., 2001] </li></ul></ul><ul><ul><li>Anisotropy [Ng and Fiume, 1997] </li></ul></ul><ul><ul><li>Incompressibility </li></ul></ul>Part 1
  36. 36. Mass-Spring Systems <ul><li>Mesh geometry influences material behavior </li></ul><ul><ul><li>Undesired anisotropy </li></ul></ul><ul><ul><li>Incorrect behavior in bending </li></ul></ul>Tetrahedral mass-spring system Part 1
  37. 37. Mass-Spring Systems <ul><li>Mesh geometry influences material behavior </li></ul><ul><ul><li>Undesired anisotropy </li></ul></ul><ul><ul><li>Incorrect behavior in bending </li></ul></ul>Part 1 Tetrahedral mass-spring system
  38. 38. Our Approach <ul><li>Goal </li></ul><ul><ul><li>As simple and efficient as mass-spring system </li></ul></ul><ul><ul><li>Speed vs precision tradeoff </li></ul></ul><ul><ul><li>Anisotropy </li></ul></ul><ul><ul><li>Incompressibility </li></ul></ul>Part 1
  39. 39. Our Approach <ul><li>Goal </li></ul><ul><ul><li>As simple and efficient as mass-spring system </li></ul></ul><ul><ul><li>Speed vs precision tradeoff </li></ul></ul><ul><ul><li>Anisotropy </li></ul></ul><ul><ul><li>Incompressibility </li></ul></ul><ul><li>Choice </li></ul><ul><ul><li>Discrete model </li></ul></ul><ul><ul><li>Uncouple forces directions and mesh geometry [Barzel, 1992] </li></ul></ul>Part 1
  40. 40. Our Approach <ul><li>Data: Geometry </li></ul>Surface mesh Part 1
  41. 41. Our Approach <ul><li>Data: Geometry </li></ul>Surface mesh Part 1 Volume mesh
  42. 42. Our Approach <ul><li>Data: Vector field </li></ul>Surface mesh Part 1 Volume mesh Vector field
  43. 43. Our Approach <ul><li>Elements </li></ul>Surface mesh Part 1 Volume mesh Vector field Barycenter Axes of interest (mechanical characteristics)
  44. 44. Our Approach <ul><li>Elements </li></ul>Surface mesh Part 1 Volume mesh Vector field Barycenter Axes of interest (mechanical characteristics) <ul><li>For each element: </li></ul><ul><li>Element deformation </li></ul><ul><li>Local frame deformation </li></ul><ul><li>Forces applied to local frame </li></ul><ul><li>Forces applied to nodes </li></ul>
  45. 45. Forces Calculations Stretch: Axial damped spring forces (each axis) Shear: Angular spring forces (each pair of axes) f 1 I 1 ’ I 1 e 1 f 1 ’ f 3 I 1 ’ I 1 e 1 e 3 I 3 I 3 ’ f 1 f 1 ’ f 3 ’ Part 1
  46. 46. Volume Conservation <ul><li>Soft constraint [Lee et al., 1995] </li></ul><ul><li>Conserve sum of barycenter-vertices distances </li></ul>f C f B f D f A Part 1
  47. 47. Volume Conservation <ul><li>Comparison with mass-spring systems </li></ul>Part 1 With volume conservation forces Mass-spring system Without volume conservation forces
  48. 48. Results <ul><li>Comparison with mass-spring systems </li></ul><ul><ul><li>No more undesired anisotropy </li></ul></ul><ul><ul><li>Correct behavior in bending </li></ul></ul>Orthotropic material (as muscle fiber) Same parameters in the 3 directions Part 1
  49. 49. Results <ul><li>Comparison with mass-spring systems </li></ul><ul><ul><li>No more undesired anisotropy </li></ul></ul><ul><ul><li>Correct behavior in bending </li></ul></ul>Part 1 Orthotropic material (as muscle fiber) Same parameters in the 3 directions
  50. 50. Results <ul><li>Different anisotropic behaviors with same tetrahedral mesh </li></ul>Horizontal Part 1
  51. 51. Results <ul><li>Different anisotropic behaviors with same tetrahedral mesh </li></ul>Diagonal Part 1
  52. 52. Results <ul><li>Different anisotropic behaviors with same tetrahedral mesh </li></ul>Part 1 Hemicircular
  53. 53. Results <ul><li>Different anisotropic behaviors with same tetrahedral mesh </li></ul>Part 1 Concentric Helicoidal
  54. 54. Results <ul><li>Different anisotropic behaviors with same tetrahedral mesh </li></ul>Part 1 Concentric Helicoidal (top view)
  55. 55. Results <ul><li>Different anisotropic behaviors with same tetrahedral mesh </li></ul>Part 1 Random
  56. 56. Validations <ul><li>Emerging behavior [Boux de Casson, 2000] </li></ul><ul><ul><li>Define a behavior at the element level </li></ul></ul><ul><ul><li>Measure the emerging behavior at the object level </li></ul></ul>Part 1
  57. 57. Validations <ul><li>Emerging behavior </li></ul>Object level behavior Part 1 f 1 I 1 ’ I 1 e 1 f 1 ’ Element level behavior (data points fit) +
  58. 58. Validations <ul><li>Multiresolution behavior [Debunne, 2000] </li></ul>Part 1
  59. 59. Validations <ul><li>Multiresolution behavior [Debunne, 2000] </li></ul>Mass-spring system Our model Part 1
  60. 60. Conclusion and Future Work <ul><li>Conclusion: Pedagogical application </li></ul><ul><ul><li>Build interactive samples of biological materials </li></ul></ul><ul><ul><ul><li>Nonlinear, anisotropic behaviors </li></ul></ul></ul><ul><ul><ul><li>Soft constraint for volume conservation </li></ul></ul></ul><ul><ul><ul><li>Efficient </li></ul></ul></ul>Part 1
  61. 61. Conclusion and Future Work <ul><li>Conclusion: Pedagogical application </li></ul><ul><ul><li>Build interactive samples of biological materials </li></ul></ul><ul><ul><ul><li>Nonlinear, anisotropic behaviors </li></ul></ul></ul><ul><ul><ul><li>Soft constraint for volume conservation </li></ul></ul></ul><ul><ul><ul><li>Efficient </li></ul></ul></ul><ul><ul><li>Experiment by varying model parameters </li></ul></ul><ul><ul><ul><li>Intuitive system image [Norman, 1988] </li></ul></ul></ul>Part 1
  62. 62. Conclusion and Future Work <ul><li>Conclusion: Pedagogical application </li></ul><ul><li>Future work: “Animated sketches” </li></ul><ul><ul><li>Draw sample </li></ul></ul>Part 1
  63. 63. Conclusion and Future Work <ul><li>Conclusion: Pedagogical application </li></ul><ul><li>Future work: “Animated sketches” </li></ul><ul><ul><li>Draw sample </li></ul></ul><ul><ul><li>Specify parameters by drawing </li></ul></ul>Part 1
  64. 64. Conclusion and Future Work <ul><li>Conclusion: Pedagogical application </li></ul><ul><li>Future work: “Animated sketches” </li></ul><ul><ul><li>Draw sample </li></ul></ul><ul><ul><li>Specify parameters by drawing </li></ul></ul>Animate! Part 1
  65. 65. Our Contributions <ul><li>Part 1: Interactive physically based animation </li></ul><ul><ul><li>Animating anisotropic elastic materials [Bourguignon and Cani, EGCAS 2000] </li></ul></ul><ul><li>Part 2: Interaction in 3D using 2D input </li></ul><ul><ul><li>Drawing in 3D [Bourguignon et al., EG 2001] </li></ul></ul><ul><ul><li>Modeling by drawing </li></ul></ul>Part 2
  66. 66. Motivation <ul><li>Most people draw </li></ul><ul><ul><li>Writing alternative </li></ul></ul><ul><ul><ul><li>Faster </li></ul></ul></ul><ul><ul><ul><li>More convenient </li></ul></ul></ul>Part 2
  67. 67. Motivation <ul><li>Most people draw </li></ul><ul><ul><li>Writing alternative </li></ul></ul><ul><ul><li>Minimal tool set </li></ul></ul>Part 2
  68. 68. Motivation <ul><li>Most people draw </li></ul><ul><ul><li>Writing alternative </li></ul></ul><ul><ul><li>Minimal tool set </li></ul></ul><ul><ul><li>Since kindergarten </li></ul></ul>Part 2
  69. 69. Motivation <ul><li>Most people draw </li></ul><ul><li>Few people sculpt </li></ul><ul><ul><li>Materials difficult to handle </li></ul></ul>Part 2
  70. 70. Motivation <ul><li>Most people draw </li></ul><ul><li>Few people sculpt </li></ul><ul><ul><li>Materials difficult to handle </li></ul></ul><ul><ul><li>Simpler with computer ? </li></ul></ul><ul><ul><ul><li>Scanning </li></ul></ul></ul><ul><ul><ul><li>Modeling </li></ul></ul></ul>Part 2
  71. 71. Motivation <ul><li>Most people draw </li></ul><ul><li>Few people sculpt </li></ul><ul><li>Drawing application: Teaching </li></ul><ul><ul><li>Example: Pr. Jean-Paul Chirossel, anatomy laboratory CHU Grenoble </li></ul></ul>Part 2
  72. 72. Motivation <ul><li>Drawing characteristics </li></ul><ul><ul><li>Visual abstraction </li></ul></ul>Human heart Part 2
  73. 73. Motivation <ul><li>Drawing characteristics </li></ul><ul><ul><li>Visual abstraction </li></ul></ul><ul><ul><li>Indication of uncertainty </li></ul></ul>Leonardo da Vinci Part 2
  74. 74. Motivation <ul><li>Drawing characteristics </li></ul><ul><ul><li>Visual abstraction </li></ul></ul><ul><ul><li>Indication of uncertainty </li></ul></ul><ul><ul><li>Limitation to single viewpoint </li></ul></ul>Part 2
  75. 75. Motivation <ul><li>Drawing characteristics </li></ul><ul><ul><li>Visual abstraction </li></ul></ul><ul><ul><li>Indication of uncertainty </li></ul></ul><ul><ul><li>Limitation to single viewpoint </li></ul></ul><ul><li>Problems </li></ul><ul><ul><li>Drawing with multiple viewpoints </li></ul></ul>Part 2
  76. 76. Motivation <ul><li>Drawing characteristics </li></ul><ul><ul><li>Visual abstraction </li></ul></ul><ul><ul><li>Indication of uncertainty </li></ul></ul><ul><ul><li>Limitation to single viewpoint </li></ul></ul><ul><li>Problems </li></ul><ul><ul><li>Drawing with multiple viewpoints </li></ul></ul><ul><ul><li>Modeling by drawing </li></ul></ul>Part 2
  77. 77. Our Contributions <ul><li>Part 1: Interactive physically based animation </li></ul><ul><ul><li>Animating anisotropic elastic materials [Bourguignon and Cani, EGCAS 2000] </li></ul></ul><ul><li>Part 2: Interaction in 3D using 2D input </li></ul><ul><ul><li>Drawing in 3D [Bourguignon et al., EG 2001] </li></ul></ul><ul><ul><li>Modeling by drawing </li></ul></ul>Part 2.1
  78. 78. Previous Work <ul><li>2D-to-3D drawing: 3D Strokes </li></ul><ul><ul><li>Input stroke and its shadow [Cohen et al., 1999] </li></ul></ul><ul><ul><ul><li>3D curves design, no drawing </li></ul></ul></ul>Part 2.1
  79. 79. Previous Work <ul><li>2D-to-3D drawing: 3D Strokes </li></ul><ul><ul><li>Input stroke and its shadow [Cohen et al., 1999] </li></ul></ul><ul><ul><li>Deep canvas [Disney, 1999] </li></ul></ul><ul><ul><ul><li>Need a 3D model </li></ul></ul></ul>Part 2.1
  80. 80. Previous Work <ul><li>2D-to-3D drawing: 3D Strokes </li></ul><ul><ul><li>Input stroke and its shadow [Cohen et al., 1999] </li></ul></ul><ul><ul><li>Deep canvas [Disney, 1999] </li></ul></ul><ul><ul><li>Billboard, terrain, etc., stroke [Cohen et al., 2000] </li></ul></ul><ul><ul><ul><li>Drawing modes adapted to landscaping only </li></ul></ul></ul>Part 2.1
  81. 81. Previous Work <ul><li>2D-to-3D drawing: 3D Strokes </li></ul><ul><li>2D-to-3D drawing: 3D Objects </li></ul><ul><ul><li>Reconstruction [Lipson and Shpitalni, 1996] </li></ul></ul><ul><ul><ul><li>No free-form drawing </li></ul></ul></ul>Part 2.1
  82. 82. Previous Work <ul><li>2D-to-3D drawing: 3D Strokes </li></ul><ul><li>2D-to-3D drawing: 3D Objects </li></ul><ul><ul><li>Reconstruction [Lipson and Shpitalni, 1996] </li></ul></ul><ul><ul><li>Sketching interface [Igarashi et al., 1999] </li></ul></ul><ul><ul><ul><li>Closed strokes only </li></ul></ul></ul>Part 2.1
  83. 83. Our Approach <ul><li>Drawing in 3D </li></ul><ul><ul><li>Augment strokes to true 3D entities </li></ul></ul><ul><ul><ul><li>Line stroke: Space curve (view-independent) </li></ul></ul></ul>Eye Edgar Degas Part 2.1
  84. 84. Our Approach <ul><li>Drawing in 3D </li></ul><ul><ul><li>Augment strokes to true 3D entities </li></ul></ul><ul><ul><ul><li>Line stroke: Space curve (view-independent) </li></ul></ul></ul><ul><ul><ul><li>Silhouette stroke: Surface contour (view-dependent) </li></ul></ul></ul>Back Edgar Degas Part 2.1
  85. 85. Our Approach <ul><li>Drawing in 3D </li></ul><ul><ul><li>Augment strokes to true 3D entities </li></ul></ul><ul><ul><li>Annotation of existing 3D models </li></ul></ul>Part 2.1
  86. 86. Our Approach <ul><li>Drawing in 3D </li></ul><ul><ul><li>Augment strokes to true 3D entities </li></ul></ul><ul><ul><li>Annotation of existing 3D models </li></ul></ul><ul><ul><li>Illustration in 3D </li></ul></ul>Part 2.1
  87. 87. Our Approach <ul><li>Drawing in 3D </li></ul><ul><li>Choices </li></ul><ul><ul><li>Represent line stroke as space curve </li></ul></ul>Part 2.1
  88. 88. Our Approach <ul><li>Drawing in 3D </li></ul><ul><li>Choices </li></ul><ul><ul><li>Represent line stroke as space curve </li></ul></ul><ul><ul><li>Represent silhouette stroke using local surface </li></ul></ul><ul><ul><ul><li>Infer local surface from user input </li></ul></ul></ul><ul><ul><ul><li>New silhouette from new viewpoint </li></ul></ul></ul>Part 2.1
  89. 89. Silhouette Stroke <ul><li>Infer local surface from user input </li></ul><ul><ul><li>Simplest: same local curvature in 3D as in 2D </li></ul></ul>Part 2.1
  90. 90. Silhouette Stroke <ul><li>Infer local surface from user input </li></ul><ul><ul><li>Simplest: same local curvature in 3D as in 2D </li></ul></ul><ul><ul><li>Modulate width as if fitting circles along curve </li></ul></ul>Part 2.1
  91. 91. Silhouette Stroke <ul><li>Infer local surface from user input </li></ul><ul><ul><li>Simplest: same local curvature in 3D as in 2D </li></ul></ul><ul><ul><li>Modulate width as if fitting circles along curve </li></ul></ul><ul><ul><li>Resulting surface </li></ul></ul>Part 2.1
  92. 92. Silhouette Stroke <ul><li>New silhouette from new viewpoint </li></ul><ul><ul><li>Approximate silhouette </li></ul></ul>Part 2.1
  93. 93. Silhouette Stroke <ul><li>New silhouette from new viewpoint </li></ul><ul><ul><li>Approximate silhouette </li></ul></ul><ul><ul><li>Represent uncertainty away from viewpoint </li></ul></ul>Part 2.1
  94. 94. Silhouette Stroke <ul><li>New silhouette from new viewpoint </li></ul><ul><ul><li>Approximate silhouette </li></ul></ul><ul><ul><li>Represent uncertainty away from viewpoint </li></ul></ul><ul><ul><li>Manage occlusion with background color </li></ul></ul>Part 2.1
  95. 95. Interface for Drawing <ul><li>Two drawing modes </li></ul><ul><ul><li>In empty space </li></ul></ul>Part 2.1
  96. 96. Interface for Drawing <ul><li>Two drawing modes </li></ul><ul><ul><li>In empty space </li></ul></ul><ul><ul><li>Relatively to other objects </li></ul></ul>Part 2.1
  97. 97. Interface for Drawing <ul><li>Video </li></ul>Part 2.1
  98. 98. Applications <ul><li>Annotation </li></ul>Part 2.1
  99. 99. Applications <ul><li>Illustration </li></ul>Part 2.1
  100. 100. Conclusion <ul><li>System for drawing in 3D </li></ul><ul><ul><li>View-dependent strokes </li></ul></ul>Part 2.1
  101. 101. Conclusion <ul><li>System for drawing in 3D </li></ul><ul><ul><li>View-dependent strokes </li></ul></ul><ul><ul><li>Useful for drawing simple scenes in 3D </li></ul></ul>Part 2.1
  102. 102. Conclusion <ul><li>System for drawing in 3D </li></ul><ul><ul><li>View-dependent strokes </li></ul></ul><ul><ul><li>Useful for drawing simple scenes in 3D </li></ul></ul><ul><ul><li>Useful for annotations </li></ul></ul>Part 2.1
  103. 103. Conclusion <ul><li>System for drawing in 3D </li></ul><ul><li>Limitations </li></ul><ul><ul><li>Switching between stroke types </li></ul></ul>Part 2.1
  104. 104. Conclusion <ul><li>System for drawing in 3D </li></ul><ul><li>Limitations </li></ul><ul><ul><li>Switching between stroke types </li></ul></ul><ul><ul><li>Plane positioning can be tedious </li></ul></ul>Part 2.1
  105. 105. Our Contributions <ul><li>Part 1: Interactive physically based animation </li></ul><ul><ul><li>Animating anisotropic elastic materials [Bourguignon and Cani, EGCAS 2000] </li></ul></ul><ul><li>Part 2: Interaction in 3D using 2D input </li></ul><ul><ul><li>Drawing in 3D [Bourguignon et al., EG 2001] </li></ul></ul><ul><ul><li>Modeling by drawing </li></ul></ul>Part 2.2
  106. 106. Motivation <ul><li>Input: Just plain strokes… </li></ul><ul><ul><li>Silhouette, sharp features ? </li></ul></ul><ul><ul><li>Texture, shading ? </li></ul></ul><ul><ul><li>Open, closed, self-intersecting ? </li></ul></ul>Part 2.2
  107. 107. Motivation <ul><li>Input: Just plain strokes… </li></ul><ul><li>Output: Manifold polyhedral surface </li></ul>Part 2.2
  108. 108. Motivation <ul><li>Input: Just plain strokes… </li></ul><ul><li>Output: Manifold polyhedral surface </li></ul><ul><li>Pen-and-paper for sculptors </li></ul><ul><ul><li>Painter and sculptor shading </li></ul></ul>Michelangelo Buonarroti Rembrandt van Rijn Part 2.2
  109. 109. Previous Work <ul><li>Painting depth as luminance [Williams, 1990] </li></ul>Part 2.2 +
  110. 110. Previous Work <ul><li>Silhouette inflation [Williams, 1991] </li></ul>Part 2.2
  111. 111. Previous Work <ul><li>Editing gradient by shading [van Overveld, 1996] </li></ul>Part 2.2
  112. 112. Previous Work <ul><li>Bump map inference [Johnston, 2002] </li></ul>Part 2.2
  113. 113. Previous Work <ul><li>Direct manipulation interface </li></ul>Artisan [Alias|wavefront, 2002] ZBrush [Pixologic, 2002] Part 2.2
  114. 114. From 2D to 2.5D <ul><li>Overview </li></ul>Strokes 2D discontinuous Part 2.2
  115. 115. From 2D to 2.5D <ul><li>Overview </li></ul>Geometry 2D discontinuous Part 2.2 Strokes
  116. 116. From 2D to 2.5D <ul><li>Overview </li></ul>Geometry Constrained triangulation 2D discontinuous Part 2.2 Strokes
  117. 117. From 2D to 2.5D <ul><li>Overview </li></ul>Geometry Non-convex hull 2D discontinuous Part 2.2 Strokes Constrained triangulation
  118. 118. From 2D to 2.5D <ul><li>Overview </li></ul>Geometry Image 2D discontinuous Part 2.2 Strokes Constrained triangulation Non-convex hull
  119. 119. From 2D to 2.5D <ul><li>Overview </li></ul>Geometry Height field 2D discontinuous 2.5D continuous Part 2.2 Strokes Image Constrained triangulation Non-convex hull
  120. 120. From 2D to 2.5D <ul><li>Find a non-convex hull </li></ul><ul><ul><li>Original drawing (polylines) </li></ul></ul>Part 2.2
  121. 121. From 2D to 2.5D <ul><li>Find a non-convex hull </li></ul><ul><ul><li>Original drawing (polylines) </li></ul></ul><ul><ul><li>Constrained Delaunay triangulation </li></ul></ul>Part 2.2
  122. 122. From 2D to 2.5D <ul><li>Find a non-convex hull </li></ul><ul><ul><li>Original drawing (polylines) </li></ul></ul><ul><ul><li>Constrained Delaunay triangulation </li></ul></ul><ul><ul><li>Non-convex hull [Watson, 1997] </li></ul></ul>Part 2.2
  123. 123. From 2D to 2.5D <ul><li>Hole marks </li></ul><ul><ul><li>In comics books production </li></ul></ul>Part 2.2 Stone #3 , Avalon Studios
  124. 124. From 2D to 2.5D <ul><li>Hole marks </li></ul><ul><ul><li>In comics books production </li></ul></ul><ul><ul><li>In our system </li></ul></ul>Part 2.2
  125. 125. From 2D to 2.5D <ul><li>Infer a height field </li></ul><ul><ul><li>Large features have large inflations </li></ul></ul>Part 2.2
  126. 126. From 2D to 2.5D <ul><li>Infer a height field </li></ul><ul><ul><li>Large features have large inflations </li></ul></ul><ul><ul><li>Use geometry information to build it </li></ul></ul>Part 2.2
  127. 127. From 2D to 2.5D <ul><li>Infer a height field </li></ul><ul><ul><li>Large features have large inflations </li></ul></ul><ul><ul><li>Use geometry information to build it </li></ul></ul><ul><ul><li>Use texture information to modulate it </li></ul></ul>Part 2.2
  128. 128. From 2D to 2.5D <ul><li>Infer a height field: First step </li></ul><ul><ul><li>Euclidean distance transform </li></ul></ul>Part 2.2
  129. 129. From 2D to 2.5D <ul><li>Infer a height field: First step </li></ul><ul><ul><li>Mapping to unit sphere [Oh et al., 2001] </li></ul></ul>Part 2.2
  130. 130. From 2D to 2.5D <ul><li>Infer a height field: First step </li></ul><ul><ul><li>Adaptive low pass filter [Williams, 1991] </li></ul></ul>Part 2.2
  131. 131. From 2D to 2.5D <ul><li>Infer a height field: Second step </li></ul><ul><ul><li>Use same filter for image </li></ul></ul>Part 2.2
  132. 132. From 2D to 2.5D <ul><li>Infer a height field: Third step </li></ul><ul><ul><li>Use previous height field as matte for image </li></ul></ul>Part 2.2
  133. 133. From 2D to 2.5D <ul><li>Infer a height field: Result </li></ul>Part 2.2
  134. 134. From 2D to 2.5D <ul><li>Fast polygonal height field approximation [Garland and Heckbert, 1995] </li></ul>Part 2.2
  135. 135. From 2D to 2.5D <ul><li>Result: Manifold polyhedral surface </li></ul>Part 2.2
  136. 136. Results <ul><li>A simple sketch of a human heart </li></ul>
  137. 137. Conclusion <ul><li>System for modeling by drawing </li></ul><ul><ul><li>Plain strokes as input </li></ul></ul><ul><ul><li>Manifold polyhedral surface as output </li></ul></ul><ul><ul><li>Using sculptor shading convention </li></ul></ul>Part 2.2
  138. 138. Conclusion <ul><li>System for modeling by drawing </li></ul><ul><ul><li>Plain strokes as input </li></ul></ul><ul><ul><li>Manifold polyhedral surface as output </li></ul></ul><ul><ul><li>Using sculptor shading convention </li></ul></ul><ul><li>Limited to a single viewpoint </li></ul>Part 2.2
  139. 139. Future Work <ul><li>From 2.5D to 3D: Iterative modeling process </li></ul>Part 2.2 Modeling by drawing Changing viewpoint
  140. 140. Future Work <ul><li>Relief metaphor </li></ul><ul><ul><li>From low to high relief </li></ul></ul><ul><ul><li>From painting to sculpture </li></ul></ul>Fourth century B.C. First century B.C. Fifteenth century Part 2.2
  141. 141. General Conclusion <ul><li>Animation of anisotropic material </li></ul><ul><ul><li>Intuitive </li></ul></ul><ul><ul><li>Efficient </li></ul></ul>
  142. 142. General Conclusion <ul><li>Animation of anisotropic material </li></ul><ul><li>Three-dimensional drawing system </li></ul><ul><ul><li>Use drawing characteristics </li></ul></ul><ul><ul><li>Good geometric detail vs modeling speed tradeoff </li></ul></ul>
  143. 143. General Conclusion <ul><li>Animation of anisotropic material </li></ul><ul><li>Three-dimensional drawing system </li></ul><ul><li>Modeling by drawing from a single viewpoint </li></ul>
  144. 144. General Future Work <ul><li>Evaluation according to ergonomics methods </li></ul><ul><li>“Drawing as a front-end to everything” [Gross and Do, 1996] </li></ul>
  145. 145. Merci de votre attention

×