3D Visualization Method for Maxillofacial Surgical Planning by Using                     X-ray and Photo Images           ...
1      Introduction     Digital dental surgery planning system has many advantages in that of predicting thepostsurgical m...
The remainder of this paper is structured as follow: In section 2 we describe the    motivation of our research. And we gi...
images on frontal view. Figure2 shows the overall flow structure of our dental surgeryplanning system.                    ...
a)                        b)Figure 3. Facial contour extraction. a) extracted facial profile on X-ray image. b)Adaptation ...
a)                b)            c)Figure 4. Maxillofacial surgery on the hard tissue contour. a) cutting and moving theman...
preprocessing, we get two wire templates on front and side views and use them tomodify a generic facial model. In this pap...
 x       x                               A =    H =   , A′ =   J ′ =                                        ...
of the feature point on soft-tissue by prediction function, and interpolate the additionalpoints by equation 4.    Maxillo...
a)            b)             c)Figure 9. 3 dimensional control points from CT data. a) soft tissue control pointson latera...
Z      a)            b)      Figure 10. Computing variation of 3D difference before and after maxillofacial      surgery. ...
Figure 11. Comparison simulated results with actual surgery. a) Input data: presurgical images. b)Estimated postsurgical i...
No.10, pp.1095-1102, Oct. 1998.[3] Hounsfield, G.N., and Ambrose, J.A. Computerized transverse axial scanning    tomograph...
Upcoming SlideShare
Loading in...5
×

doc

206

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
206
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

doc

  1. 1. 3D Visualization Method for Maxillofacial Surgical Planning by Using X-ray and Photo Images Young-In Kim*, Jung-Hyun Park**, Chang-Hun Kim* Dept. of Computer Science & Engineering, Korea University*, Dept. of Oral&Maxillofacial Surgery, Dental College, Yonsei University** {yikim,chkim}@cgvr.korea.ac.kr*, goodputt@hotmail.com** Abstract This paper describes a prototype system for predicting of human facial shape andvisualizing of realistic 3D images after maxillofacial surgery for patient with facialdeformities from three input images consisting of lateral Xray, lateral photo and frontalphoto. The basic idea is to join a accurate predicting variation of facial shape with a 3Dvisualizing postsurgical facial shape. Firstly, we draw out the important parts of face,contours and control points in the profile of a patient from Xray image. Secondly, whena surgeon designates beforehand the change of the direction and magnitude of amaxillary bones (the lower and upper jawbone) at plan step, the system forms thepostsurgical image using warping method and predicting function which calculate thechange of movement of a soft tissue. The predicting function is based on the clinicalstudy obtained from 100 patients and the warping method is implemented consideringanatomic relation of facial soft tissue. Finally we generate the three dimensional imagesof postsurgical facial shape using the deformation estimating method which predict the3dimensional movement of vertices on face. Keywords: Digital Surgery, Medical Visualization, Surgical Planning System
  2. 2. 1 Introduction Digital dental surgery planning system has many advantages in that of predicting thepostsurgical morphology and appearance of human face, or solving the problem of a fairfacial surface before the actual surgery is carried out. Early works [4,6,7] needexpensive equipment and high computation cost because it require the complex andlaborious processing steps of mapping the soft tissue structure to the 3D skull modelwhich is constructed from 2D CT or MRI data. More over these systems are notadequate for maxillofacial surgery because subtle malformation can strongly affect theappearance of a face and these systems is hard to handle to surgeon. Thus typically,the facial operation surgeon draws the patients predicted profile using X-ray to give atleast a 2D appearance of the future face. In this thesis, we propose the facial surgeryplanning system, which computes accurate 2D lateral facial contour and realistic 3Dpicture of the post surgical shape. Our system consists of two modules, one is about the virtual surgery step and theother is 3D visualization step. In virtual surgery step, we use lateral X-ray of patientand predicting function for computing predicted soft tissue profile after maxillofacialsurgery. In 3D visualization step, we use two photos, which are frontal and lateralimages of patient, and deformation estimating method for generating 3D appearance ofpost surgical facial shape. Figure1 shows the simple procedure of our surgerypredicting system.a) b) c)Figure 1: Illustration of system flow for maxillofacial surgery planning.a) Lateral photo and X-ray image of the presurgical face. b) 2D postsurgical image aftermaxillofacial surgical simulation. c) 3D postsurgical appearance.
  3. 3. The remainder of this paper is structured as follow: In section 2 we describe the motivation of our research. And we give an overview of the system in section 3. Section 4 we explain the principles of the predicting function, which is used for virtual surgery. Section5 describes the deformation estimating method of soft tissue for generating 3D appearance. Finally, we demonstrate and discuss the proposed system with experimental results obtained from the X-ray image and frontal and lateral facial photos in section 6.2 Motivation There has been much research in the field of facial surgery simulation. Most facial surgery simulation systems used today in 3D medical image data like CT or MRI. These 3D medical image data requires high cost and complex preprocessing for surgical simulation. So we choose the planar medical image data, X-ray image, traditionally it has been used in clinical area. Early works such as [9] restricted themselves to 2D lateral image generation of the malformed patients. So we propose the system which requires a simple and cheap X-ray image data and generates 3D appearance of postsurgical facial shape.3 System Overview This section explains on different procedures, data acquisition and preprocessingsteps, virtual surgery step, and 3D visualization step in the chart of Figure 2. Our datasources consist of X-ray image, frontal and lateral facial photos. First of all, an initial facial contour is extracted from the X-ray image and controlpoints is created on a lateral contour. Secondly, we simulate the maxillofacial surgery system, which enable to compute thevariations between the presurgical profile and postsurgical profile. As this resultpostsurgical profile contour is created and it is warped with the lateral facial photo.Then we warp the lateral photo to do the virtual surgery result. Finally, we execute the third pass procedure, 3D postsurgical image generation. Inour system, we individualize the generic facial model and predict the postsurgical facial
  4. 4. images on frontal view. Figure2 shows the overall flow structure of our dental surgeryplanning system. Figure 2: System overview. 3.1 Data Acquisition and Preprocessing The first pass procedure of our virtual surgery system is the data acquisition andpreprocessing step. In this step, we prepare two 2D wire frame templates composed offeature points with predefined relation for front and side views and draw out the facialprofile on X-ray image and align facial lateral photo with it. Because X-ray image maintains the 1:1 ratio of the surgery area, the lateral photo istransformed by fixed profile on X-ray. Figure 3 shows the result after malformedprofile contours extraction and adaptation of them. These images can be aligned byrotation and translation and scaling.
  5. 5. a) b)Figure 3. Facial contour extraction. a) extracted facial profile on X-ray image. b)Adaptation profile to X-ray image and photo. 3.2 Virtual Surgery Operation This section explains the process of virtual operation necessary to accomplish facialsurgery simulations. First, we cut the malformed mandible contour and move it tocorrect it. Next, the variations of the control points on facial soft tissue contoursbetween presurgery and postsurgery are computed by predicting function. Finally, non-control points consisting of soft tissue contours are interpolated by linear interpolationmethod. Figure 4 shows the process of virtual operation. Figure 4a represents the cut andmoved malformed mandible contour. Figure 4b is the result from calculating movementof the control points on soft tissue. Figure 4c shows the linear interpolating movementsof non-control points on soft-tissue contours.
  6. 6. a) b) c)Figure 4. Maxillofacial surgery on the hard tissue contour. a) cutting and moving themandible. b) calculating the movements of control point on soft tissue. c) linearinterpolating the movements of non-control point on soft tissue contours. This system makes synthetic postoperative image of post-operation by imagewarping technique. Image warping process consists of triangulation and colorinterpolation. Delaunay triangulation used on this system defines local areas on lateralphoto, each area is reorganized by virtual surgery, and color information in area isinterpolated. Figure 5 shows this process about generating warped postsurgical image. Figure 5bshows the divided regions by triangulation and Figure 5d shows the warped result ofthe modified model by virtual surgery. a) b) c) d) Figure 5. Virtual postsurgical facial image generation. a) Input data : lateral facialphoto. b) Triangulation of input image. c) Virtual operation d) Warped image of themodified model by virtual surgery 3.3 3D postsurgical image generation In this section, we explain the process for visualizing 3D postsurgical imagegeneration. First of all, we make individual face to visualize 3D images. After
  7. 7. preprocessing, we get two wire templates on front and side views and use them tomodify a generic facial model. In this paper, generic facial model with 915 vertices ismodified to make an individualized smooth facial surface. Figure 6 shows the process ofindividualizing a generic facial model. Figure 6. Individualization of generic facial model4 Computation of the Soft-Tissue Variation In this paper, we use the predicting function to calculate the variation ofpostsurgical soft tissue profile. As the variation of human soft tissue can’t be generallycomputed by linear equations, we suppose that it can be computed by non-linearequations. To formulate our predicting function, we define 8 control points on hardtissue and 10 control points on soft tissue. Figure 7 shows these control points. Controlpoints on hard tissue consist of A(ANS), B(A), C(MxI), D(MxM), E(MnI), F(B), G(Pg),H(Me) and control points on soft tissue consist of A’(Pn), B’(Sn), C’(A), D’(Ls), E’(Stms), Hard control points Soft control pointsF’(Stmi), G’(Li), H’(B’), I(Pg’), J’(Me’). Figure 7. Facial control points on hard tissue and soft tissue. This predicting function is based on the clinical study obtained from 100 patients. Formeasuring the variation between presurgical profile and postsurgical profile, we defineeq. 1 as following.
  8. 8.  x  x A =    H =   , A′ =   J ′ =   x x  y  y    y    y Eq. 1         The horizontal variation on soft tissue A’ is formulated as following.  Ax  2  A2  y      Ax   Ay  A′x = [α 1  α 16 ]    + [ β 1  β 16 ]     2  2 Eq. 2 H x  H y  H  H   x  y As the preceding equation 2 , other horizontal variation on control points from B’ to H’is formulated. Equation 3 explains the linear regression for finding values of α i , β i . Dn×1 = Z n×m ∆Am×1 Eq. 3  ∂f1 ∂f   ∆a 0    1     ∂a1 ∂am  f ( x ) = a1 x +  + a n x n ∆Am ×1 =   Z n× m =    ∆a   ∂f n ∂f n   m  ∂a  ∂a   1   m  Yi :estimated value, xi :input value , ai :parameter , n :the number of data, m : the number of parameter Using hard-tissue location changed from virtual surgery produce the soft-tissuechange by soft-tissue movement prediction functions. Each partial function for a featurepoint on soft-tissue makes the movement based on multiple feature points on hard-tissue. Table 1 shows the example of two partial functions. Table1 Two functions about hard and soft tissue relation Soft-tissue Hard-tissue VPn 0.44409 * ANS – 0.27588 * vA + 0.05905 VB’ 0.65773 * vMe – 0.13109 * hMe + 0.21406 * hMnM – 1.31458 11 soft tissue control points predicted by that of hard tissue control point is as smallas this system represent the smooth facial outline. So, additional points of soft tissue isdefined as follows: p − p′ π Weight = cos( ) , 0 ≤ Weight ≤ 1 Eq. 4 w 2 , where p is the control point coordinate, w is the maximum length effected by p, andp’ is the point on soft-tissue. [Figure 4] shows that this system determines coordinate
  9. 9. of the feature point on soft-tissue by prediction function, and interpolate the additionalpoints by equation 4. Maxillofacial surgeon generally says that an error tolerance on maxillofacial surgeryis less than 2mm. In figure 8, we represent the comparison of result on actual surgerywith estimated surgery. You can see that the differences between this two graphs arenot greater than 2 mm. Figure 8. The comparison of result between actual surgery and estimated surgery.5 Deformation Estimating Method Our system makes frontal postsurgical facial image by the deformation estimating method. 3D visualization process consists of the individualization of a generic facial model and the calculation of 3dimensional movement of soft tissue on face. For calculating the movement of control points, we define the deformation estimating method. First, we measure 3 dimensional variations of control points between soft tissue and hard tissue from CT data. Figure 8 shows the 3 dimensional control points of the soft tissue. The number of 3 dimensional control points is seven, Pn, Sn, A, Ls, Li, B’, Pg’ , except Stms, Stmi, Me.
  10. 10. a) b) c)Figure 9. 3 dimensional control points from CT data. a) soft tissue control pointson lateral view b) control points on frontal view. c) control points from –90 ° to+90° on sectional view.Secondly, we calculate the 3 dimensional variations on facial soft tissue. For thiscomputation, an axis of coordinates is established on individual facial model. Weuse the following equation to compute the postsurgical soft tissue control points. Inthis equation 5, P(x,y,z) is a control point on soft tissue contour before virtualmaxillofacial surgery and P’(x’,y’,z’) is computed position of it after virtualmaxillofacial surgery. P’’(x’’,y’’,z’’) in figure 10 is a temporary point projected onxy-plane a point P(x,y,z).  x ′   x + ∆x       y′ =  y , ∆ x = d ⋅ cosθ , ∆z = d ⋅ sin θ . Eq. 5.  z ′   z + ∆z     
  11. 11. Z a) b) Figure 10. Computing variation of 3D difference before and after maxillofacial surgery. a) lateral comutation model. b) sectional computation model.6 Results The goal was to predict the facial shape after procedure in maxillofacial surgery.Figure 11-12 show the shape of soft tissue after maxillofacial surgery. The calculationswere carried out on a Intel Pentium-III 700 processor, 256MB RAM, Microsoft WindowsNTTM Workstation 4.0 and our system was implemented by using Microsoft Visual C++TM6.0 and SGI OpenGL. We can compare a estimated postsurgical image with a actualpostsurgical image. (See Figure11). We can find the high similarity of two images. a) b) c)
  12. 12. Figure 11. Comparison simulated results with actual surgery. a) Input data: presurgical images. b)Estimated postsurgical images. c) Actual postsurgical image a) b) c) d) f) f)Figure 12. 3D images after maxillofacial surgery. a,b) presurgical frontal and lateralpictures of patient. c) Estimated postsurgical facial image. d,e) Actual postsurgicalfrontal and lateral photo. f) 3d visualization of estimated postsurgical result.7 Conclusion We present a system which enables us to predict the deformations of the facial shapeafter surgical procedures. Our system can generate a high realistic and accuratepostsurgical images by predicting function which obtained from 100 clinical studies. Inour system, maxillofacial surgeon can easily extract the outline profile and predict thepostsurgical facial shape.8 Reference [1] Haider, A. Md., Takahashi, E. and Kaneko, T., Automatic Reconstruction of 3D Human Face from CT and Color Photographs, IEICE Trans. Inf. & Syst.., vol.E81- D, No.9, pp.1287-1293, Sep. 1999. [2] Haider, A. Md., Takahashi, E. and Kaneko, T., A 3D face reconstruction method from CT image and color photographs, IEICE Trans. Inf. & Syst.., Vol.E81-E,
  13. 13. No.10, pp.1095-1102, Oct. 1998.[3] Hounsfield, G.N., and Ambrose, J.A. Computerized transverse axial scanning tomography. British Journal of Radiology, pp.1016-1022, 1973[4] Koch, R.M., Gross, M.H., Carls, F.R., von Büren, D.F., Fankhauser, G. and Parish, Y.I.H., Simulating facial surgery using finite element models, Computer Graphics(SIGGRAPH’96 Proceedings), pp.421-429, Aug. 1996[5] Lee, W-S., Thalmann, N.M., Fast head modeling for animation, Image and Vision Computing, pp.355-364, 2000.[6] Xia, J., Wang, D., Samman, N., Yeung, R.W.K., Tideman, H., Computer-assisted three-dimensional surgical planning and simulation: 3D color facial model generation, International journal of oral & maxillofacial surgery , Vol.29, No.1, pp.20-10, Feb. 2000.[7] Xia, J., Ip, H.H.S., Samman, N., Wang, D., Kot, C.S.B., Yeung, R.W.K., Tideman, H., Computer-assisted three-dimensional surgical planning and simulation: 3D virtual osteotomy, International journal of oral & maxillofacial surgery, Vol.29, No.1, pp.11-17, Feb. 2000.[8] Dr.Pss,(http://www.bit.co.kr/medical-info/html/ drpss.htm)[9] QuickCeph Systems, (http://www.quickceph.com/)

×