1   3-Dimensional Surface Visualization of Anatomical CT/MRI Slice Data                                       Shashank Sin...
2                                        CONTENTS   I. CERTIFICATE of work.  II. INTRODUCTION to project.         a. Backg...
3                                    I.   CERTIFICATE   Certified that the work contained in the report entitled “3-Dimens...
4                                                 II.   INTRODUCTION         The project aims at surface visualization and...
5    d.   DATA:     The male cadaver was frozen and cut into 1 871 axial slices (at 1 millimeter intervals) which were the...
6    e.   THE DONORS:The male cadaver is from Joseph Paul Jernigan, a 38-year-old, 199 lb, 5 11, Texas murderer who wasexe...
7There are a number of different methods used:         -   Rendering voxels in binary partitioned space         -   Marchi...
8Ray-casting sensation began with the release of a game, Wolfenstein 3D (iD Software), in 1992. InWolfenstein 3D, the play...
9Several different algorithms for ray casting exist. The one presented by Marc Levoy in many of his papers[3], is used in ...
10                                  Fig. Illustration of VOXEL cube and Slice Thicknessl.        OBJECT CHARACTERISTICS IN...
11       The preprocessing used commonly can be grouped as following operations:  i.   Volume of interest: Converts a give...
12                                             III.   METHODOLOGYThe entire process carried out under this project can be ...
13                 b.   Steps 1, 2 and 3:                 The data set from NLM server was downloaded via ftp over a perio...
14         The original data is in the form of .Z compression. When these compressed files were extracted,the outputs were...
15To appreciate the variations in images quality, shown below are the sampled zoomed screenshots fromdifferent formats and...
16 ii.         The Directory Structure of the Data Set Repository build up at IIIT Medical Imaging Lab is             depi...
17c.   Steps 4 and 5:The raw data from the ‘National Library of Medicine’ is converted to standard image formats. Theimage...
18     The formula used to determine output pixel value is:                new(x, y) = light_factor* light_factor* light_f...
19                                                               IV. RESULTS         We have provided options to vary the ...
20Fig. 4. Depicts views from different angles (other parameters remaining constant):           Viewing angle = 0 degree   ...
21Fig. 5 to 8 are the results depicting the performance of the program to cut out a view from the whole object            ...
22     Volume Cut Views of axial Images:      Fig. 11 sagittal view from axial CT slices.Slices used = 200, Cut Depth = 22...
23      Fig. 13 Cornal view from axial Cryo-Sections. Slices used = 777, Cut Depth = 594, Time = 1170.86 s.   Fig. 14 View...
24            Surface Visualization from Cryo-Sections:            Fig. 15. Skin Surface view from Female Cryo-SectionsRat...
25        Ratio = 0.60, Viewing Angle = 45 degree, Slices used = 800, Time = 3665.781            Fig. 16. Skin Surface vie...
26   Noise Removal, Image Resizing and Cropping:               Fig 17. Original ImageFig 18. Image after background Remova...
27Variations of RGB and Gray Values in the Images:      Fig 19 RGB profile in Original Image.
28 Fig 20 RGB profile in Original Image.Fig 21 Gray Value profile in CT Image.Fig 22 Gray Value profile in MRI Image.
29                                         V. DISCUSSIONThe method generates quite good quality 3-dimensioanl views of the...
30                                           VII. TIME LINE:November – December:                  -       Implemented basi...
31         I would like express my gratitude towards my project guide Dr. UmaShanker Tiwary for his         invaluable gui...
32                -     IEEE Xplore.                          http://www.ieeexplore.ieee.org/Xplore/DynWel.jsp            ...
32                -     IEEE Xplore.                          http://www.ieeexplore.ieee.org/Xplore/DynWel.jsp            ...
32                -     IEEE Xplore.                          http://www.ieeexplore.ieee.org/Xplore/DynWel.jsp            ...
Upcoming SlideShare
Loading in …5
×

Project report 3D visualization of medical imaging data

842
-1

Published on

Report of my engineering research on 3D visualisation of medical images obtained from slices of human male and female cadevars. Courtesy NIH (USA), IIIT (Allahabad)

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
842
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
33
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Project report 3D visualization of medical imaging data

  1. 1. 1 3-Dimensional Surface Visualization of Anatomical CT/MRI Slice Data Shashank Singh, Bachelor of Information Technology, Eighth Semester, Indian Institute of Information Technology – Allahabad Enrollment No. 20000048 Email: shashank@iiita.ac.in Project Supervisor: Dr. UmaShanker Tiwary Associate Professor Indian Institute of Information Technology – Allahabad Email: ust@iiita.ac.in Abstract—The aim of the project is 3D surface visualization of body organs from 2Dslices and sections of the body parts. The input data is in the form of CT/MRI axial slicesand anatomical cryo-section images of the body parts. The object of interest is extractedfrom the slice data after performing an interactive thresholding. There after surface viewsare generated using ray-casting methods. The work would serve as tool for visualizingmedical objects while fusing multi modal medical imaging data. It also aims at providingan environment for measurement and estimation of damages/ deformities in the organs. Keywords—Ray-Casting, CT, MRI, Anatomical Cryo-Section, Surface Visualization,3-D viewing, Volume cut, Segmentation, Thresholding
  2. 2. 2 CONTENTS I. CERTIFICATE of work. II. INTRODUCTION to project. a. Background b. About data source c. Visible Human Project d. Data e. Donors f. Problems with data set g. Discoveries h. License i. About 3D imaging i. Ray Casting ii. Disadvantages iii. Possible improvements j. Sources of Medical Images k. Basics and terminology l. Objects characteristics in images i. Graded composition ii. Hanging-togetherness m. Preprocessing i. Volume of interest (VOI) ii. Filtering iii. Interpolation iv. Registration v. Segmentation III. METHODOLOGY a. Steps b. Steps 1,2 and 3 i. Data Format ii. Data Directory Structure c. Steps 4 and 5 i. Threshold determination d. Steps 6 and 7 i. Ray Casting e. Step 8 IV. RESULTS V. DISCUSSIONS VI. CONCLUSIONVII. TIME LINEVIII. ACKNOWLEDGEMENT IX. REFERENCES X. RESOURCES a. Books b. web XI. COURTESY
  3. 3. 3 I. CERTIFICATE Certified that the work contained in the report entitled “3-Dimensional Surface Visualizationof Anatomical CT/MRI Slice Data”, by Shashank Singh (Enrollment Number 20000048), hasbeen carried out under my supervision as his B.Tech. project, and that this work has not beensubmitted elsewhere for a degree.________________________(Date)________________________________(Signatures of the project supervisor)Dr. Umashanker TiwaryAssociate ProfessorIndian Institute of Information Technology – ALLAHABAD( Name of the project supervisor)
  4. 4. 4 II. INTRODUCTION The project aims at surface visualization and 3-dimensional viewing of anatomical CT/MRI slices. Working as a part of ‘Fusion of Multi-Modal Medical Imaging Data’ project, I would provide support for 3d viewing of object of interest from various 2d data modalities like CT, MRI slices and anatomical section images.a. BACKGROUND: Research in three-dimensional imaging in medicine started in the 1970s. The activity in the field became very brisk in the 1980s. During this period, the technical developments – design of algorithms, software, and machine – have galloped ahead, accompanied by a considerable increase in clinical applications, but clinical validation and the study of clinical usefulness have generally lagged behind. Although the applications that were pursued for 3D imaging in the early 1980 s were for imaging the bone via CT, with advent of MRI in the mid-1980’s, soft-tissue 3D imaging became a hot topic. This also initiated research in the better utilization of functional imaging modalities such as PET in conjunction with CT and MRI for a variety of applications. The 1990’s saw the reemergence of CT with the advent of spiral technology, and the rapid acquisition of the 3D volume data made possible new 3D imaging applications for CT. The 1990s also witnessed rapid advances in the use of 3D imaging for interventional and surgery procedures. The difficult problem of acquiring image data and processing and analyzing them for dynamic organs also continued to be pursued with steady progress. CT, MRI and cryo-sections provide the doctors and researchers with great tools to view at body internals. But all these imaging modalities are restricted to planar view of the body sections. But these modalities provide us with the images that can be fed into a 3D visualization machine and generate 3D views. 3D visualization of the CT, MRI, cryo-sectional data provides a revolutionary way to look at the body organs. The shape, orientations and spatial locations of organs are far better perceived in 3D views as compared to the 2D slices form which these 3D views are generated. Such visualizations can of great help in surgical operations and treatments. The deformities and damages in the organs such as those in skull fractures or disfigurement can be better studied by doctors using 3D surface visualization tools.b. ABOUT DATA SOURCE: For obtaining faithful 3D views the data source must be reliable and truly represent the human body. That is what made us to turn to the National Library of Medicine, USA.c. VISIBLE HUMAN PROJECT: In 1989, the National Library of Medicine (NLM) began an ambitious project to create a digital atlas of the human anatomy. The NLM Planning Panel on Electronic Image Libraries recommended a project to create XRAY Computed Tomography (XRAY-CT), Magnetic Resonance Imaging (MRI) and physical sections of a human cadaver. The Visible Human Project is an effort to create a detailed data set of cross- sectional photographs of the human body, in order to facilitate anatomy visualization applications. A male and a female cadaver were cut into thin slices which were then photographed and digitized. The project is run by the U. S. National Library of Medicine (NLM) under the direction of Michael J. Ackerman. Planning began in 1989; the dataset of male was released in November 1994 and the one of the Female in November 1995.
  5. 5. 5 d. DATA: The male cadaver was frozen and cut into 1 871 axial slices (at 1 millimeter intervals) which were thenphotographed and digitized, yielding 15 GB of data. In 2000, the photos were rescanned at a higherresolution, yielding more than 65 gigabytes. The female cadaver was cut into slices at .33 millimeterintervals, resulting in some 40 gigabytes of data. Each cadaver was perfused with approximately 19 litersof 1% formalin (a type of preservative /fixative used for tissue specimens) and an anticoagulant shortlyafter death to retard deterioration. The cadavers were placed horizontally on their backs, which resulted inthe feet being pointed away from the head rather than as they would be if flat on the floor in a standingheight measurement. (This adds about 3" to the apparent height, which is included in the figures mentionedabove.) For fiduciary markers two 3 mm OD Tygon tubes filled with 4 millimolar copper sulfate solutionand 5% Omnipaque were attached with Liquid Nails to the skin surface from head to toe. The cadaver wasplaced in a plywood box and surrounded with Alpha Cradle AC660 (Smithers Medical Products, Inc.)foaming agent to hold in a fixed position The data is supplemented by axial sections of the whole body obtained by CT axial sections of thehead and neck obtained by MRI and coronal sections of the rest of the body also obtained by magneticresonance imaging. The initial aim of the Visible Human Project® is to create a digital image dataset of completehuman male and female cadavers in MRI, CT and anatomical modes.The male dataset consists of axial MR images of the head and neck taken at 4 mm intervals andlongitudinal sections of the remainder of the body also at 4 mm intervals. The resolution of the MR imagesis 256 pixels by 256 pixels. Each pixel has 12 bits of grey tone.The CT data consists of axial CT scans of the entire body taken at 1 mm intervals at a resolution of 512pixels by 512 pixels where each pixel is made up of 12 bits of grey tone. The axial anatomical images are2048 pixels by 1216 pixels where each pixel is defined by 24 bits of color, each image consisting of about7.5 megabytes of data. The anatomical cross-sections are also at 1 mm intervals and coincide with the CTaxial images. There are 1871 cross-sections for each mode, CT and anatomy, obtained from the malecadaver.The dataset from the female cadaver has the same characteristics as the male cadaver with one exception.The axial anatomical images were obtained at 0.33 mm intervals instead of 1.0 mm intervals. This results inover 5,000 anatomical images. The female dataset is about 40 gigabytes in size. Spacing in the "Z"direction was reduced to 0.33 mm in order to match the pixel spacing in the "XY" plane which is 0.33 mm.This enables developers who are interested in three-dimensional reconstructions to work with cubic voxels.A recent addition to the male dataset is the inclusion of higher resolution anatomical images. 70mm filmtaken during the original data collection phase has been digitized at a resolution of 4096 pixels by 2700pixels where each pixel is made up of 24 bits of color. As with the original anatomical images, there are atotal of 1871 of these high resolution images. The scanning, slicing and photographing took place at the University of Colorado’s HealthSciences Center.
  6. 6. 6 e. THE DONORS:The male cadaver is from Joseph Paul Jernigan, a 38-year-old, 199 lb, 5 11, Texas murderer who wasexecuted by lethal injection on August 5, 1993. At the prompting of a prison chaplain he had agreed todonate his body for scientific research or medical use, without knowing about the Visible Human Project.Some people have voiced ethical concerns over this.The female donor, 59-year-old, height 5 5.5, remains anonymous. In the press she has been described asa Maryland housewife who died from a heart attack and whose husband requested that she be part of theproject. f. PROBLEMS WITH DATA SETS:Freezing caused the brain of the man to be slightly swollen and his inner ear ossicles were lost duringpreparation of slices. Nerves are hard to make out since they have almost the same color as fat. Small bloodvessels were collapsed by the freezing process. The male had only one testicle. The reproductive organs ofthe woman are not representative of those of a young woman. g. DISCOVERIES:By studying the data set, researchers at Columbia University found several errors in anatomy textbooks,related to the shape of a muscle in the pelvic region and the location of bladder and prostate. h. LICENSE:The data may be bought on tape or downloaded free of charge; one has to specify the intended use and signa license agreement that allows NLM to use and modify the resulting application. NLM can cancel theagreement at any time, at which point the user has to erase the data files. IIIT Allahabad obtained its licenseon 5th May 2004 from the Visible Human Project Administrator at National Library of Medicine, Bethesda,Maryland USA i. ABOUT 3D IMAGING:The purpose of 3D imaging is: given a set of multidimensional images pertaining to an object / objectsystem, to output qualitative/ quantitative information about the object/ object system under study. The term volume rendering is used to describe techniques which allow the visualization of three-dimensional data. Volume rendering is a technique for visualizing sampled functions of three spatialdimensions by computing 2-D projections of a colored semitransparent volume.Currently, the major application area of volume rendering is medical imaging, where volume data isavailable from X-ray Computer Tomography (CT) scanners, Positron Emission Tomography (PET)scanners, and MRI scanners. CT scanners produce three-dimensional stacks of parallel plane images, eachof which consist of an array of X-ray absorption coefficients. Typically X-ray CT images will have aresolution of 512 * 512 * 12 bits and there will be up to 50 slides in a stack. The slides are 1-5 mm thick,and are spaced 1-5 mm apart.In the two-dimensional domain, these slides can be viewed one at a time. The advantage of CT images overconventional X-ray images is that they only contain information from that one plane. A conventional X-rayimage, on the other hand, contains information from all the planes, and the result is an accumulation ofshadows that are a function of the density of the tissue, bone, organs, etc., anything that absorbs the X-rays.The availability of the stacks of parallel data produced by CT scanners prompted the development oftechniques for viewing the data as a three-dimensional field rather than as individual planes. This gave theimmediate advantage that the information could be viewed from any view point.
  7. 7. 7There are a number of different methods used: - Rendering voxels in binary partitioned space - Marching cubes - Ray castingThere are some problems with the first two of these methods. In the first method, choices are made for theentire voxel. This can produce a "blocky" image. It also has a lack of dynamic range in the computedsurface normals, which produces images wit relatively poor shading.The marching cubes approach solves this problem, but causes some others. Its biggest disadvantage is thatit requires that a binary decision be made on the position of the intermediate surface that is extracted andrendered. Also, extracting an intermediate structure can cause false positive (artifacts that do not exist) andfalse negatives (discarding small or poorly defined features). i. Ray Casting: In Ray casting, also known as backward mapping, a ray is fired from each pixel in the view plane,and information from all the voxels, in the volume data, intersecting the current ray or pixel is gathered .Thebasic goal of ray casting is to allow the best use of the three-dimensional data and not attempt to imposeany geometric structure on it. It solves one of the most important limitations of surface extractiontechniques, namely the way in which they display a projection of a thin shell in the acquisition space.Surface extraction techniques fail to take into account that, particularly in medical imaging, data mayoriginate from fluid and other materials which may be partially transparent and should be modeled as such.Ray casting doesnt suffer from this limitation. Fig. Illustration of RAY CASTING
  8. 8. 8Ray-casting sensation began with the release of a game, Wolfenstein 3D (iD Software), in 1992. InWolfenstein 3D, the player is placed on a three dimensional maze-like environment, where he/she mustfind an exit while battling multiple opponents. Wolfenstein 3D becomes an instant classic for its fast andsmooth animation. What enables this kind of animation is an innovative approach to three dimensionalrendering known as "ray-casting."When it pertains to volume visualization, ray casting is often referred to as ray tracing. Strictly speakingthis is inaccurate, since the method of ray tracing with which most people are familiar is more complexthan ray casting. However, the basic ideas of ray casting are identical to those of ray tracing, and the resultsare very similar.Algorithms which implement the general ray casting technique described above involve a simplification ofthe integral which computes the intensity of the light arriving at the eye. The method by which this is doneis called "additive re-projection." It essentially projects voxels along a certain viewing direction. Intensitiesof voxels along parallel viewing rays are projected to provide intensity in the viewing plane. Voxels of aspecified depth can be assigned a maximum opacity, so that the depth that the volume is visualized to canbe controlled. This provides several useful features: • The volume can be visualized from any direction. • Hidden surface removal can be implemented so that, for example, front ribs can obscure back ribs. • Color can be used to enhance interpretation.Additive re-projection uses a lighting model which is a combination of reflected and transmitted light fromthe voxel. All of the approaches are a subset of the model in the following figure.In the above figure, the outgoing light is made up of: • light reflected in the view direction from the light source • incoming light filtered by the voxel • any light emitted by the voxel
  9. 9. 9Several different algorithms for ray casting exist. The one presented by Marc Levoy in many of his papers[3], is used in several commercial applications. I performed ‘Early Ray Termination’ implementation of thealgorithm. ii. Disadvantages: - Ray Casting is a computationally intensive process. - Although, ray casting is a fairly direct algorithm to implement, it does possess the significant disadvantage that random access to all of the dataset is required for each ray - the implication being that significant memory overheads are required.iii. Possible Improvements: Ray Casting is a computationally intensive process. The fastest implementations of this methodare achieved by combining several common computer graphics techniques, like early ray termination,octree decomposition and adaptive sampling [3] [7]. Early ray termination is a technique that can be used ifthe rays are traversed front-to-back. It simply ends the ray traversal after the accumulated color for that rayis above a certain threshold. Octree decomposition is a hierarchical spatial enumeration technique thatpermits fast traversal of empty space, thus saving substantial time in traversing the volume and calculatingtrilinear interpolations. Adaptive sampling tries to minimize work by taking advantage of the homogeneousparts of the volume, for each square in the image, one traverses the rays going out of the vertices of thebounding box and recursively goes down repartitioning this square into smaller ones if the difference in theimage pixel value is larger than a threshold. j. SOURCES OF IMAGES: There are several sources of digital multidimensional images in medical imaging: 2D: a digital radiograph, a tomographic slice from a data set from CT, MRI, PET, SPECT, Ultra- Sound, fMRI, magnet source imaging, and surface light scanning. 3D: time sequence radiographic images or tomographic slice images of a dynamic object, a volume tomographic slice image of a static object. 4D: time sequence of tomographic volume images of a dynamic object 5D: time sequence of tomographic volume images of a dynamic object for each of a range of values of an imaging parameter. k. BASICS AND TERMINOLOGY: Object: The original physical object under study. This may be a particular organ in the body, a human made device, a pathological entity etc. Object System: A collection of objects. For example the brain is an object system made up of object like – the white matter, the gray matter the Cerebrospinal fluid (CSF). Body Region: A finite region of 3D space within which the object system of study is embedded. Imaging Device: A device used that produces a digital image of the body region with its object contents E.g. CT scanners, MRI devices Pixel, Voxel: In a digital form the body region is virtually partitioned into small abutted cubical volume elements and the imaging device estimated an aggregate of property if the material within each such element. The volume element is called VOXEL. The 2D analog of voxel is a pixel (picture element).
  10. 10. 10 Fig. Illustration of VOXEL cube and Slice Thicknessl. OBJECT CHARACTERISTICS IN IMAGES: i. GRADED COMPOSITION: Object in any body region have heterogeneous composition. Also, the imaging device blurs object information captured in scene due to various approximations. Thus even if an object is compose of perfectly homogeneous material, the scene of its body region will invariably display a heterogeneous intensity. This property is called graded composition of object information in scenes. ii. HANGING-TOGETHERNESS (Gestalt): In spite of graded composition, humans usually perceive objects as distinct in displays. The fact that the voxels in some body region have identical intensity values which are similar to intensity values of the voxels in other objects in vicinity, does not interfere with human ability to perform a mental grouping of voxels into objects. This property is called as hanging- togetherness or Gestalt.m. PREPROCESSING:The purpose of preprocessing of medical images is to facilitate identification of objects, removal ofnoises, and restoration of lost information, convert images to suitable format to work upon.
  11. 11. 11 The preprocessing used commonly can be grouped as following operations: i. Volume of interest: Converts a given scene to another scene by reducing the size of the scene domain and/or the intensity range for the purpose of minimizing the storage space. ii. Filtering: Converts a given scene to another scene by suppressing unwanted information and/or enhancing wanted information.iii. Interpolation: Convert a given scene to another scene of a specified level discretization. Medical imaging systems collect data typically in a slice-by-slice manner. Usually, the pixel size of the scene within a slice is different from spacing between adjacent slices. In addition, often the spacing between slices may not be the same for all slices. For visualization, manipulation and analysis of such anisotropic data, they often need to be converted into data of isotropic discretization or of desired level of discrete level of discretization in any of the three (or higher) dimensions. Interpolation techniques can be divided into two categories: scene based and object based. In scene- based methods, interpolated scene intensity values are determined directly from the intensity values of the given scene. In object-based methods, some object information extracted from the given scene is used in guiding the interpolation process. Interpolation is the process by which we estimate an image value at a location in between image pixels. For example, if you resize an image so it contains more pixels than it did originally, the software obtains values for the additional pixels through interpolation. The imresize and imrotate geometric functions use two-dimensional interpolation as part of the operations they perform. Nearest neighbor interpolation Bilinear interpolation Bicubic interpolation The interpolation methods all work in a fundamentally similar way. In each case, to determine the value for an interpolated pixel, you find the point in the input image that the output pixel corresponds to. You then assign a value to the output pixel by computing a weighted average of some set of pixels in the vicinity of the point. The weightings are based on the distance each pixel is from the point. The methods differ in the set of pixels that are considered. For nearest neighbor interpolation, the output pixel is assigned the value of the pixel that the point falls within. No other pixels are considered. For bilinear interpolation, the output pixel value is a weighted average of pixels in the nearest 2-by-2 neighborhood. For bicubic interpolation, the output pixel value is a weighted average of pixels in the nearest 4-by-4 neighborhood. The number of pixels considered affects the complexity of the computation. Therefore the bilinear method takes longer than nearest neighbor interpolation, and the bicubic method takes longer than bilinear. However, the greater the number of pixels considered, the more accurate the computation is, so there is a trade-off between processing time and quality.iv. Registration: Convert a given scene to another scene of by matching it with another given scene to combine information about the same body region from multiple sources. v. Segmentation: Converts a given set of scenes to a structure/ structure system.
  12. 12. 12 III. METHODOLOGYThe entire process carried out under this project can be illustrated by following steps undertaken: a. STEPS: 1. Obtain the data set 2. Extract the compressed files 3. Convert the data files in RAW format to suitable image format (.PNG) 4. Crop, shrink or expand the images to fit onto one another precisely. 5. Perform threshold and segmentation to retain the object of interest and remove unwanted surrounding regions and background. 6. Interpolate to obtain the missing slices. 7. Perform Ray-Casting on the slices to obtain 3D surface visualization of the objects present in the slices. 8. Apply false coloring for better visual perception.The details of the above steps are discussed in the pages that follow.
  13. 13. 13 b. Steps 1, 2 and 3: The data set from NLM server was downloaded via ftp over a period of 50 days. The data includes about 40 GB of Female CT, MRI and Cryo-Section Images and 15 GB of male CT, MRI and Cryo- Sections. In addition there are some more data under the folder BWH_Harvard. Recently a 70mm male cryosection data set has been released, which too has been downloaded under male folder. i. Data Format: The format of the data set used for generating the images is shown below: Cryo-Sections Cryo-section Cryo-section CT MRI 70mm Color ( Female ) ( Male ) (Male/Female) (Male/Female) (Male) Header 0 0 0 3416 7900 Width 2048 2048 4096 512 256 Height 1216 1216 2700 512 256 Channels 3 3 3 1 1 Interlaced NO NO YES ------------ ------------ Bit Depth 8 bits 8 bits 8 bits 16 bits 16 bits Byte Order IBM PC IBM PC IBM PC Mac Mac Serial No. 1001 to 2730 1001 to 2878 1001 to 2878 1001 to 2734 1014 to 7584 Pixel Size 0.33 mm 0.33 mm 0.144 mm 0.489 mm 0.9375 mm ( mm/pixel)Slice Thickness 0.33 mm 1.00 mm 1.00 mm 0.50 mm 3.00 mm Inter-slice 0 0 0 0 0 Spacing Coordinate Offset in X,Y NONE,NONE NONE,NONE NONE,NONE NONE,NONE NONE,NONE ( in Pixels)
  14. 14. 14 The original data is in the form of .Z compression. When these compressed files were extracted,the outputs were the data set files in the .raw format. This again was not useful for either direct viewing orprocessing in MATLAB. So the raw files were converted to suitable image data format. I wrote scripts inMATLAB that performed the operation for entire data set automatically and stored the images on hard-diskin the same directory hierarchy as that of the original dataset.Script Name Purpose Reads .raw files from Male / Female Cryo-Sections (2048 x 1216)readfile_cryosections.m and converts them into .PNG files. Reads .rgb files from Male 70 mm Cryo-Sections (4096 x 2700) andreadfile_cryosections_70mm.m converts them into .PNG files. Reads .raw files from Male / Female CT (512 x 512) and covertsreadfile_ct.m them into .PNG files. 1 Reads .raw files from Male / Female MRI (256 x 256) and covertsreadfile_mri.m them into .PNG files. 21. The extracted files from MRI folder have extensions: ‘.pd’, ‘.t1’ and ‘.t2’. Before applying above scriptchange the ‘.raw’ in the script file to appropriate extension.2. The extracted files from CT folder have extensions: ‘.fre’, ‘.fro’. Before applying above script changethe ‘.raw’ in the script file to appropriate extension.I chose the ‘.PNG’ format for saving the final images. This was done after an analytical comparison of theproperties of different images formats. The study of which is as follows: 1. While ‘.JPG’ and ‘.GIF’ formats resulted in good degree of compression, they resulted in loss of information evidently visible as blurring at high level resolution. 2. ‘.TIFF’ and ‘.BMP’ maintained good degree of image details. But the problem was compression ratio. The size of the output images in this format was almost same as that of the original ‘.RAW’ File. 3. The ‘.PNG’ appeared most suited one as it compresses the file by a factor of 2 as well as maintains the image quality comparable to that of ‘.TIFF’ or ‘.BMP’.To compare the space requirements of different Storage Formats, the sizes of different files are given belowfor a sample file ‘avf1013a.raw.Z’ and its converted images: File name Size in KB s avf1013a.raw.Z 3943 avf1013a.raw 7296 avf1013a.raw.JPG 169 avf1013a.raw.TIFF 7319 avf1013a.raw.BMP 7297 avf1013a.raw.PNG 2988
  15. 15. 15To appreciate the variations in images quality, shown below are the sampled zoomed screenshots fromdifferent formats and the sizes of entire image relative to the original ‘.Z’ compress file:Fig 1c ‘.TIFF’ format SIZE Ratio: 1.85 Fig. 1b ‘.BMP’ format SIZE Ratio: 1.85Fig. 1a ‘.JPG’ format SIZE Ratio: 0.043 Fig. 1d ‘.PNG’ format SIZE Ratio: 0.76Fig. 1f ‘.JPG’ format SIZE Ratio: 0.699 Fig. 1e ‘.PNG’ format SIZE Ratio: 0.033Fig.1. The Zoomed Sections from different image formats. Note the blurred vessels in the JPG format (marked with arrows). Also note that TIFF is sharpest image but PNG is slightly poor in sharpness but much smaller in size. (The SIZE Ratio is ratio of size of the file to the ratio of the original compressed file in ‘.Z’ or ‘.gz’ format.)
  16. 16. 16 ii. The Directory Structure of the Data Set Repository build up at IIIT Medical Imaging Lab is depicted below: IIIT Medical Imaging Data Set (courtesy: vhnet.nlm.nih.gov) Female Male BWH_Harvard Utils Fullcolor Fullcolor cryo Head Head MRI_CT_DICOMM Thorax MRI_CT_tiffs Thorax Abdomen Abdomen Pelvis Pelvis Thighs Thighs Legs Legs Fullbody Fullbody Radiological Radiological MRI MRI Frozen_CTNormal_CT f Normal_CT 70 mm Fullbody y
  17. 17. 17c. Steps 4 and 5:The raw data from the ‘National Library of Medicine’ is converted to standard image formats. Theimages include CT scan slices and anatomical images of male and female cadavers. Then an interactivemethod is used to determine appropriate thresholds for different objects in the images. Now, based onthese thresholds, ray-casting generates the surface views of the objects in slices. Threshold determination [1], [5]: Since medical images are best judged by human eyes, weprovide the user to judge and decide the threshold himself. The user selects sample regions in theimages that correspond to his object of interest. Now based on the pixel value ranges in the samples, athreshold is decided and operated upon the image. Based on segmentation results, the user canresample the pixels until sufficiently good segmentation is obtained. The values of threshold arerecorded and operated on the rest of images.d. Steps 6 and 7: Ray-Casting [2],[3],[4],[5]: First, the images are rotated to correspond to the requested viewingdirection. Then a viewing plane is determined, which involves determining its orientation, distance,shape and dimensions.Now rendering by scene based method consists following three basic steps: a) Projection. b) Hidden part removal. c) Shading.These are needed to impart a sense of three dimensionality to the rendered image. Additional cues forthree dimensionality may be provided by techniques such as stereoscopic display, motion parallax byrotation of objects, shadowing, and texture mapping.If ray casting is the method of projection chosen, then points equally spaced along the ray are sampledand at each such point, the scene intensity is estimated using appropriate interpolation method. Hiddenpart removal is then done by stopping at the first sampled point encountered along each ray thatsatisfies the threshold criterion. The value of shading assigned to the pixel corresponding to the ray isdetermined by different methods.We determine the pixels on the projection plane. To start with, all these pixels are assigned abackground value of zero. Next, to obtain the rendered value for each pixel on the viewing plane, a rayis assumed to start from that pixel and is traced into the volume data till either a volume elementsatisfying the constraints is encountered or the opposite boundary is reached. The front most volumeelement satisfying the threshold is recorded on to the pixel on the viewing plane.This serves the first two steps mentioned above for surface rendering. First, based on the voxelencountered the pixel value and position on the projection plane is determined. Second, by stopping atthe first encountered voxel all the background voxels are ignored, which serves as a hidden surfaceremoval.This ends the ray trace for one pixel. The same process is repeated for the all the other pixels on theviewing plane.Now to adjust for depth perception, the distance of the selected volume element from the viewing planeis used. Here we have used a weighted combination of the selected volume element value and itsdistance from the viewing plane. This provides for depth perception while maintaining the localvariations in the value of the pixels on the object.
  18. 18. 18 The formula used to determine output pixel value is: new(x, y) = light_factor* light_factor* light_factor*(1-ratio) + pixel_factor* pixel_factor* pixel_factor*ratio. Here, new(x, y) = resultant pixel value on the output plane. light_factor = scan depth normalized to 0 to 1. pixel_factor = pixel value at scan point normalizedto to 0 to 1. ratio = determines relative contribution of depth and pixel value factors.Here, to obtain the final output value, the light_factor and pixel_factor cubed before multiplying withtheir ratios. The cubing operations allows for better depth perception as it increases the rate of variationof pixel intensity as depth increases.The above procedure is applied to all the slices in the stack. Each slice contributes one output line, thewidth of which is kept proportional to the contributing slice-thickness.The lines corresponding to missing intermediate slices are interpolated from adjacent slice scans.To incorporate the volume cut views, we used masks to determine which all volume elements shouldnot contribute to the final rendering. The cuts can be obtained along two intersecting planes passingthrough the object volume.e. Steps 8:To further assist output appreciation we have added false coloring to the rendered outputs. The bonesare represented in ‘chrome’ color while the intersection points of the object volume and the cuttingplanes are shown with ‘green’.Ray Casting generates very accurate and precise surface renderings with no false informationgeneration or loss of small features. In spite of the good feature the one major drawback that doesn’tallow real time renderings is that Ray Casting is computationally very expensive for the followingreasons: 1. The need to sample point along the ray, 2. The need to interpolate, 3. The need to store the entire scene in memory.
  19. 19. 19 IV. RESULTS We have provided options to vary the values of different processing parameter, the effects of which are evident in the images below. Fig 1 to 3 depict the effect of varying ‘ratio’ values: Fig. 1. Generated 3d surface of Female Skull. ratio = 0.01, number of slices used = 200, viewing angle = 45 Fig. 2. Generated 3d surface of Female Skull. ratio = 0.3, number of slices used = 200, viewing angle = 45 Note the aberrations introduced due to increased contribution from original pixel values as value of ‘ratio’ increases. Fig. 3. Generated 3d surface of Female Skull. ratio = 0.7, number of slices used = 200, viewing angle = 45Note the further aberrations introduced due to increased contribution from original pixel values as value of ‘ratio’ increases further.
  20. 20. 20Fig. 4. Depicts views from different angles (other parameters remaining constant): Viewing angle = 0 degree viewing angle = 40 degree Viewing angle = 64 degree viewing angle = 98 degree Viewing angle = 125 degree viewing angle = 180 degree viewing angle = 223 degree viewing angle = 250 degree viewing angle = 317 degree viewing angle = 352 degree Fig. 4. Generated 3d surface of Female Skull. ratio = 0.01, number of slices used = 223
  21. 21. 21Fig. 5 to 8 are the results depicting the performance of the program to cut out a view from the whole object as the depth of cutting plane and its direction varies: Fig. 5. Generated 3d surface of Female Skull. Fig. 8. Generated 3d surface of Female Skull. Cut angle = 180, cut depth = 200, viewing angle = 210 Cut angle = 260, cut depth = 137, viewing angle = 280 Fig. 6. Generated 3d surface of Female Skull. Fig. 9. Generated 3d surface of Female Skull. Cut angle = 170, cut depth = 230, viewing angle = 180 Cut angle = 0, cut depth = 20, viewing angle = 137 Fig. 7. Generated 3d surface of Female Skull. Cut angle = 0, cut depth = 176, viewing angle = 135 Fig. 10. Generated 3d surface of Female Skull. Cut angle = 290, cut depth = 135, viewing angle = 310
  22. 22. 22 Volume Cut Views of axial Images: Fig. 11 sagittal view from axial CT slices.Slices used = 200, Cut Depth = 220, Time = 99.516 s. Fig. 12. sagittal view from axial Cryo-Sections.Slices used = 777, Cut Depth = 966, Time = 1179.75 s.
  23. 23. 23 Fig. 13 Cornal view from axial Cryo-Sections. Slices used = 777, Cut Depth = 594, Time = 1170.86 s. Fig. 14 View at 45 degree from axial Cryo-Sections.Slices used = 777, Cut Depth = 1148, Time = 9974.344 s.
  24. 24. 24 Surface Visualization from Cryo-Sections: Fig. 15. Skin Surface view from Female Cryo-SectionsRatio = 0.30, Viewing Angle = 45 degree, Slices used = 790, Time = 3755.985 Fig. 16. Skin Surface view from Female Cryo-Sections
  25. 25. 25 Ratio = 0.60, Viewing Angle = 45 degree, Slices used = 800, Time = 3665.781 Fig. 16. Skin Surface view from Female Cryo-SectionsRatio = 0.40, Viewing Angle = 0 degree, Slices used = 1747, Time = 15101.406
  26. 26. 26 Noise Removal, Image Resizing and Cropping: Fig 17. Original ImageFig 18. Image after background Removal and cropping.
  27. 27. 27Variations of RGB and Gray Values in the Images: Fig 19 RGB profile in Original Image.
  28. 28. 28 Fig 20 RGB profile in Original Image.Fig 21 Gray Value profile in CT Image.Fig 22 Gray Value profile in MRI Image.
  29. 29. 29 V. DISCUSSIONThe method generates quite good quality 3-dimensioanl views of the surfaces of the objects. Thegray scale views can be color coded to provide natural perception. But the scanning process isquite time consuming. So it can not be used to generate views in real time. To test the reliability ofthe method we generated surface views of the skull from 0 to 360 degree at 1 degree interval. Theoutput images are saved as different files. These can be used to provide interactive viewing of theskull from any viewing direction. The volume cut operation and subsequent rendering providegood views of the object features which had been hidden behind outer surfaces. The surfacevisualization is applicable to color cryo-sections. The surface views of the skin are shown. TheSagittal and coronal slices were generated from axial data. But the segmentation of color imagesposed a real problem. In the Beginning it appeared to a simple task based on edge-detection andcolor component based threshold. In fact color component based threshold has been successfullyused to remove the Gelatin background from the cryo-sections, in spite of the variations inbackground color, shade and texture within the image. But Segmenting out the body organs is areal mess. The color values as well as boundaries of the organs are so much indistinguishable thatat a few instances even human eye is not able to judge out the organs from each other. The taskappears mammoth for a fully automatic method. It appears that faithful segmentation of organs ororgan system of interest is not possible with automatic methods and human assistance to a tediouslevel is apparent. Methods for human – computer interactions and fuzzy connected segmentationmay be developed to try out better segmentation results. VI. CONCLUSIONI have generated true and precise surface views from CT slice data. Surface views of HumanCadaver form cryo-sections had also been achieved. Other than these I have also been ale togenerate sagittal, coronal slices form axial CT, MRI, Cryo-sections slices. The Axial Slices data isalso used to generate volume cut views at any depth and from any viewing angle. Limitation isthat all the views are parallel projections of the volume.The results generated are exceptionally well in quality. Though the process is computationallyexpensive but the accuracy of the results supersedes this drawback. But there is a scope forimproving the computational efficiency.The surface views generated corresponding to different viewing angles’ and ’volume cut depths’can provide useful resource for studying shapes and size deformities of the organs in the body.
  30. 30. 30 VII. TIME LINE:November – December: - Implemented basic operations like Gamma Correction, Noise Removal, Histogram Equalization, FFT. - Work on building an interface for preprocessing the images. - Inter-Slice Interpolation implemented.January - February: - Collect resources, study and explore about different 3D visualization methods, their utility in case of medical images. - Learn from similar projects at other universities – their experiences and difficulties. - Search for suitable data source. Finally found the repository at NLM.March: - 5th March 2004, applied for license to use data set from VHP at NLM. - 10th March 2004, began data download from the NLM ftp server. - Trying Marching Cube and Surface Rendering Algorithm but had to dropped them later - Switched to Ray-CasingApril 2nd – 15th - Extracted compressed data set files.. - Converted raw files to .PNG format. - Segmented CT slices to obtain skull. - Obtained image intensity value profiles along a line through the middle of the CT, MRI and Cryo-section images. - Finished with implementation of Ray Casting.April 12th – 30th - Tested implementation on test images. - Applied the method to real CT slice data set. - Obtained first complete skull 3D surface after 400s. - Improved upon the implementation to bring down the rendering time to 220s. - Applied method to incorporate effect of pixel values variations and the variations in lighting due to depth into the rendered surface. - Extended method to color anatomical cryo-sections. - Completed Male and Female Cadaver Data set download over a period of 50 days at about 30 KBps average download speed. VIII. ACKNOWLEDGMENT
  31. 31. 31 I would like express my gratitude towards my project guide Dr. UmaShanker Tiwary for his invaluable guidance and support. I would also like to acknowledge the support and help that my fellow students and the people at the institute extended to me. I worked as a junior project assistant to the MHRD funded project titled as ‘Fusion of Multi-Modal Medical Imaging Data’ and the job assigned to him was ‘3-Dimensional Surface Visualization of Anatomical CT/MRI Slice Data’ We owe the data courtesy to the ‘Visible Human Project’ [6] group at the National Library of Medicine, Bethesda, MD 20894, USA. IX. REFERENCES [1] Thomas Schiermann, Ulf Tiede, Karl Heinz Hohne, “Segmentation of the Visible Human for High Quality Volume based Visualization” in Medical Image Analysis , Vol.1, No. 4, pp. 263-271, 1997. [2] Levoy, M., “Efficient Ray Tracing of Volume Rendering” ACM Transaction on Graphics 1990:9(3):245-261. [3] Levoy M., “Displays of Surfaces from Volume Data,” in IEEE Computer Graphics and Applications 1988; (3): 29- 37. Trans. Biomed. Eng., vol. 49, no. 10, pp. 1204–1210, Oct. 2002. [4] Tiede, U., Hohne, K., Bomans, M., Pommert, A., Riemer, M. Wiebecke, G.,” Investigation of Medical 3D- Rendering Algorithms,” IEEE Computer Graphics and Applications, 1990; 10(2), 41-53. [5] Jayaram K. Udupa, Gabor T.Herman.,” 3D Imaging in Medicine” CRC Press, Inc Boca Raton, FL, USA, Year 2000 ISBN: 0-8493-4294-5; [6] The National Library of Medicines Visible Human Project.,” www.nlm.nih.gov/research/visible/visible_human.html”. [7] Levoy M., “Efficient Ray Tracing Alogrothm” ACM Transations on Graphics, pages 245-261, July 1990 X. RESOURCESBOOKS: 1) 3D Imaging in Medicine Edited by: Jayaram K. Udupa, Ph.D. University of Pennsylvania Philadelphia, PA Gabor T. Herman, Ph.D. University of Pennsylvania Philadelphia, PA 2) Computer Graphics: Principles and Practice By: James D. Foley, Andries van Dam, Steven K. Feiner, John F. HughesWEB: - The National Library of Medicines Visible Human Project. www.nlm.nih.gov/research/visible/visible_human.html
  32. 32. 32 - IEEE Xplore. http://www.ieeexplore.ieee.org/Xplore/DynWel.jsp - GE Medical Systems. http://www.gemedicalsystems.com - Visualization Tool Kit resource page. - Image Processing Fundamentals http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/fip-Contents.html . XI. COURTESYResources Courtesy: Medical Image Processing Lab, Indian Institute of Information Technology, Allahabad.Data Set Courtesy: Visible Human Project® National Library of Medicine Building 38A, Room B1N-30 8600 Rockville Pike Bethesda, MD 20894
  33. 33. 32 - IEEE Xplore. http://www.ieeexplore.ieee.org/Xplore/DynWel.jsp - GE Medical Systems. http://www.gemedicalsystems.com - Visualization Tool Kit resource page. - Image Processing Fundamentals http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/fip-Contents.html . XI. COURTESYResources Courtesy: Medical Image Processing Lab, Indian Institute of Information Technology, Allahabad.Data Set Courtesy: Visible Human Project® National Library of Medicine Building 38A, Room B1N-30 8600 Rockville Pike Bethesda, MD 20894
  34. 34. 32 - IEEE Xplore. http://www.ieeexplore.ieee.org/Xplore/DynWel.jsp - GE Medical Systems. http://www.gemedicalsystems.com - Visualization Tool Kit resource page. - Image Processing Fundamentals http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/fip-Contents.html . XI. COURTESYResources Courtesy: Medical Image Processing Lab, Indian Institute of Information Technology, Allahabad.Data Set Courtesy: Visible Human Project® National Library of Medicine Building 38A, Room B1N-30 8600 Rockville Pike Bethesda, MD 20894

×