An Efficient Brain Tumor Detection Algorithm based on Segmentation for MRI Sy...
Thesis_BurkhardtK
1. Development of an in silico Methodfor Visualization
of Pelvic Fractures and Reconstruction
Kate Burkhardt
Dr. Ferris Pfeiffer
Department of Biological Engineering
University of Missouri
May 12, 2016
Abstract:
The goal of this project is to develop a computational method for segmentation
and manipulation of pelvic CT data. Multiple methods were explored so that
pelvic fractures could be visualized, and analyzed systematically. Current
methods for evaluation of complex pelvic fractures are insufficient and thus
obsolete. Therefore, computational in silico methods are required which can allow
for a more complete 3D visualization for patient anatomy. Computer programs
utilized in this project include Amira, Fortran, and 3D Slicer. The final product
hopes to enhance the recognition for the classification of acetabular fractures.
2. ii
Acknowledgements
I would like to express my appreciation to my cooperators Dr. Brett Crist,
Associate Professor of Orthopaedic Surgery at the University of Missouri Hospital
and Dr. Lauren Cook,OrthopaedicSurgeon at the University of MissouriHospital.
They were the inspiration and idea from which the project came.
I would also like to thank my mentor Dr. Ferris Pfeiffer, Assistant Professor and
Head of Biomechanics and Bioengineering Comparative Orthopaedic Laboratory
for graciously giving me the opportunity to take on this research project.
3. iii
Table of Contents
Cover and Abstract…………………………………………………….i
Acknowledgements…………………………….………………...……ii
Table of Contents………………………………...………………….…iii
1. Introduction…………………………………………………………….1
1.1 Judet and Letournel Classification………………...……….…..1-2
2. Methodology…………………………………………………………...3
2.1 Amira 3D Software…………………………………………....…3
2.1.1 Home User Interface…………………………………….3-4
2.1.2 Segmentation Editor…………………………………….4-6
2.1.3 Surface Generation……………………………………....6-7
2.1.4 Surface Editor…………………………………………....7
2.1.5 STL File Formatting …………………………………….7-8
2.2 Microsoft Visual Studio and Fortran……………….…………8-10
3. Results…………………………………………………………………10-12
4. Discussion……………………………………………………….…….12-14
5. Conclusion…………………………………………………………….14
6. References………………………………………………………..……15-16
4. 1
1. Introduction
Fractures of the acetabulum, or socket of the hip bone where the femur hits,
primarily occur in young adults as a result of high-energy trauma or low-energy
trauma in elderly. Many of these traumatic injuries are caused from car accidents
or falling from a significant height and can be associatedwith other life threatening
damage. The displaced fracture fragments are also dangerous as they lead to
abnormal pressure distribution and exponential breakdown of the articular
cartilage surface.
It wasn’t until the middle of the 1900’s that acetabulum fractures were surgically
treatable [1]. The operative outcomes at this time were still not ideal, which called
for more investigation. Around 1960, Judet and Letournel developed a
standardized imaging protocol to assist in acetabular fracture classification. This
took an agonizing amount of research because all injuries are different which
makes diagnosis difficult. Still, the Judet and Letournel radiographic landmark
and fracture classification is the most common system in use today [2].
Although there is a standard classificationsystem, conceptualizing2D imagesinto
a 3D model is still challenging. This project uses a computational in silico method
to segment and manipulate pelvic CT data. This allows for systematic analyzation
as well as a more complete 3D visualization of patient anatomy. Both CT data sets
and 3D models are utilized in an interactive program for image analysis and
scientific visualization.
1.1 The Judet and Letournel Classification of Acetabular Fractures
Classifying acetabular fractures is important so the type of fracture can be
categorized and communicated for patient treatment and research purposes.
There are other classification systems such as the Orthopedic Trauma Association
5. 2
scheme and the more recent Harris System. Despite these new systems, the 1964
Judet and Letournel classification method is still preferred and will also be the
method demonstrated in this project.
The Judet and Letournel classification system is a combination of anatomic and
radiographic pattern findings. Alton et al. addresses that, “Specific fracture
patterns emerged base on mechanism of injury, the vector of injury force
application, anatomy of the innominate bone, and its mechanical properties.” In
this system there are 10 fracture patterns, five elementary and five associated [3].
Figure 1 shows the break lines of each pattern. It should be noted these are generic
cases and real fractures tend to violate the classification system.
Figure 1. Judet and Letournel classification system of 10 acetabular fracture patters [4].
The simple patterns on the left, and more complex on the right
6. 3
2. Methodology
A search was conducted to identify patients treated at the Department of
Orthopaedic Surgery for acetabular fractures over the last ten years. X-ray and CT
scans were obtained from this patient database that best embodied the 10 fracture
patterns of Judet and Letournel. All patient information and identifiers were
removed from the images and distributed to the Bioengineering Department and
Dr. Ferris Pfeiffer for further analyzation.
2.1 Amira 3D Software for Life Sciences
Amira 3D was the initial software program used in this study. Amira is a
multifaceted software platform for visualizing, manipulating, and understanding
data from computed tomography, microscopy, MRI, and other imaging
techniques. Amira allowsfor advanced 3D imaging in research areas ranging from
molecular and cellular biology to neuroscience and bioengineering [5].
2.1.1 Amira: Home User Interface
Figure 2 shows the Amira graphical user interface with uploaded tutorial data.
The 3D viewer is where the main graphics can be seen in the image viewing space.
All interactions including rotation and manipulation take place here. The object
pool is used for the iconic representation of the data objects, compute modules,
display modules, and measurement modules. Lines connecting the module icons
represent dependencies among them, not data flow. These data dependencies
among models are considered their network. The property area is where the
details of the module and its parameters are displayed. These are essentially
‘settings’ that have default values,but can almost always be controlledby the user.
Finally the console help window is shared with the integrated hypertext help
7. 4
browser. Here, information and error messages are shown as an output of Amira
executing [6].
Each Ct dataset includes anywhere between 75 and 200 scans of the pelvis. In this
project the axial CT scans are used for segmentation in Amira, as it is the easiest
view for visualization. These scans are then imported into Amira via a DICOM
loader, which gives information about the number of files, thickness of the scan
slices,load index, etc. An AmiraMesh binary will represent this set of data and can
be located in the object pool.
2.1.2 Amira: Segmentation Editor
From the image stack dataset, another AmiraMesh binary file will be created that
holds the segmentation information for those particular images. This will be an
object type LabelField, which is a cubic grid with the same dimensions as the
underlying image volume. For each voxel, or array of elements of volume, the
Figure 2. Amira graphical user interface tutorial. (A) 3D viewer. (B) Object pool.
(C) Property area. (D) Console help.
(B)
(C)
(A)
(D)
8. 5
LabelField contains a label indicating the region to which the voxel belongs to [7].
The user can manually modify a LabelField using Amira’s graphical interface
Segmentation Editor seen in Figure 3.
The Segmentation Editor is a tool that allows interactive segmenting of 3D image
data. Image Segmentation is considered the process of dividing an image into
different sub-regions, such as different organs, tissue types, etc. This is done by
selecting the desired voxels and assigning them to a particular material. This
information isstored as a labelin the LabelFieldobject [7]. Segmenting isvery time
consuming, but Amira offers a number of tools to help aid this process.
Figure 3 is one example of the segmented pelvis CT scans used in this project. This
is just one of close to 100 slices that is segmented for this particular set. Each color
coordinates with a specific material, in this case different bone types or fragments
of those bones. On the left side of the viewer is the right femur encapsulated by
Figure 3. Amira Segmentation view. Coloring indicates material. Yellow: right femur.
Blue: left femur. Purple: pelvis. Green: posterior column fragment.
9. 6
the acetabular dome. On the bottom is a glimpse of the coccyx, which is a
triangular bone at the base of the spine. The right side of the viewer shows the left
femur cut at the mid-level joint. The green indicates one of the fragments that is
considered a posterior column fracture pattern. This segmentation process was
done for all 10 cases of the Judet and Letournel fracture patterns.
2.1.3 Amira: Surface Generation
After the process of segmentation is complete for the entire dataset, each material
is saved as its own separate AmiraMesh LabelField.Next, a toolcalled SurfaceGen
is invoked as a direct dependency to that particular LabelField. SurfaceGen is a
module that computes a triangular approximation of the interfaces between
different or same material types in a LabelField with either uniform or stacked
coordinates [8]. The SurfaceGen must be applied to start the surface extraction.
This then creates a dependency to a new default file called a HxSurface Binary (or
.surf), which is displayed as a surface view and can be seen in Figure 4.
Figure 4. Amira SurfaceGen of each independent LabelField. Yellow: right femur. Purple:
pelvis. Light blue: vertebrae of lumbar spine. Dark blue: left femur. Green: posterior column
fragment. Pink: posterior wall fragment.
10. 7
Figure 4 shows each separate AmiraMesh LabelField as a surface file: right femur,
left femur, pelvis, vertebrae of the lumbar spine, posterior column fragment, and
posterior wall fragment. Each material was uploaded and applied into a surface
view in the 3D viewer that was previously explained. This is the same fracture
pattern, posteriorcolumn posteriorwall, which is shown in the segmentation view
in Figure 3. The vertebrae and posterior wall fragment can now been seen in the
3D image.
2.1.4 Amira: Surface Editor
The surface files obtained can be very large, especially for substantial materials
such as the pelvis. Thus, the each material must be simplified in the Surface
Simplification Editor. This feature of the Property Editor is used to reduce the
number of triangles in an arbitrary surface model according to a user defined
value. For example, the pelvis will usually be made up of around 300,000-400,000
faces and will be reduced to 100,000 faces. If the fragment piece has more than
30,000 faces it will be reduced to 20,000 faces. Next, the surface is refined with the
image segmentation editor to smooth the data. This provides a better surface
appearance of the material. Finally, the connectivity is recomputed to rid any last
deformations of the material caused from the reduction of the faces [8].
2.1.5 Amira: STL File Formatting
All of these surface files must be saved, but unfortunately the HxSurface Binary
file is not completely universally compatible and mostly used in Amira. Instead
these surface files will be saved in a different format call an STL file. STL
technically stands for stereolithography, but has also acquired acronyms such as
‘standard triangle language’and ‘standard tessellationlanguage’. The STL file was
introduced to be a bridge from CAD to other software programs. It describes only
the surface geometry of a 3D object without any representation of color, texture,
11. 8
other common CAD model attributes. The STL file format is now widely used for
3D printing, modeling, computer-aided manufacturing and much more [9]. It is
imperative that each material (bone or fragment of bone) is saved as a STL file
because this format is necessary for manipulation.
2.2 Microsoft Visual Studio and Fortran
Visual Studio is an integrated development environment, or IDE, created by
Microsoft. It can be used to edit, build, run, and debug applications as well as
facilitate the development process in more ways than one [10]. Visual Studio can
also be used to build web applications, XML Web services, and mobile
applications and more. Visual Basic, Visual C#, and Visual C++ all use the same
integrated development environment, which enables tool sharing and eases the
creation of mixed-language solutions [11]. In this project Microsoft Visual Studio
is used to compile Fortran programmed files.
Fortran is one of the oldest programming languages and was first published in
1957. The name Fortran comes from the acronym “formula translation” because it
was designed to allow easy translation of math formulas into code. Its objective
was to create a programming language that would be simple to learn, suitable for
a wide variety of applications, machine independent, and would allow complex
mathematical expressions to be stated similarly to regular algebraic notation.
Fortran was very innovative for its time because of its compiler that transforms
source code written in a programming language into another computer target
language [12].
For this project, mentor Dr. FerrisPfeiffer wrote the Fortran programming file.The
file is the basis of how the pelvis fragments are manipulated in 3D and
reconstructed to mock where they would be positioned after surgery. Although,
the results may seem simple, it took over 300 lines of Fortran code to achieve
12. 9
manual fragment manipulation as well as a new origin for such manipulation. A
small amount of the Fortran programming code can be seen in Figure 5. This is
considered the source file and is grouped into a project that contains all the
information and sources needed to compile, such as all of the STL files. Selecting
“BuildSolution” from the VisualStudio’sBuildmenu will create an executablefile
that will run, compile, and debug [10].
To overview the Fortran program, first variables are declared and a STL file that
was saved from Amira is opened and read. Only one file is able to be opened and
thus manipulated at a time. Next, four points are taken from around the right
acetabular dome and four points are taken from around the left acetabular dome.
The pointscan been found in the Amira Segmentation Editor viewer seenin Figure
3. Note that the points are in 3D space and therefore have an x, y, and z coordinate.
Following, the center points for the left and right acetabulum are then both
calculated. The center of the acetabulum can be visualized as where the center of
Figure 5. Fortran transformation code file for single fragment manipulation seen
in Microsoft Visual Studio compiler.
13. 10
the head of the femur is. The calculation process is therefor similar to finding the
center of a sphere. The calculation steps include: constructing a vector of constant
values from the four input points, constructing a matrix using the four input
points, computing the determinants of sub matrices of the matrix, computing the
determinant then the inverse of the matrix, and finally determining the center of
the acetabulum. Next, a calculation is made to translate the origin and place it in
between the left and right acetabulum center.
This new origin is very important because it keeps a consistent basis for
manipulating the acetabular fractures. The right femur, left femur, pelvis, and
vertebrae will all stay at this new transformed ‘zero’ axial position. On the other
hand, the fragments willbe manuallymovedin the Fortran file.The userwill input
a desired value for the translated x, y, z axis and rotation positions. The rotation
function is determined from the middle of the two centers of the left and right
acetabulum. These new values for the manipulated fragments are written into the
Fortran files and are compiledas the translated STL file. Once again these files will
be opened in Amira to observe the manual 3D manipulation and means of proper
reconstruction.
3. Results
The results are easy to see in Amira, as they are the 3D models at their original
position and acetabular fragment reconstructed position. Two examples of
completedcases can be seen in Figure 6 and Figure 7, which show both the original
and reconstructed acetabular fracture (the femurs and vertebrae were left out to
simplify examination). The results acquired were sufficient enough for the means
of this project, but they were not ideal. Amira is an excellent medical imaging
program, but could not accomplish the ‘drag and drop’ mechanism that was
desired to use to reconstruct the acetabular fractures. Unfortunately, that kind of
14. 11
program was out of the realm of knowledge, funding, and time frame for this
project. Reconstructing the acetabular fractures in Fortran and Microsoft Visual
Studio was also not ideal. The manual input to transform the fragment piece was
a guess and check process. Every time the user would make a manipulation in the
Fortran file it would have to be opened in Amira to see the 3D reconstruction. This
method was very time consuming and tedious.
Initially it was expected that a new computer program for Fortran would be
written and developed. This new algorithm would have been semi-automated
instead of completely manual and thus more user friendly. It was anticipated that
this would be accomplished by prompting the user to input multiple coincident
points on the pelvis and fracture. This would have been much easier for the user
since points can be pulled right off the Segmentation Editor screen in Amira. Once
again, time constraints did not allow for this to be completed. Regardless, the
methods used were still successful for the needs and expectations of this project.
Figure 6. Anterior column fracture pattern show as 3D models in Amira software.
Left: original. Right: reconstructed.
15. 12
4. Discussion
This project will further be used under Dr. Brett Crist and Dr. Lauren Cook at the
Department of Orthopaedic Surgery and University of Missouri Hospital. It is
intended to be used by residents to better recognize the 10 Judet and Letournel
acetabulum fracture patterns. Amira was too expensive and out of the budget to
distribute to all the residents, so a different program called 3D Slicer was utilized.
Slicer is a free, open source software package for visualization and image analysis.
Some of the notable Slicer highlights include: Robust DICOM capabilities,
interactive segmentation, volume rendering, 4D image viewer, and flexible
layouts and sliver viewers [13]. It is particularly useful because it allows the
upload of both STL files and DICOM image files. Figure 8 shows the 3D slicer
interface. The top right corner shows the STL files created in Amira and Fortran
and is very similar to the 3D viewing window in Amira. The user will use the
menu on the left and be able to click on and off the visualization of the fragments.
This way they will be able to see the original and reconstructed views at different
Figure 7. Transverse posterior column fracture pattern shown as 3D models in Amira
software. Left: original. Right: reconstructed.
16. 13
times. In the other 3 boxes are the axial, sagittal, and coronal views from the CT
scans for that particular case. Each CT data set is scrollable, so the user can look
through the CT scans at their own pace and recognize fracture pattern lines.
The goal is for the residents to be able to analyze the 3 views of the CT images and
be able to visualize the pelvis in 3D and how to reconstruct it. The hope is that this
interactive program will improveretention and reorganization for the acetabulum
fracture patterns. For ease of the residents,the program will bedistributedon flash
drives. A PowerPoint presentation will preview and introduce the acetabular
fracture patterns. The user will then be able to click on a hyperlink and 3D slicer
will open to the exact screen as seen in Figure 8.
Before even using the software, an initial test will be given to address current
ability to identify fracture patterns and potential surgical approaches correctly.
After this, residents will be allowed to use the software for one month. A second
test will then take place at the end of that month. At this point, the residents and
Figure 9. Anterior column fracture pattern show as 3D models in Amira software.
Left: original. Right: reconstructed.
17. 14
fellows will be asked to avoid studying or reading about acetabular fractures for
one month, unless needed to prepare for a case. A final test will then be
administered to help access long term retention of information.
5. Conclusion
Although the software used to build the 3D original and reconstructed models is
difficult to use and time consuming, it was still sufficient to create a learning tool
for this project. At this point the program has not yet been introduced to the
residents at the University of Missouri, but it is expected to be by the end of May.
In July after the study, there will be an evaluation of overall improvement in the
percentage of fractures and approaches correctly identified, as well as intra- and
inter- observer variability. This program will be reevaluated and the results will
be published in a medical journal.
18. 15
6. References
[1] Thacker, MihiM. "AcetabulumFractures." MedScape.WebMD LLC,14 Nov. 2014.
Web. 19 Apr. 2016. <http://emedicine.medscape.com/article/1246057-
overview#a5>.
[2] Scheinfeld, Meir H., Michael Spektor, Laura L. Avery, Joshua Dym, and Derek F.
Amanatullah. "Acetabular Fractures: What Radiologists Should Know and How 3D
CT Can Aid Classification." RadioGraphics:. Radiological Society of North America,
Mar.-Apr. 2015. Web. 19 Apr. 2016. <http://pubs.rsna.org
/doi/full/10.1148/rg.352140098>.
[3] Alton,TimothyB., andAlbert O.Gee. "ClassificationsinBrief:LetournelClassification
for Acetabular Fractures." National Center for Biotechnology Information. U.S. National
Library of Medicine, 9 Nov. 2013. Web. 21 Apr. 2016.
<http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3889427/>.
[4] Pagenkopf, Eric, Andrew Grose, George Patel, and David L. Helfet. "Acetabular
Fractures in the Elderly: Treatment Recommendations." ResearchGate. Hospital for
Special Surgery, 12 July 2006. Web. 21 Apr. 2016.
<https://www.researchgate.net/publication/23218476_Acetabular_Fractures_in_th
e_Elderly_Treatment_Recommendations>.
[5] Amira 3D Software for Life Sciences. Computer software. FEI. FEI, 2016. Web. 21 Apr.
2016. <http://www.fei.com/software/amira-3d-for-life-sciences/>.
[6] "Amira - Hands on Basic Tutorial." Stanford School of Medicine (n.d.): n. pag. Web. 10
Apr. 2016.
[7] Weber. "Overview of the Segment Editor." N.p., n.d. Web. 22 Apr. 2016.
<http://ftp.tuebingen.mpg.de/pub/kyb/bweber/zib/share/doc/hxgi/GIOvervie
w.html#A1>.
[8] "Amira Reference Guide." University of Delaware. Visage Imaging, 2009. Web. 3 Apr.
2016. <http://www2.udel.edu/ctcr/sites/udel.edu.ctcr/files/Amira%20
Reference%20Guide.pdf>.
[9] "STL (file Format)." Wikipedia. Wikimedia Foundation, 2016. Web. 22 Apr. 2016.
<https://en.wikipedia.org/wiki/STL_(file_format)>.
19. 16
[10] "Getting Started with Visual Studio Fortran." Getting Started with Visual Studio
Fortran. Lahey Computer Systems, 2016. Web. 22 Apr. 2016.
<http://www.lahey.com/docs/lgf12help/LFGetStartVS.htm>.
[11] "Introducing Visual Studio." Developer Network. Microsoft, 2016. Web. 22 Apr.
2016. <https://msdn.microsoft.com/en-us/library/fx6bk1f4(v=vs.90).aspx>.
[12] "The FORTRAN Programming Language." The Language Guide. N.p., 1999. Web.
22 Apr. 2016. <http://groups.engin.umd.umich.edu/CIS/course.des/cis400/
fortran/fortran.html>.
[13] "Introduction." SlicerWebRSS. BWH & 3D Slicer Contributors, 2016. Web. 22 Apr.
2016. <https://www.slicer.org/pages/Introduction>.