1. BACKGROUND
Physicians applying regional anesthesia often use ultrasound to aid in nerve blocks
and drug delivery. Two-dimensional ultrasound images are conventionally used for
this task, requiring the physician to mentally reorient these images. A three-
dimensional ultrasound system allows the physician to view a nerve plexus as it
appears in the body. Yet, existing three-dimensional ultrasound systems are extremely
costly. Combining the tracking ability of the XBOX Kinect (Microsoft, Redmond, WA)
with LabVIEW (National Instruments, Austin, TX) programming, static three-
dimensional images can be generated at a substantially lower cost.
Novel 3-D Brachial Plexus Reconstruction from 2-D Ultrasound
Using XBOX Kinect Tracking
P.J. Boutros1, C.X. Lee1, S.J. Mathews1, P.J. Wilkens1, D.R. Peterson1,2, J.H. McIsaac3
1Biomedical Engineering Program, University of Connecticut, Storrs, CT
2Biodynamics Laboratory, University of Connecticut Health Center, Farmington, CT
3Hartford Hospital, Hartford, CT
In LabVIEW, image registration was used to transform all data sets onto one coordinate
system. Ultrasound images were translated in the x- and z-planes, according to the
Kinect tracking data.
Anterior view of the proximal portion of the brachial plexus, including the common carotid
artery. The first image on the left is the isometric view without application of the Kinect
tracking data and pixel threshold criterion, while the other image is the same
reconstruction but with tracking and filtering applied.
KINECT TRACKING
In this user interface, a clinician is able
to control file-paths and image
processing values. A text file containing
ultrasound information and patient data
is also exported.
MATERIALS AND METHODS
Assuming the patient to be a rigid body and minute wrist rotations of the individual
taking the ultrasound being minimal, the ultrasound probe was considered an
extension of the hand and tracked through the XBOX Kinect. The relative position of
the hand was recorded and down-sampled to match the sampling rate of the ultrasound
(i.e., 7.5 fps). Converting the ultrasound movie into a series of JPEG images, an image
registration technique was applied to translate the images according to the position
data obtained from the XBOX Kinect. Implementing a pixel threshold criterion, the
newly processed data set was also filtered. Both the tracking and reconstruction
algorithms were performed in LabVIEW (version 2011) and the processed images were
viewed in the Biomedical Engineering Startup Kit (version 3.0) add-on.
RESULTS AND DISCUSSION
Using the described reconstruction modality, static three-dimensional images of the
brachial plexus were reconstructed from eight-second, linear, anteroposterior ultrasound
scans.
CONCLUSION
Our reconstruction method yielded static three-dimensional images of diagnostic
significance after incorporating the Kinect position data. Motion artifacts and image
noise were also able to be minimized. Future work will apply this technique to other soft
tissues and allow the ability to isolate and reconstruct any region of interest.
Ultrasound Data
XBOX Kinect
Position Data
Interact with 3D
Image Series
LabVIEW
3D Viewer
3D Reconstruction
to 3D Image
Series
Data Processing
and Filtering
IMAGE RECONSTRUCTION AND USER INTERFACE
Using an M-Turbo Ultrasound Machine (SonoSite Inc., Bothell, WA),
scans of a custom-built imaging phantom (i.e., submerged PVC tube
in an aquarium) were performed. The ultrasound video was
deconstructed by LabVIEW into a series of JPEGs and, because the
framerate sampling of both imaging systems were inherently
different, a Henderson 23-term filter was used to remodel the
position data from the Kinect. After remodeling, the position data
set was resampled to match the frame rate of the ultrasound imager.
An image registration technique was applied to transform the
deconstructed JPEGs into one coordinate system, according to the
resampled Kinect position data. An image filter was also used to
account for noise and motion artifacts based on pixel intensity. The
final, processed set of images were then viewed using LabVIEW.
SYSTEM VALIDATION
The first image on the left is the isometric view of PVC reconstruction without
application of the Kinect tracking data and pixel threshold criterion, while the other two
images are the same PVC reconstruction but with tracking and filtering applied.
RESULTS OF SYSTEM VALIDATION
Using the reconstruction algorithm, note the decrease of image noise in both the PVC
and Brachial Plexus scans. Also note, the formation of the brachial nerves and carotid
artery.
PVC IMAGE RECONSTRUCTION
SOFT TISSUE RECONSTRUCTION
The Microsoft XBOX Kinect unit contains a 640x480 Color CMOS (RGB) camera, an
infrared projector, and a 320x240 infrared CMOS camera. The infrared projector
projects a micro-pattern of infrared points onto a field of view and the deformation of
the micro-pattern is used to calculate depth.
The LabVIEW interface is used to provide real-time visualization of an individual’s
movement and posture. Up to six individuals can be tracked at one time.