1. 
Abstract— Vibroarthrography, a diagnostic method for
interpreting the sounds emitted by a knee during movement, has
been studied for several joint disorders since 1902. However, to
our knowledge, the usefulness of joint sound signatures for
management of Juvenile Idiopathic Arthritis (JIA) has not been
investigated. To study these signatures for cases of JIA we
designed and built VibroCV, a platform to capture
vibroarthrograms from four accelerometers; electromyograms
(EMG) and inertial measurements from four Trigno Lab
wireless EMG modules; and joint angles from two Sony Eye
cameras and six light emitting diodes using computer vision and
commercially available off-the-shelf parts. This article explains
the design of this turn-key platform in detail, and provides a
sample recording captured from a subject with JIA.
I. INTRODUCTION
First studied by Blodgett in 1902 [1], the diagnostic
potential of knee-joint sounds has been demonstrated in
several different studies. For example, in 1987, McCoy et al
showed that the amplitude of knee joint sounds may decrease
after meniscal resection when 86% of a population of 170
subjects with meniscal tears exhibited changes in the
vibroarthrographic signal [2]–[4]. Since then, different
research groups have developed vibroarthrographic signal
processing techniques [3], [5]. Others have focused on systems
approaches. Our group has applied vibroarthrography to assess
progress in joint rehabilitation post-injury with sit-to-stand and
flexion/extension exercises of the knee [6], and recently we
developed a wearable system to measure knee joint sounds in
the context of different physical activities [7].
Diseases studied with vibroarthrography have involved
cartilage degeneration such as osteoarthritis [3]. However, to
our knowledge, the effect of juvenile idiopathic arthritis (JIA)
on the vibroarthrographic signal has not been studied even
though the disease tends to degrade synovial joints like the
knee when untreated. Known as juvenile rheumatoid arthritis
until relatively recently, JIA is any form of arthritis of
unknown cause that lasts more than 6 weeks and has an onset
before age 16 [8]. It is the most common type of arthritis in
children [8], and affects 7-21 out of every 100,000 children in
the US and Northern Europe. The disease can persist into
adulthood, has adverse long-term effects on joints, and does
not respond to standard therapy in 30% of patients [9].
Furthermore, while the main goal of treatment is to achieve
disease remission, approximately 30-50% of patients relapse
*This work was supported by the Center for Pediatric Innovation, a
research partnership between Children’s Healthcare of Atlanta and the
Georgia Institute of Technology.
A. D. Wiens, Ann Johnson, Sahithi Bonala, and Omer T. Inan are with
Electrical and Computer Engineering at Georgia Institute of Technology,
Atlanta, GA 30308 USA (314-610-9194; andrew.wiens@gatech.edu).
after discontinuation of methotrexate, the most common
standard therapy [10].
In response to the recent introduction of several highly-
successful anti-TNF (tumor necrosis factor) biologic drugs
such as infliximab, adalimumab, etanercept, and golimumab,
US guidelines now recommend switching to biologic therapy
after four months of unsuccessful treatment with standard
therapy [9]. Still, this is a long time for most juvenile patients
to wait. It is exacerbated by the fact that the process of iterating
toward an effective treatment plan typically takes many
months and requires several costly return visits to the
rheumatologist for subjective evaluation. In severe cases, such
as a 2-year-old girl who had JIA-induced growth retardation
resulting in a height of 3 standard deviations below the mean
for that age [9], months of ineffective treatment is completely
unacceptable.
Sampath Prahalad is with the Department of Human Genetics at Emory
University School of Medicine, Atlanta, GA 30322 USA.
VibroCV: A Computer Vision-Based Vibroarthrography Platform
with Possible Application to Juvenile Idiopathic Arthritis*
Andrew D. Wiens, Student Member, IEEE, Ann Johnson, Sahithi Bonala, Sampath Prahalad, and
Omer T. Inan, Senior Member, IEEE
Figure 1. Diagram of the experiment and instruments with a pediatric subject.
Four ultra-low noise accelerometers are configured as contact microphones
on the knees. A data acquisition unit (not pictured) records the knee sounds
while four wireless modules capture electromyograms (EMG) and 3-axis
acceleration on the quads and calves. Simultaneously, two high-speed
cameras operating at 120 frames per second (FPS) record the angles of the
left and right knees via six 10mm bright green light-emitting diodes (LED)
powered by lithium coin cells. Custom software running on a laptop (not
pictured) determines the location of the LEDs in each video frame, syncs the
information from all sensors, and streams data to the file system.
KNEE
ANGLE
LED
MARKER
ACCELEROMETER
WIRELESS
EMG & IMU
120 FPS
CAMERA
FRONT SIDE

2. To reduce the amount of time it takes to arrive at an
effective treatment plan after initial diagnosis, there have been
efforts to uncover personalized biomarkers of JIA. Biomarkers
in blood, urine, and saliva such as S100 proteins and MRP8/14
have been identified that predict remission and response to
treatment with varying degrees of success [10]. However, all
require sample collection and lab tests. Vibroarthrography
could provide a noninvasive biomarker of JIA disease that can
be easily measured outside of the clinic and without trained
personnel. If vibroarthography can provide information about
JIA disease state, it would constitute the first quantitative
metric that can provide day-to-day feedback between visits to
the clinic at very low cost.
To study the usefulness of vibroarthography for JIA
assessment we developed VibroCV, a fully-integrated system
for capturing time-synchronized vibroarthrographic and
related physiologic signals with a standard PC. This paper
describes the design, implementation, and testing of VibroCV
and briefly describes its future use in our pediatric human
subjects research study of JIA.
II. METHODS
This preliminary human-subjects research study was
approved by the Institutional Review Boards of Georgia
Institute of Technology and Emory University.
A. Equipment
An overview of the technique is shown in Figure 1, and a
block diagram of VibroCV appears in Figure 2a. All hardware
was connected to a laptop PC (Latitude E6000 series, Dell Inc.,
Round Rock, TX) running Windows 7 (Microsoft, Redmond,
WA). Four wireless electromyogram (EMG) sensors with
built-in 3-axis linear inertial measurement units (IMUs) were
used to capture muscle activity and lower body movements
(Trigno Lab, Delsys Inc., Natick, MA). In particular, the IMUs
are useful for joint angle estimation when marker based joint
angles are not available, i.e. when walking. The EMG receiver
provided a USB connection to a PC. Four single axis analog
miniature low-noise piezoelectric accelerometers were used
for capturing vibrations of the joints and recording the
vibroarthrogram signal (3225F7, Dytran Instruments Inc.,
Chatsworth, CA). These accelerometers were chosen because
they are Integrated Electronic Piezoelectric (IEPE) sensors
with low mass (0.85 grams), wide bandwidth (2 Hz—10 kHz,
±10%), and low noise (700 µgrms) properties. The
accelerometers were attached to the lateral sides of each patella
with elastic kinesiology tape (Kinesio Tex Gold, Kinesio,
Albuquerque, NM) and recorded on a PC with a USB data
acquisition unit (USB-4432, National Instruments
Corporation, Austin, TX). This unit was chosen for its high
resolution (24 bits), high sample rate (102.4 kS/s), ANSI C
API (NI-DAQmx), and built-in IEPE capability, which
allowed us to eliminate the need for a separate 4-channel
analog IEPE signal conditioner. Two USB cameras were used
to capture joint angles with a marker-based approach
(PlayStation Eye, Sony Computer Entertainment, Tokyo,
Japan). This camera was chosen because its light sensitivity is
optimal for LED marker-based capture (i.e. PS Move) and it
has a high framerate (120 frames per second at 320 x 240
pixels resolution), fixed-focus lens, wide field of view (75° for
wide zoom setting), a third-party high-framerate driver with a
dynamically-loaded library (DLL) and example code (CL-
Eye, Code Laboratories, Henderson, NV), and low cost. An
example image from the left camera is shown in Figure 3c.
Each camera was mounted on a tripod with cyanoacrylate glue
(LOC1365882, Loctite, Düsseldorf, Germany) and a ¼-20 nut
(08424, The Home Depot, Atlanta, Georgia). All hardware
was installed on a cart for quick clinical setup as shown in
Figure 2b (FR1006C-COM, Mainstays, Bentonville, AR). A
generic stool with adjustable height was also used for sit-to-
stand exercises.
B. LED Markers with Snap Mounts
To capture joint angles, LED markers were made as shown
in Figure 3a and placed laterally on the femur, tibia, and at the
knee joint as shown in Figure 1. The markers were made from
available commercial off-the-shelf electronics components.
Each marker had a bright green 10mm diffused LED (844,
Adafruit Industries, New York, NY) connected to a switch
(2750409, RadioShack, Fort Worth, TX) and a lithium coin
cell battery (CR2032, Dantona Industries, Wantagh, NY)
directly without a resistor. Wires were glued to the positive
and negative terminals of the coin cell battery with conductive
graphite glue (6400146, RadioShack, Fort Worth, TX). The
coin cell was then wrapped in 2:1 heat-shrink tubing (21-8765,
Figure 2. A) Block diagram of the necessary hardware and software for VibroCV. Data from all sensors are sent to VibroCV via dynamically-loaded libraries
(DLLs) and streamed to a fast solid-state disk (SSD) for later analysis. OpenCV provides the libraries for graphics and computer vision capabilities. B) Photo
of the hardware loaded onto a cart for a human-subjects research (HSR) study of juvenile idiopathic arthritis (JIA) in the clinic.
B)A)
EMG/IMUEMG/IMUEMG/IMU
4x
EMG/IMU
EMG/IMUEMG/IMUEMG/IMU
4x
ACCEL.
EMG/IMU
2x
CAMERA
USB 2.0
TRIGNO
BASE
NI DAQ VibroCV
SSD
TRIGNO
DLL
NI DAQ
DLL
CL-EYE
DLL
OpenCV

3. MCM Electronics, Springboro, OH), and a snap connector
from electrocardiogram (ECG) test leads (ECG-PRO-3-
WAY-CABLE, Olimex Ltd, Plovdiv, Bulgaria) was attached
with cyanoacrylate glue (LOC1365882, Loctite, Düsseldorf,
Germany) to one side. The snap connectors fit to disposable
ECG electrodes (2660, 3M Company, Maplewood, MN) to
provide easy mounting to the body. The electrodes were not
electrically connected to a circuit. The switch and LED were
also attached with cyanoacrylate glue to the other side of the
coin cell. Cyanoacrylate glue is very strong and results in a
very durable light marker when applied liberally and allowed
to dry overnight. Two examples of finished LED markers are
shown in Figure 3b, and an example of their use in practice is
shown in Figure 3c.
C. Software
VibroCV was implemented as a multithreaded C++
application and compiled for Win32 (Visual C++, Microsoft
Corporation, Redmond, WA). Raw video frames were
captured directly from each camera at a nominal rate of 120
Hz with CL-Eye (Code Laboratories, Henderson, NV).
Accelerometer waveforms from the data acquisition unit were
streamed to VibroCV via the NI-DAQmx ANSI C library
(National Instruments Corporation, Austin, TX), which was
configured to sample each accelerometer at 102.4 kHz. EMG
and IMU time series were obtained from the Trigno Lab
sensors with EMGworks and the Trigno Digital SDK (Delsys
Inc., Natick, MA).
All data were streamed to disk as raw binary files for later
processing. A solid-state hard drive allowed continuous
writing to the file system at high bandwidth, and the total CPU
usage by VibroCV was less than 25% across two hyper-
threaded cores (Ivy Bridge, Intel Corporation, Santa Clara,
California). A second Visual C++ application was written to
convert these binary files to MATLAB format (MathWorks,
Natick, MA) and to compress the raw camera frames with
H.264 (Windows 7 built-in codec) to save disk space for
storage after the experiments were completed.
Finally, OpenCV was used to create separate GUI
windows showing live previews of the cameras and scrolling
plots of the EMG, IMU, and accelerometer time series. Green
circles around the three LED markers, two magenta lines
connecting the markers, and white text of the computed angle
in degrees were drawn on each camera preview with OpenCV
as shown in Figure 3d. This helped the study coordinator check
each camera’s field of view and that OpenCV was registering
each marker when the cameras were set up in the clinic.
III. RESULTS AND DISCUSSION
A test vibroarthrogram of a healthy pediatric subject’s
knee as captured with VibroCV appears in the bottom of
Figure 4b, and the knee joint angles obtained with the marker-
based computer vision method appear in the top. The high
framerate of the cameras used in this work provided the
benefit of true 120 Hz measurements of joint angles. The low
resolution of the cameras at that framerate (320 x 240 pixels)
resulted in some jitter because single-pixel changes in the
estimated location of the markers caused relatively larger
changes in the estimated joint angle than higher resolution
cameras would produce. However, low-pass filtering of the
120 Hz joint angle measurements would be appropriate given
the low frequency of sit-to-stand exercise (< 1 Hz) and would
adequately suppress such high-frequency jitters.
VibroCV was also very convenient to use with pediatric
patients, and the bright LEDs engaged young subjects much
more than the wireless IMU system in our tests. One of our
three initial test subjects was very excited to wear the LED
markers and watch the output of the camera preview as shown
in Figure 3d while he performed the experiment. Joint angles
with LED markers could thus allow easier data collection with
young subjects.
A test recording of wrist joint auscultation of an adult with
JIA is shown in the time and time-frequency domains in
Figure 4a for comparison to the healthy vibroarthrogram in
the bottom of Figure 4b. The healthy vibroarthrogram
contains no distinct clicks while the diseased wrist joint
produced large clicks during each flexion/extension cycle.
IV. CONCLUSION
The design, implementation, and a preliminary recording
obtained with a new vibroarthrography platform based on
computer vision and accelerometers called VibroCV were
presented. Trial experiments with two subjects subjectively
validated the performance of the system. Optical marker-
Figure 3. A) Schematic of the LED markers. Inexpensive off-the-shelf
components are soldered into a simple on-off circuit and glued together to
form a compact package. An ECG snap connector and electrode are used to
provide convenient and safe mechanical attachment to the subject’s skin. B)
Photo of completed LED markers. C) Raw image from the camera during one
recording in a bright room. D) One joint angle as displayed on-screen in
VibroCV. The software locates the three brightest spots in the image using
OpenCV and computes the inner angle in degrees. The three LEDs are bright
and distinct even with overhead fluorescent lights turned on.
ON/OFF
SWITCH
CR2032 COIN CELL
& HEATSHRINK
1000 MCD 10 mm
DIFFUSED
GREEN LED
ECG SNAP
CYANOACRYLATE GLUE
ECG ELECTRODE FOR SKIN ADHESION
A)
B)
C) D)

4. based joint angles appear to offer significant benefits over
IMU-based joint angles, namely, more convenient placement
on pediatric subjects. Quantitative comparisons of the optical
system to traditional goniometers or IMU systems could be a
topic of a future investigation. Next steps include
characterization of the vibration signal characteristics of noise
from sensor rubbing and kinesiology tape adhesive from our
hardware platform as described in the vibroarthrography
literature where researchers obtained superior results with
accelerometers compared to air microphones by filtering
mechanical noise out of the signals after characterizing the
shape of unwanted noise in the time-frequency domain [3].
Finally, VibroCV will be used in the clinic to study the
feasibility of vibroarthrography for assessment of patients
with JIA. Clicks identified in a similar wrist auscultation
signal from an adult with JIA suggest that pediatric JIA
patients may exhibit similar changes at the knee, especially
because the knee is a larger joint than the wrist.
Vibroarthrography could potentially improve treatment for
patients with juvenile idiopathic arthritis by providing
feedback between doctor visits to allow the patient’s
physician to respond to disease changes and modify the
treatment plan more quickly. This upcoming pilot study
should be relevant to any rheumatologist, biomedical
engineer, or patient suffering from JIA or related
inflammatory diseases.
REFERENCES
[1] W. E. Blodgett, “Auscultation of the Knee Joint,” Bost. Med. Surg.
J., vol. 146, no. 3, pp. 63–66, 1902.
[2] G. F. McCoy, J. D. McCrea, D. E. Beverland, W. G. Kernohan, and
R. a Mollan, “Vibration arthrography as a diagnostic aid in diseases
of the knee. A preliminary report.,” J. Bone Joint Surg. Br., vol. 69,
no. 2, pp. 288–293, 1987.
[3] Y. F. Wu, S. Krishnan, and R. M. Rangayyan, “Computer-aided
diagnosis of knee-joint disorders via vibroarthrographic signal
analysis: A review,” Crit. Rev. Biomed. Eng., vol. 38, no. 2, pp.
201–224, 2010.
[4] S. C. Abbott and M. D. Cole, “Vibration arthrometry: a critical
review.,” Crit. Rev. Biomed. Eng., vol. 41, no. 3, pp. 223–42, 2013.
[5] Y. Shen, R. M. Rangayyan, G. D. Bell, C. B. Frank, Y. T. Zhang,
and K. O. Ladly, “Localization of knee joint cartilage patholog by
multichannel vibroarthrography,” Med. Eng., vol. 17, no. 8, pp.
583–594, 1995.
[6] C. Teague, S. Hersek, H. Töreyin, M. L. Millard-stafford, M. L.
Jones, G. F. Kogler, M. N. Sawka, and O. T. Inan, “Novel Methods
for Sensing AcousticalEmissions from the Knee for Wearable Joint
Health Assessment,” IEEE Transactions on Biomedical
Engineering, In Press, 2016.
[7] H. Toreyin, S. Hersek, C. Teague, and O. Inan, “A Proof-of-
Concept System to Analyze Joint Sounds in Real Time for Knee
Health Assessment in Uncontrolled Settings,” IEEE Sens. J., In
Press, 2016.
[8] S. Rangaraj, “Diagnosing juvenile idiopathic arthritis,” Paediatr.
Child Health (Oxford)., vol. 1, no. 12, pp. 552–557, 2007.
[9] F. La Torre, M. Cattalini, B. Teruzzi, A. Meini, F. Moramarco, and
F. Iannone, “Efficacy of adalimumab in young children with
juvenile idiopathic arthritis and chronic uveitis: a case series.,”
BMC Res. Notes, vol. 7, no. 1, p. 316, 2014.
[10] C. L. Duurland and L. R. Wedderburn, “Current developments in
the use of biomarkers for juvenile idiopathic arthritis.,” Curr.
Rheumatol. Rep., vol. 16, no. 3, p. 406, 2014.
Figure 4. A) Time series (top) and short-time Fourier transform (bottom) of
joint sounds of the wrist of an adult male subject with JIA during flexion-
extension movements. This figure is shown for reference as an example of
the clicks that can occur in a diseased joint, although it cannot be compared
directly to the knee example since it is a different joint and the subject was an
adult. B) Knee-joint vibrations from the four accelerometers (bottom) and the
joint angles (top) of the left knee (red) and right knee (blue) during a trial sit-
to-stand recording of a male juvenile subject without JIA. These waveforms
were recorded with VibroCV and show the capability of the system. Clicks
and pops are not present in these time series presumably because the subject
was healthy. Future recordings will be made on male and female children
with and without JIA.
16.7 s
90
170
+3.9 m/s2
-3.9 m/s2
23 s
A)
B)
0 Hz
25 kHz
-10 m/s2
+10