This document discusses the MELODY project which uses ultra wideband technology for medical sensing, localization, and communications. The project aims to develop wireless technologies to continuously monitor vital signs, perform high resolution medical imaging without contact, localize objects inside the body, and transmit high data rate signals from implanted devices. Specific applications discussed include a wireless capsule endoscope for gastrointestinal imaging, localization of implants, and using medical radar to measure heart rate and blood pressure. The project has developed technologies to enable full HD video transmission from an ingestible capsule and localization of implants within millimeters.

1. Medical sensing, localization, and communications using
ultra wideband technology
(MELODY)
Ilangko Balasingham
Project Leader
http://www.melody-project.info

2. Project info
• Consortium core
–
–
–
–
Oslo universitetssykehus HF
NTNU
Forsvarets forskningsinstitutt
University of Oslo (2008-2012)
• Industry panel
• International expert panel
– National Institute of Information and
Communication Technology, Japan
– Nagoya Institute of Technology, Japan
– Technical University of Dresden, Germany
– GE Healthcare, UK
– SORIN Group, France
– Given Imaging, USA
– OmniVision, Norway
–
–
–
–
–
Novelda
IBM
ABB
Atmel
Hospitality
(2008-2012)
(2008-2012)
(2008-2012)
(2008-2012)
(2008-2012)
NFR: StorIKT/VERDIKT program 01.09.2008 – 31.12.2015 :
- Total budget: 48 M.kr. (NFR: 36 M.kr.)
- 8 PhD and 17 Postdoc man-years

3. Results (2008-2012)
•
•
•
•
•
•
24 journals (4 in level 2)
1 book chapter
72 peer-reviewed full conference papers (level 1)
6 abstracts with poster/oral presentations
7 popular articles in Norwegian and international media
2 patents filed, 3 DOFIs submitted
• 4 master theses
• 4 PhD defended - 2 PhD theses submitted
• 12 Postdoc projects completed
• Int. visitors: 2 prof’s, 2 PhDs, 4 master students
• 4 of our PhD students spent 3-12 months abroad
• Organized special sessions in 4 international conferences and annual
workshops with international talks given by international experts

4. Some of the future challenges
• Increased cost due to readmissions and follow up treatment and
care of chronic sick patients (diabetes, cardio vascular, cancer, etc.)
to hospitals (“svingdør-pasienter)
•
Improved personalized healthcare (long term monitoring and customized
treatment on an individual basis) in ubiquitous manner
Wireless health technology?

5. Problem
• Can wireless technology be used
– to measure vital signs (heart rate, respiration, blood
pressure) continuously and remotely without any
contact?
– to do high resolution, cost effective imaging without
any contact with non ionizing radiation?
– to localize and track objects inside the human body
without imaging?
– to transmit high data rate sensor signals from devices
implanted deep inside the human body at receivers
located outside in a robust, reliable manner?

6. MELODY Overview

7. Ultra wideband (UWB) technology
(3.1 – 10.6 GHz)
• Characteristics
– ultra-short pulses, low duty cycle, fine time resolutions – Imaging,
sensing, localization, tracking
– very large bandwidth, extremely low power spectral density,
excellent propagation, low interference generation and good
interference rejection, coexistence with conventional systems,
almost undetectable – wireless communication
– Possibility to address all three application domains such as
communications, localization and sensing within one single
technology

8. Ultra Wide Band (UWB)
• Regulations
– 3.1-10.6 GHz
– EIRP<-41.3 dBm
– Max <0.5 mW
• Bandwidth
– > 500 MHz or
– 20% of center
frequency
Largest unlicensed bandwidth ever released!

9. MELODY Research Fields (2008-2012-2015)
UWB Technology
Sensing/Imaging
Localization/Tracking
Signal Proc./Commun.
Blood pressure, HR, etc.
Radar imaging techniques
High penetration in tissues
Beams with mm range
Distance measurements
Accuracy in the mm scale
Active echo engine
Algorithms of localization
Channel Modeling
Joint source-channel coding
Modulation, pulse shaping
Cognitive networks

10. UWB radio interfaces for on-body sensor network
EEG
IR-UWB
3.1–4.8 GHz
ECG
Relay
node
PDA
SpO2
WBAN controller
EMG
Patient monitor
MB-OFDM UWB
3.1–10.6 GHz

11. In-body sensor network
LOW DATA RATE
IMPLANT SENSOR
TO THE WIRELESS BODY AREA
NETWORK CONTROLLER
IR-UWB
3.1–4.8 GHz
TO THE PATIENT
MONITOR
CAPSULE ENDOSCOPE
Heart applications – leadless pacemaker, etc.
Brain applications – Parkinson, Alzheimer, etc.

12. Capsule video endoscope
• Use for examination of gastrointestinal track
for bleeding, inflammation, tumor, cancer,
etc.
– ca. 15% of male and female above 50 years old
are likely to get colorectal cancer
– early detection can cure or extend the life with a
few years – screening the entire population
above 50 years!
• Fiber optic cable – problems to reach small
intestine – huge discomfort for the patient!

13. Wireless capsule endoscopy
• Capsule Endoscope




A small camera the size of a pill that can be swallowed
Enables visual inspection of small intestines
Diagnosis of gastrointestinal diseases
Significantly less discomfort to patients
• State of the Art




Trasmits still pictures (external video construction)
Slow motion (typically 8 hours)
No navigation system
Localization and tracking with accuracy in the centimeter scale

14. Required characteristics for improvement
•
High data rate
 73.8 Mbps for raw HD data
•
Extremely low power consumption
 On the order of 1 mW
•
Circuitry simplicity/integrability
 0.18 m CMOS technology
•
Reduced physical dimension
 11 mm × 26 mm2
•
Electromagnetic radiation safety
 SAR limits, overheating below 1 °C
Impulse Radio Ultra
Wideband (IR-UWB)
Technology

15. Capsule Endoscope
Particularly useful for inspection of the small bowel
15

16. Example of Capsule Endoscope Video
Capsule endoscope video quality (256 × 256 pixels, 2 fps)

17. Experimental Feasibility Verification
A series of in-vivo video transmissions in porcine chirurgical models
Transmission Characteristics
 UWB transceivers
 4224–4752 MHz, 528 MHz bandwidth
 80 Mbps, 1280×720 pixel/30 fps
17

18. Wireless full HD video transmission
• See the demo at http://www.melodyproject.info

19. GB 1220466.5 Video Camera Pill
Opportunities to commercialize the invention
Size: 11 × 26 mm
Transmission frequency: 402405 MHz
Bandwidth: 300 kHz
Size: less than 11 × 26 mm
Transmission frequency: 1063–3841 MHz
Bandwidth: at least 500 MHz
Data Rate: 800 kbps
Image Rate: 2 to 10 fps
Image Resolution: 256 × 256 pixels
Data Rate: 80 Mbps
Image Rate: 30 fps
Image Resolution: 1920 × 1080 pixels
Power consumption: 100 mW
Operating life: 8 hours
Power consumption: estimated 1 mW
Operating life: more than 8 hours
Possibility of smaller batteries
Possibility of remote control

20. Video compression: Encoder
• Frame-by-frame video coding applied due to low complexity
requirement: limited size and limited power- and storage
capacity
• Capsule moves slowly → can reduce frame rate to 10-15 fps.
Frame interpolation in receiver enhances viewing experience
• Each frame should be coded with as low a rate as possible while
maintaining adequate quality
• Coder built on: Differential pulse coded modulation (DPCM)
which removes correlation between pixels
Very Low Complexity Low Rate Image Coding for the Wireless Endoscope, US 61/717,963

21. Video coding: Encoder architecture

22. Original vs. compressed
Original
Compressed
0.67 bpp with 2x2 decimation: Compression ratio ≈ 97%

23. Localization
– Need to know where
anomalies are located
– Adapt frame rate
– Control transmission power
and save energy by turning
device on and off

24. Methods for in-body localization
• Electromagnetic: (Received signal strength)
PPM has constant amplitude. If path loss is known →
Received signal strength from several sensors indicated
where capsule is
– RF power undergoes extreme link dependent shadowing
•
Fixed magnetic field (Ferro magnet)
–
–
–
–
Magnetic field not absorbed by human body
Can determine orientation of capsule as well
Possibility to steer capsule from outside
On-board magnet makes endoscope larger and heavier

25. Results: Localization
• Cramer-Rao lower bounds calculated for
electromagnetic- and magnetic based localization
• Electromagnetic: Localization accuracy to 1 cm
• Magnetic: Localization accuracy to 2 mm
• Tracking of capsule makes localization more accurate.
Principle built on multi-model Kalman filtering and
matched filter detection

26. Application of medical radar

27. Medical radar measurements
of human heart
• Penetrating human body with
body-contact antennas
• Useable frequency range:
0.5 – 3 GHz

28. 1 GHz CW Radar measurements, single
channel
Radar (blue curve)
ECG (red curve)
The phase is related to
the motion of the heart.
The instantaneous
frequency is related to
the velocity of the heart
movement.

29. Radar imaging
UWB radar (0.5 – 3 GHz)
Antenna array
Combining UWB
measurements from the
elements of the antenna
array gives a radar image
Antenna

30. Time-lapsed imaging results
Images are sequenced to a video with frame rate 25 Hz
Heartbeat signals at different locations

31. Principle for Measurements (1)
Quasi-linear relationship between radius r(t) and pressure P(t)

32. Multiple clutter objects

33. Results on blood pressure estimation
• The backscattered signal from the aorta contains necessary
information for estimating its diameter in time and frequency
domains.
• Trade-off between high frequency and bandwidth in order to
achieve high resolution; high frequencies involve high
attenuation.
• Optimal points have been identified with good specificity and
accuracy.
• Studying to remove ”artifact” due to the physical heart motion
embedded in the contraction/dilation cycle of the aorta.

34. Concluding remarks
• Demonstrated that UWB technology for
– Wireless capsule endoscope
• Ultra low power communication architecture
– source coding, channel coding, pulse shapes, modulations, and extremely simpler transmitter
and simple receiver architectures based channel model and channel state information
– new MAC protocol, cognitive network architecture, white-space detection scheme, and
resource allocation (frequency, power)
• Algorithms for localization and tracking for both electromagnetic (1 cm) and magnetic
schemes (0.5 mm)
– Medical radars for heart rate with finer details of opening and closing of heart
values, preliminary studies on blood pressure estimation
• New design architecture for the future wireless capsule endoscope:
– diagnostics
• improved imaging sensor for anomaly detection, RF based tomography for cancer tissue
imaging
• targeted drug delivery: wireless control with improved localization and tracking using
diversity techniques and range estimation (multiple receivers, path loss, etc.) power
control/transmission, nano particles, etc.