Image Fusion is the process of combining the information from two or more input images onto a single fused image. Fusion of Medical Images may help the radiologist and doctors to get the complete information about the body organ in a single image. This improves diagnosis.

1. Dr. Nishant Jain
(M.Tech & Ph.D. from IIT Roorkee)
Fusion of BioMedical Images
Assistant Professor, Department of ECE
Jaypee University of Information Technology
Waknaghat, Solan, HP (India) -173234

2. Contents
1. Acquisition of Images.
2. Medical Imaging Modalities.
3. Introduction to Image Fusion
4. Need of Registered images for Image Fusion.
5. Fusion of CT and MR images.
6. Image fusion method based on wavelet.
7. Online Dataset for Image Fusion.
8. Hands-on in MATLAB.

3. Acquisition of Digital Images
Single Imaging Sensor / one Pixel
Incoming energy is transformed into a voltage. This voltage is digitized to
obtain a digital quantity.
3
Sensor ADC
Analog
Voltage Signal
Input
Energy
Digital
Value

4. Acquisition of Digital Images
Array of Imaging Sensors forms an Image
4

5. Acquisition of Digital Images
Single Imaging Sensor / one Pixel
By changing the sensors that are sensitive to some specific input energy,
we can form different images like X-Rays, MRI, Thermal images, etc
5
Sensor ADC
Analog
Voltage Signal
Input
Energy
Digital
Value

6. Examples of Imaging Modalities
1. X-rays
2. Computed Tomography (CT)
3. Magnetic Resonance Imaging (MRI)
4. Ultrasound (US)
5. Thermal Imaging

7. Imaging Modalities
Different imaging modalities scan different
parameters or features of the objects.
Example: Normal Camera scan the person
using visible spectrum and give image with
actual colors; whereas Thermal Camera can the
thermal radiation or temperature of the
person.
Brighter colors like red and orange indicate
high temperature whereas dark colors like dark
blues and black indicate low temperature.
Images taken from:
https://www.photofunny.net/efectos/grandes/thermography_photo_effect.jpg
https://www.flir.com/discover/public-safety/thermal-imaging-for-detecting-elevated-body-temperature/

8. Appearance of the body organs also differs from one Medical Imaging Modalities to
another.
For example: CT image of the brain shown in (a) gives better information of bone
structure, whereas information of less dense objects like soft tissues of the brains,
tumors, etc. are not properly visible and have poor contrast.
MR image of the brain shown in (b) gives poor information of bone structure, whereas
information of less dense objects like soft tissues of the brains, tumors, etc. are visible
better in comparison to CT and have good contrast.
(a) (b)
Medical Imaging Modalities

9. CT Image MR Image Fused Image
Image that contains the information from all the medical imaging modalities would
be the perfect one for the Doctors or the Radiologists to diagnose the abnormality
or to plan the surgery.
This image that captures information from all the modalities available is known as
Fused image.
Medical Imaging Modalities

10. CT Image MR Image Fused Image
Fusion of CT image with MR image give a fused image having information of both
bones and soft tissues in the same image.
This helps Doctors to locate the position of the Tumor in brain with respect to the
bone with high accuracy.
Medical Imaging Modalities

11. Image Fusion
Image Fusion is the process of combining the information from two or more
input images onto a single image in such a way as to reduce uncertainty,
minimize redundancy in the output, and maximize relevant information
pertaining to an application or a task.

12. Need of Registered images for Image Fusion
Input Image (a) Input Image (b)

13. Need of Registered images for Image Fusion
Input Image (a) Input Image (b) Fused Image without Registration

14. Need of Registered images for Image Fusion
Input Image (a) Input Image (a)
aligned or registered with
Input Image (b)
Input Image (b)

15. Need of Registered images for Image Fusion
Input Image (b) Fused Image after Registration
Input Image (a)
aligned or registered with
Input Image (b)

16. Image Fusion
Methods
Multiscale
Decomposition
Pyramid
Transform
Discrete
Wavelet
Transform
Non- Multiscale
Decomposition
Estimation
Theory Based
Method
Artificial
Neural
Networks
Pixel Level
Weighted
Averaging
Image Fusion Methods

17. Image fusion method based on Wavelets
Wavelets based Image fusion method is a 3 Step Process:
1. Transformation of input images using Discrete Wavelet Transform (DWT)
into multi- resolution images.
2. Fusion of multi-resolution images using fusion rule.
3. Inverse wavelet transformation to get the fused image.

18. Image fusion method based on wavelets
Transformation of input images using Discrete Wavelet Transform (DWT)
into multi- resolution images:
Wavelet transform transforms the input images into multi-resolution images,
so that all information from coarse to fine can be fused effectively.
L : Low Sub-Band Image
H: High Sub-Band Image
IXY
: Image I obtained after
filtering rows with X filter and
columns with Y filter

19. Image fusion method based on wavelets
Transformation of input images using Discrete Wavelet Transform (DWT)
into multi-resolution images:
2-Level Wavelet Transformed Image
LL LH
LH
HL HH
HL HH
Image LL is the Approximate Image
All the remaining images are known as detailed
images

20. Image fusion method based on wavelets
Transformation of input images using Discrete Wavelet Transform (DWT)
into multi-resolution images:

21. DWT of MR Image

22. Image fusion method based on wavelets
Wavelets based Image fusion method is a 3 Step Process:
1. Transformation of input images using Discrete Wavelet Transform (DWT)
into multi- resolution images.
2. Fusion of multi-resolution images using fusion rule.
3. Inverse wavelet transformation to get the fused image.

23. Image fusion method based on wavelets
Fusion of multi-resolution images using fusion rule:
Fusion Rule defines how the two pixels from the two input images are used to
get the value of the corresponding pixel in the fused image.
Since Approximate image contains low frequency content of the input images,
whereas Detailed images contain high frequency content of the input image,
therefore fusion rule for both the approximate and detailed images are
different.

24. Image fusion method based on wavelets
Fusion rule for Approximate Image (ILL
): Minimum Fusion Rule
20 15 16 10 5
22 17 7 18 7
3 6 23 32 1
45 32 12 21 20
2 34 32 5 2
2 10 36 20 15
25 15 4 18 17
31 6 21 30 2
46 34 11 20 30
1 4 35 6 2
2 10 16 10 5
25 15 4 18 7
3 6 21 30 1
45 32 11 20 20
1 4 32 5 2
Input Image A Input Image B Fused Image
Approximate
Images:

25. Image fusion method based on wavelets
Fusion rule for Approximate Image (ILL
):
Other fusion rules that can be used are mean, max, Global
Energy Maximization, etc.

26. Image fusion method based on wavelets
Fusion rule for Detailed Images:
The simplest rule that can be used to fuse the pixels of detailed images is MAX
Rule
20 15 16 10 5
22 17 7 18 7
3 6 23 32 1
45 32 12 21 20
2 34 32 5 2
2 10 36 20 15
25 15 4 18 17
31 6 21 30 2
46 34 11 20 30
1 4 35 6 2
Input Image A Input Image B Fused Image
20 15 36 20 15
25 17 7 18 17
31 6 23 32 2
46 34 12 21 30
2 34 35 6 2
Detailed
Images:

27. Image fusion method based on wavelets
Wavelets based Image fusion method is a 3 Step Process:
1. Transformation of input images using Discrete Wavelet Transform (DWT)
into multi- resolution images.
2. Fusion of multi-resolution images using fusion rule.
3. Inverse wavelet transformation to get the fused image.

28. Image fusion method based on wavelets
Inverse wavelet transformation to get the fused image:
Fused image having all the information from the two original images is
obtained using inverse wavelet transform on fused sub-band images.

29. Online Dataset for Image Fusion
http://www.med.harvard.edu/AANLIB/

30. Hands-on in MATLAB
Type waveletAnalyzer in the command window of the MATLAB

31. Hands-on in MATLAB

32. Hands-on in MATLAB
wfusimg

33. Thank You
Website: www.nishantjainlab.in
GitHub Account: https://github.com/nishantjain86
Linkedin: www.linkedin.com/in/nishantjain86
Email Id: nishantjain86@gmail.com
Facebook Id: nishantjain86
YouTube Channel: nishantjainlab