This document discusses determining the importance of figures in medical journal articles. It presents the results of a user test where a physician ranked the importance of figures in 50 medical articles. Key findings include that diagnostic images, images of higher magnification and resolution, and images located towards the end of articles tended to be most important. Compound figures containing multiple sub-images were difficult to rank. The results could help improve information retrieval systems and visualization tools by enabling selection of the most representative figures from articles.

1. Determining the importance of figures
in journal articles to find representative
images
Henning Müller, Antonio
Foncubierta-Rodriguez,
Chang Lin, Ivan Eggel

2. Introduction
 Images are essential for diagnosis and
treatment planning
Diversity of modalities and protocols is growing
 Medical images represent largest amount of
medical information produced
 Medical literature carries much of the medical
knowledge
Images are essential for medical articles
 Images can help understand content of an article quickly
 Much quicker than reading a text to determine relevance
Search in the literature is frequent 2

3. Motivation
 Image retrieval and classification make
medical visual content accessible for search
Images from PACS or from the medical literature (as
in the ImageCLEF image retrieval campaign)
 When searching the image needs to be put
into context
Text and visual data to understand context
Often more than a single image
 Identification of important figures
Use of visual content to determine relevance quickly
Limit search to important figures 3

4. Examples for using figure importance
 Interfaces and visualization require choices
 Mobile information search has small interface
 Creation of ground truth and rules for
importance ranking to select in the best way
 Often the first N are taken
4

5. Database used
 ImageCLEF 2012 dataset
 ~75,000 articles of PubMed Central, ~300,000 images
 Subset was used
 ~800 articles contained more than 8 images
 Final choice of 50 articles containing 641 diagnostic
images
5

6. Types of figure importance
 Five categories – sorted by importance
Most important figure – 1 figure that best
describes the article
Essential figures – 0-3 figures
Important figures – 0-3 figures
Little importance figures – 0+ figures
Totally unimportant figures – 0+ figures
6

7. User test
 User test participant profile:
45-year-old physician, surgeon
Working with medical images on a daily basis
 User test scenario:
Searching for relevant articles on the topic of the
text
 Which figures contain the most important
information about the article?
 Requires to read and understand article
 All systematic findings or rules for the
ranking were written down 7

8. Results
 All 50 articles were analyzed and all 641
images manually ranked by importance
Usually over 1 hour per article
 Rules and findings were noted and discussed
 Statistics on figure importance and places in
the text were made
 Compound figures were difficult to judge
Information content is high as they contain different
subfigures and links between them
Compound figure separation new sub task in
ImageCLEF 8

9. Rules for determining importance
 Diagnostic images are more important than
graphs/system overviews
 Diversity in the results set is important
Not several times the same modality if others exist
 Size/resolution matters
Image quality is importance
 Position in article
Most importance in the conclusions, then results
 Common vs. rare modality
Common modalities are often easier to understand 9

10. More rules for the importance
 Magnification
Highest magnification most important
 MRI for soft tissue diseases, CT for bone
related ones
Modality adapted to the disease, can be derived
from RadLex terms
 Compound vs. other
Compound figure contains much information
 In non-diagnostic images, statistical
graphs most important
10

11. Results
 Distribution of figure importance
11

12. Results
 Importance vs. position of the figure in the
article based on relative position in the text
12

13. Example for figure importance
13

14. Conclusions
 Relative importance of visual content in
scientific articles is important for many
applications
First study to investigate this in a user test
 Can improve IR system performance
Rank images based on position or remove some
images to search a smaller sub space
 Can improve visualization
For small interfaces such as mobile phones and
tablets
To combine text and images in an optimal way 14

15. Future work
 Repeat the same test with a second
person to measure inter-rater
disagreement
Quantify the subjectivity of the task
 Model the rules and optimize an automatic
selection strategy
Modality classification to filter out unwanted
images
Clustering and use of cluster centers to increase
diversity
Find the place of figures in the text and rank
based on this 15

16. Thank you for your attention!
Further information:
henning.mueller@hevs.ch
http://medgift.hevs.ch/
http://www.khresmoi.eu/
http://www.mitime.org/width/
16