SlideShare a Scribd company logo
1 of 16
Download to read offline
Determining the importance of figures
in journal articles to find representative
images



                            Henning Müller, Antonio
                            Foncubierta-Rodriguez,
                            Chang Lin, Ivan Eggel
Introduction
  Images are essential for diagnosis and
  treatment planning
  Diversity of modalities and protocols is growing
  Medical images represent largest amount of
  medical information produced
  Medical literature carries much of the medical
  knowledge
  Images are essential for medical articles
   Images can help understand content of an article quickly
    Much quicker than reading a text to determine relevance
  Search in the literature is frequent                        2
Motivation
  Image retrieval and classification make
   medical visual content accessible for search
  Images from PACS or from the medical literature (as
   in the ImageCLEF image retrieval campaign)
  When searching the image needs to be put
  into context
  Text and visual data to understand context
  Often more than a single image
  Identification of important figures
  Use of visual content to determine relevance quickly
  Limit search to important figures                    3
Examples for using figure importance
  Interfaces and visualization require choices
   Mobile information search has small interface
  Creation of ground truth and rules for
  importance ranking to select in the best way
   Often the first N are taken




                                                    4
Database used
  ImageCLEF 2012 dataset
   ~75,000 articles of PubMed Central, ~300,000 images
  Subset was used
   ~800 articles contained more than 8 images
   Final choice of 50 articles containing 641 diagnostic
   images




                                                            5
Types of figure importance
  Five categories – sorted by importance
  Most important figure – 1 figure that best
   describes the article
  Essential figures – 0-3 figures
  Important figures – 0-3 figures
  Little importance figures – 0+ figures
  Totally unimportant figures – 0+ figures



                                                6
User test
  User test participant profile:
  45-year-old physician, surgeon
  Working with medical images on a daily basis
  User test scenario:
  Searching for relevant articles on the topic of the
    text
   Which figures contain the most important
    information about the article?
  Requires to read and understand article
  All systematic findings or rules for the
   ranking were written down                             7
Results
  All 50 articles were analyzed and all 641
  images manually ranked by importance
  Usually over 1 hour per article
  Rules and findings were noted and discussed
  Statistics on figure importance and places in
  the text were made
  Compound figures were difficult to judge
  Information content is high as they contain different
   subfigures and links between them
  Compound figure separation new sub task in
   ImageCLEF                                               8
Rules for determining importance
  Diagnostic images are more important than
  graphs/system overviews
  Diversity in the results set is important
  Not several times the same modality if others exist
  Size/resolution matters
  Image quality is importance
  Position in article
  Most importance in the conclusions, then results
  Common vs. rare modality
  Common modalities are often easier to understand      9
More rules for the importance
  Magnification
  Highest magnification most important
  MRI for soft tissue diseases, CT for bone
  related ones
  Modality adapted to the disease, can be derived
   from RadLex terms
  Compound vs. other
  Compound figure contains much information
  In non-diagnostic images, statistical
  graphs most important
                                                     10
Results
  Distribution of figure importance




                                       11
Results
  Importance vs. position of the figure in the
  article based on relative position in the text




                                                   12
Example for figure importance




                                13
Conclusions
  Relative importance of visual content in
  scientific articles is important for many
  applications
  First study to investigate this in a user test
  Can improve IR system performance
  Rank images based on position or remove some
   images to search a smaller sub space
  Can improve visualization
  For small interfaces such as mobile phones and
   tablets
  To combine text and images in an optimal way     14
Future work
  Repeat the same test with a second
  person to measure inter-rater
  disagreement
  Quantify the subjectivity of the task
  Model the rules and optimize an automatic
  selection strategy
  Modality classification to filter out unwanted
   images
  Clustering and use of cluster centers to increase
   diversity
  Find the place of figures in the text and rank
   based on this                                       15
Thank you for your attention!



Further information:
henning.mueller@hevs.ch
http://medgift.hevs.ch/
http://www.khresmoi.eu/
http://www.mitime.org/width/
                                            16

More Related Content

What's hot

Unlocking the value of health data - Presentation at the Congressional Lunche...
Unlocking the value of health data - Presentation at the Congressional Lunche...Unlocking the value of health data - Presentation at the Congressional Lunche...
Unlocking the value of health data - Presentation at the Congressional Lunche...Sandeep Purao
 
Research Methods for Computational Statistics
Research Methods for Computational StatisticsResearch Methods for Computational Statistics
Research Methods for Computational StatisticsSetia Pramana
 
Student performance prediction batch 15 cse DUET.AC.BD
Student performance prediction batch 15 cse DUET.AC.BDStudent performance prediction batch 15 cse DUET.AC.BD
Student performance prediction batch 15 cse DUET.AC.BDMd.Golam Muktadir
 
Data Management Lab: Data mapping exercise instructions
Data Management Lab: Data mapping exercise instructionsData Management Lab: Data mapping exercise instructions
Data Management Lab: Data mapping exercise instructionsIUPUI
 

What's hot (6)

Unlocking the value of health data - Presentation at the Congressional Lunche...
Unlocking the value of health data - Presentation at the Congressional Lunche...Unlocking the value of health data - Presentation at the Congressional Lunche...
Unlocking the value of health data - Presentation at the Congressional Lunche...
 
Research Methods for Computational Statistics
Research Methods for Computational StatisticsResearch Methods for Computational Statistics
Research Methods for Computational Statistics
 
Data Extraction
Data ExtractionData Extraction
Data Extraction
 
Student performance prediction batch 15 cse DUET.AC.BD
Student performance prediction batch 15 cse DUET.AC.BDStudent performance prediction batch 15 cse DUET.AC.BD
Student performance prediction batch 15 cse DUET.AC.BD
 
Medical Writing
Medical WritingMedical Writing
Medical Writing
 
Data Management Lab: Data mapping exercise instructions
Data Management Lab: Data mapping exercise instructionsData Management Lab: Data mapping exercise instructions
Data Management Lab: Data mapping exercise instructions
 

Viewers also liked

The Value of Visualization by @columnfive
The Value of Visualization by @columnfiveThe Value of Visualization by @columnfive
The Value of Visualization by @columnfiveColumn Five
 
Kyeongan Kwon - PhD Dissertation 2016
Kyeongan Kwon - PhD Dissertation 2016Kyeongan Kwon - PhD Dissertation 2016
Kyeongan Kwon - PhD Dissertation 2016Karl Kwon, Ph.D.
 
Data Visualization for Journalists and Media
Data Visualization for Journalists and MediaData Visualization for Journalists and Media
Data Visualization for Journalists and MediaDavid Newbury
 
Storyboarding for Data Visualization Design
Storyboarding for Data Visualization DesignStoryboarding for Data Visualization Design
Storyboarding for Data Visualization Designspatialhistory
 
Data Visualization & Information Design: One Learner's Perspective
Data Visualization & Information Design: One Learner's PerspectiveData Visualization & Information Design: One Learner's Perspective
Data Visualization & Information Design: One Learner's PerspectiveSheila B. Robinson
 
Data Visualization 101: How to Design Charts and Graphs
Data Visualization 101: How to Design Charts and GraphsData Visualization 101: How to Design Charts and Graphs
Data Visualization 101: How to Design Charts and GraphsVisage
 
Social Media Reporting & Data Visualization Best Practices
Social Media Reporting & Data Visualization Best PracticesSocial Media Reporting & Data Visualization Best Practices
Social Media Reporting & Data Visualization Best PracticesBrian Honigman
 
Visual Data Representation Techniques Combining Art and Design
Visual Data Representation Techniques Combining Art and DesignVisual Data Representation Techniques Combining Art and Design
Visual Data Representation Techniques Combining Art and DesignLogo Design Guru
 

Viewers also liked (11)

The Value of Visualization by @columnfive
The Value of Visualization by @columnfiveThe Value of Visualization by @columnfive
The Value of Visualization by @columnfive
 
Agile Tour 2012 (Sierre) - Quelques pratiques du Management 3.0
Agile Tour 2012 (Sierre) - Quelques pratiques du Management 3.0Agile Tour 2012 (Sierre) - Quelques pratiques du Management 3.0
Agile Tour 2012 (Sierre) - Quelques pratiques du Management 3.0
 
Kyeongan Kwon - PhD Dissertation 2016
Kyeongan Kwon - PhD Dissertation 2016Kyeongan Kwon - PhD Dissertation 2016
Kyeongan Kwon - PhD Dissertation 2016
 
CALPACT Webinar: Using Infographics and Data Visualization
CALPACT Webinar:  Using Infographics and Data VisualizationCALPACT Webinar:  Using Infographics and Data Visualization
CALPACT Webinar: Using Infographics and Data Visualization
 
Data Visualization for Journalists and Media
Data Visualization for Journalists and MediaData Visualization for Journalists and Media
Data Visualization for Journalists and Media
 
Storyboarding for Data Visualization Design
Storyboarding for Data Visualization DesignStoryboarding for Data Visualization Design
Storyboarding for Data Visualization Design
 
Data Visualization & Information Design: One Learner's Perspective
Data Visualization & Information Design: One Learner's PerspectiveData Visualization & Information Design: One Learner's Perspective
Data Visualization & Information Design: One Learner's Perspective
 
A Framework for Semantic Business Process Management in E-Government
A Framework for Semantic Business Process Management in E-Government   A Framework for Semantic Business Process Management in E-Government
A Framework for Semantic Business Process Management in E-Government
 
Data Visualization 101: How to Design Charts and Graphs
Data Visualization 101: How to Design Charts and GraphsData Visualization 101: How to Design Charts and Graphs
Data Visualization 101: How to Design Charts and Graphs
 
Social Media Reporting & Data Visualization Best Practices
Social Media Reporting & Data Visualization Best PracticesSocial Media Reporting & Data Visualization Best Practices
Social Media Reporting & Data Visualization Best Practices
 
Visual Data Representation Techniques Combining Art and Design
Visual Data Representation Techniques Combining Art and DesignVisual Data Representation Techniques Combining Art and Design
Visual Data Representation Techniques Combining Art and Design
 

Similar to Determining Importance of Medical Images in Journal Articles

Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...Health Informatics New Zealand
 
Key Topics in Health Care Technology EvaluationThe amount of new i.docx
Key Topics in Health Care Technology EvaluationThe amount of new i.docxKey Topics in Health Care Technology EvaluationThe amount of new i.docx
Key Topics in Health Care Technology EvaluationThe amount of new i.docxsleeperfindley
 
To prepare for this Assignment· Review the article, Sources of
To prepare for this Assignment· Review the article, Sources ofTo prepare for this Assignment· Review the article, Sources of
To prepare for this Assignment· Review the article, Sources ofTakishaPeck109
 
Diversity and Depth: Implementing AI across many long tail domains
Diversity and Depth: Implementing AI across many long tail domainsDiversity and Depth: Implementing AI across many long tail domains
Diversity and Depth: Implementing AI across many long tail domainsPaul Groth
 
Week 5 Lab 3· If you choose to download the software from http.docx
Week 5 Lab 3· If you choose to download the software from http.docxWeek 5 Lab 3· If you choose to download the software from http.docx
Week 5 Lab 3· If you choose to download the software from http.docxcockekeshia
 
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
IJCER (www.ijceronline.com) International Journal of computational Engineerin...IJCER (www.ijceronline.com) International Journal of computational Engineerin...
IJCER (www.ijceronline.com) International Journal of computational Engineerin...ijceronline
 
Translating Evidence into Practice Data Collection Assignment.pdf
Translating Evidence into Practice Data Collection Assignment.pdfTranslating Evidence into Practice Data Collection Assignment.pdf
Translating Evidence into Practice Data Collection Assignment.pdfsdfghj21
 
C O N C E P T A N A L Y S I SClinical reasoning concept a.docx
C O N C E P T A N A L Y S I SClinical reasoning concept a.docxC O N C E P T A N A L Y S I SClinical reasoning concept a.docx
C O N C E P T A N A L Y S I SClinical reasoning concept a.docxclairbycraft
 
An investigation on combination methods
An investigation on combination methodsAn investigation on combination methods
An investigation on combination methodsAli HOSSEINZADEH VAHID
 
(1) Critique Template for a Qualitative StudyNURS 6052Week.docx
(1)  Critique Template for a Qualitative StudyNURS 6052Week.docx(1)  Critique Template for a Qualitative StudyNURS 6052Week.docx
(1) Critique Template for a Qualitative StudyNURS 6052Week.docxkatherncarlyle
 
The purpose of this initial paper is to briefly describe your sear.docx
The purpose of this initial paper is to briefly describe your sear.docxThe purpose of this initial paper is to briefly describe your sear.docx
The purpose of this initial paper is to briefly describe your sear.docxhelen23456789
 
Formulating a literature review
Formulating a literature reviewFormulating a literature review
Formulating a literature reviewLynn Hendricks
 
Walden ADHD Translating Evidence into Practice Data Collection Assignment.pdf
Walden ADHD Translating Evidence into Practice Data Collection Assignment.pdfWalden ADHD Translating Evidence into Practice Data Collection Assignment.pdf
Walden ADHD Translating Evidence into Practice Data Collection Assignment.pdfsdfghj21
 
and Practice in Nursing.docx
and Practice in Nursing.docxand Practice in Nursing.docx
and Practice in Nursing.docxwrite5
 
Big Data - To Explain or To Predict? Talk at U Toronto's Rotman School of Ma...
Big Data - To Explain or To Predict?  Talk at U Toronto's Rotman School of Ma...Big Data - To Explain or To Predict?  Talk at U Toronto's Rotman School of Ma...
Big Data - To Explain or To Predict? Talk at U Toronto's Rotman School of Ma...Galit Shmueli
 
Melissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AIMelissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AImelissadata
 

Similar to Determining Importance of Medical Images in Journal Articles (20)

Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
 
MVilla IUI 2012 Lisbon
MVilla IUI 2012 LisbonMVilla IUI 2012 Lisbon
MVilla IUI 2012 Lisbon
 
Devry nr500 entire course
Devry nr500 entire courseDevry nr500 entire course
Devry nr500 entire course
 
Systematic review
Systematic reviewSystematic review
Systematic review
 
Key Topics in Health Care Technology EvaluationThe amount of new i.docx
Key Topics in Health Care Technology EvaluationThe amount of new i.docxKey Topics in Health Care Technology EvaluationThe amount of new i.docx
Key Topics in Health Care Technology EvaluationThe amount of new i.docx
 
To prepare for this Assignment· Review the article, Sources of
To prepare for this Assignment· Review the article, Sources ofTo prepare for this Assignment· Review the article, Sources of
To prepare for this Assignment· Review the article, Sources of
 
Diversity and Depth: Implementing AI across many long tail domains
Diversity and Depth: Implementing AI across many long tail domainsDiversity and Depth: Implementing AI across many long tail domains
Diversity and Depth: Implementing AI across many long tail domains
 
Week 5 Lab 3· If you choose to download the software from http.docx
Week 5 Lab 3· If you choose to download the software from http.docxWeek 5 Lab 3· If you choose to download the software from http.docx
Week 5 Lab 3· If you choose to download the software from http.docx
 
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
IJCER (www.ijceronline.com) International Journal of computational Engineerin...IJCER (www.ijceronline.com) International Journal of computational Engineerin...
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
 
Translating Evidence into Practice Data Collection Assignment.pdf
Translating Evidence into Practice Data Collection Assignment.pdfTranslating Evidence into Practice Data Collection Assignment.pdf
Translating Evidence into Practice Data Collection Assignment.pdf
 
C O N C E P T A N A L Y S I SClinical reasoning concept a.docx
C O N C E P T A N A L Y S I SClinical reasoning concept a.docxC O N C E P T A N A L Y S I SClinical reasoning concept a.docx
C O N C E P T A N A L Y S I SClinical reasoning concept a.docx
 
An investigation on combination methods
An investigation on combination methodsAn investigation on combination methods
An investigation on combination methods
 
(1) Critique Template for a Qualitative StudyNURS 6052Week.docx
(1)  Critique Template for a Qualitative StudyNURS 6052Week.docx(1)  Critique Template for a Qualitative StudyNURS 6052Week.docx
(1) Critique Template for a Qualitative StudyNURS 6052Week.docx
 
The purpose of this initial paper is to briefly describe your sear.docx
The purpose of this initial paper is to briefly describe your sear.docxThe purpose of this initial paper is to briefly describe your sear.docx
The purpose of this initial paper is to briefly describe your sear.docx
 
Formulating a literature review
Formulating a literature reviewFormulating a literature review
Formulating a literature review
 
Walden ADHD Translating Evidence into Practice Data Collection Assignment.pdf
Walden ADHD Translating Evidence into Practice Data Collection Assignment.pdfWalden ADHD Translating Evidence into Practice Data Collection Assignment.pdf
Walden ADHD Translating Evidence into Practice Data Collection Assignment.pdf
 
and Practice in Nursing.docx
and Practice in Nursing.docxand Practice in Nursing.docx
and Practice in Nursing.docx
 
Sample Of Research Essay
Sample Of Research EssaySample Of Research Essay
Sample Of Research Essay
 
Big Data - To Explain or To Predict? Talk at U Toronto's Rotman School of Ma...
Big Data - To Explain or To Predict?  Talk at U Toronto's Rotman School of Ma...Big Data - To Explain or To Predict?  Talk at U Toronto's Rotman School of Ma...
Big Data - To Explain or To Predict? Talk at U Toronto's Rotman School of Ma...
 
Melissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AIMelissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AI
 

More from Institute of Information Systems (HES-SO)

Classification of noisy free-text prostate cancer pathology reports using nat...
Classification of noisy free-text prostate cancer pathology reports using nat...Classification of noisy free-text prostate cancer pathology reports using nat...
Classification of noisy free-text prostate cancer pathology reports using nat...Institute of Information Systems (HES-SO)
 
Machine learning assisted citation screening for Systematic Reviews - Anjani ...
Machine learning assisted citation screening for Systematic Reviews - Anjani ...Machine learning assisted citation screening for Systematic Reviews - Anjani ...
Machine learning assisted citation screening for Systematic Reviews - Anjani ...Institute of Information Systems (HES-SO)
 
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Institute of Information Systems (HES-SO)
 
Studying Public Medical Images from Open Access Literature and Social Network...
Studying Public Medical Images from Open Access Literature and Social Network...Studying Public Medical Images from Open Access Literature and Social Network...
Studying Public Medical Images from Open Access Literature and Social Network...Institute of Information Systems (HES-SO)
 
Risques opérationnels et le système de contrôle interne : les limites d’un te...
Risques opérationnels et le système de contrôle interne : les limites d’un te...Risques opérationnels et le système de contrôle interne : les limites d’un te...
Risques opérationnels et le système de contrôle interne : les limites d’un te...Institute of Information Systems (HES-SO)
 
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...Institute of Information Systems (HES-SO)
 
Le système de contrôle interne : Présentation générale, enjeux et méthodes
Le système de contrôle interne : Présentation générale, enjeux et méthodesLe système de contrôle interne : Présentation générale, enjeux et méthodes
Le système de contrôle interne : Présentation générale, enjeux et méthodesInstitute of Information Systems (HES-SO)
 
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...Institute of Information Systems (HES-SO)
 
NOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
NOSE: une approche Smart-City pour les zones périphériques et extra-urbainesNOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
NOSE: une approche Smart-City pour les zones périphériques et extra-urbainesInstitute of Information Systems (HES-SO)
 

More from Institute of Information Systems (HES-SO) (20)

MIE20232.pptx
MIE20232.pptxMIE20232.pptx
MIE20232.pptx
 
Classification of noisy free-text prostate cancer pathology reports using nat...
Classification of noisy free-text prostate cancer pathology reports using nat...Classification of noisy free-text prostate cancer pathology reports using nat...
Classification of noisy free-text prostate cancer pathology reports using nat...
 
Machine learning assisted citation screening for Systematic Reviews - Anjani ...
Machine learning assisted citation screening for Systematic Reviews - Anjani ...Machine learning assisted citation screening for Systematic Reviews - Anjani ...
Machine learning assisted citation screening for Systematic Reviews - Anjani ...
 
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
 
L'IoT dans les usines. Quels avantages ?
L'IoT dans les usines. Quels avantages ?L'IoT dans les usines. Quels avantages ?
L'IoT dans les usines. Quels avantages ?
 
Studying Public Medical Images from Open Access Literature and Social Network...
Studying Public Medical Images from Open Access Literature and Social Network...Studying Public Medical Images from Open Access Literature and Social Network...
Studying Public Medical Images from Open Access Literature and Social Network...
 
Risques opérationnels et le système de contrôle interne : les limites d’un te...
Risques opérationnels et le système de contrôle interne : les limites d’un te...Risques opérationnels et le système de contrôle interne : les limites d’un te...
Risques opérationnels et le système de contrôle interne : les limites d’un te...
 
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
 
Le système de contrôle interne : Présentation générale, enjeux et méthodes
Le système de contrôle interne : Présentation générale, enjeux et méthodesLe système de contrôle interne : Présentation générale, enjeux et méthodes
Le système de contrôle interne : Présentation générale, enjeux et méthodes
 
Crowdsourcing-based Mobile Application for Wheelchair Accessibility
Crowdsourcing-based Mobile Application for Wheelchair AccessibilityCrowdsourcing-based Mobile Application for Wheelchair Accessibility
Crowdsourcing-based Mobile Application for Wheelchair Accessibility
 
Quelle(s) valeur(s) pour le leadership stratégique ?
Quelle(s) valeur(s) pour le leadership stratégique ?Quelle(s) valeur(s) pour le leadership stratégique ?
Quelle(s) valeur(s) pour le leadership stratégique ?
 
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
 
Challenges in medical imaging and the VISCERAL model
Challenges in medical imaging and the VISCERAL modelChallenges in medical imaging and the VISCERAL model
Challenges in medical imaging and the VISCERAL model
 
NOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
NOSE: une approche Smart-City pour les zones périphériques et extra-urbainesNOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
NOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
 
Medical image analysis and big data evaluation infrastructures
Medical image analysis and big data evaluation infrastructuresMedical image analysis and big data evaluation infrastructures
Medical image analysis and big data evaluation infrastructures
 
Medical image analysis, retrieval and evaluation infrastructures
Medical image analysis, retrieval and evaluation infrastructuresMedical image analysis, retrieval and evaluation infrastructures
Medical image analysis, retrieval and evaluation infrastructures
 
How to detect soft falls on devices
How to detect soft falls on devicesHow to detect soft falls on devices
How to detect soft falls on devices
 
FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS
FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSISFUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS
FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS
 
MOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLS
MOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLSMOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLS
MOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLS
 
Enhanced Students Laboratory The GET project
Enhanced Students Laboratory The GET projectEnhanced Students Laboratory The GET project
Enhanced Students Laboratory The GET project
 

Recently uploaded

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 

Recently uploaded (20)

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 

Determining Importance of Medical Images in Journal Articles

  • 1. Determining the importance of figures in journal articles to find representative images Henning Müller, Antonio Foncubierta-Rodriguez, Chang Lin, Ivan Eggel
  • 2. Introduction  Images are essential for diagnosis and treatment planning Diversity of modalities and protocols is growing  Medical images represent largest amount of medical information produced  Medical literature carries much of the medical knowledge Images are essential for medical articles  Images can help understand content of an article quickly  Much quicker than reading a text to determine relevance Search in the literature is frequent 2
  • 3. Motivation  Image retrieval and classification make medical visual content accessible for search Images from PACS or from the medical literature (as in the ImageCLEF image retrieval campaign)  When searching the image needs to be put into context Text and visual data to understand context Often more than a single image  Identification of important figures Use of visual content to determine relevance quickly Limit search to important figures 3
  • 4. Examples for using figure importance  Interfaces and visualization require choices  Mobile information search has small interface  Creation of ground truth and rules for importance ranking to select in the best way  Often the first N are taken 4
  • 5. Database used  ImageCLEF 2012 dataset  ~75,000 articles of PubMed Central, ~300,000 images  Subset was used  ~800 articles contained more than 8 images  Final choice of 50 articles containing 641 diagnostic images 5
  • 6. Types of figure importance  Five categories – sorted by importance Most important figure – 1 figure that best describes the article Essential figures – 0-3 figures Important figures – 0-3 figures Little importance figures – 0+ figures Totally unimportant figures – 0+ figures 6
  • 7. User test  User test participant profile: 45-year-old physician, surgeon Working with medical images on a daily basis  User test scenario: Searching for relevant articles on the topic of the text  Which figures contain the most important information about the article?  Requires to read and understand article  All systematic findings or rules for the ranking were written down 7
  • 8. Results  All 50 articles were analyzed and all 641 images manually ranked by importance Usually over 1 hour per article  Rules and findings were noted and discussed  Statistics on figure importance and places in the text were made  Compound figures were difficult to judge Information content is high as they contain different subfigures and links between them Compound figure separation new sub task in ImageCLEF 8
  • 9. Rules for determining importance  Diagnostic images are more important than graphs/system overviews  Diversity in the results set is important Not several times the same modality if others exist  Size/resolution matters Image quality is importance  Position in article Most importance in the conclusions, then results  Common vs. rare modality Common modalities are often easier to understand 9
  • 10. More rules for the importance  Magnification Highest magnification most important  MRI for soft tissue diseases, CT for bone related ones Modality adapted to the disease, can be derived from RadLex terms  Compound vs. other Compound figure contains much information  In non-diagnostic images, statistical graphs most important 10
  • 11. Results  Distribution of figure importance 11
  • 12. Results  Importance vs. position of the figure in the article based on relative position in the text 12
  • 13. Example for figure importance 13
  • 14. Conclusions  Relative importance of visual content in scientific articles is important for many applications First study to investigate this in a user test  Can improve IR system performance Rank images based on position or remove some images to search a smaller sub space  Can improve visualization For small interfaces such as mobile phones and tablets To combine text and images in an optimal way 14
  • 15. Future work  Repeat the same test with a second person to measure inter-rater disagreement Quantify the subjectivity of the task  Model the rules and optimize an automatic selection strategy Modality classification to filter out unwanted images Clustering and use of cluster centers to increase diversity Find the place of figures in the text and rank based on this 15
  • 16. Thank you for your attention! Further information: henning.mueller@hevs.ch http://medgift.hevs.ch/ http://www.khresmoi.eu/ http://www.mitime.org/width/ 16