Explaining
                          Semantic Search Results of
                          Medical Images in MEDICO
       ...
Reality check
Mittwoch, 15. Juli 2009
„Trust me. I know
                          what I am doing!“
Mittwoch, 15. Juli 2009
„Trust me. I know
                          what I am doing!“
Mittwoch, 15. Juli 2009
Goal of Medico Project
                          Development of
                          • intelligent
                  ...
Goal of Medico Project
                          Development of
                          • intelligent
                  ...
Goal of Medico Project
                          Development of
                          • intelligent
                  ...
RadSem
                      •   Tool to support medical doctors (esp. radiologists) in
                          annotati...
Intended Users of
                               RadSem
                      • Medical doctors
                      • Me...
MEDICO System Architecture




                                7
Mittwoch, 15. Juli 2009
MEDICO System Architecture




                                7
Mittwoch, 15. Juli 2009
MEDICO System Architecture




                                7
Mittwoch, 15. Juli 2009
MEDICO Ontology Hierarchy




                                      8
Mittwoch, 15. Juli 2009
MEDICO Ontology Hierarchy




                                      8
Mittwoch, 15. Juli 2009
Foundational Model of
                         Anatomy FMA
                      •   developed and maintained by Structura...
ICD-10 in OWL

                      •   Problem: No disease terminology available in OWL
                      •   Establ...
Example
  annotation



       • FMA
       • ICD 10


Mittwoch, 15. Juli 2009
Example
  annotation              Region of
                          Interest




       • FMA
       • ICD 10


Mittwoch...
Example
  annotation              Region of
                          Interest




       • FMA
       • ICD 10


Mittwoch...
Example
  annotation              Region of
                          Interest




       • FMA
       • ICD 10


Mittwoch...
Example
  annotation              Region of
                          Interest




       • FMA
       • ICD 10


Mittwoch...
Explainer

                            User Interface

                                             Originator




       ...
Explainer

                            User Interface

                                             Originator


         ...
Explanation
                                                               knowledge

                                    ...
Explainer

                            User Interface

                                             Originator




       ...
Explainer

                            User Interface

                                              Originator
          ...
• Query
                                                 expansion
                                                 with o...
Motivations for
                   explanations in RadSem
                      •   Test whether the Search Engine works
 ...
Motivations for
                   explanations in RadSem
                      •   Test whether the Search Engine works
 ...
Motivations for
                   explanations in RadSem
                      •   Test whether the Search Engine works
 ...
Motivations for
                   explanations in RadSem

                      •   Help users to improve their search

 ...
Motivations for
                   explanations in RadSem

                      •   Help users to improve their search
  ...
Motivations for
                   explanations in RadSem

                      •   Help users to improve their search
  ...
What are
                          explanations?




Mittwoch, 15. Juli 2009
What are
                          explanations?
                 Explanations are answers
                       to quest...
When are questions
                being asked?




Mittwoch, 15. Juli 2009
When are questions
                being asked?
                          Whenever expectations
                          ...
Explanation goals
                   •      Transparency             How did the system reach an answer?

                ...
When are explanations
                     good explanations?
                   • Short and easy to overlook
            ...
When are explanations
                     good explanations?
                   • Short and easy to overlook
            ...
Kinds of explanations

                      • Action explanations and justifications:
                          „How do se...
Action explanations
                   • Action explanations explain the activities of
                          the respe...
Why-explanations

               •          Why-explanations provide causes or justifications for
                         ...
Concept Explanations
               •          The goal of concept explanations is to build links between
                ...
Explainer

                            User Interface

                                              Originator
          ...
• Dijkstra
                                              algorithm
                                              estimates...
Example search




Mittwoch, 15. Juli 2009
Exploration interface




Mittwoch, 15. Juli 2009
Exploration interface




Mittwoch, 15. Juli 2009
„Bridge concepts“




Mittwoch, 15. Juli 2009
„Bridge concepts“




Mittwoch, 15. Juli 2009
FMA
  problem


       • Same
               concept,
               different
               labels




Mittwoch, 15. Jul...
FMA
  problem


       • Same
               concept,
               different
               labels




Mittwoch, 15. Jul...
Label problems of FMA




Mittwoch, 15. Juli 2009
User experiment wrt
                   explanations in RadSem
                      •   Test whether the Search Engine wor...
User experiment wrt
                   explanations in RadSem
                      •    Test whether the Search Engine wo...
Future Work
                      • Selection of proper labels wrt different
                          user groups
       ...
Take home messages




Mittwoch, 15. Juli 2009
Take home messages
              • RadSem is a complex annotation and search tool.




Mittwoch, 15. Juli 2009
Take home messages
              • RadSem is a complex annotation and search tool.
              • Goals and kinds of expl...
Take home messages
              • RadSem is a complex annotation and search tool.
              • Goals and kinds of expl...
Take home messages
              • RadSem is a complex annotation and search tool.
              • Goals and kinds of expl...
Take home messages
              • RadSem is a complex annotation and search tool.
              • Goals and kinds of expl...
Thank you!

                    Explaining
                    Semantic Search Results of
                    Medical Imag...
Upcoming SlideShare
Loading in …5
×

Explaining Semantic Search Results of Medical Images in MEDICO

2,226 views

Published on

Abstract. The research project MEDICO aims at developing an intelligent, robust and scalable semantic search engine for medical images and is designated for different kinds of users, such as medical doctors, medical IT professionals, patients and citizens, and policy makers. Since semantic search results are not always self-explanatory various kinds of explanation are necessary to satisfy different user goals. Our prime concern is to provide intuitive justifications for inexperienced users in the medical domain using semantic networks as form of depiction. In addition, we provide several interaction styles enabling a deeper insight into the medical knowledge.

Published in: Technology, Health & Medicine
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,226
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
42
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Explaining Semantic Search Results of Medical Images in MEDICO

  1. 1. Explaining Semantic Search Results of Medical Images in MEDICO Björn Forcher, Manuel Möller, Michael Sintek, and Thomas Roth-Berghofer Mittwoch, 15. Juli 2009
  2. 2. Reality check Mittwoch, 15. Juli 2009
  3. 3. „Trust me. I know what I am doing!“ Mittwoch, 15. Juli 2009
  4. 4. „Trust me. I know what I am doing!“ Mittwoch, 15. Juli 2009
  5. 5. Goal of Medico Project Development of • intelligent • robust and • scalable semantic search engine for medical images Mittwoch, 15. Juli 2009
  6. 6. Goal of Medico Project Development of • intelligent • robust and • scalable semantic search engine for medical images Mittwoch, 15. Juli 2009
  7. 7. Goal of Medico Project Development of • intelligent • robust and • scalable semantic search engine for medical images Mittwoch, 15. Juli 2009
  8. 8. RadSem • Tool to support medical doctors (esp. radiologists) in annotating and searching for medical images (and text) • Part of the MEDICO project (funded by BMWi in the research programme THESEUS) • Developed together with medical experts (who have to use the tool to annotate real images) 5 Mittwoch, 15. Juli 2009
  9. 9. Intended Users of RadSem • Medical doctors • Medical IT professionals • Patients and citizens • Policy makers Mittwoch, 15. Juli 2009
  10. 10. MEDICO System Architecture 7 Mittwoch, 15. Juli 2009
  11. 11. MEDICO System Architecture 7 Mittwoch, 15. Juli 2009
  12. 12. MEDICO System Architecture 7 Mittwoch, 15. Juli 2009
  13. 13. MEDICO Ontology Hierarchy 8 Mittwoch, 15. Juli 2009
  14. 14. MEDICO Ontology Hierarchy 8 Mittwoch, 15. Juli 2009
  15. 15. Foundational Model of Anatomy FMA • developed and maintained by Structural Informatics Group at University of Washington • contains more than 70.000 anatomical entities (classes) • more than 1.5 million relations between the entities • most comprehensive human ontology 9 Mittwoch, 15. Juli 2009
  16. 16. ICD-10 in OWL • Problem: No disease terminology available in OWL • Established standard: International Classification of Diseases (WHO), but only available in semi- structured formats • Approach: Crawler for online version of ICD-10 generates light-weight OWL ontology 10 Mittwoch, 15. Juli 2009
  17. 17. Example annotation • FMA • ICD 10 Mittwoch, 15. Juli 2009
  18. 18. Example annotation Region of Interest • FMA • ICD 10 Mittwoch, 15. Juli 2009
  19. 19. Example annotation Region of Interest • FMA • ICD 10 Mittwoch, 15. Juli 2009
  20. 20. Example annotation Region of Interest • FMA • ICD 10 Mittwoch, 15. Juli 2009
  21. 21. Example annotation Region of Interest • FMA • ICD 10 Mittwoch, 15. Juli 2009
  22. 22. Explainer User Interface Originator Basic explanation scenario Mittwoch, 15. Juli 2009
  23. 23. Explainer User Interface Originator Problem solving Basic explanation scenario knowledge Mittwoch, 15. Juli 2009
  24. 24. Explanation knowledge Explainer User Interface Originator Problem solving Basic explanation scenario knowledge Mittwoch, 15. Juli 2009
  25. 25. Explainer User Interface Originator Basic explanation scenario Mittwoch, 15. Juli 2009
  26. 26. Explainer User Interface Originator Semantic Search Basic explanation scenario Mittwoch, 15. Juli 2009
  27. 27. • Query expansion with ontology concepts • Count path Explainer length from search to User Interface found concept Originator Semantic Search Basic explanation scenario Mittwoch, 15. Juli 2009
  28. 28. Motivations for explanations in RadSem • Test whether the Search Engine works correctly • Test whether the ontologies are correctly modelled • Learn about the medical domain • Justify results in order to increase trust Mittwoch, 15. Juli 2009
  29. 29. Motivations for explanations in RadSem • Test whether the Search Engine works correctly Medical IT • Test whether the ontologies are professionals correctly modelled • Learn about the medical domain • Justify results in order to increase trust Mittwoch, 15. Juli 2009
  30. 30. Motivations for explanations in RadSem • Test whether the Search Engine works correctly Medical IT • Test whether the ontologies are professionals correctly modelled • Learn about the medical domain Patients and • Justify results in order to increase citizens trust Mittwoch, 15. Juli 2009
  31. 31. Motivations for explanations in RadSem • Help users to improve their search • Activate passive knowledge • Users learn how to use the engine concerning ontologies Mittwoch, 15. Juli 2009
  32. 32. Motivations for explanations in RadSem • Help users to improve their search Medical • Activate passive knowledge doctors • Users learn how to use the engine concerning ontologies Mittwoch, 15. Juli 2009
  33. 33. Motivations for explanations in RadSem • Help users to improve their search Medical • Activate passive knowledge doctors • Users learn how to use the engine Patients and concerning ontologies citizens Mittwoch, 15. Juli 2009
  34. 34. What are explanations? Mittwoch, 15. Juli 2009
  35. 35. What are explanations? Explanations are answers to questions. Mittwoch, 15. Juli 2009
  36. 36. When are questions being asked? Mittwoch, 15. Juli 2009
  37. 37. When are questions being asked? Whenever expectations are not met. Mittwoch, 15. Juli 2009
  38. 38. Explanation goals • Transparency How did the system reach an answer? • Justification Why is the answer a good answer? • Relevance Why is the question relevant? • Conceptualisation What is the meaning of a concept? • Learning Teach the user about the given domain. Sørmo, F., Cassens, J., Aamodt, A.: Explanation in Case-Based Reasoning – Perspectives and Goals, 2005. Mittwoch, 15. Juli 2009
  39. 39. When are explanations good explanations? • Short and easy to overlook • Innovative • Relevant • Convincing • Different perspectives and follow-up questions • Natural W. R. Swartout and J. D. Moore. Explanation in second generation expert systems. In J. David, J. Krivine, and R. Simmons, editors, Second Generation Expert Systems, pages 543–585, Berlin, 1993. Springer Verlag. Mittwoch, 15. Juli 2009
  40. 40. When are explanations good explanations? • Short and easy to overlook • Innovative • Relevant • Convincing • Different perspectives and follow-up questions • Natural W. R. Swartout and J. D. Moore. Explanation in second generation expert systems. In J. David, J. Krivine, and R. Simmons, editors, Second Generation Expert Systems, pages 543–585, Berlin, 1993. Springer Verlag. Mittwoch, 15. Juli 2009
  41. 41. Kinds of explanations • Action explanations and justifications: „How do search concepts relate to found concepts?“ • Concept explanations Mittwoch, 15. Juli 2009
  42. 42. Action explanations • Action explanations explain the activities of the respective system (originator). Action explanations: “Why was this seat post selected?” – “For the given price, only one other seat post was available. But this was too short. • In RadSem: Reconstructive explanations based on search and found concepts. Mittwoch, 15. Juli 2009
  43. 43. Why-explanations • Why-explanations provide causes or justifications for facts or events. • Examples: • Justification: “Why does the universe expand?” – “Because we can observe a red shift of the light emitted by other galaxies.” • Cause: “Because the whole matter was concentrated at one point of the universe and because the whole matter moves away from each other Mittwoch, 15. Juli 2009
  44. 44. Concept Explanations • The goal of concept explanations is to build links between unknown and known concepts. • Variations: • Definition: “What is a bicycle?” – “A bicycle is a land vehicle with two wheels in line. Bicycles are a form of human powered vehicle.” • Functional mapping: “What is a bicycle?” – “A bicycle serves as a means of transport.” • Prototypical usage of individual things or actions: “What is a bicycle?” – “The thing, that man over there just crashed with.” • … Mittwoch, 15. Juli 2009
  45. 45. Explainer User Interface Originator Semantic Search Basic explanation scenario Mittwoch, 15. Juli 2009
  46. 46. • Dijkstra algorithm estimates semantic search Explainer User Interface Originator Semantic Search Basic explanation scenario Mittwoch, 15. Juli 2009
  47. 47. Example search Mittwoch, 15. Juli 2009
  48. 48. Exploration interface Mittwoch, 15. Juli 2009
  49. 49. Exploration interface Mittwoch, 15. Juli 2009
  50. 50. „Bridge concepts“ Mittwoch, 15. Juli 2009
  51. 51. „Bridge concepts“ Mittwoch, 15. Juli 2009
  52. 52. FMA problem • Same concept, different labels Mittwoch, 15. Juli 2009
  53. 53. FMA problem • Same concept, different labels Mittwoch, 15. Juli 2009
  54. 54. Label problems of FMA Mittwoch, 15. Juli 2009
  55. 55. User experiment wrt explanations in RadSem • Test whether the Search Engine works correctly Medical IT • Test whether the ontologies are professionals correctly modelled • Learn about the medical domain Patients and • Justify results in order to increase citizens trust Mittwoch, 15. Juli 2009
  56. 56. User experiment wrt explanations in RadSem • Test whether the Search Engine works correctly Medical IT • Test whether the ontologies are professionals correctly modelled • Learn about the medical domain Patients and • Justify results in order to increase citizens trust → Results supported our motivations for providing explanations. Mittwoch, 15. Juli 2009
  57. 57. Future Work • Selection of proper labels wrt different user groups • Search for alternative paths • Exploration of paths • Tailoring of paths • Dictionary for lexical concepts • Links to Wikipedia Mittwoch, 15. Juli 2009
  58. 58. Take home messages Mittwoch, 15. Juli 2009
  59. 59. Take home messages • RadSem is a complex annotation and search tool. Mittwoch, 15. Juli 2009
  60. 60. Take home messages • RadSem is a complex annotation and search tool. • Goals and kinds of explanations are a useful tool in designing a software system in an explanation-aware manner. Mittwoch, 15. Juli 2009
  61. 61. Take home messages • RadSem is a complex annotation and search tool. • Goals and kinds of explanations are a useful tool in designing a software system in an explanation-aware manner. Explainer • Basic explanation scenario helps User identify communication partners Originator Mittwoch, 15. Juli 2009
  62. 62. Take home messages • RadSem is a complex annotation and search tool. • Goals and kinds of explanations are a useful tool in designing a software system in an explanation-aware manner. Explainer • Basic explanation scenario helps User identify communication partners Originator • Exploration interface with concept explanations support domain understanding. Mittwoch, 15. Juli 2009
  63. 63. Take home messages • RadSem is a complex annotation and search tool. • Goals and kinds of explanations are a useful tool in designing a software system in an explanation-aware manner. Explainer • Basic explanation scenario helps User identify communication partners Originator • Exploration interface with concept explanations support domain understanding. • Justification interface provides action explanations, which counteract encapsulation and information hiding. Mittwoch, 15. Juli 2009
  64. 64. Thank you! Explaining Semantic Search Results of Medical Images in MEDICO Thomas Roth-Berghofer Senior researcher, trb@dfki.de German Research Centre for Artificial Intelligence DFKI GmbH Mittwoch, 15. Juli 2009

×