Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Image understanding and artificial intelligence

154 views

Published on

Thèse "Image understanding and artificial intelligence" par Isabelle BLOCH, lors de la journée Futur & Ruptures du 31 janvier 2019. Une journée scientifique pour présenter l’ensemble des travaux de thèses aboutis portant sur des thématiques prospectives du programme de l’IMT.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Image understanding and artificial intelligence

  1. 1. Image Understanding and Artificial Intelligence Isabelle Bloch LTCI, T´el´ecom ParisTech, Universit´e Paris-Saclay isabelle.bloch@telecom-paristech.fr 2019 I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 1 / 22
  2. 2. What is image understanding? From the 1960’s to today: Miller and Shaw (1968): survey of linguistic methods for picture processing, defined as analysis and generation of pictures by computers, with or without human interaction. Clowes (1971): linguistic approach for picture interpretation (pattern description language). Reiter (1989): interpretation = logical model of sets of axioms. Ralescu (1995): image understanding = verbal description of the image contents. Bateman (2010): needs for a semantic layer for spatial language. Xu et al. (2014): image interpretation = assigning labels or semantics representations to regions of a scene. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 2 / 22
  3. 3. What is image understanding? Here: Beyond individual object recognition. Objects in their context, spatial arrangement. Global scene interpretation. Semantics extraction. Providing verbal descriptions of image content. Dynamic scenes: recognition and description of actions, gestures, emotions.. Inference, higher level reasoning. Important role of Artificial Intelligence. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 2 / 22
  4. 4. A few examples A lot of work on image annotation: object → several objects → scene. Magritte, 1928 I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
  5. 5. A few examples A lot of work on image annotation: object → several objects → scene. Millet et al., 2005 (rules, spatial reasoning...) Region without spatial relations with spatial relations 1 sky sky 2 grass tree 3 tree tree 4 building building I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
  6. 6. A few examples A lot of work on image annotation: object → several objects → scene. Venus? Thanks to H. Maˆıtre I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
  7. 7. A few examples A lot of work on image annotation: object → several objects → scene. “Show and tell”: Vinyals et al., 2015 (convolutional neural networks, deep learning) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
  8. 8. A few examples A lot of work on image annotation: object → several objects → scene. Kulkarni et al., 2013 I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
  9. 9. Skubic et al., 2003 (fuzzy modeling of spatial relations) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 4 / 22
  10. 10. Ogiela et al. 2002, Trzupek et al. 2010 (graphs and grammars) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 5 / 22
  11. 11. An abnormal structure is present in the brain. A peripheral non-enhanced tumor is present in the left hemisphere. Atif et al., 2014 (spatial reasoning, abduction) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 6 / 22
  12. 12. Morimitsu et al., 2015 (graphs, Bayesian tracking, hidden Markov models) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 7 / 22
  13. 13. Data vs. knowledge Is everything in the data? Powerful methods and impressive results. Accessibility of data. Size and number of data. Cost of learning. Importance of knowledge. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 8 / 22
  14. 14. Imperfect information, multiple nature of information I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 9 / 22
  15. 15. Models for image understanding From models to interpretation Develop mathematical models to represent knowledge (context, expert, spatial organization...), information contained in images (geometry, statistics, shape, appearance...), and to combine them (fusion process), ⇒ operational and efficient algorithms for image understanding Semantic gap. Knowlegde representation and reasoning. Pathological or unexpected cases. Conversely: from images to models example: individual anatomical models, virtual patient. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 10 / 22
  16. 16. CdR LVR ThR PuR GPR ClR V3 CC icR Knowledge Graphs [COLLIOT, DERUYVER, ...] Stochastic grammars [ZHU, MUMFORD,..] Ontologies [DAMERON,HU,...] Formal representation Data Reasoning Decision I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 11 / 22
  17. 17. Examples, within various collaborations with hospitals and companies: I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 12 / 22
  18. 18. Symbolic and structural representations and reasoning Morphology Duality Mathematical Adjunction Uncertainty modeling Spatial relations Knowledge representation Reasoning (revision, fusion, abduction, spatial reasoning) Preference modeling Image understanding Math−Music ... Set, functions Images Fuzzy sets Graphs Hypergraphs FCA Logics Satisfaction systems CNN Image processing and analysis Structural representations (data and knowledge) Lattices Learning I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 13 / 22
  19. 19. Modeling fuzzy spatial relations Mathematical models: combining fuzzy sets and mathematical morphology. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 14 / 22
  20. 20. With Alessandro Delmonte and Pietro Gori I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 15 / 22
  21. 21. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 16 / 22
  22. 22. Graph-based reasoning and constraint satisfaction problem, with Jamal Atif, Geoffroy Fouquier and Olivier Nempont I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 17 / 22
  23. 23. Conceptual graphs and complex spatial relations, with Carolina Vanegas I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 18 / 22
  24. 24. Image interpretation as an abduction problem, with Jamal Atif and Yifan Yang K |= (γ → ϕ) Finding the “best” explanation to the observations taking into account expert knowledge. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 19 / 22
  25. 25. With Yongchao Xu, Thierry G´eraud, Hadrien Bertrand, Roberto Ardon, Matthieu Perrot Specialized Layers Fine feature maps Coarse feature maps Base network architecture n 0 n-1R nG n+1B N SegmentationInput Original 3D volume I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 20 / 22
  26. 26. Adult Child Ref. segmentation 3D U-Net Transfer learning I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 21 / 22
  27. 27. Research in Artificial Intelligence at LTCI - IMAGES Team Covers several topics in the general AI cartography (see #FranceIA): Machine learning. Pattern recognition. Interaction. Knowledge representation. Decision and uncertainty management. Reasoning. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 22 / 22

×