1. The document discusses principles of human cognition that can be utilized for automated image analysis, including context-driven and object-oriented thinking. 2. It describes a Cognitive Network Language (CNL) that allows programming through multiple choices, natural formulation, and directed local processing to evolve object hierarchies. 3. The document provides examples of context-driven image segmentation and analysis in various domains like cell biology, medical imaging, and earth observation.

1. Principles of Human Cognition Utilized for Automated Image Analysis Gerd Binnig 3.Nov.2009

2. I think therefore I am

3. Context Driven Image Analysis Context driven through CNL Object oriented Knowledge based context ?? YES

4. Which Object ?

5. Which Object ?

6. Context

7. Context

8. Object Oriented and Knowledge Based Thinking We have abstract knowledge about classes of objects and their relations Fork! Knife! Plate between

9. Context Objects – From the Concrete to the Vague Objects 1. CONTEXT Objects 2. CONTEXT Object

10. Context Navigation Image Plate ? ? ? ? Fork Handle Knife Context navigation - most important Handle Blade +1 0 +1 0 0

11. Context Navigation Context slowly builds up The easy and well defined “objects” first Image Plate ? ? ? ? Fork Handle Knife Handle

12. Only vague objects ?

13. Network of Objects The objects are undefined The relations are concrete

14. Cognition Network Language - CNL Context driven Knowledge based Object based Local Script based

15. Process Hierarchy programming by multiple choice . Knowledge Hierarchy natural formulation . Navigation for directed local processing . Object Hierarchy evolves in the course of processing The Elements of CNL MRS, Chessboard, Merge, Classification, .. Fork! Knife! Plate between

16. Context driven Knowledge based Object based Object-Pixel unification Local Script based Multi-Image capability Up to 4 Dimensions The Elements of CNL – Version 8

17. Process Hierarchy programming by multiple choice . Knowledge Hierarchy natural formulation . Navigation for directed local processing . Object Hierarchy evolves in the course of processing The Elements of CNL – V8 MRS, Chessboard, Merge, Classification, .. Region O + P

18. Real Space / Size of Objects www.definiens.com www.definiens.com 1nm 1  m 1mm 10 100 10 100 0.1 100 1m 10 1Km Atom transistor Organ Person Ship Car House City Cell Forest m=meter Traveling through the Dimensions of Space 100

19. Phase Contrast Mic. Cells 3D-Confocal Microscopy Cell biology 3D-Confocal Microscopy Tissue Molecular Pathology 3D PET/CT Small animal Biological Research and Drug Development CT Mouse High Content Sreening Cells Proliferation index Tissue Cancer Biomarker Tissue

20. Biopsy Tissue Clinical Applications Serum Cells X-Ray Mammography CT Organs MRI low res. Organs CT Head/Neck MRI Ventricles CT Lymph Nodes

21. Context Driven Processing – Cell Cultures -1 +1 3. Micro- Nucleus Context Objects : Nuclei Cells 1. Nuclei 2. Cells Image

22. Context Objects – Blue, and Red Areas; Nucleus, Cell Blue stain red stain Different stains – different contrasts Blue area Red area -1 -1 Nucleus Cell Membrane +1 -1 Nuc. Cell Membrane +1 -1 Context Driven Processing – Tissue

23. Spinal Cord +1 Context Driven Processing – CT Spine Context Objects : Spinal cord Liver Kidney Spleen Spine

24. Solution: Anatomical Context Anatomy segmentation Context-free lymph node segmentation produces many false positives Reduced false positive rate + =

26. Hannover – University – Detecting HOUSES

27. Screenshots from Internet (Google)

28. Hannover – University – Detecting HOUSES What is easy and well defined? ..and can be used as context?

29. Meadow Potential shadow of tree 0 Context Driven Processing – Earth Observation 0 tree Direction = -55 ° Image 1. 2. 3. 4. 5.

30. Context Driven Processing – Earth Observation

31. Segmentation Result from Street-Model (also centerline)

32. Segmentation Result from Image – Houses, Trees, Roads and Meadows

33. Centerlines of Roads Imprinted into Segmentation Result

34. Conflicts – Streets Run Through Houses

35. Finding Conflicts Automatically – in Blue

36. Demo

37. Different kind of context object

38. 8:35 Umtata - South Africa 7604 x 4660 pixels

39. 8:35 Multi-Resolution Segmentation

40. 10:00 + Merge

41. 4:30 Multi-Resolution Segmentation on Half Resolution

42. 3:45 Pixel-Based

43. Major Roads Within and Outside the City (South Africa)

44. Major Roads Within and Outside the City (South Africa)

45. Rural Road Network near Mvezo, South Africa

46. Munich - 100 MPixels

47. 32:00

48. 34:00

49. 16:00

50. 3:01 Half Resolution

51. Different kind of context object

52. Local Processing

53. Local Processing

54. Local Processing

55. Yield in Case of Dependencies Context objects need to be very reliable Chance for success: 0.95*0.95*0.95*0.95*0.95*0.95 = 0.73 Probabilities multiply

59. From GIS to GIN (From geographic Information Systems to a gigantic Geographic Information Network)

60. From GIS to GIN A network of organizations, individuals, and autonomous machines

61. From GIS to GIN Data provider Government Institution University Satellite Service provider Airplane sensor sensors people people people Drone (Internet) sensor Lidar sensor sensor The Emerging Network Relatively new (in red): New sensors, internet service providers, contribution of private individuals networking of sensors and their data, sensors on people, automated data creation

62. GIN with Intelligent Processing and Autonomous Machines (IP and AM) Data provider Government Institution University Satellite Airplane sensor sensor sensor sensor people people people sensors Drone Service provider (Internet) IP AM warning Lidar IP IP, AM IP Distributed sensors AM Company, Organization

63. Autonomous Machines ( AM ) Automatic data creation Unmanned vehicles satellites, drones stationary cameras Automatic data analysis Definiens- nothing else

64. Statement: automated data analysis is possible Analysis gets better and better Analysis gets simpler and simpler Combination of different data

65. Statement: automated data analysis is possible Analysis gets better and better XD Analysis gets simpler and simpler Combination of different data

66. Statement: automated data analysis is possible Analysis gets better and better XD Analysis gets simpler and simpler Combination of different data

67. Example for Simplicity: Combination of Infrared and Lidar Infrared=vegetation

68. Digital Surface Model (DSM) Lidar: Elevation=vegetation or building

69. Classification of Aerial Image with DSM (Buildings and Vegetation) Buildings=Elevation-Vegetation

70. Intelligent automation is possible Consequences? … not only for image analysis

71. From GIS to GIN (geographic information network) A network of organizations, individuals, and autonomous machines

72. More data New types of data More automated data generation More private use of geographical information Private contribution More networking of different data More networking of organizations and people The future context for automated image analysis

73. From GIS to GIN (Geographic Information Network) Data provider Government Institution University Satellite Service provider Airplane sensor sensor people people people Drone (Internet) sensor Lidar sensor sensor

74. More data New types of data More automated data generation More private use of geographical information Private contribution More networking of different data More networking of organizations and people Automated intelligent data processing Automated data and event communication The future context for automated image analysis

75. GIN Plus Intelligent Machines (IM) Data provider Government Institution University Satellite Airplane sensor sensor sensor sensor people people people sensor Drone Service provider (Internet) IM IM warning Lidar

76. GIN Plus IM Plus Pervasive Computing Data provider Government Institution University Satellite Airplane sensor sensor sensor sensor people people people sensor Drone Lidar Service provider (Internet) IM warning IM

77. Explosion of data: intelligent data processing is a solution More data are turned into information Easier access to relevant information Explosion of information: context driven data management required Consequence of the consequences