Sascha Seifert, Siemens Healthcare, pour la journée e-health 2013

1,169 views

Published on

Published in: Education, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,169
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Sascha Seifert, Siemens Healthcare, pour la journée e-health 2013

  1. 1. Answers for life.Unrestricted © Siemens AG 2013 All rights reserved.Theseus Medico andimaging in the digital diagnosisDr. Sascha SeiferteHealth DaySierreJune 2013
  2. 2. Page 2Unrestricted © Siemens AG 2013 All rights reserved.Page 2 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOTHESEUS-MEDICO consortium2FraunhoferIGDSiemens CTErlangen &MunichAverbisLudwig-MaximillianUniversity MunichUniversityHospitalErlangenTransinsightDFKI• 11/2007 – 05/2012• 67.5 man-years,• grant 50% government, 50% industry• 3 research centers, 1 hospital• 3 companies (Siemens & 2 SME)• Lead with Siemens AG
  3. 3. Page 3Unrestricted © Siemens AG 2013 All rights reserved.Page 3 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOCodesControl1980 - 2000physicsRobust Learning MethodsSemantic Web Standards2000-2010dataOpen Internet databasesBusiness intelligenceBig Data analyticsSemantic interoperability2010 - futurecontentQuelle: Gartner. Hype Cycle for Healthcare Provider Technologies andStandards, July 2010Theseus-Medico(Semantic Web for Medicine)Google „understands“ now context; knowledge graph with 570 million elements, 18 billionfacts, launched in 2012RSNA.org drives semantic webstandard for radiologyEvolution of medical data processingComprehending Software
  4. 4. Page 4Unrestricted © Siemens AG 2013 All rights reserved.Page 4 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOThe THESEUS-MEDICO approachSemantic Web1Radiologist / Clinician• Content Understanding• Content Linking• Content Search• Knowledge explosion• Internet, books, articles• Data overload• Unstructured data: Images, Texts• Structured data: Lab values, Medication 1by Tim Berners-Lee
  5. 5. Page 5Unrestricted © Siemens AG 2013 All rights reserved.Page 5 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGO
  6. 6. Page 6Unrestricted © Siemens AG 2013 All rights reserved.Page 6 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOApplication Prototype (National IT-Summit 2011)
  7. 7. Page 7Unrestricted © Siemens AG 2013 All rights reserved.Page 7 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOText Mining FormalizationTextMiningSegmentation FormalizationSemanticProcessingofMedicalDataMedical Images Radiology Reports Treatment Plans Online Knowledge Expert KnowledgeMedical ImageAnnotation Representational Ontology: OWLUpper Ontology:time, space, organization, person, eventmoredomainspecificmorelikelytobechangedInformation ElementOntologyimages, texts,volumes, …AnnotationOntologyClinical Ontology-doctor, nurse, patient-medical case-DICOM Ontologymid-levelontologylow-levelontologyMedical OntologiesFMAICD-10WebsiteRadLexFMAExten.MappingstoexternalsourcesICD-10mapping&mergingVisualCharac.MappingstoexternalsourcesannotationThesauri & TaxonomiesextractionDisease-SymptomNavigationMulti-ModalInteractionQuality Control Intelligent DiagnoseOntology EngineeringInformation Extractionfrom Medical TextsSemantic Search Semantic Reporting Image & Text LinkageIntelligent Healthcare ApplicationsIntelligent applications using knowledge servicesKnowledge Services InfrastructureKnowledge Extraction and FormalizationKnowledge Management OverviewClinical Workflow Components2,5 TB of 750 patients(≙ ~ 7000 series) 6000 rad reports
  8. 8. Page 8Unrestricted © Siemens AG 2013 All rights reserved.Page 8 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOIMAGE AND TEXT UNDERSTANDING
  9. 9. Page 9Unrestricted © Siemens AG 2013 All rights reserved.Page 9 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOSupport systems for disease recognitionAnatomical Body Regions and OrgansLungs, heart, liver, spleen, kidneys, prostate, urinary bladder,esophagus, pancreas and several anatomical landmarksValve function, coroanry stenosis, osteolytic tumors, livertumors, lymph node cancerImageParsing
  10. 10. Page 10Unrestricted © Siemens AG 2013 All rights reserved.Page 10 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOIncrease of knowledge about the image contentImageContentDetection SegmentationPrior knowledge# Images 3-fold C.V. [mm] Runtime [s/vol]Heart* 457 1,30 3,55Liver 346 1,07 6,00Spleen 203 2,14 9,90Right kidney 199 1,03 0,40Left kidney 197 1,15 0,40Left lung 166 2,64 1,70Richt lung 163 2,35 1,80Urinary bladder 141 1,35 1,008 organs, 19 landmarks,3 body regions <1 min (09/2010)Generic Image Parsing
  11. 11. Page 11Unrestricted © Siemens AG 2013 All rights reserved.Page 11 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOKnowledge based image processing for anatomy understandingMean error 1,80±1,17 mmMean error 1,70±0,71 mmPanceas segmentation considering splenic veiniThoracic lymph nodesEsophagus
  12. 12. Page 12Unrestricted © Siemens AG 2013 All rights reserved.Page 12 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOOsteoblastic OsteolyticTotal Volumes 30 20Total Annotations 172 42False Positives per Patient 3.5 3.7Overall Sensitivity 83% 88%Mean Sensitivity 80% 93%Overall Positive Predictive Value 58% 35%Mean Positive Predictive Value 65% 49%+9 monthsBaselineTh8osteolyticvolume 10%lower end-plateTh8mixedvolume 95%whole vertebraFractureprogressionTh11osteolyticvolume 28%vertebra backnewPrior knowledge: vertebrae / discsConstrains lesions search and automatically findcorresponding lesions in prior examsSemantic detection and follow-up of spine lesions for bone lesionsResults may vary. Data on file
  13. 13. Page 13Unrestricted © Siemens AG 2013 All rights reserved.Page 13 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOSemantic detection of coronary findings• Automatic vessel tracing and labeling of coronaryvessels (Random forest)[Gülsün2008]• Automatic estimation of lumina by Random ForestRegressor (10x faster than segmentation)• Characteristics curves along vessel (e.g., degree ofcalcification)• Identification of potential stenosis• Additional information such as FFR(measured/simulated) is semantically linkedcalcifiednon-calcified mixed overallbylesionsensitivity 96.55% 89.23% 91.78% 94.75%FPR 1.50 2.30 0.87 4.67byvesselsensitivity 98.67% 94.44% 92.16% 96.47%specificity 79.12% 54.35% 81.39% 71.27%NPV 99.58% 99.21% 99.30% 99.37%10-fold cross validation with 256 CCTA volumes runtime <2 min / volumeKnowledge pipeline:centerlinelumenstenosisclassificationAccelerate and quantify readingResults may vary. Data on file
  14. 14. Page 14Unrestricted © Siemens AG 2013 All rights reserved.Page 14 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOVesselMEDICO 14… use knowledge about the anatomy (shape) ofthe liver to register accurately.Registration of multi modal and multi phase examinationsInstead of just comparing grey-values…
  15. 15. Page 15Unrestricted © Siemens AG 2013 All rights reserved.Page 15 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOMEDICO 15Registration of multi modal and multi phase examinations
  16. 16. Page 16Unrestricted © Siemens AG 2013 All rights reserved.Page 16 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOMEDICO 16Fusion view of head-neck tomographiesSegmenting bones Rigid bone registration “Adherence” of soft tissueUsing knowledge about anatomy of bones and soft tissue
  17. 17. Page 17Unrestricted © Siemens AG 2013 All rights reserved.Page 17 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOSemantic Computer Aided Detection• Horizontal and vertical integration of computer aided detection• Disadvantage of current solutions:• Specific sub systemsw/o information exchange /consolidation• „Syntactic“ interoperabilityReportingSemanticSearchBone lesion CADBreast CADLiver CADPatient contextBone (e.g. Spine) segmentationTHESEUS-MEDICO:Standardized, semantic informationusing common or (machine-) convertiblevocabulary
  18. 18. Page 18Unrestricted © Siemens AG 2013 All rights reserved.Page 18 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOThree Information Dimensionsof Radiology Reports - Challenges1 Anatomical / Spatial information• Location of a finding, e.g. affected organ, lymph nodes• Spatial modifier, e.g. left, right, axilliary2 Pathological Information• Pathological interpretation of the highlighted finding,e.g. size (enlarged lymph nodes), density (lung nodule),number of occurrences, etc.3 Temporal Information• Provides information about the difference/changesof the current findings in relation to past findings, e.g.In comparison to prior examination…Compound words (German): Stemming• Split into sub words before semantic mapping to reduce solution space• Data reduction of 90% in German languageExample:Unchanged, not pathologically enlarged axilliary,mediastinal and hilar lymph nodes.FINDINGS:CHEST (EXAMPLE)The lungs demonstrate bilateral areas of pulmonary consolidation, involvingpredominantly the right upper lobe and to a lesser extent the left upper lobe posteriorlingula and superior segment of the lower lobes with additional patchy opacities in theright lung base and right middle lobe. Findings are compatible with multi lobarpneumonia. There are bilateral right greater than left pleural effusions, small in size.There is diffuse anasarca present with 3rd spacing of fluid. There is an old healeddisplaced right clavicle fracture noted. There are sub centimeter hypodense nodules inboth thyroid lobes. There are sub centimeter lymph nodes in the mediastinum measuringup to 9mmin size in the precarinal space, probably reactive. There are calcified lymphnodes in the hila bilaterally, from prior granulomatous disease. There is bibasilar passiveatelectasis adjacent to the effusions, with calcified granulomata in both atelectatic lowerlobes. The heart is globally enlarged, with coronary artery, aortic valve calcificationspresent. Additional calcified granulomata are shown in the anterior segment of the rightupper lobe. The main pulmonary artery segment is dilated up to 3.3cm in size,suggesting pulmonary artery hypertension.Myo|kard|itisHerz|muskel|entzünd|ungInflamm|ation of the heart musclemusclemyomuskelmusculinflamm-itisinflamentzündKONZEPTsubwort herzheartcardcorazoncardINFLAMMATIONMUSCLEHEART
  19. 19. Page 19Unrestricted © Siemens AG 2013 All rights reserved.Page 19 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGO100 report texts from lymphomacases were manually annotatedRadMiner™ Report Search, Averbis GmbHAverbis Annotator(internal)ValidationPrecision=0.921Recall=0.935F1=0.928Orig: Radlex 2.0Stem: with stemmingSem: Radlex extendedSemantic Report SearchNatural Language Processing + Semantic MappingResults may vary. Data on file
  20. 20. Page 20Unrestricted © Siemens AG 2013 All rights reserved.Page 20 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOKnowledge RepresentationRepresent segmentations with semanticsManuel Möller, DFKI Kaiserslautern, manuel.moeller@dfki.deSemanticAnnotationMedicalOntologyLink bySemanticConceptsfromOntology
  21. 21. Page 21Unrestricted © Siemens AG 2013 All rights reserved.Page 21 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOAnnotationOntologyConsolidated data model: knowledge exchange instead of data exchange usingsemantic web standards (RDF and OWL)new* SPARQL 1.1 (recursive queries)*W3C Working Draft 05 January 2012Ontology TermsSnomed CT 395036FMA 83281Radlex 34895Reference OntologiesContentPhysicalRef.PhysicalRef.ReferenceOntologiesRepresentation LanguageThe whole picture
  22. 22. Page 22Unrestricted © Siemens AG 2013 All rights reserved.Page 22 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOSEMANTIC READING
  23. 23. Page 23Unrestricted © Siemens AG 2013 All rights reserved.Page 23 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOSemantic Reading – Compare Studies based on Anatomy instead ofFrame of ReferenceSemantics enables registration as expected by radiologists.Locating correspondinganatomical structuresby conceptsAnatomy is compared withthe same anatomyBenefits•synchronized scrolling•compare findings overtime•compare similar patientsbronchialbifurcation
  24. 24. Page 24Unrestricted © Siemens AG 2013 All rights reserved.Page 24 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOText-to-Image Linking(Navigation support)• Hyperlinks encode semantic information and enable tobidirectional jumps• Close semantic gap between image and text basedsystems.• Improve dialogue of radiologist and clinicians (ideal forradiol. demonstrations, easy access of priors)Anatomy aware findings labeling• Benefit from image understanding.• More meaningful findings names,understandable by computers• Enables the system to infer knowledgeSemantic Reading – Intelligent applications
  25. 25. Page 25Unrestricted © Siemens AG 2013 All rights reserved.Page 25 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOAnatomy aware literature referencesImage understanding enables to display literaturereferences matching the underlying anatomy, e.g. theBosnial classification for renal cysts.Example subjects:• Bosniak renal cyst classification• Fleischner pulmonary nodule management• Couinaud liver segments• Lung segments• Hydronephrosis classification• TNMSemantic lesion progressionRelated findings retrieved withsemantic reasoningSemantic Reading – Intelligent applications
  26. 26. Page 26Unrestricted © Siemens AG 2013 All rights reserved.Page 26 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGODISEASE MODELING
  27. 27. Page 27Unrestricted © Siemens AG 2013 All rights reserved.Page 27 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOclinical dataMedical ImagesReportslymph node of headLingual lymph nodemandibular lymph nodelymph nodelymph node of trunkmalar lymph nodesize modifiershrunken Annotationsfacial lymph nodeenlargedMEDICO-Annotation-OntologyWhatweareusingOurextensionUse of external knowledgelymphomahasLeadingSymptomsymptomdiseaseis-ais-aenlarged lymph nodeDisease-Symptom-OntologyDiSy:hasModifierDiSy:locatedInInferlikelydiseasesClinical RecommendationsNext ExaminationsFrom Annotations to Diagnosis
  28. 28. Page 28Unrestricted © Siemens AG 2013 All rights reserved.Page 28 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGODifferentialDiagnosisDefinitionsProbabilitiesCorrelationSource: Herold, Innere Medizin,2011Analysis of available Clinical Knowledge
  29. 29. Page 29Unrestricted © Siemens AG 2013 All rights reserved.Page 29 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGO5 DiseasesHodgkin-LymphomaNon-Hodgkin-LymphomaCorrectal CarcinomaReactive Lymphadenitis40 SymptomsIf possible with definition andprobabilitiesEncoding withRadLex or SNOMED CT10 Dummy PatientsInformation about LeadingSymptomsTest Data Set provided by experts
  30. 30. Page 30Unrestricted © Siemens AG 2013 All rights reserved.Page 30 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGORanking factors: Age, gender, specific incidence Leading symptoms Symptom intensity Reappearing symptoms Relative importance of symptoms Ratio of present and absent symptomsof a diseaseTowards a Ranking of Likely Diseases in Terms of Precision and Recall Heiner Oberkampf, Sonja Zillner, Bernhard Bauer, Matthias Hammon, Netmed2012.Ranking of Likely Diseases
  31. 31. Page 31Unrestricted © Siemens AG 2013 All rights reserved.Page 31 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGORanked symptoms list is providedRecommendations of next examinationsDecision Support
  32. 32. Page 32Unrestricted © Siemens AG 2013 All rights reserved.Page 32 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOSEMANTIC SEARCH
  33. 33. Page 33Unrestricted © Siemens AG 2013 All rights reserved.Page 33 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOMedical images can only be searched using:• meta data in so-called DICOM-headers(patient name, acquisition date, imaging modality etc.)• indirectly by searching corresponding radiology reports‘Content’ of the images can not beused for• quality control• data mining for clinical /epidemiological studies• decision support• workflow improvements• reporting supportSemantic Limitations of Todays Hospital ITwrt Medical Images
  34. 34. Page 34Unrestricted © Siemens AG 2013 All rights reserved.Page 34 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGO
  35. 35. Page 35Unrestricted © Siemens AG 2013 All rights reserved.Page 35 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGO„Find patients with similar liver lesion, enlarged (pathological) lymph nodes in the thorax, hemoglobin value low andpatient age greater than 65“Resulting patientswithsimilar findingsReferencelesionFindings histogramfor queryrefinementAccessingreports and labvaluesQuerytermsIntegrated Semantic Image SearchPET-CT 32%MRI 12%Acquisition methodsCHOP14DXBEAM_CTreatmentsIMVP16Germany• DSHNHL2004-2 (FLYER) Phase 3• DSHNHL2006-1B (ACT-2) Phase 3• DSHNHL2002-1 (Mega-CHOEP) Phase 3USA• UCLA-0406049-01 Phase 3Clinical trials
  36. 36. Page 36Unrestricted © Siemens AG 2013 All rights reserved.Page 36 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGO„Search for images and textsof patients with thickened wallof intestine andhemoglobin value low“(cancer?)Including lab valuesUnderstanding the anatomy:intestine expands to rectum, colon,sigmoid, cecum, …Full text search is not enough!Integrated Semantic Image Search
  37. 37. Page 37Unrestricted © Siemens AG 2013 All rights reserved.Page 37 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOImage Retrieval - Results• 111 liver lesion with 6105 pairs annotated according to similarity• Annotated with 5 similarity levels; Leave-One-Out validationMAP: 0.78nDCG(10): 0.850.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.000.250.500.751.00RecallPrecisionsearchinputoutputHammon, M.; Dankerl, P.; Costa, M.; Tsymbal, A.; Seifert, S.; Sühling, M.;Uder, M. & Cavallaro, A. (2012), Computer-aided decision support for thecharacterization of liver lesions in CT scans, in Proceedings of theEuropean Congress of Radiology (ECR)‘.
  38. 38. Page 38Unrestricted © Siemens AG 2013 All rights reserved.Page 38 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGO• transform data into actionable knowledge by image, text and speech understanding• harmonize CAD by using standardized vocabulary (CAD group, SCR, and 3rd party)• improve interoperability by describing content in a standardized (semantic web) way• enable inter-modal semantic navigation, i.e. between image, text and other clinical data• Understanding content is the key for proactive context-sensitive workflow support• organize information semantically and prepare for data analytics (understanding first)• formalize medical knowledge instead of programming• enable reuse of third party knowledge networks / databases (publishers, clinical trials, ICD10, ….)Summary
  39. 39. Page 39Unrestricted © Siemens AG 2013 All rights reserved.Page 39 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGODr. Sascha SeifertH SY TI / GermanyHartmannstr. 1691052 ErlangenE-mail:saschaseifert@siemens.comContactDisclaimer:The MEDICO prototypes are under development and not commercially available, and theirfuture availability cannot be ensured. The prototypes should not be used for any patientdiagnosis or therapy. MEDICO is not related to the commercial hospital information systemMedico.Acknowledgements:The MEDICO project is supported in part by the THESEUS program, which is funded by theGerman Federal Ministry of Economics and Technology under the grant number01MQ07016. The responsibility for this demonstration lies with the authors.
  40. 40. Page 40Unrestricted © Siemens AG 2013 All rights reserved.Page 40 Dr. Sascha Seifert / Healthcare, Imaging & Therapy, SYNGOsyngo. It’s all about you.

×