Slides presented at the Molecular Med Tri-Con 2018 Precision Medicine, "Emerging Role of Radiomics in Precision Medicine" (http://www.triconference.com/Precision-Medicine/) Abstract The goal of this talk is to discuss the role of data standards, and specifically the Digital Imaging and Communication in Medicine (DICOM) standard, in supporting radiomics research. From the clinical images, to the storage of image annotations and results of radiomics analysis, standardization can potentially have transformative effect by enabling discovery, reuse and mining of the data, and integration of the radiomics workflows into the healthcare enterprise.

1. Data Standards in Radiomics
Research
Andrey Fedorov, PhD
Molecular Medicine Tri-Conference
February 13, 2018
https://fedorov.github.io

2. Disclosures
Grant support from NIH/NCI under the Informatics Technology for Cancer
Research (ITCR) program:
● U24 CA180918 Quantitative Image Informatics for Cancer Research (QIICR) -
PIs Fedorov, Kikinis - http://qiicr.org
● U24 CA194354 Quantitative Radiomics System Decoding the Tumor
Phenotype - PI Aerts - https://radiomics.io
2

3. Radiomics analysis workflow
3
Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RGPM, Granton P, et al. Radiomics:
extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:
441–446. doi:10.1016/j.ejca.2011.11.036
Kumar V, Gu Y, Basu S, Berglund A, Eschrich S a., Schabath MB, et al. Radiomics: the process and the
challenges. Magn Reson Imaging. Elsevier Inc.; 2012;30: 1234–1248. doi:10.1016/j.mri.2012.06.010

4. Images Regions Features Classes
Radiomics analysis “data view”
4

5. Images Regions Features Classes
“Swap” pipeline components
5

6. Images
Features Classes
Aggregate results
Images
Images
Regions
6

7. Images Regions Features Classes
Integration with patient record
7

8. Weber GM, Mandl KD, Kohane IS. Finding the
missing link for big biomedical data. JAMA.
2014;311(24):2479–2480
8

9. https://www.forbes.com/sites/gilpress/2016/03/23/dat
a-preparation-most-time-consuming-least-enjoyable-d
ata-science-task-survey-says/#5bbab7d56f63
“Cleaning Big Data: Most Time-Consuming, Least
Enjoyable Data Science Task, Survey Says”
9
Data science is more about data than methods

10. TCIA image directory
10

11. Standard for images
Digital Imaging and Communication in Medicine (DICOM) is the standard for
communication of medical imaging information and related data
● Compatibility with acquisition and archival tools
● Harmonized with other standards (HL7 CDA, JSON, XML, REST, FHIR)
● History of development and adoption since 1983
● Adopted by virtually all manufacturers of medical imaging equipment
● Open international community of stakeholders
● Continuously evolving standard
11

12. DICOM
Clinical images
12

13. DICOM Image
Pixel data:
● Sparse measurements sampling a 2d plane or 3d
volume
Non pixel data (a.k.a. “metadata”, or “header”):
● Patient identification, dates, image acquisition details,
unique identifiers of the dataset, pointers to related
evidence, body part imaged, …
13

14. TCIA image directory
14

15. TCIA analysis results directory
15

16. Example: Image Segmentations - Research formats
16
● encoding of pixel data
● common tasks
○ visualization, comparison of the results, image
fusion, quantitative measurements
● image processing domain-specific formats
○ “just enough” for typical image analysis tasks
○ image spacing, origin and direction cosines
○ NIFTI, NRRD, MHD, Analyze, …
● non-volumetric formats
○ planar
○ voxel intensity values
○ JPG, PNG, TIFF, …

17. Segmentations Composite context
● Patient identification, dates, image acquisition
details, unique identifiers of the dataset,
pointers to related evidence, annotation of the
body part imaged
Structured metadata about analysis result
● Structure segmented, tissue type, body
location, segmentation approach, references to
the source images
Research formats
17

18. DICOM - preparing for the unknown, since 1983
● Standard for images and image-related evidence
● Object definitions
○ Object type defines required and optional attributes
● For all object types: Composite context is formalized and required
○ Dates, patient IDs, study, series - for every object
● Unique identifiers
● References to related evidence
○ Provenance of data acquisition
○ Provenance of data analysis
● Fixed syntax
○ (hierarchical) list of attribute/value pairs
● Normalized semantics
○ Common data elements / lexicons / ontologies 18

19. Example: Image Segmentations - DICOM Segmentation image
19
In addition to the spatial labeling of
the voxels of interest:
● composite context
● semantics
● segmentation approach
● single file, efficient storage,
multiple occupancy voxels
● overlay color

20. No guessing about the meaning
20

21. Measurements (e.g., radiomics features)
● DICOM structured reporting
template
● Hierarchy of measurement groups
assigned per segmentation
● Coded and structured
○ Source image and segmentation
○ Anatomy
○ Quantity
○ Units
● Composite context
21

22. Radiomics feature extraction
● Free open source implementation of
radiomics features in Python
● Preprocessing filters
● Integrated with 3D Slicer
● Integration with MeV underway
22

23. Head and neck cancer use case
● Images and related evidence in DICOM:
○ PET/CT image data
○ segmentations
○ PET quantitative features
○ Clinical data (demographics, Dx, Tx)
● Data: TCIA QIN-HEADNECK collection
● Tools: public in dcmqi and 3D Slicer
● Demo: http://qiicr.org/dicom4miccai/
Beichel et al. 2016. Semiautomated segmentation of head and neck
cancers in 18F-FDG PET scans: A just-enough-interaction approach.
Medical physics 43:2948. DOI: 10.1118/1.4948679.
23

24. non-DICOM
Conversion to DICOM
DICOM
DICOM
dcmqi
JSON
Composite context is propagated from the
source image data
Analysis-specific metadata is
defined by the user and
parametrized by a JSON-Schema
Segmentation image volume
in any format readable by ITK
(NRRD, NIfTI, Analyze, MHD)
24
https://github.com/qiicr/dcmqi
Free open source, binaries for
Win/Mac/Linux, Docker image

25. Conversion from DICOM - free open source tools available
25
XML, JSON
SQL tables,
CSV, Excel
Volumetric research
formats
(NRRD, NIfTI,
MHD)
PDF, HTML
(human-readable
reports)

26. DICOM4QI
● Connectathon and demonstration of interoperability of standardized
analysis results at the RSNA, since 2015
○ 4 types of QI DICOM objects (segmentations, parametric maps, volumetric measurements,
tractography)
○ 11 platforms participated (including 5 commercial)
● Free open source tools that support the standard for QI analysis results
○ 3D Slicer (desktop platform and application)
○ ePAD (browser-based application)
○ MITK (desktop platform and application)
○ OHIF LesionTracker / Cornerstone (browser-based application and toolkit)
○ vtk-dicom (desktop)
26
https://qiicr.gitbooks.io/dicom4qi

27. 27

28. NIH Cancer Research Data Commons
Response to Blue Ribbon panel
recommendation: “to collect, share, and
interconnect a broad array of large
datasets so that researchers, clinicians,
and patients will be able to both
contribute and analyze data, facilitating
discovery that will ultimately improve
patient care and outcomes”
● Domain-specialized nodes
● Containerized standardized tools
● Node-specific consistent identifiers
and semantics
Hinkson IV, Davidsen TM, Klemm JD, Kerlavage AR, Kibbe WA. A
Comprehensive Infrastructure for Big Data in Cancer Research: Accelerating
Cancer Research and Precision Medicine. Front Cell Dev Biol. 2017;5:83 28

29. Few of the many challenges that need to be addressed
● Radiomics data sharing is in its infancy
● No tools to query/discover/interpret standardized data
● Support in commercial tools is very limited
○ —> data collected during routine clinical care is not standardized
● Clinical trial datasets are not standardized
○ Annotations often are not even machine readable
● Adherence to the standards is inconsistent
● No ontologies to describe research outputs (radiomics, filters, parameters)
● Standardization/harmonization of clinical data
29

30. In conclusion
● Data standardization is required
○ Cloud computing
○ Data-thirsty AI methods
○ Funders, publishers, regulators are increasingly requiring data sharing
● DICOM can help standardize radiomics outputs
○ Capabilities demonstrated
○ Supporting open source tools are emerging
○ Adoption is growing
● Joint effort is required to address standardization
○ Multi-domain expertise is needed
○ Community building
○ Aggregate datasets for analysis
30

31. Thank you!
Questions? Comments? More info?
andrey.fedorov@gmail.com
http://qiicr.org
31