Artificial Intelligence in Radiation Oncology.pptx
•
1 like•1,464 views
The document discusses artificial intelligence applications in radiation oncology, including automatic delineation of organs-at-risk using deep learning models like OARNet. It also discusses radiomics approaches for clinical decision support and outcomes prediction using features extracted from medical images with techniques like spiculation quantification for lung cancer screening.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Artificial Intelligence in Radiation Oncology.pptx
Similar to Artificial Intelligence in Radiation Oncology.pptx (20)
More from Wookjin Choi
More from Wookjin Choi (20)
Recently uploaded
Recently uploaded (20)
Artificial Intelligence in Radiation Oncology.pptx
- 1. Artificial Intelligence in Radiation Oncology Wookjin Choi, PhD Assistant Professor of Radiation Oncology Sidney Kimmel Medical College at Thomas Jefferson University Wookjin.Choi@Jefferson.edu Nov 4, 2022 @ KOSHIS
- 2. Acknowledgements Memorial Sloan Kettering Cancer Center • Wei Lu PhD • Sadegh Riyahi, PhD • Jung Hun Oh, PhD • Saad Nadeem, PhD • Eric Aliotta, PhD • Joseph O. Deasy, PhD • Andreas Rimner, MD • Prasad Adusumilli, MD Stony Brook University • Allen Tannenbaum, PhD University of Virginia School of Medicine • Jeffrey Siebers, PhD • Victor Gabriel Leandro Alves, PhD University of Maryland School of Medicine • Howard Zhang, PhD • Wengen Chen, MD, PhD • Charles White, MD Thomas Jefferson University • Yevgeniy Vinogradskiy, PhD • Hamidreza Nourzadeh, PhD • Adam P. Dicker, MD 2 NIH/NCI Grant R01 CA222216, R01 CA172638; NIH/NCI Cancer Center Support Grant P30 CA008748 and 5P30 CA056036; and ViewRay Technologies, Inc. The ESTRO Falcon project team, Scott Kaylor of EduCase, Benjamin Nelms of Proknow for the multi- delineator contour data presented in this work
- 3. Outline • Introduction • Auto Delineation and Variability Analysis - OARNet - CIRDataset - … • Radiomics - Clinical decision support and outcomes prediction - Spiculation Quantification - PathCNN - … • Summary 3
- 4. Outline • Introduction • Auto Delineation and Variability Analysis - OARNet - CIRDataset - … • Radiomics - Clinical decision support and outcomes prediction - Spiculation Quantification - PathCNN - … • Summary 4
- 5. What is AI? 5 Artificial intelligence (AI) is intelligence demonstrated by machines, as opposed to the natural intelligence displayed by animals and humans. Source: Nvidia
- 6. What is Radiation Oncology? 6
- 7. The Radiation Oncology Team • Radiation Oncologist - The doctor who prescribes and oversees the radiation therapy treatments • Medical Physicist - Ensures that treatment plans are properly tailored for each patient, and is responsible for the calibration and accuracy of treatment equipment • Dosimetrist - Works with the radiation oncologist and medical physicist to calculate the proper dose of radiation given to the tumor • Radiation Therapist - Administers the daily radiation under the doctor’s prescription and supervision • Radiation Oncology Nurse - Interacts with the patient and family at the time of consultation, throughout the treatment process and during follow-up care 7
- 8. 8
- 9. AI in Radiation Oncology 9 Huynh et al. Nat Rev Clin Oncol 2020
- 10. Automatable tasks in radiation oncology for the modern clinic 10 Netherton et al. Oncology 2021
- 11. Hype cycle for three major innovations in radiation oncology 11 Netherton et al. Oncology 2021
- 12. Outline • Introduction • Auto Delineation and Variability Analysis - OARNet - CIRDataset - … • Radiomics - Clinical decision support and outcomes prediction - Spiculation Quantification - PathCNN - … • Summary 12
- 13. Auto Delineation and Variability Analysis 13
- 15. OARNet: auto-delineate organs-at-risk (OARs) in head and neck (H&N) CT image 15 Soomro, Nourzadeh, Choi et al., arXiv:2108.13987.
- 16. OARNet results 16 Soomro, Nourzadeh, Choi et al., arXiv:2108.13987.
- 17. CIRDataset: A large-scale Dataset for Clinically-Interpretable lung nodule Radiomics and malignancy prediction 17 Choi et al., MICCAI 2022 https://github.com/choilab-jefferson/CIR
- 18. 18 Nodule (Class0), spiculation (Class1), and lobulation (Class2) peak classification metrics Training Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑ Class0 Class1 Class2 Class0 Class1 Class2 Mesh Only 0.009 0.010 0.013 0.507 0.493 0.430 Mesh+Encoder 0.008 0.009 0.011 0.488 0.456 0.410 Validation Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑ Class0 Class1 Class2 Class0 Class1 Class2 Mesh Only 0.010 0.011 0.014 0.526 0.502 0.451 Mesh+Encoder 0.014 0.015 0.018 0.488 0.472 0.433 Testing LIDC-PM N=72 Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑ Class0 Class1 Class2 Class0 Class1 Class2 Mesh Only 0.011 0.011 0.014 0.561 0.553 0.510 Mesh+Encoder 0.009 0.010 0.012 0.558 0.541 0.507 Testing LUNGx N=73 Network Chamfer Weighted Symmetric ↓ Jaccard Index ↑ Class0 Class1 Class2 Class0 Class1 Class2 Mesh Only 0.029 0.028 0.030 0.502 0.537 0.545 Mesh+Encoder 0.017 0.017 0.019 0.506 0.523 0.525 Segmentation Results
- 19. Delineation Variability Quantification and Simulation A framework for radiation therapy variability analysis 19 RT plan Structure Set CT Image Dose Distribution Structure Sets DV simulation ASSD GrowCut RW Other delineators SV analysis DV analysis Geometric Dosimetric Variability analysis Human DV Simulated DV Consensus SS OARNet Choi et al., AAPM, 2019.
- 20. Delineation Variability Quantification and Simulation (Results) • DVH variability not predicted by geometric measures • Large human variability 20 100% 50% 0% 100% 50% 0% Human ASSD GrowCut RW Right Parotid Left Parotid Choi, Nourzadeh et al., AAPM, 2019.
- 21. Human ASSD GrowCut RW 100% 50% 0% Education 100% 50% 0% Clinic 21
- 22. Dose Volume Coverage Map (DVCM) 22 Plan Variability Setup Variability Delineation Variability Choi, Nourzadeh et al., AAPM, 2020.
- 23. Auto contouring for Pancreatic Adenocarcinoma MR guided adaptive RT (ongoing) • Pancreatic Adenocarcinoma • 5-Fraction Stereotactic MRgRT with on-table adaptive replanning • To segment future fractions for early replanning - 25 pts data collected and continue to collect more data. - Target volumes (GTV, CTV, PTV) - OARs (Bowel_Small, Bowel_Large, Kidney_L, Kidney_R, Liver, Spinal Cord, Stomach, Esophagus, Pancreas, Gallbladder, Lungs) 23 Input MR Image Manual Contouring Auto Contouring
- 24. Outline • Introduction • Auto Delineation and Variability Analysis - OARNet - CIRDataset - Delineation Variability Analysis - Auto contouring for Pancreatic Adenocarcinoma MRgRT • Radiomics - Clinical decision support and outcomes prediction - Spiculation Quantification - PathCNN - … • Summary 24
- 25. Radiomics Clinical decision support and outcomes prediction 25 Tseng et al. Frontiers in Oncology 2018 Aerts et al. Nature Comm. 2014
- 26. Radiomics Framework 26 Image Registration • Multi-level rigid • Deformable • Pre/Post-CT • MSE, MI Tumor Segmentation • Adaptive region growing • Level set • Grow cut • Morphology filter • Multi-modality image segmentation Feature Extraction • Intensity distribution • Spatial variations (texture) • Geometric properties • Jacobian feature from DVF • Feature selection Predictive Model • ROC analyses • Prediction models • Validation • Tumor response • Recurrence • Survival Source codes: https://github.com/taznux/radiomics-tools • Automated Workflow - Integrate all the radiomics components - 3D Slicer, ITK (C++), Matlab, R, and Python - Scalable: support multicore & GPU computing
- 27. Radiomics Framework 27 Image Registration • Multi-level rigid • Deformable • Pre/Post-CT • MSE, MI Tumor Segmentation • Adaptive region growing • Level set • Grow cut • Morphology filter • Multi-modality image segmentation Feature Extraction • Intensity distribution • Spatial variations (texture) • Geometric properties • Jacobian feature from DVF • Feature selection Predictive Model • ROC analyses • Prediction models • Validation • Tumor response • Recurrence • Survival Deep Learning Model • Automated Workflow - Integrate all the radiomics components - 3D Slicer, ITK (C++), Matlab, R, and Python - Scalable: support multicore & GPU computing
- 28. Radiomics Framework 28 Image Registration • Multi-level rigid • Deformable • Pre/Post-CT • MSE, MI Tumor Segmentation • Adaptive region growing • Level set • Grow cut • Morphology filter • Multi-modality image segmentation Feature Extraction • Intensity distribution • Spatial variations (texture) • Geometric properties • Jacobian feature from DVF • Feature selection Predictive Model • ROC analyses • Prediction models • Validation • Tumor response • Recurrence • Survival Deep Learning Model • Automated Workflow - Integrate all the radiomics components - 3D Slicer, ITK (C++), Matlab, R, and Python - Scalable: support multicore & GPU computing
- 29. Radiomics Framework 29 Image Registration • Multi-level rigid • Deformable • Pre/Post-CT • MSE, MI Tumor Segmentation • Adaptive region growing • Level set • Grow cut • Morphology filter • Multi-modality image segmentation Feature Extraction • Intensity distribution • Spatial variations (texture) • Geometric properties • Jacobian feature from DVF • Feature selection Predictive Model • ROC analyses • Prediction models • Validation • Tumor response • Recurrence • Survival Deep Learning Model • Automated Workflow - Integrate all the radiomics components - 3D Slicer, ITK (C++), Matlab, R, and Python - Scalable: support multicore & GPU computing
- 30. Radiomics Framework 30 Image Registration • Multi-level rigid • Deformable • Pre/Post-CT • MSE, MI Tumor Segmentation • Adaptive region growing • Level set • Grow cut • Morphology filter • Multi-modality image segmentation Feature Extraction • Intensity distribution • Spatial variations (texture) • Geometric properties • Jacobian feature from DVF • Feature selection Predictive Model • ROC analyses • Prediction models • Validation • Tumor response • Recurrence • Survival Deep Learning Model • Automated Workflow - Integrate all the radiomics components - 3D Slicer, ITK (C++), Matlab, R, and Python - Scalable: support multicore & GPU computing
- 31. Radiomics Framework 31 Image Registration • Deep Learning Optical fl ow • Action like flow • Differential warp • Dynamic filtering • … Tumor Segmentation • U-Net • Prob. U-Net • UANet • … Feature Extraction • AlexNet • ResNet • VGG • LeNet • …. Predictive Model • ROC analyses • Prediction models • Validation • Tumor response • Recurrence • Survival • Automated Workflow - Integrate all the radiomics components - 3D Slicer, ITK (C++), Matlab, R, and Python - Scalable: support multicore & GPU computing
- 32. Spiculation Quantification for Lung Cancer Screening 32 𝜖𝑖: = log 𝑗,𝑘 𝐴 ([𝜙(𝑣𝑖), 𝜙(𝑣𝑗), 𝜙(𝑣𝑘)]) 𝑗,𝑘 𝐴 ([𝑣𝑖, 𝑣𝑗, 𝑣𝑘]) Area Distortion Map Spherical Mapping Malignant nodules Benign nodules Early detection of lung cancer by LDCT can reduce mortality Known features correlated with PN malignancy Size, growth rate (Lung-RADS) Calcification, enhancement, solidity → texture features Boundary margins (spiculation, lobulation), attachment → shape and appearance features
- 33. ACR Lung-RADS 1.0 Category Baseline Screening Malignancy 1 No PNs; PNs with calcification Negative <1% chance of malignancy 2 Solid/part-solid: <6 mm GGN: <20 mm Benign appearance <1% chance of malignancy 3 Solid: ≥6 to <8 mm Part-solid: ≥6 mm with solid component <6 mm GGN: ≥20 mm Probably benign 1-2% chance of malignancy 4A Solid: ≥8 to <15 mm Part-solid: ≥8 mm with solid component ≥6 and <8 mm Suspicious 5-15% chance of malignancy 4B Solid: ≥15 mm Part-solid: Solid component ≥8 mm >15% chance of malignancy 4X Category 3 or 4 PNs with suspicious features (e.g., enlarged lymph nodes) or suspicious imaging findings (e.g., spiculation) >15% chance of malignancy 33 ACR: American College of Radiology Lung-RADS: Lung CT Screening Reporting and Data System
- 34. ACR Lung-RADS 1.1 Category Baseline Screening Malignancy 1 No PNs; PNs with calcification Negative <1% chance of malignancy 2 Perifissural: <10mm (<524 mm3) Solid: <6mm (<113 mm3) new < 4mm (<34 mm3) part-solid: <6 mm total diameter (<113 mm3) GGN: <30 mm (<14137 mm3) Benign appearance <1% chance of malignancy 3 Solid: ≥6 to <8 mm (≥113 to <268 mm3) Part-solid: ≥6 mm total diameter (≥113 mm3) with solid component <6 mm (<113 mm3) GGN: ≥30 mm (≥14137 mm3) Probably benign 1-2% chance of malignancy 4A Solid: ≥8 to <15 mm (≥268 to <1767 mm3) Part-solid: ≥ 6 mm (≥113 mm3) with solid component ≥6 mm to <8 mm (≥113 to <268 mm3) Endobronchial nodule Suspicious 5-15% chance of malignancy 4B Solid: ≥15 mm (≥1767 mm3) Part-solid: a solid component ≥ 8 mm (≥268 mm3) >15% chance of malignancy 4X Category 3 or 4 PNs with suspicious features (e.g., enlarged lymph nodes) or suspicious imaging findings (e.g., spiculation) >15% chance of malignancy 34 ACR: American College of Radiology, Lung-RADS: Lung CT Screening Reporting and Data System
- 35. Spiculation Quantification (Results) 35 Number of Spiculations and Radiologists spiculation score (RS) for different pulmonary nodules 1 2 3 4 5 0 1 4 8 14 Choi et al. in CMPB 2021 https://github.com/choilab-jefferson/LungCancerScreeningRadiomics
- 36. Spiculation Quantification (Model Validation) 36
- 37. Spiculation Quantification (Results: Comparison) 37 Choi et al. in CMPB 2021 https://github.com/choilab-jefferson/LungCancerScreeningRadiomics
- 38. Progression-free survival Prediction after SBRT for early-stage NSCLC 38 Thor, Choi et al. ASTRO 2020 • 412 patients treated between 2006 and 2017 • PETs and CTs within three months prior to SBRT start. • The median prescription dose was 50Gy in 5 fractions.
- 39. Progression-free survival Prediction (Results) • PET entropy, CT number of peaks, CT major axis, and gender. • The most frequently selected model included PET entropy and CT number of peaks - The c-index in the validation subset was 0.77 - The prediction-stratified survival indicated a clear separation between the observed HR and LR - e.g., a PFS of 60% was observed at 12 months in HR vs. 22 months in LR. 39 Thor, Choi et al. ASTRO 2020
- 40. CIRDataset: A large-scale Dataset for Clinically-Interpretable lung nodule Radiomics and malignancy prediction 40 Choi et al., MICCAI 2022 https://github.com/choilab-jefferson/CIR
- 41. 41 Training Network AUC Accuracy Sensitivity Specificity F1 Mesh Only 0.885 80.25 54.84 93.04 65.03 Mesh+Encoder 0.899 80.71 55.76 93.27 65.94 Validation Network AUC Accuracy Sensitivity Specificity F1 Mesh Only 0.881 80.37 53.06 92.11 61.90 Mesh+Encoder 0.808 75.46 42.86 89.47 51.2 Testing LIDC-PM N=72 Network AUC Accuracy Sensitivity Specificity F1 Mesh Only 0.790 70.83 56.10 90.32 68.66 Mesh+Encoder 0.813 79.17 70.73 90.32 79.45 Prediction Results Testing LUNGx N=73 Network AUC Accuracy Sensitivity Specificity F1 Mesh Only 0.733 68.49 80.56 56.76 71.60 Mesh+Encoder 0.743 65.75 86.11 45.95 71.26
- 42. PathCNN: interpretable convolutional neural networks for survival prediction and pathway analysis applied to glioblastoma 42 • CNNs have achieved great success • A lack of interpretability remains a key barrier • Moreover, because biological array data are generally represented in a non-grid structured format • PathCNN An interpretable CNN model on integrated multi-omics data using a newly defined pathway image. Oh, Choi et al. Bioinformatics 2021, joint first author Source code: https://github.com/mskspi/PathCNN.
- 43. PathCNN: interpretable CNNs (results) 43 Cancer PathCNN Logistic regression SVM with RBF Neural network MiNet GBM 0.755 ± 0.009 0.668 ± 0.039 0.685 ± 0.037 0.692 ± 0.030 0.690 ± 0.032 LGG 0.877 ± 0.007 0.816 ± 0.036 0.884 ± 0.017 0.791 ± 0.031 0.854 ± 0.027 LUAD 0.637 ± 0.014 0.581 ± 0.028 0.624 ± 0.034 0.573 ± 0.031 0.597 ± 0.042 KIRC 0.709 ± 0.009 0.654 ± 0.034 0.684 ± 0.027 0.702 ± 0.028 0.659 ± 0.030 Comparison of predictive performance with benchmark methods in terms of the area under the curve (AUC: mean ± standard deviation) over 30 iterations of the 5-fold cross validation Note: AUCs for PathCNN were obtained with three principal components. Bold = Highest AUC for each dataset. SVM, support vector machine; RBF, radial basis function; MiNet, Multi-omics Integrative Net; GBM, glioblastoma multiforme; LGG, low-grade glioma; LUAD, lung adenocarcinoma; KIRC, kidney cancer. Oh, Choi et al. Bioinformatics 2021, joint first author Source code: https://github.com/mskspi/PathCNN.
- 44. 44 A matrix of adjusted P-values. The row represents the 146 KEGG pathways ordered on pathway images. The columns represent the first two principal components of each omics type. The red color indicates key pathways with adjusted P-values < 0.001
- 45. Heart Toxicity Prediction using PET/CT radiomics for Lung Radiotherapy (ongoing) • Pre & Post - PET/CT Radiomics - Delta-radiomics between pre and post PET/CT - 70 pts from ACRIN 6668/RTOG 0235, 39 pts from CU/BU - Manual and auto heart contouring • Pre only - CT coronary artery calcium scoring - PET/CT Radiomics - Collected TJU 50 pts data and continue to collect more data - Manual and auto heart contouring 45
- 46. Heart Toxicity Preliminary Results 46 Miller et al. Radiotheray and Oncology, 2022 Comparison of Overall Survival Data by SUVmean Change. The median OS for patients with a negative SUVmean cardiac change (dark grey) was 413 days and the median OS for patients with a positive SUVmean cardiac change was 585 days (lightgrey). Kaplan-Meier Analysis. Patients with a positive cardiac SUVmean change (light grey) demonstrate a significantly higher surviving fraction in comparison to those patients with a negative cardiac SUVmean change (black).
- 47. Summary • Introduction • Auto Delineation and Variability Analysis - OARNet: H&N OAR auto segmentation - CIRDataset: volume segmentation and 3D mesh model generation - Delineation Variability Analysis - Auto contouring for Pancreatic Adenocarcinoma MRgRT • Radiomics - Clinical decision support and outcomes prediction - Spiculation Quantification - CIRDataset: end-to-end model to predict PN malignancy - PathCNN: Multiomics - Heart Toxicity Prediction • Summary 47
- 48. Selected Publications 1. Jung Hun Oh*, Wookjin Choi* et al., “PathCNN: interpretable convolutional neural networks for survival prediction and pathway analysis applied to glioblastoma”, Bioinformatics, 2021, *joint first author 2. Wookjin Choi et al., “ Reproducible and Interpretable Spiculation Quantification for Lung Cancer Screening”, Computer Methods and Programs in Biomedicine”, 2021 3. Noemi Garau, Wookjin Choi, et al., “ External validation of radiomics‐based predictive models in low‐dose CT screening for early lung cancer diagnosis”, Medical Physics, 2020 4. Jiahui Wang, Wookjin Choi et al., “Prediction of anal cancer recurrence after chemoradiotherapy using quantitative image features extracted from serial 18F-FDG PET/CT”, Frontiers in oncology, 2019 5. Wookjin Choi et al., “Radiomics analysis of pulmonary nodules in low-dose CT for early detection of lung cancer”, Medical Physics, 2018 6. Sadegh Riyahi, Wookjin Choi, et al., “Quantifying local tumor morphological changes with Jacobian map for prediction of pathologic tumor response to chemo-radiotherapy in locally advanced esophageal cancer”, Physics in Medicine and Biology, 2018 7. Shan Tan, Laquan Li, Wookjin Choi, et al., “Adaptive region-growing with maximum curvature strategy for tumor segmentation in 18F-FDG PET”, Physics in Medicine and Biology, 2017 8. Wookjin Choi et al., “Individually Optimized Contrast-Enhanced 4D-CT for Radiotherapy Simulation in Pancreatic Ductal Adenocarcinoma”, Medical Physics, 2016 9. Wookjin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection based on Three-dimensional Shape-based Feature Descriptor”, Computer Methods and Programs in Biomedicine, 2014 48 Complete list of publications: https://scholar.google.com/citations?user=iHgsGLUAAAAJ
- 49. Looking for a Post-Doctoral Research Fellow Developing Interpretable Predictive Models for Radiation Therapy • PI: Wookjin Choi, PhD - Wookjin.Choi@Jefferson.edu • 2 Years • Machine Learning/Deep Learning: Radiomics (PET/CT & MR) and Bioinformatics • Computational Medical Physics: Development of Predictive Models and Automated Workflows, and Improve Clinical Workflow • Internal or Extramural Research Funding Opportunities Qualifications • Ph.D. in Computer Science, Electrical Engineering, Medical Physics, or related field required
- 50. Thank You! Q & A
- 52. Short-term Future Works • Develop interpretable radiomic features - Improve spiculation quantification and multi-institution validation - Multimodal data integration • Variability aware auto-delineation - Variability quantification and simulation using generative models - AI-guided interactive delineation editing • Integrate the radiomics framework into TPS - Eclipse (C#) and MIM (Python) 52
- 53. … PACS Server Cloud Computation … Server Local Computation TPS1 TPS2 SIM Client RIS EHR DICOM communication Jefferson Radiomics Framework De-ID 53
- 54. Long-term Future Works • Comprehensive Framework for Cancer Imaging - Multi-modal imaging - Response prediction and evaluation (Pre, Mid, and Post) - Longitudinal analysis of tumor change during treatment (MRgRT) - Shape analysis (e.g., Spiculation) - Deep learning models • Automation of Clinical Workflow - Big Data Analytics: EMR, PACS, ROIS, Genomics, etc. - Provide an informatics platform for comprehensive cancer therapy 54
Editor's Notes
- I would like to thank everyone who has helped me in the projects
- How to generalize it
- How to generalize it
- Radiation has been an effective tool for treating cancer for more than 100 years More than 60 percent of patients diagnosed with cancer will receive radiation therapy as part of their treatment Radiation oncologists are cancer specialists who manage the care of cancer patients with radiation for either cure or palliation Radiation therapy or radiotherapyis a therapy using ionizing radiation, generally provided as part of cancer treatment to control or kill malignant cells and normally delivered by a linear accelerator Radiation therapy has multiple sources to be used for delivery. Most radiation therapy treatments are delivered using photons which are either delivered with Gamma Rays (such as radioisotopes used in brachytherapy) and X-rays (generated by a linear accelerator) Additional sources include particle beams such as protons, neutrons and electrons Photons Gamma Rays Emitted from a nucleus of a radioactive atom Cobalt treatment machine Radioisotopes used in brachytherapy X-rays Generated by a linear accelerator when accelerated electrons hit a target Particle Beams Protons Neutrons Electrons
- The radiation therapy treatment team works closely to ensure that patients are receiving safe, quality treatment.
- a general overview of the radiation therapy workflow with brief descriptions of expected applications of artificial intelligence (AI) at each step. The workflow begins with the decision to treat the patient with radiation therapy, followed by a simulation appointment during which medical images are acquired for treatment planning. Subsequently, the patient-specific treatment plan is created, and then the plan is subjected to approval, review and quality assurance (QA) measures prior to delivery of radiation to the patient. The patient then receives follow-up care. AI has the potential to improve radiation therapy for patients with cancer by increasing efficiency for the staff involved, improving the quality of treatments, and providing additional clinical information and predictions of treatment response to assist and improve clinical decision-making.
- Automatable tasks in radiation oncology for the modern clinic. The extent to which each skill set is used or task is performed in this figure is not indicated and may be dependent on each clinical practice. In order to group essential tasks performed during the treatment planning process, “Physical,” “Knowledge,” and “Social” skill domains were created and are indicated by green, magenta, and blue ellipses, respectively. Skills or tasks are indicated by circles within each colored domain and may be shared between domains. Based on works cited in this review, tasks which may be automated are within the “Automatable” domain.
- (triangle: Monte Carlo; square: Inverse optimization/IMRT; circle: deep learning-based contouring). The curve depicts expectations by the target audience (those in radiation oncology and medical physics) as a function of time. Yellow, magenta, cyan, green, and blue portions of the curve denote “innovation trigger,” “peak of inflated expectations,” “trough of disillusionment,” “slope of enlightenment,” and “productivity plateau” regions, respectively.
- 10분
- Objective: To auto-delineate organs-at-risk (OARs) in head and neck (H&N) CT image sets via a compact high-performance knowledge-based model. Approach: A 3D deep learning model (OARnet) is developed and used to delineate 28 H&N OARs on CT images. OARnet utilizes a densely connected network to detect the OAR bounding-box, then delineates the OAR within the box. It reuses information from any layer to subsequent layers and uses skip connections to combine information from different dense block levels to progressively improve delineation accuracy. Training uses up to 28 expert manual delineated (MD) OARs from 165 CTs. Dice similarity coefficient (DSC) and the 95th percentile Hausdorff distance (HD95) with respect to MD is assessed for 70 other CTs. Mean, maximum, and root-mean-square dose differences with respect to MD are assessed for 56 of the 70 CTs. OARnet is compared with UaNet, AnatomyNet, and Multi-Atlas Segmentation (MAS). Wilcoxon signed-rank tests using 95% confidence intervals are used to assess significance.
- Depiction of IRT architecture based on Voxel2Mesh. The standard UNet based voxel encoder/decoder (top) extracts features from the input CT volumes while the mesh decoder deforms an initial spherical mesh into increasing finer resolution meshes matching the target shape. The mesh deformation utilizes feature vectors sampled from the voxel decoder through the Learned Neighborhood (LN) Sampling technique and also performs adaptive unpooling with increased vertex counts in high curvature areas. We extend the architecture by introducing extra mesh decoder layers for spiculation and lobulation classification. We also sample vertices (shape features) from the final mesh unpooling layer as input to Fully Connected malignancy prediction network. We optionally add deep voxel-features from the last voxel encoder layer to the malignancy prediction network.
- A framework for radiation therapy variability analysis, human delineation variability (DV) and simulated DV generated by auto delineation (AD) methods were analyzed using geometric measurements and dosimetric consequence, as well as dosimetric consequences of setup variability also evaluated using RTRA. If multiple humans delineated SSs are not available, consensus SS will be generated by 10 ASSD delineated SSs (5 SSs with 2mm SD and 5 SSs with 5mm SD).
- The MD Dice was 0.7±0.2 (μ±σ). AD Dice values (ASSD, GrowCut, and RW) were 0.5±0.2, 0.7±0.2, and 0.8±0.1, respectively. HDs were 35.4±45.2, 27.3±19.1, 29.3±19.9, and 14.6±10.3. The simulated DV increased with increasing the seed standard deviations or variability level. The dosimetric effect was largest for MD DVs (larger OAR DVH confidence intervals and larger HD), even though the MD Dice was greater than the ASSD and GrowCut Dice values. GrowCut DV resulted in less dosimetric variation than RW, unlike the geometric indices.
- 5 Different 400 Plan
- Elekta 1.5T ViewRay 0.35T
- How to generalize it
- 20분 The pre-planning prediction of dosimetric tradeoffs to assist physicians, patients and payers alike to make better informed decisions about treatment modality and dose prescription [30], [31], [32]. The integration of dosimetric information with orthogonal data (e.g. genomics, diagnostic imaging, electronic medical records) to build accurate outcomes models of Tumor Control Probability (TCP) and Normal Tissue Complication Probability (NTCP). Although early research has shown this to be a promising approach, this area of decision support is not yet ready for routine clinical use [3], [5], [33], [34], [35], [36], [37]. Radiomics, which is a branch of medical imaging analytics that relies upon primary extraction of quantitative imaging features (e.g. texture) to predict various clinical phenomena.
- I open sample automated workflow and essential components to public
- I open sample automated workflow and essential components to public
- I open sample automated workflow and essential components to public
- I open sample automated workflow and essential components to public
- I open sample automated workflow and essential components to public
- I open sample automated workflow and essential components to public
- Why spherical mapping because nodule has a spherical topology and we want to simplify its representation First non-trivial eigenfunction of the Laplace-Beltrami operator Conformal mapping from surface 𝑆 to unit sphere 𝒮 2 : 𝜙:𝑆→ 𝒮 2 Compute area distortion 𝜖 𝑖 to detect base ( 𝜖 𝑖 =0) and apex (max. negative 𝜖 𝑖 )
- The Lung Imaging Reporting and Data System (Lung-RADS) was developed by the American College of Radiology (ACR) to standardize the screening of lung cancer on CT images. As shown in the Table, the Lung-RADS categorization is mainly based on PN size (the average of the longest and shortest diameters on axial slice) with some consideration to calcification, appearance type (solid, part-solid, and non-solid or ground glass nodule/GGN), and additional suspicious features. We also performed Lung-RADS categorization based on the PN contour and the physician’s annotations. To match the original LIDC-IDRI diagnosis, categories 3 and lower are labeled as benign and category 4 (4A, 4B, and 4X) as malignant.
- The Lung Imaging Reporting and Data System (Lung-RADS) was developed by the American College of Radiology (ACR) to standardize the screening of lung cancer on CT images. As shown in the Table, the Lung-RADS categorization is mainly based on PN size (the average of the longest and shortest diameters on axial slice) with some consideration to calcification, appearance type (solid, part-solid, and non-solid or ground glass nodule/GGN), and additional suspicious features. We also performed Lung-RADS categorization based on the PN contour and the physician’s annotations. To match the original LIDC-IDRI diagnosis, categories 3 and lower are labeled as benign and category 4 (4A, 4B, and 4X) as malignant.
- Spiculations are spikes on the surface of PN and are important predictors of malignancy in lung cancer
- state of the art
- Our spiculation measures improved the radiomics model for malignancy prediction Model 9 is also mine
- 412 patients treated between 2006 and 2017 were included. Patients had to have PETs and CTs available within three months prior to SBRT start. The median prescription dose was 50Gy in 5 fractions. The planning-CT gross tumor volumes (GTVs) were propagated onto the pre-treatment PETs and CTs using b-spline deformable image registration. PET intensity features (90th percentile, entropy, maximum, mean, peak, robust mean absolute deviation, SD, valley) and CT shape features (compactness, diameter, elliptic axes, elongation, flatness, number of lobules/peaks/spicules, sphericity, surface area, surface to volume ratio, volume) were extracted. Data were split into training and hold-out validation subsets (n = 283, 123; 70%, 30%). In the training subset, the imaging features and six patient characteristics (age, gender, histology, performance status, prior surgery, tumor location) were tested for association with PFS using Cox Proportional Hazards regression with re-sampling (bootstrapping with 1000 samples). Significance was denoted at p≤0.0019 (corrected for 26 tests). A bootstrapped forward-stepwise multivariate analysis was undertaken including only non-strongly correlated predictors (Spearman’s rank, |Rs|<0.70). The most frequently selected model was explored in the validation subset in which model performance was assessed using the c-index and the prediction-stratified high and low risk tertiles (HR, LR) of the observed PFS were compared.
- Results Nineteen of the 20 identified candidate predictors were either PET or CT features (p-value range: 3E-9, 1.2E-3). The intra-imaging modality correlation between features was strong (median |Rs|: PET: 0.93; CT: 0.76) and only four features were passed on to multivariate analysis: PET entropy, CT number of peaks, CT major axis, and gender. The most frequently selected model included PET entropy and CT number of peaks; the c-index in the validation subset was 0.77 and the prediction-stratified survival indicated a clear separation between the observed HR and LR: e.g. a PFS of 60% was observed at 12 months in HR vs. 22 months in LR. Conclusion This PET and CT-based model identified the SUV distribution randomness (entropy) and spiculated tumor pattern on CTs as the most important features in predicting PFS in early stage NSCLC. The associated performance on the hold-out validation subset was good and its use has the potential to further improve the prediction of response to SBRT for this patient population. This model will be used to identify high-risk patients based on the predicted PFS in an upcoming phase II study on adjuvant immunotherapy.
- Depiction of IRT architecture based on Voxel2Mesh. The standard UNet based voxel encoder/decoder (top) extracts features from the input CT volumes while the mesh decoder deforms an initial spherical mesh into increasing finer resolution meshes matching the target shape. The mesh deformation utilizes feature vectors sampled from the voxel decoder through the Learned Neighborhood (LN) Sampling technique and also performs adaptive unpooling with increased vertex counts in high curvature areas. We extend the architecture by introducing extra mesh decoder layers for spiculation and lobulation classification. We also sample vertices (shape features) from the final mesh unpooling layer as input to Fully Connected malignancy prediction network. We optionally add deep voxel-features from the last voxel encoder layer to the malignancy prediction network.
- To address these issues, we propose a novel method, called PathCNN, that constructs an interpretable CNN model on integrated multi-omics data using a newly defined pathway image. PathCNN showed promising predictive performance in differentiating between long-term survival (LTS) and non-LTS when applied to glioblastoma multiforme (GBM). The adoption of a visualization tool coupled with statistical analysis enabled the identification of plausible pathways associated with survival in GBM. In summary, PathCNN demonstrates that CNNs can be effectively applied to multi-omics data in an interpretable manner, resulting in promising predictive power while identifying key biological correlates of disease.
- Motivation Convolutional neural networks (CNNs) have achieved great success in the areas of image processing and computer vision, handling grid-structured inputs and efficiently capturing local dependencies through multiple levels of abstraction. However, a lack of interpretability remains a key barrier to the adoption of deep neural networks, particularly in predictive modeling of disease outcomes. Moreover, because biological array data are generally represented in a non-grid structured format, CNNs cannot be applied directly. Results To address these issues, we propose a novel method, called PathCNN, that constructs an interpretable CNN model on integrated multi-omics data using a newly defined pathway image. PathCNN showed promising predictive performance in differentiating between long-term survival (LTS) and non-LTS when applied to glioblastoma multiforme (GBM). The adoption of a visualization tool coupled with statistical analysis enabled the identification of plausible pathways associated with survival in GBM. In summary, PathCNN demonstrates that CNNs can be effectively applied to multi-omics data in an interpretable manner, resulting in promising predictive power while identifying key biological correlates of disease.
- The cardiac SUVmean change was calculated as the post-treatment SUVmean minus the pretreatment SUVmean. SUVmean cardiac change was predictive of OS with a HR of 0.811 (95% CI 0.68–0.96, p = 0.017).
- How to generalize it
- Radiation oncology information systems