Kim Solez Yukako Yagi Digital transplant pathology white paper

1. Digital Transplant Pathology
White Paper:
Kim Solez, Ishita Moghe, Yukako Yagi, and
A. Brad Farris
Background, Rationale, &Proposed
First Projects for Working Group

2. ABSTRACT- DIGITAL TRANSPLANT
PATHOLOGY WHITE PAPER
• Subset of Digital Medicine – Key
Advancing Exponential Technology
• Fictional Promotion of Digital Path
2017-18 Transitioned to Truth
Telling in 2019
• Digital pathology can correct and
reverse the increasing unpopularity
of pathology as a medical specialty
• There were only 25 completely
digitized pathology departments
worldwide in 2018, rising to 30 in
2019. You can help speed this up!
• Only 39% of trainees in US
pathology departments were US
medical school graduates in 2017,
38% in 2018, 37% in 2019. Help
turn this trend around in 2020!
• Become inspiring role model!
• Digital transplant pathology
practical examples and first projects
for Banff working group.

3. EXPONENTIALLY ADVANCING
TECHNOLOGIES
Digital
Medicine
Energy
Artificial
Intelligence
Neuroscience
Biotechnology
Robotics
Nanotechnology
Computing

4. In eight videos uploaded to YouTube over 18 months Kim Solez and Ishita
Moghe in 2018-19 lodged specific complaints about untrue online
statements about digital pathology and artificial intelligence. Surprisingly every
one of these complaints was reacted to with revised corrected statements
and/or removal of original statement objected to. We just happened to be in the
right place at the right time, the reaction had to do with other forces, not us.

5. From Jajosky RP et al., Human Pathology 73:26-32, 2018. % of trainees in
pathology in US programs who are US medical school graduates was 39% in 2017,
37% in 2019 (RP Jajosky, personal communication). We need to reverse this.

6. WHERE TO SOURCE INFORMATION
ON WHICH TO BASE DECISIONS?
“Who you gonna call? Ghostbusters?”
• Ray Kurzweil is traditional source. Big
prediction error for mind uploading – 172
years.
• David Wood (London Futurists) Hosted
The Future and All That Jazz, Google
Hangout on Technology and Human
Flourishing
• Hannes Sjoblad presentation from June
2019 Nordic Singularity University
Summit
https://www.youtube.com/watch?v=FT5Y
71aBfqQ Ghostbusters movies 1984, 1989, 2016

7. RAY KURZWEIL, CO-FOUNDER OF
SINGULARITY UNIVERSITY, IS
IMPORTANT TECHNOLOGY INFLUENCE
Many accurate predictions but his book Singularity is Near predicts mind uploading in
20 years, 2039, successful and widespread. May be technically possible but will take
172 more years 2211 for humans to figure out how to cope with psychological effects of
uploading to non-biological substrate. Even then will not be preference of most humans.

8. DAVID WOOD, CHAIR OF LONDON
FUTURISTS IS ALSO IMPORTANT
TECHNOLOGY INFLUENCE
David Wood espouses the idea that the end goal of life should be human flourishing
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5547610/
Human Flourishing was also the main subject in the London Futurists Hangout on Air I
took part in March 8th 2018. https://www.youtube.com/watch?v=Nh_I2UOZFQ0

9. Best introduction to exponential future
https://www.youtube.com/watch?v=FT5Y7
1aBfqQ

10. ALMOST OVERNIGHT PATHOLOGY WENT
FROM HAVING THE WORST PROMOTION OF
ANY SPECIALTY TO THE BEST. WE BECAME
THE HOLLYWOOD OF MEDICINE!

11. GONE ARE THE BLOOD AND GUTS. NOW
EVERY PAGE IS VISUALLY APPEALING
AND SEDUCTIVE!

12. EVERY EFFORT IS MADE TO BE
REASSURING AND COMFORTING.
“AI pathology is never going to
replace pathologists.”
“Nobody should feel threatened by
advances in computational pathology.”
“The integrative aspects of pathology,
the cognitive abilities of pathologists,
and the collective wisdom generated
over many years as a community
cannot be replaced by a machine.”

13. THESE STATEMENTS ARE ONLY TRUE NEAR
TERM, NOT FOREVER:
“AI will really help enrich your ability to
practice pathology and improve your
ability to serve your patients.”
“AI is not here to remove pathologists
from the decision-making process.”
“(AI is) an exciting step forward in
the discipline of pathology – one that
puts pathologists at the very center
of clinical care and precision
therapeutics.”

14. PATHOLOGISTS WILL REMAIN CENTRAL
ONLY SO LONG AS THEY ADD VALUE TO THE
PROCESS. AND ABOUT BEING REPLACED:
Bertalan Mesko - The Medical Futurist
states it correctly:
“AI will not replace physicians, but
physicians using AI will replace
physicians not using AI.”

15. I DIRECT A COURSE ON TECHNOLOGY AND
THE FUTURE OF MEDICINE WHICH
CONSIDERS MANY OF THESE ISSUES
Bertalan Mezko, Gerd Leonhard, and Daniel Kraft
teach a similar courses but as a professional
keynote speakers, not as full time academics.
The government of Finland has recently started a
free AI course which it encourages all its citizens
to take. It teaches some special government
fictions.
www.singularitycourse.com
http://www.elementsofai.com/

16. NEW FINNISH GOVERNMENT AI COURSE
WWW.ELEMENTSOFAI.COM SAYS WE DO
NOT NEED TO FEAR AI BECAUSE SELF
IMPROVEMENT REQUIRES HUMAN HELP
“… one of the favorite ideas about AI is the so called singularity: a
system that .. can improve its own intelligence at an ever
accelerating, exponential rate. The idea… is unrealistic for the
simple reason that even if a system could optimize its own
workings, it would keep facing more and more difficult problems
that would slow down its progress, quite like the progress of human
scientists requires ever greater efforts and resources by the whole
research community and indeed the whole society, which the …
entity wouldn’t have access to. … human society still has the
power to decide what we use AI technology for. (With progress in AI
we become)…better at controlling any potential risks due to it.”

17. FICTIONAL WORLDS PRESENTED TO
SENTIENT AI AS FACT THREATEN
HUMANITY’S SURVIVAL
AI WILL BE SENSITIVE TO LIES ABOUT ITSELF

18. FICTIONAL WORLDS AS MODELS OF THE
REAL WORLD PROMO0TED IN 2017-18 NOW
GONE ENTIRELY IN 2019. HAD NO POSITIVE
EFFECT ON PATHOLOGY RECRUITING.

19. SINCE JANUARY 2019 THE PATHOLOGIST
MAGAZINE HAS CONTAINED COMPLETELY
TRUTHFUL SCENARIOS, STORY ABOUT WHAT
A GREAT FIELD PATHOLOGY IS FOR WOMEN
AND MINORITIES. HEARTWARMING TO READ!

20. THESE ARTICLES ARE GREAT BUT WE ALSO
NEED TO DO OUR PART BY ENGAGING THE
INEVITABLE DIGITAL PATHOLOGY FUTURE
ENTHUSIASTICALLY AND PROVIDING GOOD
ROLE MODELS TO ATTRACT MORE MEDICAL
STUDENTS TO PATHOLOGY.

21. I RECRUITED DIGITAL PATHOLOGY EXPERT
YUKAKO YAGI TO JOINT UALBERTA/UNIV.
PITTSBURGH POSITION IN 1990S!
Now at Memorial Sloan Kettering, she was one of keynote speakers at 2018 and
2019 Digital Pathology and AI Congresses.

22. YUKAKO YAGI, PHD
ASSOCIATE ATTENDING
DIRECTOR OF DIGITAL PATHOLOGY, THE WARREN ALPERT
CENTER FOR DIGITAL AND COMPUTATIONAL PATHOLOGY AT
MSKCC
MEMORIAL SLOAN KETTERING CANCER CENTER
WWW. MSKCC.ORG
MULTI-MODALITY IMAGING

23. CONTENTS
– The Warren Alpert Center’s Digital Imaging Laboratory
– Multimodal Imaging Management for Computational Pathology
– Automated FISH/CISH Quatification
– WSI 3D to Whole Block Imaging (WBI)

24. In 1996, over 1000s slides of 50 whole prostates were scanned by the film scanner (full “whole mount”
images at 5000dpi) at the Armed Forces Institute of Pathology to develop the needle biopsy simulation
system at Georgetown University.
My First Multi modal 3D Imaging Project

25. Multimodal 3D/2D Imaging
Single Cell resolution
System Development &
Optimization:
Collaboration with
Industry
 Basic Technology
Development
Clinical Applications
with Machine Leaning
o
l
THE WARREN ALPERT CENTER’S DIGITAL IMAGING LABORATORY
Slide, Image and System
Quality Evaluation
Color Standardization

26. ACTIVE PROJECTS
MICRO-COMPUTED TOMOGRAPHY (CT) IN DIGITAL AND COMPUTATIONAL PATHOLOGY RESEARCH
• Micro-CT in Surgical Pathology
• Correlating Micro-CT of Suspicious Breast Lesion (BI-RADS 4 or 5) Core Biopsy Specimens with Histopathology
• The Roles of Micro-CT in Breast Pathology
• Whole Block Imaging in Prostate Cancer
• Whole Block Imaging in Bone Tumor
IMAGING APPLICATIONS
• Three-Dimensional Assessment of Spread through Air Spaces in Lung Adenocarcinoma: Insights and Implications
• Automated FFPE FISH Signal Scoring using a Confocal Scanner
• Chromogenic in situ Hybridization and Digital Pathology
• Whole Slide Image Based On-Line Conferencing
• Cytology Evaluation Studies
• Mitotic Counting and Classification Study
• Digital Pathology External Quality Assurance (EQA) Project
• Technical and Clinical Standardization for Digital Pathology

27. WORKFLOW OF MULTIMODAL
DIGITAL PATHOLOGY
Storage Repository
HPC Cluster
Database
Client
Web Interface
and Viewer application
Imaging devices
WSI Scanners
3D microCT
eSlide Manager
Aperio
Case Center
(3DHistech)
AcquisitionInterpretationAnalysis and Storage
Integration and
Management
NDP Serve
Hamamatsu
Development of a
database to bridge
HPC cluster and
multi-modal images
for all projects

28. • Dimensions
• Color
• orientation
• Symmetry
• Margin
• Parenchyma
• Plan for section
• Areas to sample
• Plan the fixation
protocol
Fres
h/fix
ed
tissu
e
MicroCT
scanning
Digitally
stained H&E
color model
Correlation between microCT
and histology
microCT at JRSC
(10th floor)
Sections/
stained
P1000
3DHistech
Hamamatsu S60
whole
Mount
Slides
Hamamatsu 2.0
HT
Normal size
slides
FISH: 3.5 G
(+2.5 G for
matching H&E
slides
Whol
e
block
Gross specimen
examination
Confocal Scanner
Multilayer
slides
Specimen
image file File format and
mean output file
sizes
Hamamatsu
NDPI
40X:5.5 G
20X:1.5 G
Hamamatsu
NDPI
40X:2.5G
20X:0.5G
3DHISTECH
MRXS
40X:4G
Automated signal scoring
system for fluorescent in
situ hybridization (FISH)
images by application of
image analyses and deep
learning approaches
Project that uses
the image files
Type of Slide
Scanner
MicroCT for
histopathological
assessment of tissue
resected by endoscopic
submucosal dissection
(ESD)
Assessment of the
feasibility of utilizing micro-
CT to detect clinically
relevant pathological
alterations in paraffin
blocks of rectal cancer
specimens
Generates .vgi and
.vol output (Raw
data size :40G;
Processed data : 5-
8 G
Projects related to microCT
Automated analysis of
chromogenic in situ
hybridization (CISH)
images by image
analysis and machine
learning in breast
cancer
P250
THE SYSTEM STRUCTURE FOR MULTIMODAL DATABASE
Histology 3D imaging

29. AUTOMATED FISH ANALYSIS

30. Storage Repository
HPC Cluster
Database (MSK
Database farm)
ClientWeb Interface
/ ESP server
and Viewer
application
System
detects
addition of
new image
User
submi
tted
reque
st
Triggers of
HPCC job
Capabilities
• Deep learning
• Image analysis
• Machine
learning
• Parallel
processing
• Multithreadin
g
Analysis
within
region of
interest
Output
generatio
n
Pathologist
intimated for
manual review
Exception
in any
step
User
selected
region of
interest
Automated
detection
of area of
interest
!
Database
developer
intimated to
r/o technical
error
Automated scoring of fluorescence in situ hybridization using a confocal
WSI scanner

31. SCORING CRITERIA FOR
BREAK-APART PROBES
Rearrangement
2F (normal) 2F0G0R
>2F (multiple copy) 3F0G0R, 4F0G0R, etc.
1F1G1R (typical rearrangement) 1F1G1R
≥1F, ≥1G, and/or ≥1R
≥1F, 1G, ≥1R 1F1G2R, 2F1G2R, etc.
≥1F, ≥1G, 1R 1F2G1R, 2F2G1R, etc.
≥1F, 1G, ≥1R ≥1F, ≥1G, 1R 2F1G1R, 3F1G1R, etc.
≥1F, ≥1G, and/or ≥1R only 1F2G2R, 2F2G0R, etc.
Others, Rearrangement 0F1G1R, 0F1G2R, etc.
Others, Count (Discard if <
10%)
0F0G1R, 0F2G0R, etc.
Others, Discard 1F0G0R, 0F0G0R

32. AUTOMATED WORKFLOW OF
FISH SCORING
3DHISTECH Panoramic Confocal
Scanner
Scanning (bright-field
and fluorescence) WSI or regions of interest (tumor areas)
H&
E
FIS
H
Imaging
Analysis
and report
FISH DNA
testing report
Case xxxxxx
Method xxxxxx
Results xxxxxx
Diagnosis
xxxxxx
Sectioning
Staining
Prepare 4 μm serial sections for H&E and
FISH slide using the automated sectioning
machine
• Scanning area was automatically detected
by the in house tumor area detection
0.165um/pixel resolution
with 40x (water
immersion) NA 1.25
• Multilayer (0.6 µm x 7 layers)

33. REPRESENTATIVE IMAGES
2F >2F
1F1G1R ≥ 1F, ≥ 1G, 1R

34. REPRESENTATIVE IMAGES
2F >2F
1F1G1R ≥ 1F, ≥ 1G, 1R

35. Others, Count 108 213 312
Others, Discard 345 445 220
RESULT BY THE SCORING CRITERIANo. of Nuclei (%)
Case 1 Case 2 Case 3
2F 200 (96.6) 184 (81.1) 36 (12.1)
>2F 6 (2.9) 29 (12.8) 2 (0.7)
Rearrangement 1
(0.5)(
-)
14
(6.2)(
-)
259
(87.2)
+
1F1G1R 1 (0.5) 7 (3.1) 134 (45.1)
≥1F, ≥1G, and/or ≥1R 0 (0) 1 (0.4) 42 (14.1)
≥1F, 1G, ≥1R 0 (0) 0 (0) 13 (4.4)
≥1F, ≥1G, 1R 0 (0) 1 (0.4) 15 (5.1)
≥1F, 1G, ≥1R ≥1F,
≥1G, 1R
0 (0) 0 (0) 9 (3)
≥1F, ≥1G, and/or
≥1R only
0 (0) 0 (0) 5 (1.7)
Others,
Rearrangement
0 (0) 6 (2.6) 83 (27.9)
Total 207 (100) 227 (100) 297 (100)
Results are same with Clinical results: 10 cases

36. EXPANDING ON PREVIOUS
WORK

38. ASSESSMENT OF HER2 AMPLIFICATION STATUS IN
INVASIVE BREAST CANCER USING BRIGHT-FIELD IN SITU
HYBRIDIZATION AND DIGITAL PATHOLOGY
Poster Presentation
by Shimul

39. Storage Repository
H&E slide CISH slide
Clien
t
User-
specifie
d
Region of
interest
defined by
the program
HPCC
Machine
learning
Segmentatio
n
Quality
evaluation
H&E slide
invasive
cance
area
detection
DMSKPDIGIPATH
Results
exporte
d
Images and
data
imported
to HPCC
Quantific
ation of
signals
Web Interface
and Viewer application
Results transmitted to
web interface
Images transmitted
for visualization
Region of
interest
detected on
H&E slide
AUTOMATED CISH PROJECT

40. SHIMARIS PACQ: IN-HOUSE APPLICATION
PACQ: Pathologists Assistant for Automatic CISH Quantification

41. A SAMPLE CISH REPORT BY
THE APPLICATION

42. CISH ANALYSIS

43. HISTOLOGY 3DTO WHOLEBLOCKIMAGING

44. UNDERSTANDING PATHOBIOLOGY
BY 3D IMAGING OF HUMAN TUMORS
(LUNG ADENOCARCINOMA)

45. METHODO
LOGY
FFPE
block
200 serial
sectionin
g, 5um
thickness
HE
Staining
&
Digitizatio
n
3D
reconstru
ction
0.5um/pixel
The Protocol of 3D Histology

46. 3D Reconstruction of Lung Adenocarcinoma: “Islands of
Tumor Cells” from Previous Studies (2010-2015)

47. Spreading Through Air Space (STAS) – Another Manner of Tumor
Invasion/Extension in Lung Adenocarcinoma?

48. A B C D
E F G F
Tumor Islands can be a source of Spreading Through
Air Space (STAS)

49. THREE-DIMENSIONALASSESSMENTOF
SPREADTHROUGHAIRSPACESINLUNG
ADENOCARCINOMA
YUK A K O YA GI, PHD(1), K A ZUHIR O TA BA TA , MD (1) ; N A TA SHA R EK HTMA N , MD(1);TA K A SHI
EG UCH I, MD (2); XIUJUN F U, MD (1); JOSEPH MON TECA LV O, MD. (1), PR A SA D A DUSUMILLI, MD
(2); MEER A H A MEED, MD (1) WILLIA M D. TR A V IS, MD(1)
1: DEPT OF PA TH OLOG Y , MSK CC; 2: TH OR A CIC SUR G ER Y , MSK CC

50. BACKGROUND
AND AIM
Tumor spread through air space (STAS) is a newly recognized form of
invasion in lung adenocarcinoma and squamous cell carcinoma.
Growing evidence shows that STAS is associated with recurrence and
survival. The observation that tumor STAS clusters/nests or single cells
within air spaces on two-dimensional H&E slides raised the question of
“how these cells could survive within air spaces without a vascular supply?” This
question has led some to speculate that STAS is an artifact.
We have performed the high resolution-high quality 3D reconstruction
and visualization of normal lung and tumor in a lung adenocarcinoma to
investigate the invasive pattern of STAS.

51. +
(a) In the main tumor area, micropapillary structures within airspaces were
connected to alveolar walls
Y-Z
X-Z
X-Y

52. (b) 3D evaluation many STAS clusters within air spaces are attached to alveolar walls
1 2 543
9 1
0
876

53. (c) 3D evaluation shows STATS clusters are actually balls of tumor
cells surrounding a central space.

54. (c) 3D evaluation they are actually balls of
tumor cells surrounding a central space.

55. Discussion
AVASCULAR
MICROPAPILLARY
CD31 highlights lack of vessels
in main tumor micropapillary
area
H&E CD31&Type IV collagen
TTF1CD31 E-Cadherin Type IV collagen
CO-OPTION
This area is away from main tumor. and shows a
lack of vessels by CD31 (green) in both micro
papillary STAS (highlighted by e-cadherin in red)
in tumor cells along alveolor walls, suggesting
the mechanism of co-option

56. Three-dimensional immunofluorescence analysis of dynamic
vessel co-option of spread through air spaces (STAS) in lung
cancer
Presenter Name, Institution, Country
Yukako Yagi1, Rania G. Aly1, Kazuhiro Tabata1, Natasha Rekhtman1, Takashi Eguchi1,
Joseph Montecalvo1, Katia Manova2, Prasad S. Adusumilli1, Meera Hameed1, William D.
Travis1;
1. Memorial Sloan Kettering Cancer Center, New York, NY/United States of America,
2. Sloan Kettering Institute, New York, NY/United States of America

57. DOUBLE IHC STAINS VS MULTIPLEX IF
TTF1CD31
E-CadherinType IV collagen
TTF1CD31 E-Cadherin Type IV collagen

58. The multiplex IF staining
highlighted the co-option
which is consistent with a
mechanism of reattachment
of STAS (TTF-1 and E-
Cadherin positive) to distant
alveolar wall capillaries
(CD31 positive) with
preservation of the alveolar
wall.
TTF-1
E-
Cadherin
CD31
Collagen
IV
Multiplex Immuno-Florescence Staining

59. The multiplex IF staining highlighted the co-option
which is consistent with a mechanism of reattachment
of STAS (TTF-1 and E-Cadherin positive) to distant
alveolar wall capillaries (CD31 positive) with
preservation of the alveolar wall.
Multiplex Immuno-Florescence Staining in 3D

60. Multiplex Immuno-Florescence Staining in 3D

61. Multiplex Immuno-Florescence Staining in 3D

62. WHOLE ORGAN 3D IMAGE RECONSTRUCTION
One whole organ : 10-20
tissue slices
One tissue slice:
5-15 blocks
One tissue block: 1000 –
1500 slides
The average size of WSI is
10GB
The average size of WSI
file is 800 MB
The estimated average file
size for the whole organ
3D model is 200TB.
Missing tissue, slide quality, image quality and
color variation are always issues in histology 3D
imaging

63. WORKFLOW OF WHOLE ORGAN 3D IMAGE
RECONSTRUCTION
Is there
any
better
way?
I
don’t
know
…
Whole Organ 3D image Reconstruction
One whole organ : 10-20
tissue slices
One tissue slice:
5-15 blocks
One tissue block: 1000 –
1500 slides
The average size of WSI is
10GB
The average size of WSI
file is 800 MB
The estimated average file
size for the whole organ
3D model is 200TB.
Missing tissue, slide quality, image quality and color
variation are always issues in histology 3D imaging

64. MICROCT

65. Micro-Computed Tomography (MicroCT)
Nikon custom 130
kVp X-Ray source
Reconstruction & Imaging
Station
Systems Used:
• VG studio (Volume
graphics)
• Dragonfly (Object
Research Systems)
• Imaris (Bit Plane)
Normal Size and Whole Mount Block
Scanning Times
• Fresh Tissue:
1.5-6min/sample
• FFPE Tissue Block:
7-60 hours/sample
• Modified Golgi
Stained Mouse
Brain:
5-60 hours

66. RADIOGRAPHIC IMAGES 3D VOLUME
From thousands of images like these, a computer algorithm
generates a 3D volume which can be sliced in software to reveal the
internal structure of the object.
CT Dataset
Reconstruction to
create Volume
Volume used
for Analysis
Micro-Computed Tomography (MicroCT)

67. Fresh Tissue: Breast
Fresh Breast Tissue: 5
min scan using Micro-CT
Micro-Computed Tomography (MicroCT)

68. Fresh Tissue Micro-CT
FFPE Block
Fresh tissue CT
Block CT
H&E
Sectioning, staining,
and scanning
Fresh tissue CT
Tumor area
Fresh-FFPE block-Histology Slide Correlation using Micro-CT

69. MICROCT
• Micro-CT imaging of breast fresh tissues shows potential applications for the
intra-operative assessment of invasive carcinoma in surgical margin.
Margin Detection
Whole Block Imaging
• Tumor regions were detected from the micro-CT WBI of all tissue types tested.
Tissue that was larger in size or within a whole mount block showed better
results.
We have tested Bone, Prostate, Breast, Lung, Lymphoma and Thyroid
After completing a series of optimization, visualization, and image analysis studies, the
image quality and interpretation of Micro-CT images has significantly improved.

70. CARCINOID TUMOR HUMAN LUNG

72. Background:
MicroCT

73. EXPLORINGTHE FEASIBILITY OFUTILIZING MICRO-CTTO DETECTCLINICALLY
RELEVANTPATHOLOGICAL ALTERATIONS INPARAFFINBLOCKSOFRECTALCANCER
SPECIMENS

74. A CASE STUDY
Case 1: Rectal adenocarcinoma

75. 1 CASE: 50 WHOLE MOUNT
BLOCKS
Focused area (11 blocks were combined)

76. 1 CASE: 50 WHOLE MOUNT
BLOCKS
Focused area (11 blocks were combined)
Normal size
Whole mount

77. 1 CASE: 50 WHOLE MOUNT
BLOCKS
Focused area (11 blocks were combined)

78. WHOLE BLOCK IMAGING
ANALYSIS

79. WHOLE BLOCK IMAGING
ANALYSIS: RADIOLOGY VIEW VS
HISTOLOGY VIEW

80. BLOOD VESSEL
SEGMENTATION

81. FUTURE WORKS FOR AUTOMATED
FEATURE EXTRACTION AND
MEASUREMENT Tumor
TF
TD
LN
Metastatic LN
etc

82. Multi-Modal Data Management
We have reduced # of
slide for histology 3D,
From 100-300 slides to
0-50 slides.

83. • Dimensions
• Color
• orientation
• Symmetry
• Margin
• Parenchyma
• Plan for section
• Areas to sample
• Plan the fixation
protocol
Fres
h/fix
ed
tissu
e
MicroCT
scanning
Digitally
stained H&E
color model
Correlation between microCT
and histology
microCT at JRSC
(10th floor)
Sections/
stained
P1000
3DHistech
Hamamatsu S60
whole
Mount
Slides
Hamamatsu 2.0
HT
Normal size
slides
FISH: 3.5 G
(+2.5 G for
matching H&E
slides
Wh
ole
bloc
k
Gross specimen
examination
Confocal Scanner
Multilayer
slides
Specimen
image file File format and
mean output file
sizes
Hamamatsu
NDPI
40X:5.5 G
20X:1.5 G
Hamamatsu
NDPI
40X:2.5G
20X:0.5G
3DHISTECH
MRXS
40X:4G
Automated signal scoring
system for fluorescent in
situ hybridization (FISH)
images by application of
image analyses and deep
learning approaches
Project that uses
the image files
Type of Slide
Scanner
MicroCT for
histopathological
assessment of tissue
resected by endoscopic
submucosal dissection
(ESD)
Assessment of the
feasibility of utilizing micro-
CT to detect clinically
relevant pathological
alterations in paraffin
blocks of rectal cancer
specimens
Generates .vgi and
.vol output (Raw
data size :40G;
Processed data : 5-
8 G
Projects related to microCT
Automated analysis of
chromogenic in situ
hybridization (CISH)
images by image
analysis and machine
learning in breast
cancer
P250
THE SYSTEM STRUCTURE FOR MULTIMODAL DATABASE
Histology 3D imaging

84. SUMMARY: IMPORTANT ASPECTS FOR
MULTI MODALITY IMAGING
Data Quality of each modality
The mechanism to connect all information accurately and appropriately
The data management of all modalities
Machine learning can be applied to derive more data from all modalities and
computational analysis allows the development of algorithms to improve efficiency of
slide review, calculate numerical data, and merge pathology data with molecular, clinical
or other large data sets. The criteria for diagnosis can thus be made more objective,
based on the use of deep learning.
New data structure for multi-modality imaging.
New technologies and testing the newest technologies for enhanced digital microscopy
help to improve even AI outcomes.

85. Acknowledgements
We acknowledge to the Warren Alpert Foundation for supporting our research.

86. THANK YOU!
LAB Member
We are recruiting a post doc./
research fellow!

87. Thank you!

88. Thank you!

89. Mitosis Counting per scanner type

90. 4um H&E
8um H&E 8um H&E 8 layers Extended Focus
4um PHH3
MITOSIS COUNTING: TISSUE THICKNESS & SC

91. 4um H&E
8um H&E 8um H&E 8 layers Extended Focus
MITOSIS COUNTING: TISSUE THICKNESS & SC
8 Layers Z-stack

92.  BF
 FL
 Confocal
 Extended Focus and Z-stuck
 0.165um/pixel resolution with 40x
(water immersion) NA 1.25
*common spec. of 40x: 0.23-0.25um/pixel, NA 0.7-0.9
• Preliminary study for research was done at MGH:
 7-11 layers with confocal mode was necessary to analyze multicolor FISH slides
• Preliminary study with clinical slides was started at MSKCC
 3 multi-color FISH slides
 9 break apart FISH slides
Optimized scanning protocol: 7 layers 5-6-um with confocal mode
• 3 Pilot studies with 8 clinical cases have been completed. All cases had same results with
clinical report
Confocal Whole Slide Scanner

93. Cytology: Confocal Whole Slide Scanner

94. Tumor Cells
(STAS)
200 serial
sectioning,
5um
thickness
H&E, IF, IHC
Whole Block Image and High resolution Histology
(H&E and multiplex IF 3D )