Session by Sabyasachi Mukhopadhyay Kolkata Lead, Facebook Developer Circle GDE in ML Intel Software Innovator Visiting Faculty, SCIT Pune Co-Founder & Chief Research Officer, Twelit MedTech Pvt. Ltd

1. Sabyasachi Mukhopadhyay
Kolkata Lead, Facebook Developer Circle
GDE in ML
Intel Software Innovator
Visiting Faculty, SCIT Pune
Co-Founder & Chief Research Officer, Twelit MedTech Pvt. Ltd
Early Diagnosis of Chronic Diseases by
Smartphone AI

2. What problem should we solve?
Use the process below to Identify, Articulate and
Probe new Problems worth solving

3. Goal: What is a problem worth solving?
Pick a theme e.g. ‘Digital Tech in Ag’
1. Resource Map
A. People and Orgs
B. Users and beneficiaries
C. Breakthroughs and risks
2. Problem Canvas
3. Solution Canvas
4. Findings Plot

4. 1. Resources Map
A. People and Orgs
B. Users and beneficiaries
C.Breakthroughs and risks

5. 1A. Orgs/Lists
• Orgs, Companies, Startups, Govt. dept
• Data sets and information sources
• Partners
• People, VCs, Influentials
• Known Published Challenges (e.g. UN, Gates, NSF, NIH,
DBT/DST)

6. 1B. Users and beneficiaries
• Graph of users and use cases
• Influencers
• Buyers/sponsors
• Partners
• Where do we need research?
• Speak with experts!

7. 1C. Breakthroughs, benefits and risks
• Breakthroughs, Parallel tech, Trends
• Payoffs, results, benefits  IMPACT
• Friction – regulation, biases, etc.
• Financial risks
• Execution risks/ limiting factors (e.g., time, distance, lack
of existing infrastructure)

8. 2. Problem Canvas

9. 3. Solution Canvas

10. 4. Findings Plot
• Take a guess on two major factors that matter to you (usually cost and time).Cost is a proxy for
complexity etc. Time can be proxy for Engg time, regulation, market penetration etc.

11. MILESTONES
Map of users, resource requirements, dependencies, risks
and workarounds, steps, decision points
----------

12. Guides

13. Heilmeier questions*
1. What are you trying to do? Articulate objectives without using jargon.
2. How is it done today? What are the limits of current practice?
3. What’s new in your approach? Why do you think it will be successful?
4. Who cares?
5. If you’re successful, what difference will it make?
6. What are the risks and payoffs?
7. How much will it cost?
8. How long will it take?
9. What are the midterm and final “exams” to check for success?
* 9 questions to develop a meaningful project

14. LEAN CANVAS

15. Let’s Dive In

16. Introduction
 Optical Methods for tissue characterization
Light scattering
form Biological
tissue
Elastic Scattering
spectra from
biological tissue
Morphological information: Shape and size,
refractive index, intra-cellular organelles etc.
Micro-optical property: Fractal parameters,
disorder strength
Inelastic Scattering: Fluorescence and Raman Spectroscopy
Biochemical Information

17. Present interest: Light scattering based inverse analysis models – quantification of sub-
diffractional changes in tissue refractive index fluctuations.
 Monofractal approximation may not be realistic in biological tissue having wide range of
dimensions and the complex nature of the spatial correlations
 Spatial distribution of refractive index (RI) in biological
tissue observed to exhibit statistical self-similarity
 Analyzed using fractal-Born approximation
--------- Phys. Rev. Lett. 97, 138102 (2006).
Multifractal?
Multiple scaling exponents Multi-resolution analysis
 Scale invariant, power law dependence, single scaling
exponent

18. • Fractal / Self-similarity are process / geometry, which repeat its pattern in different scale of space
or time.
• Fractal dimension: measure the space filling capacity in geometry.
Fractals (self-similar) and Multifractals
( )P 
  

2 EH D  
 Follows power law: ;  being the spatial frequency. Fractal properties quantified by a single
scaling exponent, , where 𝑫 𝑬 = Euclidian Dimension, H = Hurst exponent; 0 < H <1.
 Fractal dimension: 𝐷𝑓 = 𝐷 𝐸 − 𝐻 + 1.

19. • Observations-
• Power law scaling in wide range of spatial frequency with
background randomness-complexity of RI fluctuations
• Multiple scaling exponent is observed- Fourier analysis is
not adequate---
---- signature of multifractality?
DIC image: Connective Tissue
Unfolding the image
to 1D Spatial Refractive
Index (RI) variation
Fourier analysis
Observing multifractality in spatial refractive index fluctuations
 Required a better multiresolution analysis to
characterize such Complex self-similar behavior.
 Multifractal Detrended Fluctuation Analysis
(MFDFA) is one of the state of the art
multi-resolution analysis can be used.

20. Data
  NixxiY
i
k
k ...,1,)(
1
 
)/int( sNNs 
 

s
i
iyisY
s
sF
1
22
)(])1[(
1
),(   
q
N
q
s
q
s
sF
N
sF
/1
2
1
2/2
),(
2
1
)(






 

)(
)( qh
q ssF 
Multifractal Detrended Fluctuation Analysis (MFDFA)
Profile Y(i) Divide Y(i)
Local Fluctuation Fluctuation Function
Log-Log Plot of s vs. Fq(s) h(q)=
log[Fq(s)]
log(𝑠)
τ 𝑞 = 𝑞ℎ 𝑞 − 1 α = τ′ 𝑞 f(α) = qα - τ 𝑞
H. Eugene Stanley et al, Physica A 316 (2002) 87 – 114.
 Width of the singularity spectrum (f(α)): 𝝈
 N - length of the fluctuation
series
 s - length of each segments
 Order of moments q
)(sFq
Negative q capture
small fluctuations
 Classical scaling exponent, 𝜏(q)
 Hurst exponent, h(q)

21. Confirmation of multifractality: MFDFA analysis on DIC image
Multifractality confirmed:
---- Significant variation of slope with moment q - 𝒍𝒐𝒈[Fq(s)] vs. log(s)

22. Quantification of Multifractality in Different Grades of precancerous Tissue
 Differences more prominent for negative q-values
h(q=2)  at Higher grade
 Width of singularity spectrum (σ)  at Higher grade
Small scale (sub-m) fluctuations!
Stronger multifractality
Finer index fluctuations (roughness / heterogeneity)

23. Tissue Region Hurst exponent
( mean h(q=2) ± standard deviation)
Width of singularity spectra
(mean 𝛔 ± standard deviation)
Gr-I Gr-II Gr-III Gr-I Gr-II Gr-III
Connective Tissue 0.54 ± 0.03 0.50 ± 0.04 0.36 ± 0.08 0.60 ± 0.10 0.68 ± 0.13 0.88 ± 0.07
Spatial refractive index variation in biological tissue exhibits multifractality
direct evidence (MFDFA on DIC image)
 Need of the hour: Extracting multifractal information from Light scattering
-------- clinically amenable and in-situ deployable
Key results
MFDFA results on different pathology grades
SPIE Proceedings, 2014

24. Scheme of extracting multifractality from light scattering spectra
Elastic scattering spectra from
tissue
Fourier domain inverse pre-
processing
Subjected to MFDFA
Strength of Multifractality = 𝛔;
Hurst exponent: h(q=2)
𝜼/
𝝆 : representative index
inhomogeneity distribution in
spatial scale ρ

25. Extraction of multifractality from tissue light scattering spectra
Scattering spectra from tissue
Experimental light scattering spectroscopic system
Multiple power law exponent
--- signature of multifractality?
Wave length variation [I()]; Fixed 

26. Sample Preparation
Biopsied cervical precancer tissue slices (The histopathologically
characterized grade I, grade II, grade III precancer tissue) and normal tissues
were collected from Ganesh Shankar Vidyarthi Memorial (GSVM) Medical
College, Kanpur, India (age of patients between 35 – 60 years; ntotal = 35, with
ngrade I = 14, ngrade II = 6, ngrade III = 9; four biopsies from the normal
counterparts, nnormal = 6). The Standardized histological preparation of the
excised tissues involving fixation, dehydration, imbedding in wax, sectioning
under a rotary microtome with thickness ∼5 μm, lateral dimension ∼4 mm ×
6 mm, is followed by performing subsequent de-waxing. The consent for the
use of all the intact tissue (human cervix with cancer and normal) samples in
our study was obtained from the Ethical Committee, G. S. V. M.
Medical College and Hospital, Kanpur, India. The sample preparation methods
follow approved guidelines in our study.

27. Experimental System
The spatial distribution of tissue refractive index was recorded by a differential interference1
contrast (DIC) microscope (Olympus IX-81, USA). At a magnification of 60X, these DIC2
Images were recorded by a CCD camera (ORCA-ERG, Hamamatsu, 1344 X 1024 pixels pixel3
dimension 6.45 m). The elastic scattering spectra from the multiple sites of the biopsied tissue4
sections were recorded by the angle resolved spectral light scattering measurements (Figure 1).5
In brief, light emitted from a Xe-lamp (HPX-2000, Ocean Optics, USA) was collimated by a6
combination of lenses and illuminated the tissue sample at the centre of a goniometric7
arrangement (spot size ~1-mm-diameter). The collimated scattered light from sample was8
focused into a collecting fiber probe coupled to a spectrometer (USB4000FL, Ocean Optics,9
USA) for wavelength resolved signal detection. The recordings of spectra were performed (360-10
800nm) with a spectral resolution of 2.05 nm, where the angular range was kept at 10 150 11
with an interval of10. For the inverse multifractal study the spectra were recorded at12
backscattering angle  = 150o
13

28. Schematic of the spectral light scattering measurement. Xe lamp: excitation source; A:
aperture; L1: collimating lens;L2: illuminating lens; L3 and L4: collecting lenses; f1: Focal
length of collimating lens L1; f2: focal length of illuminating lens L2 ; f3 and f4: Focal
lengths of collecting lenses L3 and L4 respectively.

29. Extraction and quantification of tissue multifractal parameters
 Light scattering intensity
from Grade I tissue
 Detrended refractive index
Fluctuations extracted from
Fourier domain preprocessed
signal
 Multifractality
confirmed:-
Significant variation of
slopes with moment q-
𝒍𝒐𝒈[Fq(s)] vs. log(s)
 Prominent variation in negative q-values Small scale (sub-m) fluctuations

30. Summary of results on tissue
 Consistency of the multifractal trends
 statistical significance of the observed differences in the multifractal parameters
No. of tissue samples studied: Healthy-20,Grade I -15, Grade II- 15 and Grade III -25
 Hurst exponent, h(q=2)  --- implies overall reduction in the depth correlation of refractive index
 Strength of multifractality, σ -- indicates the increased of roughness / inhomogeneity of depth
resolved index variations
 Possible reasons: fibrous network in connective tissue gets fragmented due to the shortening /
breaking of the building blocks, the collagen fibers / micro-fibril with the progression of cancer /
precancer.
J. Opt. 18(12), 125301 (2016)
Normal Grade-I Grade-II Grade-III
Hurst Exponent(h(q=2)) 0.630.02 0.560.05 0.480.03 0.410.04
Singularity Spectrum
Width (∆α)
0.860.01 0.900.03 0.960.04 0.990.01

31. Our initial dataset consists of 75 samples. We have used Monte Carlo cross-validation,
where we randomly split the dataset into training and testing dataset. This process has
been repeated 100 times. The size of training and testing dataset varies for each split;
we just ensured that a minimum (2) no of samples have been included in both training
and testing dataset for each split. For each such split, the model has been fit to the
training data, and predictive accuracy has been assessed using just the validation data
(testing data). Then the results of the testing data have been averaged over the splits.
The advantage of this method is that the proportion of the training/validation split is not
dependent on the number of iterations and the predictive accuracy is more or less
independent of the samples used for training dataset. We additionally have used 9
unknown samples taken at different time than the dataset and the prediction done by
our model has been compared by manual verification. Results of the unknown samples
also have been averaged and presented in the results. The above process has been
repeated for SVM and HMM.

32. Support vector machine (SVM) based multiclass classification on extracted
multifractal parameters from tissue samples.

33. SVM based tissue classification based on light scattering –derived multifractal tissue optical
properties. The horizontal surfaces of these figures have been sectioned into 4  4 rectangles.
The accurate and inaccurate prediction of each stages (normal, grade I, grade II, grade III)
have been represented by diagonal and off diagonal rectangles respectively.
Normal
Grade I
Grade II
Grade III
0
10
20
30
40
50
60
70
80
90
100
Normal
Grade I
Grade II
Grade III
98.5
14.29
10
0
1.5
57.14
35
0
0
28.57
55
0
0
0
0
100
Normal
Grade I
Grade II
Grade III

34. The flowchart of HMM based model on multifractal tissue optical properties
derived from light scattering spectra.

35. HMM based tissue classification based on light scattering –derived multifractal tissue optical
properties. The horizontal surfaces of these figures have been sectioned into 4  4 rectangles.
The accurate and inaccurate prediction of each stages (normal, grade I, grade II, grade III) have
been represented by diagonal and off diagonal rectangles respectively.

36. The results demonstrate that binary classification between normal and
cancerous tissues (grade III) are very good both in SVM and HMM. Meanwhile
in multiclass classification cases, when precancerous grades (grade I, grade II)
are to be classified along with normal and precancerous tissues (grade III),
abstract parameters achieved using HMM performs better than the SVM. The
presence of noise in the obtained signal damages the SVM performance as
SVM clearly classifies based on the kernel formed after considering all the
multifractal parameters. While in the case of HMM, the Markov model finds
abstract parameters by controlling the actual multifractal parameters and
produces prediction based on the derived abstract parameters. As a
consequence HMM avoids the noise added to the signal and able to produce
better multiclass classification results than SVM.

37. Probing depth resolved multifractality in human retina from Optical Coherence
Tomography (OCT) image
 OCT can extract depth resolved index variation
 Analyzed via MFDFA to extract alteration of multifractality due to progress of
diabetic macular edema (DME) and age related macular degeneracy.
• Cropped different layers of retina
 Acquired in-vivo retinal OCT
 Extracted depth resolved optical index variations of different layers

38. Variation of depth resolved optical index from OCT Images of in -vivo human retina
 ILM: Inner limiting
membrane
 NFL: Nerve fiber layer
 GCL: Ganglion cell layer
 IPL: Inner plexiform layer
 INL: Inner nuclear layer
 OPL: Outer plexiform layer
 ONL: Outer nuclear layer
 ELM: External limiting
membrane
 IS/OS: Inner and outer
photoreceptor
 RPE: Retinal pigment
epithelium
 OPR: Outer segment
PR/RPE complex
Diabetic macular edema

39. Quantification of multifractality in depth resolved optical index variation in in-vivo
human retinal layers
Prominent difference in negative q indicates DME
related index alters mainly in small length scale
Analysis performed on Outer plexiform layer(OPL)
 h(q=2)  in OPL of diabetic macular edema (DME): indicates reduction of optical index
correlation in progress of DME
 σ  in OPL of diabetic macular edema (DME): indicates increase of strength of multifractality or
heterogeneity in layers with progress of DME
variation of 𝒍𝒐𝒈[Fq(l)] vs. log(l) slope
variation of h(q):
multifractality

40. Retinal Layers h(q=2) σ
Healthy AMD Healthy AMD
NFL 0.87 ± 0.04 0.66 ± 0.01 0.62 ± 0.08 0.93 ± 0.09
GCL 0.81 ± 0.03 0.61 ± 0.02 0.85 ± 0.02 1.57 ± 0.07
IPL 0.72 ± 0.02 0.66 ± 0.03 0.77 ± 0.05 1.23 ± 0.08
OPL 0.62 ± 0.01 0.53 ± 0.02 0.65 ± 0.09 1.00 ± 0.10
ONL 0.61 ± 0.03 0.51 ± 0.02 1.02 ± 0.07 1.31 ± 0.06
Choroid 0.76 ± 0.04 0.57 ± 0.03 0.82 ± 0.08 1.27 ± 0.12
Results of multifractality in healthy and age related macular degeneracy (AMD) in
in-vivo human retinal layers
 Hurst exponent, h(q=2)  --- implies overall reduction in the depth correlation of refractive index
 Strength of multifractality,σ  -- indicates the increased of roughness / inhomogeneity of depth
resolved index variations
J. Biomed. Opt. 21(9), 096004 (2016)

41. [1] N.Das, S.Mukhopadhyay, N.Ghosh, J.Chhablani, A.Richhariya, K.D.Rao, N. K.Sahoo, “Investigation of Alterations in multifractality
in Optical Coherence Tomographic Images of In Vivo Human Retina”; Journal Of Biomedical Optics 21(9), 096004 (Sep 09, 2016).
[2] S.Mukhopadhyay, N.Das, I.Kurmi, A.Pradhan, N.Ghosh, P.K.Panigrahi, “Tissue multifractality and hidden Markov model based
integrated framework for optimum precancer dectection”, Journal Of Biomedical Optics 22(10), 105005 (Oct 19, 2017).
[3] S.Mukhopadhyay et al., “Recurrence Quantifications as Potential Bio-markers for Diagnosis of Pre-Cancer”, SPIE Proceeding, SPIE
Photonics West, 2017, USA.
[4] S.Mukhopadhyay et al., “Optical Diagnosis of Cervical Cancer by Intrinsic Mode Functions”, SPIE Proceeding, SPIE Photonics
West, 2017, USA.
[5] S.Pratiher, S.Mukhopadhyay, R.Barman, S.Pratiher, A.Pradhan, N.Ghosh, P.K.Panigrahi, “Optical Diagnosis of Cervical Cancer by
Higher Order Spectra and Boosting”, SPIE Proceeding, SPIE Photonics West, 2017, USA.
[6] S.Mukhopadhyay et al., “Optical diagnosis of colon and cervical cancer by support vector machine”, SPIE Proceeding, SPIE
Photonics Europe, Belgium, 2016, Europe.
[7] S.Mukhopadhyay et al., “S-TRANSFORM BASED FLUCTUATION ANALYSIS- A METHOD FOR PRECANCER
DETECTION”, IEEE Conference Proceeding, Microcom-2016, India.
[8] S.Mukhopadhyay et al., “Wavelet and multi-fractal based analysis on DIC images in epithelium region to detect and diagnose the
cancer progress among different grades of tissues”, SPIE Proceeding, SPIE Photonics Europe, Belgium, 2014, Europe.
[9]S.Mukhopadhyay et al., “Pre-cancer Detection by Wavelet Transform and Multi-fractality in various grades of DIC Stromal
Images”, SPIE Proceeding, SPIE Photonics West, 2014, USA.
[10] S.Mukhopadhyay et.al., “Efficacy of hidden Markov model over support vector machine on multiclass classification of healthy and
cancerous cervical tissues”, SPIE Proceeding, SPIE Photonics West, 2018, USA.
[11]S.Mukhopadhyay et al., “A two-stage framework for DIC image denoising and Gabor based GLCM feature extraction for pre-cancer
diagnosis”, SPIE Proceeding, SPIE Photonics West, 2018, USA.
My Publications Related To Early Stage Disease Detection

42. 9
Impact on BOP Level
Pain free Early Stage Cancer Diagnosis and Low Cost Portable Device
First Solution
Generate report in a few minutes where biopsy examination takes several
days
Second Solution
Automatic Solutions and Highly Efficient
Third Solution
Stay Away From Tobacco : Be Safe in the Sun : Eat Healthy and Get Active

43. Smart Phone based Cancer & DME Solutions

44. Unique Selling Proposition
100 Times Cheaper than Biopsy and
Accuracy >95%
Portable & Lightweight and 80 Times
Faster than Biopsy
Replacing Painful and Cost Effective
Biopsy

45. Business Model
B2B Product for Ophthalmologists
DeviceSaaS
Rs100/Scan One Time
Rs20000

46. Growth Strategiesowth Strategy
FutureCurrent
Ongoing Clinical Trail
in Hospitals
Doctors Collaboration 50-
50 Revenue Sharing Model

47. Competitive Landscape of Our Innovation Twelit
Low Cost
Accuracy >95%
Smart Device
Data Driven
Cloud Based

48. Media Reports on Published Research: New technique for early detection of human eye
diseases
Televisions:
[1]Zee News: http://zeenews.india.com/health/this-new-technique-can-detect-human-eye-
diseases-early-1962133
[2]Odisha TV: https://odishatv.in/technology/new-technique-for-early-detection-of-human-
eye-diseases-184520/
[3]News World India: https://newsworldindia.in/lifestyle/now-early-detection-of-human-
eye-diseases-is-possible-know-how/240282/
[4]NewsX: http://www.newsx.com/health-and-science/50839-new-technique-for-early-
detection-of-human-eye-diseases
Global News Site:
Yahoo News: https://in.news.yahoo.com/technique-early-detection-human-eye-diseases-
081403358.html
Science Magazine:
Nature India: http://www.natureasia.com/en/nindia/article/10.1038/nindia.2016.169

49. Media Reports on Published Research: New technique for early detection of human
eye diseases
Newspapers
[1]The Hindu: http://www.thehindu.com/sci-tech/science/Indian-
scientists%E2%80%99-novel-approach-to-diagnose-retinal-
diseases/article17004895.ece#comments
[2]Business Standard: http://www.business-standard.com/article/news-ians/new-
technique-for-early-detection-of-human-eye-diseases-116122700377_1.html
[3]The Indian Xpress: https://theindianxpress.com/20056/
[4]The Economic Times: http://economictimes.indiatimes.com/news/science/new-
technique-for-early-detection-of-human-eye-
diseases/articleshow/56198412.cms?from=mdr

50. 24
24. Cancer Research Success
Newspapers & Research Magazines:
[1]The Hindu:http://www.thehindu.com/sci-tech/science/diagnosing-early-stage-
cervical-cancer-using-artificial-intelligence/article22267117.ece
[2]Business Standard:http://www.business-standard.com/article/news-
ians/computer-based-optical-method-detects-early-stage-cervical-cancer-
117122800553_1.html
[3]The Indian Express: http://indianexpress.com/article/lifestyle/health/computer-
based-optical-method-detects-early-stage-cervical-cancer-5002420/
[4]Analytics India Magazine: https://analyticsindiamag.com/ai-cancer-detection-india-
light-scatter-algorithm/
[5] Nature India
:https://www.natureasia.com/en/nindia/article/10.1038/nindia.2017.148
Stay Away From Tobacco : Be Safe in the Sun : Eat Healthy and Get Active
AI based early stage cancer detection

51. IBPS,Banking,IAS,UPSC Exams Current Affair
Syllabus
[1] http://bankexamportal.com/daily-current-affairs/24-
december-2017
[2]http://iasexamportal.com/civilservices/daily-current-
affairs/24-12-2017

52. 25
Award On AI Based Early Stage Disease Detection
My futuristic research on ‘AI Based Early Stage Disease
Detection’ was nominated among top 10 finalists in Global
IMT X-Prize international competition in USA. Dr.Barmak
Heshmat, Research Scientist, MIT was one of the organizers of
IMT X-Prize global competition. The panel comprised of
eminent scientists and artists from MIT, NASA, Marvel
Studios (Franchises of block buster Hollywood movies like
X-Men, Spider Men). The competition theme was to produce
futuristic ideas for 'AI (Artificial Intelligence) for Good'. I was
among top 10 finalists but ended as a runner up.

53. [1] We are currently expanding our investigations towards in-vivo deployment of
this integrated approach for precancer detection using tissue light scattering
spectra.
[2] Industrial implications are going on at LVPEI, Hyderabad for early stage DME
and diabetic retinopathy detection of eyes.
Ongoing and Future Studies

54. @SM2017Official
@sabyasachi_unique
https://www.linkedin.com/in/sabyasachi-mukhopadhyay-303a1027/
@sabyasachi_mukhopadhyay
Join me:

55. THANK YOU
For your time & we’ll see you soon