Whispers of Speckles (Part I: Building Computational Imaging Frameworks for Acoustic and Optical Speckle Imaging) + (Part II: Enlightenment from Shallow to Complex Reasoning with Deep Learning)
Whispers of Speckles
(Part I: Building Computational Imaging Frameworks for Acoustic and Optical Speckle Imaging)
(Part II: Enlightenment from Shallow to Complex Reasoning with Deep Learning)
Presented at the Workshop on Machine Learning for Medical Image Analysis (WMLMIA), IIT Mandi, 25 June 2015.
Deep Learning - What's the buzz all aboutDebdoot Sheet
Deep learning is a genre of machine learning algorithms that attempt to solve tasks by learning abstraction in data following a stratified description paradigm using non-linear transformation architectures.
When put in simple terms, say you want to make the machine recognize some Mr. X with Mt. E in the background, then this task is a stratified or hierarchical recognition task. At the base of the recognition pyramid would be kernels which can discriminate flats, lines, curves, sharp angles, color; higher up will be kernels which use this information to discriminate body parts, trees, natural scenery, clouds, etc.; higher up will use this knowledge to recognize humans, animals, mountains, etc.; and higher up will learn to recognize Mr. X and Mt. E and finally the apex lexical synthesizer module would say that Mr. X is standing in front of Mt. E. Deep learning is all about how you make machines synthesize this hierarchical logic and also learn these representative kernels all by itself.
Deep learning has been extensively used to efficiently solve these kinds of problems from handwritten character recognition (NYU, U Toronto), speech recognition (Microsoft, Google Voice), lexical ordered speech synthesis (Google Voice, iPhone Siri), object and poster recognition (Cortica), image retrieval (Baidu), content filtering (Youtube, Metacafe, Twitter), product visibility tracking (GazeMetrix), computational medical imaging (IITKgp).
This talk will focus on the buzz around this topic and how firm does the buzz hold on to the claims it boasts of?
Detection of Retinal Vessels in Fundus Images through Transfer Learning of Ti...Debdoot Sheet
This document proposes a method for detecting retinal vessels in fundus images through transfer learning of tissue-specific photon interaction statistical physics. It discusses the physics of retinal imaging, defines a statistical physics model of tissue-photon interaction (TPI), and presents a framework for learning the TPI model through training images to perform vessel detection on test images. Performance is assessed using metrics like accuracy, sensitivity and specificity, demonstrating the method can detect vessels with accuracy exceeding 94%.
Deep Learning of Tissue Specific Speckle Representations in Optical Coherence...Debdoot Sheet
"Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology" presented at the Deep Learning for Biomedical Imaging session at the 2015 IEEE International Symposium on Biomedical Imaging (ISBI 2015) at Brooklyn, New York, USA.
The document discusses ultrasonic histology, a new technique for in vivo tissue characterization and diagnosis. It aims to address the limitations of existing diagnostic methods like CT, MRI, and IVUS, which have drawbacks like radiation exposure, long imaging times, and inability to characterize tissue composition. Ultrasonic histology applies concepts from statistical physics and machine learning to ultrasonic backscatter data to compute tissue-specific parameters related to speckle statistics. These parameters are then used to predict tissue types and characterize atherosclerotic plaques, providing a reliable, non-invasive method for ultrasonic tissue analysis and histology.
Can Generative Adversarial Networks Model Imaging Physics?Debdoot Sheet
Ultrasound imaging makes use of backscattering of waves during their interaction with scatterers present in biological tissues. Simulation of synthetic ultrasound images is a challenging problem on account of inability to accurately model various factors of which some include intra-/inter scanline interference, transducer to surface coupling, artifacts on transducer elements, inhomogeneous shadowing and nonlinear attenuation. Current approaches typically solve wave space equations making them computationally expensive and slow to operate. We propose a generative adversarial network (GAN) inspired approach for fast simulation of patho-realistic ultrasound images. We apply the framework to intravascular ultrasound (IVUS) simulation. A stage 0 simulation performed using pseudo B-mode ultrasound image simulator yields speckle mapping of a digitally defined phantom. The stage I GAN subsequently refines them to preserve tissue specific speckle intensities. The stage II GAN further refines them to generate high resolution images with patho-realistic speckle profiles. We evaluate patho-realism of simulated images with a visual Turing test indicating an equivocal confusion in discriminating simulated from real. We also quantify the shift in tissue specific intensity distributions of the real and simulated images to prove their similarity.
Full paper: https://arxiv.org/abs/1712.07881
Transfer Learning of Tissue Photon Interaction in Optical Coherence Tomograph...Debdoot Sheet
1) The document presents a method for characterizing oral mucosal tissue using optical coherence tomography (OCT) through transfer learning of tissue photon interaction (TPI).
2) TPI is manifested in OCT speckle intensity statistics and attenuation coefficients, which are used as features in a machine learning model to classify tissue types in cross-validated experiments.
3) The results demonstrate high accuracy in classifying epithelium and sub-epithelium tissues, indicating potential for in vivo oral mucosal tissue characterization and diagnosis of cancers or pre-cancers.
DevFest19 - Early Diagnosis of Chronic Diseases by Smartphone AIGaurav Kheterpal
Session by Sabyasachi Mukhopadhyay
Kolkata Lead, Facebook Developer Circle
GDE in ML
Intel Software Innovator
Visiting Faculty, SCIT Pune
Co-Founder & Chief Research Officer, Twelit MedTech Pvt. Ltd
This paper presents an automated method for extracting regional data from positron emission tomography (PET) images. The method uses magnetic resonance (MR) images co-registered to PET images to individualize a standard set of regions of interest (ROIs). The ROIs are refined based on each individual's MR image to account for anatomical variations. Time-activity curves obtained with the automated method are highly correlated with those obtained manually, with differences generally within 8%. The automated method eliminates variability between raters, reduces analysis time, and provides accurate PET data quantification as an alternative to manual delineation of ROIs.
Deep Learning - What's the buzz all aboutDebdoot Sheet
Deep learning is a genre of machine learning algorithms that attempt to solve tasks by learning abstraction in data following a stratified description paradigm using non-linear transformation architectures.
When put in simple terms, say you want to make the machine recognize some Mr. X with Mt. E in the background, then this task is a stratified or hierarchical recognition task. At the base of the recognition pyramid would be kernels which can discriminate flats, lines, curves, sharp angles, color; higher up will be kernels which use this information to discriminate body parts, trees, natural scenery, clouds, etc.; higher up will use this knowledge to recognize humans, animals, mountains, etc.; and higher up will learn to recognize Mr. X and Mt. E and finally the apex lexical synthesizer module would say that Mr. X is standing in front of Mt. E. Deep learning is all about how you make machines synthesize this hierarchical logic and also learn these representative kernels all by itself.
Deep learning has been extensively used to efficiently solve these kinds of problems from handwritten character recognition (NYU, U Toronto), speech recognition (Microsoft, Google Voice), lexical ordered speech synthesis (Google Voice, iPhone Siri), object and poster recognition (Cortica), image retrieval (Baidu), content filtering (Youtube, Metacafe, Twitter), product visibility tracking (GazeMetrix), computational medical imaging (IITKgp).
This talk will focus on the buzz around this topic and how firm does the buzz hold on to the claims it boasts of?
Detection of Retinal Vessels in Fundus Images through Transfer Learning of Ti...Debdoot Sheet
This document proposes a method for detecting retinal vessels in fundus images through transfer learning of tissue-specific photon interaction statistical physics. It discusses the physics of retinal imaging, defines a statistical physics model of tissue-photon interaction (TPI), and presents a framework for learning the TPI model through training images to perform vessel detection on test images. Performance is assessed using metrics like accuracy, sensitivity and specificity, demonstrating the method can detect vessels with accuracy exceeding 94%.
Deep Learning of Tissue Specific Speckle Representations in Optical Coherence...Debdoot Sheet
"Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology" presented at the Deep Learning for Biomedical Imaging session at the 2015 IEEE International Symposium on Biomedical Imaging (ISBI 2015) at Brooklyn, New York, USA.
The document discusses ultrasonic histology, a new technique for in vivo tissue characterization and diagnosis. It aims to address the limitations of existing diagnostic methods like CT, MRI, and IVUS, which have drawbacks like radiation exposure, long imaging times, and inability to characterize tissue composition. Ultrasonic histology applies concepts from statistical physics and machine learning to ultrasonic backscatter data to compute tissue-specific parameters related to speckle statistics. These parameters are then used to predict tissue types and characterize atherosclerotic plaques, providing a reliable, non-invasive method for ultrasonic tissue analysis and histology.
Can Generative Adversarial Networks Model Imaging Physics?Debdoot Sheet
Ultrasound imaging makes use of backscattering of waves during their interaction with scatterers present in biological tissues. Simulation of synthetic ultrasound images is a challenging problem on account of inability to accurately model various factors of which some include intra-/inter scanline interference, transducer to surface coupling, artifacts on transducer elements, inhomogeneous shadowing and nonlinear attenuation. Current approaches typically solve wave space equations making them computationally expensive and slow to operate. We propose a generative adversarial network (GAN) inspired approach for fast simulation of patho-realistic ultrasound images. We apply the framework to intravascular ultrasound (IVUS) simulation. A stage 0 simulation performed using pseudo B-mode ultrasound image simulator yields speckle mapping of a digitally defined phantom. The stage I GAN subsequently refines them to preserve tissue specific speckle intensities. The stage II GAN further refines them to generate high resolution images with patho-realistic speckle profiles. We evaluate patho-realism of simulated images with a visual Turing test indicating an equivocal confusion in discriminating simulated from real. We also quantify the shift in tissue specific intensity distributions of the real and simulated images to prove their similarity.
Full paper: https://arxiv.org/abs/1712.07881
Transfer Learning of Tissue Photon Interaction in Optical Coherence Tomograph...Debdoot Sheet
1) The document presents a method for characterizing oral mucosal tissue using optical coherence tomography (OCT) through transfer learning of tissue photon interaction (TPI).
2) TPI is manifested in OCT speckle intensity statistics and attenuation coefficients, which are used as features in a machine learning model to classify tissue types in cross-validated experiments.
3) The results demonstrate high accuracy in classifying epithelium and sub-epithelium tissues, indicating potential for in vivo oral mucosal tissue characterization and diagnosis of cancers or pre-cancers.
DevFest19 - Early Diagnosis of Chronic Diseases by Smartphone AIGaurav Kheterpal
Session by Sabyasachi Mukhopadhyay
Kolkata Lead, Facebook Developer Circle
GDE in ML
Intel Software Innovator
Visiting Faculty, SCIT Pune
Co-Founder & Chief Research Officer, Twelit MedTech Pvt. Ltd
This paper presents an automated method for extracting regional data from positron emission tomography (PET) images. The method uses magnetic resonance (MR) images co-registered to PET images to individualize a standard set of regions of interest (ROIs). The ROIs are refined based on each individual's MR image to account for anatomical variations. Time-activity curves obtained with the automated method are highly correlated with those obtained manually, with differences generally within 8%. The automated method eliminates variability between raters, reduces analysis time, and provides accurate PET data quantification as an alternative to manual delineation of ROIs.
Variational formulation of unsupervised deep learning for ultrasound image ar...Shujaat Khan
Recently, deep learning approaches have been successfully used for ultrasound (US) image artifact removal. However, paired high-quality images for supervised training are difficult to obtain in many practical situations. Inspired by the recent theory of unsupervised learning using optimal transport driven CycleGAN (OT-CycleGAN), here, we investigate the applicability of unsupervised deep learning for US artifact removal problems without matched reference data. Two types of OT-CycleGAN approaches are employed: one with the partial knowledge of the image degradation physics and the other with the lack of such knowledge. Various US artifact removal problems are then addressed using the two types of OT-CycleGAN. Experimental results for various unsupervised US artifact removal tasks confirmed that our unsupervised learning method delivers results comparable to supervised learning in many practical applications.
Possible future avenues for ophthalmic imaging combining advanced techniques and deep learning. "Bubbling under the surface, and inspiration from ‘bioimaging’ in general"
Darwin’s Magic: Evolutionary Computation in Nanoscience, Bioinformatics and S...Natalio Krasnogor
In this talk I will overview ten years of research in the application of evolutionary computation ideas in the natural sciences. The talk will take us on a tour that will cover problems in nanoscience, e.g. controlling self-‐organizing systems, optimizing scanning probe microscopy, etc., problems arising in bioinformatics, such as predicting protein structures and their features, to challenges emerging in systems and synthetic biology. Although the algorithmic solutions involved in these problems are different from each other, at their core, they retain Darwin’s wonderful insights. I will conclude the talk by giving a personal view on why EC has been so successful and where, in my mind, the future lies.
A BRIEF OVERVIEW ON DIFFERENT ANIMAL DETECTION METHODSsipij
Researches based on animal detection plays a very vital role in many real life applications. Applications
which are very important are preventing animal vehicle collision on roads, preventing dangerous animal
intrusion in residential area, knowing locomotive behavioural of targeted animal and many more. There
are limited areas of research related to animal detection. In this paper we will discuss some of these areas
for detection of animals.
Unsupervised Deconvolution Neural Network for High Quality Ultrasound ImagingShujaat Khan
High quality US imaging demand large number of measurements that can increase the cost, size and power requirements. Therefore, low-powered, portable and 3D ultrasound imaging system require reconstruction algorithms that can produce high quality images using fewer receive measurements. Number of model specific methods has been proposed which doesn't work under perturbation. For instance, compressive deconvolution ultrasound which provide a reasonable quality with limited measurements however, it has its own down-sides such as high computation cost and accurate estimation of point spread function (PSF). An other major limitation of conventional methods is that they require RF or base-band signal which is difficult to obtain from portable US systems. To deal with the aforementioned issues, in this study we designed a novel deep deconvolution model for image domain-based deconvolution. The proposed deep deconvolution (DeepDeconv) model can be trained in an unsupervised fashion, alleviate the need of paired high and low quality images. The model was evaluated on both the phantom and in-vivo scans for various sampling configurations. The proposed DeepDeconv significantly enhance the details of anatomical structures and using unsupervised learning on average it achieved 2.14dB, 4.96dB and 0.01 units gain in CR, PSNR and SSIM values respectively, which are comparable to the supervised method.
International Journal of Image Processing (IJIP) Volume (3) Issue (6)CSCJournals
This paper proposes a method for face hallucination using eigen transformation in transform domains. Face hallucination aims to enhance the resolution of facial images using super resolution techniques. The proposed method performs eigen transformation on low resolution face images that have been transformed using wavelet transform or discrete cosine transform. This avoids iterative optimization techniques, making the method faster than other learning-based super resolution approaches. The results show that eigen transformation can be effectively applied in transform domains for face hallucination. This suggests it may be suitable for super resolving compressed images with minor modifications. The method provides an efficient way to enhance facial images for applications like face recognition and detection.
Deep Learning for Biomedical Unstructured Time SeriesPetteriTeikariPhD
1D Convolutional neural networks (CNNs) for time series analysis, and inspiration from beyond biomedical field. Short intro for various different steps involved in Time Series Analysis including outlier detection, imputation, denoising, segmentation, classification and forecasting.
Available also from:
https://www.dropbox.com/s/cql2jhrt5mdyxne/timeSeries_deepLearning.pdf?dl=0
Trends in deep learning in 2020 - International Journal of Artificial Intelli...gerogepatton
The International Journal of Artificial Intelligence & Applications (IJAIA) is a bi monthly open access peer-reviewed journal that publishes articles which contribute new results in all areas of the Artificial Intelligence & Applications (IJAIA). It is an international journal intended for professionals and researchers in all fields of AI for researchers, programmers, and software and hardware manufacturers. The journal also aims to publish new attempts in the form of special issues on emerging areas in Artificial Intelligence and applications.
Beyond Broken Stick Modeling: R Tutorial for interpretable multivariate analysisPetteriTeikariPhD
This document provides information about Petteri Teikari, including his educational background and affiliation with the Singapore Eye Research Institute. It then lists several papers and resources related to broken stick modeling, nonlinear multivariate analysis, and variable importance measures in random forests. Specific topics covered include dynamic modeling of multivariate processes, joint frailty models, additive modeling, outcome weighted deep learning for combination therapies, survival trees, correlation and variable importance, and developing model-agnostic variable importance measures. Links are provided to papers, code implementations, and visualization resources.
Using physics-based OCT Monte Carlo simulation and wave optics models for synthesising new OCT volumes for ophthalmic deep learning.
Alternative download link:
https://www.dropbox.com/s/ax15qy47yi76eex/OCT_MonteCarlo.pdf?dl=0
Short intro for some design considerations around hyperspectral retinal imaging. Both for research-grade desktop setups built around supercontinuum laser and AOTF tunable filter, and for mobile low-cost retinal imagers.
Available also from:
https://www.dropbox.com/s/5brchl9ntqno0i9/hyperspectral_retinal_imaging.pdf?dl=0
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Est...harmonylab
公開URL:https://arxiv.org/abs/1908.10357
出典:Cheng B, Xiao B, Wang J, Shi H, Huang T S, Zhang L : Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5386-5395 (2020) https://arxiv.org/abs/1908.10357
概要:高解像度特徴量ピラミッドを用いて人物の大きさに考慮したBottom-Up型の姿勢推定手法の一つです.HRNetの特徴マップ出力と,転置畳み込みによるアップサンプリングされた高解像度な出力で構成されています.COCO test-devにおいて,中人数以上で従来のBottom-Up型手法を2.5%AP上回り,後処理などを含めない場合においてBottom-Up型でSOTA (70.5%AP)を達成しました.
Morphing Aircraft Workshop 26/03/13 - Presentation on AuxeticsFabrizio Scarpa
This document summarizes research on using auxetic materials and structures for aircraft morphing applications. It describes chiral honeycomb concepts that allow bending through node rotations, curved zero-Poisson's ratio honeycombs made of PEEK for large deformations, gradient cellular structures for variable stiffness, a Kirigami wingbox concept for MAVs/UAVs using cuts in materials, and elastomeric auxetic skins with high deformability and tunable stiffness. The document outlines several auxetic configurations that could enable shape morphing and discusses their potential for applications like boundary layer control, continuous camber variation, recoverable deformations, and vibration damping.
IRJET - Automating the Identification of Forest Animals and Alerting in Case ...IRJET Journal
This document describes a proposed system to automatically identify and monitor forest animals using deep learning and computer vision techniques. The system would collect images using cameras traps and use a convolutional neural network (CNN) to identify animals in the images. It would be trained on a dataset of 1500 images across 5 animal categories. If the system identifies an animal encroaching on villages, it would trigger an alert to notify the forest department. The system aims to automate time-consuming manual animal identification tasks and provide alerts about potential human-animal conflicts. It could help conservation efforts by monitoring wildlife populations over time more efficiently.
Introduction to resting state fMRI preprocessing and analysisCameron Craddock
from Australia Connectomes course 2018 in Melbourne, Australia. A brief introduction to CPAC and an in depth lecture on how to preprocessing functional MRI data.
Sabyasachi Mukhopadhyay discusses using multifractal analysis to characterize changes in biological tissue. He presents a method for extracting refractive index fluctuations from light scattering spectra of tissue samples and applying multifractal detrended fluctuation analysis (MFDFA) to quantify multifractality. Analysis of cervical precancer tissue samples found increasing multifractality (wider singularity spectra) at higher grades, suggesting greater refractive index inhomogeneity. Support vector machines and hidden Markov models were used to classify tissue samples based on multifractal parameters, with hidden Markov models achieving better multiclass classification performance. The method was also applied to optical coherence tomography images of the human retina to analyze depth-resolved multifractality changes related to diabetic mac
Practical Considerations in the design of Embedded Ophthalmic DevicesPetteriTeikariPhD
Practical level introduction for ophthalmic device design.
How in the future, multimodal measurement devices will be replacing unimodal devices with simple decision tree indices.
Some examples of embedded cameras, measurement illumination, stimulus presentation illumination are shown.
Alternative download link:
https://www.dropbox.com/s/lt76ohoeusopkoo/practicalConsiderations_embeddedOphthalmicDevices.pdf?dl=0
Using off-the-shelf ultrasound imagers, and transition to portable system-on-chip ultrasound imagers such as Butterfly IQ.
Embedded devices such as Butterfly IQ can be further improved by integrating deep learning / artificial intelligence at device level, and naturally at the post-processing and analysis levels
Alternative download link:
https://www.dropbox.com/s/rlwv7m29mh6y2w6/pupillometry_throughTheEyelids.pdf?dl=0
Wen-li Wu presented on the application of transmission small angle X-ray scattering (tSAXS) in the semiconductor industry. tSAXS can measure the average pitch size and line width of deep sub-micrometer line gratings with sub-nanometer precision. Additional information like line height and sidewall angle can be extracted from tSAXS data by measuring diffraction peak intensities and positions while rotating the sample. Examples were presented of using tSAXS to determine line edge roughness, standing wave roughness in photoresist patterns, and dimensions of FinFET structures.
Teleportation belongs to Quantum Physics, Quantum Teleportation is a process by which quantum information (e.g. the exact state of an atom or photon) can be transmitted (exactly, in principle) from one location to another, with the help of classical communication and previously shared quantum entanglement between the sending and receiving location.
Variational formulation of unsupervised deep learning for ultrasound image ar...Shujaat Khan
Recently, deep learning approaches have been successfully used for ultrasound (US) image artifact removal. However, paired high-quality images for supervised training are difficult to obtain in many practical situations. Inspired by the recent theory of unsupervised learning using optimal transport driven CycleGAN (OT-CycleGAN), here, we investigate the applicability of unsupervised deep learning for US artifact removal problems without matched reference data. Two types of OT-CycleGAN approaches are employed: one with the partial knowledge of the image degradation physics and the other with the lack of such knowledge. Various US artifact removal problems are then addressed using the two types of OT-CycleGAN. Experimental results for various unsupervised US artifact removal tasks confirmed that our unsupervised learning method delivers results comparable to supervised learning in many practical applications.
Possible future avenues for ophthalmic imaging combining advanced techniques and deep learning. "Bubbling under the surface, and inspiration from ‘bioimaging’ in general"
Darwin’s Magic: Evolutionary Computation in Nanoscience, Bioinformatics and S...Natalio Krasnogor
In this talk I will overview ten years of research in the application of evolutionary computation ideas in the natural sciences. The talk will take us on a tour that will cover problems in nanoscience, e.g. controlling self-‐organizing systems, optimizing scanning probe microscopy, etc., problems arising in bioinformatics, such as predicting protein structures and their features, to challenges emerging in systems and synthetic biology. Although the algorithmic solutions involved in these problems are different from each other, at their core, they retain Darwin’s wonderful insights. I will conclude the talk by giving a personal view on why EC has been so successful and where, in my mind, the future lies.
A BRIEF OVERVIEW ON DIFFERENT ANIMAL DETECTION METHODSsipij
Researches based on animal detection plays a very vital role in many real life applications. Applications
which are very important are preventing animal vehicle collision on roads, preventing dangerous animal
intrusion in residential area, knowing locomotive behavioural of targeted animal and many more. There
are limited areas of research related to animal detection. In this paper we will discuss some of these areas
for detection of animals.
Unsupervised Deconvolution Neural Network for High Quality Ultrasound ImagingShujaat Khan
High quality US imaging demand large number of measurements that can increase the cost, size and power requirements. Therefore, low-powered, portable and 3D ultrasound imaging system require reconstruction algorithms that can produce high quality images using fewer receive measurements. Number of model specific methods has been proposed which doesn't work under perturbation. For instance, compressive deconvolution ultrasound which provide a reasonable quality with limited measurements however, it has its own down-sides such as high computation cost and accurate estimation of point spread function (PSF). An other major limitation of conventional methods is that they require RF or base-band signal which is difficult to obtain from portable US systems. To deal with the aforementioned issues, in this study we designed a novel deep deconvolution model for image domain-based deconvolution. The proposed deep deconvolution (DeepDeconv) model can be trained in an unsupervised fashion, alleviate the need of paired high and low quality images. The model was evaluated on both the phantom and in-vivo scans for various sampling configurations. The proposed DeepDeconv significantly enhance the details of anatomical structures and using unsupervised learning on average it achieved 2.14dB, 4.96dB and 0.01 units gain in CR, PSNR and SSIM values respectively, which are comparable to the supervised method.
International Journal of Image Processing (IJIP) Volume (3) Issue (6)CSCJournals
This paper proposes a method for face hallucination using eigen transformation in transform domains. Face hallucination aims to enhance the resolution of facial images using super resolution techniques. The proposed method performs eigen transformation on low resolution face images that have been transformed using wavelet transform or discrete cosine transform. This avoids iterative optimization techniques, making the method faster than other learning-based super resolution approaches. The results show that eigen transformation can be effectively applied in transform domains for face hallucination. This suggests it may be suitable for super resolving compressed images with minor modifications. The method provides an efficient way to enhance facial images for applications like face recognition and detection.
Deep Learning for Biomedical Unstructured Time SeriesPetteriTeikariPhD
1D Convolutional neural networks (CNNs) for time series analysis, and inspiration from beyond biomedical field. Short intro for various different steps involved in Time Series Analysis including outlier detection, imputation, denoising, segmentation, classification and forecasting.
Available also from:
https://www.dropbox.com/s/cql2jhrt5mdyxne/timeSeries_deepLearning.pdf?dl=0
Trends in deep learning in 2020 - International Journal of Artificial Intelli...gerogepatton
The International Journal of Artificial Intelligence & Applications (IJAIA) is a bi monthly open access peer-reviewed journal that publishes articles which contribute new results in all areas of the Artificial Intelligence & Applications (IJAIA). It is an international journal intended for professionals and researchers in all fields of AI for researchers, programmers, and software and hardware manufacturers. The journal also aims to publish new attempts in the form of special issues on emerging areas in Artificial Intelligence and applications.
Beyond Broken Stick Modeling: R Tutorial for interpretable multivariate analysisPetteriTeikariPhD
This document provides information about Petteri Teikari, including his educational background and affiliation with the Singapore Eye Research Institute. It then lists several papers and resources related to broken stick modeling, nonlinear multivariate analysis, and variable importance measures in random forests. Specific topics covered include dynamic modeling of multivariate processes, joint frailty models, additive modeling, outcome weighted deep learning for combination therapies, survival trees, correlation and variable importance, and developing model-agnostic variable importance measures. Links are provided to papers, code implementations, and visualization resources.
Using physics-based OCT Monte Carlo simulation and wave optics models for synthesising new OCT volumes for ophthalmic deep learning.
Alternative download link:
https://www.dropbox.com/s/ax15qy47yi76eex/OCT_MonteCarlo.pdf?dl=0
Short intro for some design considerations around hyperspectral retinal imaging. Both for research-grade desktop setups built around supercontinuum laser and AOTF tunable filter, and for mobile low-cost retinal imagers.
Available also from:
https://www.dropbox.com/s/5brchl9ntqno0i9/hyperspectral_retinal_imaging.pdf?dl=0
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Est...harmonylab
公開URL:https://arxiv.org/abs/1908.10357
出典:Cheng B, Xiao B, Wang J, Shi H, Huang T S, Zhang L : Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5386-5395 (2020) https://arxiv.org/abs/1908.10357
概要:高解像度特徴量ピラミッドを用いて人物の大きさに考慮したBottom-Up型の姿勢推定手法の一つです.HRNetの特徴マップ出力と,転置畳み込みによるアップサンプリングされた高解像度な出力で構成されています.COCO test-devにおいて,中人数以上で従来のBottom-Up型手法を2.5%AP上回り,後処理などを含めない場合においてBottom-Up型でSOTA (70.5%AP)を達成しました.
Morphing Aircraft Workshop 26/03/13 - Presentation on AuxeticsFabrizio Scarpa
This document summarizes research on using auxetic materials and structures for aircraft morphing applications. It describes chiral honeycomb concepts that allow bending through node rotations, curved zero-Poisson's ratio honeycombs made of PEEK for large deformations, gradient cellular structures for variable stiffness, a Kirigami wingbox concept for MAVs/UAVs using cuts in materials, and elastomeric auxetic skins with high deformability and tunable stiffness. The document outlines several auxetic configurations that could enable shape morphing and discusses their potential for applications like boundary layer control, continuous camber variation, recoverable deformations, and vibration damping.
IRJET - Automating the Identification of Forest Animals and Alerting in Case ...IRJET Journal
This document describes a proposed system to automatically identify and monitor forest animals using deep learning and computer vision techniques. The system would collect images using cameras traps and use a convolutional neural network (CNN) to identify animals in the images. It would be trained on a dataset of 1500 images across 5 animal categories. If the system identifies an animal encroaching on villages, it would trigger an alert to notify the forest department. The system aims to automate time-consuming manual animal identification tasks and provide alerts about potential human-animal conflicts. It could help conservation efforts by monitoring wildlife populations over time more efficiently.
Introduction to resting state fMRI preprocessing and analysisCameron Craddock
from Australia Connectomes course 2018 in Melbourne, Australia. A brief introduction to CPAC and an in depth lecture on how to preprocessing functional MRI data.
Sabyasachi Mukhopadhyay discusses using multifractal analysis to characterize changes in biological tissue. He presents a method for extracting refractive index fluctuations from light scattering spectra of tissue samples and applying multifractal detrended fluctuation analysis (MFDFA) to quantify multifractality. Analysis of cervical precancer tissue samples found increasing multifractality (wider singularity spectra) at higher grades, suggesting greater refractive index inhomogeneity. Support vector machines and hidden Markov models were used to classify tissue samples based on multifractal parameters, with hidden Markov models achieving better multiclass classification performance. The method was also applied to optical coherence tomography images of the human retina to analyze depth-resolved multifractality changes related to diabetic mac
Practical Considerations in the design of Embedded Ophthalmic DevicesPetteriTeikariPhD
Practical level introduction for ophthalmic device design.
How in the future, multimodal measurement devices will be replacing unimodal devices with simple decision tree indices.
Some examples of embedded cameras, measurement illumination, stimulus presentation illumination are shown.
Alternative download link:
https://www.dropbox.com/s/lt76ohoeusopkoo/practicalConsiderations_embeddedOphthalmicDevices.pdf?dl=0
Using off-the-shelf ultrasound imagers, and transition to portable system-on-chip ultrasound imagers such as Butterfly IQ.
Embedded devices such as Butterfly IQ can be further improved by integrating deep learning / artificial intelligence at device level, and naturally at the post-processing and analysis levels
Alternative download link:
https://www.dropbox.com/s/rlwv7m29mh6y2w6/pupillometry_throughTheEyelids.pdf?dl=0
Similar to Whispers of Speckles (Part I: Building Computational Imaging Frameworks for Acoustic and Optical Speckle Imaging) + (Part II: Enlightenment from Shallow to Complex Reasoning with Deep Learning)
Wen-li Wu presented on the application of transmission small angle X-ray scattering (tSAXS) in the semiconductor industry. tSAXS can measure the average pitch size and line width of deep sub-micrometer line gratings with sub-nanometer precision. Additional information like line height and sidewall angle can be extracted from tSAXS data by measuring diffraction peak intensities and positions while rotating the sample. Examples were presented of using tSAXS to determine line edge roughness, standing wave roughness in photoresist patterns, and dimensions of FinFET structures.
Teleportation belongs to Quantum Physics, Quantum Teleportation is a process by which quantum information (e.g. the exact state of an atom or photon) can be transmitted (exactly, in principle) from one location to another, with the help of classical communication and previously shared quantum entanglement between the sending and receiving location.
Education in a Globally Connected WorldLarry Smarr
The document discusses how advances in technology are enabling more globally connected education and research collaboration. It provides examples of optical networks and dedicated fiber links allowing universities to share high-definition media and remotely access scientific instruments and environments. Global partnerships are being formed to leverage these technologies and better prepare students for an increasingly interconnected world.
This document provides information about the "Raman and Luminescence Submicron Spectroscopy" Laboratory located at the V. Lashkaryov Institute of Semiconductor Physics, National Academy of Science, Ukraine. The laboratory contains several lasers, spectrometers, microscopes, and temperature control equipment used to perform Raman and luminescence spectroscopy and mapping on semiconductor nanostructures with submicron spatial resolution. The laboratory studies properties such as chemical composition, strain, temperature, carrier mobility and concentration in nanostructures for applications in microelectronics and optoelectronics. Team members and their areas of research interest are also listed.
The document provides an overview of nanotechnology, discussing its history, current state, and future prospects. It defines nanotechnology as involving research and engineering at the nanoscale (1-100 nanometers). The document outlines major government funding through initiatives like the National Nanotechnology Initiative, as well as university and commercial research. It discusses various applications of nanotechnology across different industries.
The document summarizes the history and development of nanotechnology. It discusses how the concept was first developed by Richard Feynman in 1959, and the term was coined by Norio Taniguchi in 1974. It then outlines key milestones and advancements in the 1980s and beyond that helped establish nanotechnology as a field, including the invention of the scanning tunneling microscope in 1981 and discoveries of fullerenes in 1985 and carbon nanotubes. The document also provides examples of how nanotechnology is being applied in biology and medicine, such as using atomic force microscopes to image cells, optical tweezers to manipulate organisms, and quantum dots for labeling parasites.
Understanding of light sensing organs in biology creates opportunities for the development of novel optic systems that cannot be available with existing technologies. The insect's eyes, i.e., compound eyes, are particularly notable for their exceptional interesting optical characteristics, such as wide fields of view and infinite depth-of-field. While the construction of man-made imaging systems with these characteristics is of interest due to potential for applications in micro air vehicles (MVAs) and clinical endoscopes, currently available devices offer only limited capabilities due to their use of compound lens systems in planar geometries. In this presentation, I discuss a complete set of materials, design layouts and integration schemes for digital cameras that mimic fully hemispherical compound eyes. Certain of the concepts extend recent advances in ‘stretchable electronics’ that provide previously unavailable options in design. I also discuss another interesting hierarchical micro- and nanostructures that can be found in eyes of night-active insects such as moth and mosquito. I present research trends on fabrication methods, optical characteristics, and various applications for artificial micro-/nanostructures that resemble ‘moth eye’ structure.
This document describes research into using fiber optic sensing in laser catheters to differentiate between tissues in real-time during minimally invasive surgery. Experiments were conducted using a porcine model and tissue phantoms to test a fiber optic sensor's ability to distinguish between blood and different tissue types, as well as measure the distance to a surface. The results demonstrated clear differentiation between blood and tissues, discrimination of different tissue types, and detection of surface contact over 1 mm away. This fiber optic sensing technique shows potential for smart surgical tools to increase safety for procedures where tactile feedback is limited.
This document discusses the use of terahertz radiation, which has wavelengths between 0.03-3 mm, for non-destructive testing applications. Terahertz radiation can penetrate many materials and is reflected by metals, making it useful for imaging applications. Terahertz time-domain spectroscopy is introduced, which uses ultrafast lasers and photoconductive antennas to generate and detect broadband terahertz pulses. This allows the extraction of material properties from terahertz spectra. The document outlines several applications of terahertz imaging and spectroscopy for quality control, security screening, biomedical imaging, and analyzing materials like paper, pharmaceuticals, and chemicals. Specifically, it discusses using terahertz techniques to monitor degradation in ancient paper samples
Analytical methods and instrumentation syllabusThivya Prasad
This document outlines the syllabus for the course OBT751 - Analytical Methods and Instrumentation. The syllabus covers 5 units: 1) Spectrometry, 2) Molecular Spectroscopy, 3) NMR and Mass Spectrometry, 4) Separation Methods, and 5) Electroanalysis and Surface Microscopy. It also lists textbooks references for the course prepared by D.Thivya Prasad of Mount Zion College of Engineering and Technology.
CZECH NANO SHOW - Marketa Borovcova - CEITEC Jan Fried
CEITEC is a scientific center in Brno, Czech Republic focused on life sciences, advanced materials, and technologies. Its aim is to establish itself as a recognized European center of science through collaboration. It has over 110 researchers across 9 groups studying advanced nanotechnologies and microtechnologies and 78 researchers across 4 groups studying advanced materials. CEITEC collaborates with universities, research institutes, and companies both within the Czech Republic and internationally. It produces scientific publications, receives research grants, and has launched its first startup company based on a patented interferometric imaging system.
Professor Dionne explores the unique and enabling properties of nano-sized materials, with applications ranging from highly efficient solar-renewable technologies to optical computers and cloaks of invisibility.
The outstanding properties of metamaterials open the door of opportunity for a number of exciting practical applications. Fascinating applications such as: perfect lenses that break the diffraction limit of conventional lenses, optical quantum storage, and invisibility cloaking.
The Singularity: Toward a Post-Human RealityLarry Smarr
06.02.13
Talk to UCSD's Sixth College
Honor's Course on Kurzweil's The Singularity is Near
Title: The Singularity: Toward a Post-Human Reality
La Jolla, CA
Similar to Whispers of Speckles (Part I: Building Computational Imaging Frameworks for Acoustic and Optical Speckle Imaging) + (Part II: Enlightenment from Shallow to Complex Reasoning with Deep Learning) (20)
This presentation was provided by Racquel Jemison, Ph.D., Christina MacLaughlin, Ph.D., and Paulomi Majumder. Ph.D., all of the American Chemical Society, for the second session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session Two: 'Expanding Pathways to Publishing Careers,' was held June 13, 2024.
How to Setup Warehouse & Location in Odoo 17 InventoryCeline George
In this slide, we'll explore how to set up warehouses and locations in Odoo 17 Inventory. This will help us manage our stock effectively, track inventory levels, and streamline warehouse operations.
Walmart Business+ and Spark Good for Nonprofits.pdfTechSoup
"Learn about all the ways Walmart supports nonprofit organizations.
You will hear from Liz Willett, the Head of Nonprofits, and hear about what Walmart is doing to help nonprofits, including Walmart Business and Spark Good. Walmart Business+ is a new offer for nonprofits that offers discounts and also streamlines nonprofits order and expense tracking, saving time and money.
The webinar may also give some examples on how nonprofits can best leverage Walmart Business+.
The event will cover the following::
Walmart Business + (https://business.walmart.com/plus) is a new shopping experience for nonprofits, schools, and local business customers that connects an exclusive online shopping experience to stores. Benefits include free delivery and shipping, a 'Spend Analytics” feature, special discounts, deals and tax-exempt shopping.
Special TechSoup offer for a free 180 days membership, and up to $150 in discounts on eligible orders.
Spark Good (walmart.com/sparkgood) is a charitable platform that enables nonprofits to receive donations directly from customers and associates.
Answers about how you can do more with Walmart!"
Temple of Asclepius in Thrace. Excavation resultsKrassimira Luka
The temple and the sanctuary around were dedicated to Asklepios Zmidrenus. This name has been known since 1875 when an inscription dedicated to him was discovered in Rome. The inscription is dated in 227 AD and was left by soldiers originating from the city of Philippopolis (modern Plovdiv).
How to Make a Field Mandatory in Odoo 17Celine George
In Odoo, making a field required can be done through both Python code and XML views. When you set the required attribute to True in Python code, it makes the field required across all views where it's used. Conversely, when you set the required attribute in XML views, it makes the field required only in the context of that particular view.
Pengantar Penggunaan Flutter - Dart programming language1.pptx
Whispers of Speckles (Part I: Building Computational Imaging Frameworks for Acoustic and Optical Speckle Imaging) + (Part II: Enlightenment from Shallow to Complex Reasoning with Deep Learning)
2. Whispers of Speckle
Part I: Building Computational Imaging
Frameworks for Acoustic and Optical
Speckle Imaging
Dr. Debdoot Sheet
Assistant Professor
Department of Electrical Engineering
Indian Institute of Technology Kharagpur
www.facweb.iitkgp.ernet.in/~debdoot/
3. Inspiration
“A wonderful fact to reflect
upon, that every human
creature is constituted to be
that profound secret and
mystery to every other.”
- Charles Dickens
(A Tale of Two Cities)
“If you want to find the
secrets of the universe, think
in terms of energy,
frequency and vibration.”
- Nikola Tesla
25 June 2015 3Whispers of Speckles [Debdoot Sheet] - WMLMIA
4. Motivation
Whispers of Speckles [Debdoot Sheet] - WMLMIA 4
D. Sheet (2014), PhD Thesis
25 June 2015
Text books
R. K. Das (2012), PhD Thesis
A. Barui (2011), PhD Thesis
5. Introduction
• Human body consists of organs and
systems made up of different tissues.
• Pathological conditions and
abnormalities affect their normal
functioning.
• Critical soft tissue abnormalities include
– Plaque formation in the blood vascular
system.
– Lesions in the breast.
– Degeneration of the retina.
– Wounds in the skin.
• Traditional practice of Histopathological
diagnosis requires invasive Biopsy /
Excision for tissue collection
– Not possible in vessels in living Humans
– Improper sampling from Breast lesion
affects diagnostic outcome
– Not possible in retina in living Humans
– Not possible in healing wounds.
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 5
6. ACHIEVING IN SITU HISTOLOGY OF
VASCULAR PLAQUES
625 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA
7. • Atherosclerosis
– Plaque builds up in arteries
– Forms anywhere in the vascular system
• Cardiovascular diseases (CVD)
• In vivo Imaging of Plaques
– CT Angiography (CTA)
– MR Angiography (MRA)
– Intravascular Ultrasound (IVUS)
– Intravascular OCT (IV-OCT)
– Intravascular Near-Infrared Spectroscopy
(NIR)
• Plaque Vulnerability Assessment
– Calcification, fibrosis identification
– Lipid pool and Necrosis burden estimation
Source: NIH – National Heart,
Lung, and Blood Institute
Blood Vascular System
25 June 2015 7Whispers of Speckles [Debdoot Sheet] - WMLMIA
• Spectral analysis of received ultrasonic echo
signal
– Lizzi et al., 1983
– Nair et al., 2001
– Kawasaki et al., 2002
– Virtual Histology (Volcano Corp.)
– iMap (BostonScientific)
• Texture analysis of B-mode image/signal
– Katouzian et al., 2008, 2010, 2012 (Prog. Hist. /
PH)
– Esclara et al., 2009
– Seabra et al., 2011
• Limitations
– Unable to identify heterogeneous tissue
composition
– Cannot discriminate between dense fibrous tissue
and calcification
– Fails to discriminate true necrosis from shadows
8. Backdrop
8
White light source
350nm 750nm
Power
Stained tissue section
Tissue specific spectrum
350nm 750nm
Power
Calcified
Fibrotic
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA
9. 9
White light source
350nm 750nm
Power
Stained tissue section
Tissue specific spectrum
350nm 750nm
Power
Calcified
Fibrotic
: Probing energy (Light)
: Physiological property (Tissue type)
f
1
f
Inferring tissue
type based on
color
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA
10. 10
Computation Modelling of Tissue Energy
Interaction for In situ Histopathology
Computed histology
: Probing energy (Acoustic)
: Tissue type (Backscatterer density)
f
1
f
Inferring tissue
type based on
backscattering
response
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA
11. Limited Resolution Challenge
11
r1
r2
r3
P. M. Shankar, “A general statistical model for ultrasonic backscattering from tissues”,
IEEE Trans. Ultrasonics, Ferroelectrics, Freq. Control., vol. 47, no. 3, pp. 727-736,
May 2000.
11 rr f
22 rr f
33 rr f
Ultrasound
signal
backscattered
within a
resolution cell
i
i
r
r
fE
E
Signal sensed by the
transducer
irfEf 1
ˆ
Estimated functional ensemble of
backscatterer density
ˆ Improper estimation of tissue type in
inhomogeneous media
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA
12. Statistical Physics in Acoustic Imaging
r1
r2
r3
r1
r2
m=0.5
Ω1
Ω2
r
P(r)
m=1.0
Ω1
Ω2
Ω3
r
P(r)
P. M. Shankar, “A general statistical model for ultrasonic backscattering from
tissues”, IEEE Trans. Ultrasonics, Ferroelectrics, Freq. Control., vol. 47, no. 3,
pp. 727-736, May 2000.
2
12
exp
2
,| r
m
m
rm
mr m
mm
N
25 June 2015 12Whispers of Speckles [Debdoot Sheet] - WMLMIA
13. Statistical physics of ultrasonic backscattering
Lipidic
r
P(r)
Fibrotic
r
P(r)
Calcified
r
P(r)
V. Dumane and P. M. Shankar, “Use of frequency
diversity and Nakagami statistics in ultrasonic
tissue characterization”, IEEE Trans. Ultrasonics,
Ferroelectrics, Freq. Control, vol. 48, no. 4, pp.
1139-1146, Jul. 2001
F. Destrempes, J. Meunier, M. . F. Giroux, G.
Soulez, G. Cloutier, “Segmentation in ultrasonic b-
mode images of healthy carotid arteries using
mixture of Nakagami distributions and stochastic
optimization”, IEEE Trans. Med. Imaging, vol. 28,
no. 2, pp. 215-229, Feb. 2009.
25 June 2015 13Whispers of Speckles [Debdoot Sheet] - WMLMIA
14.
32121
221121
,,, 1
,
1
,,
1
,
111
)(,|,|
,|
;),(,),,,,,(||
L
l
lll
L
l
lll
L
l
lll
mrpmrpp
mrpp
ymprfyrp
NN
N
Mathematical intractability, the problem
)(
)(
)|(
)|( yP
rp
yrp
ryp The probabilistic decision making framework
Scales unknown
Correlation among scales unknown
No. components unknown
Prior probab. of each comp. unknown
25 June 2015 14Whispers of Speckles [Debdoot Sheet] - WMLMIA
15. Proposed Solution
Statistical physics model of ultrasonic backscattering
Set of signal received by the transducer
Training set of annotated examples to be used for supervised learning
Supervised learner for learning tissue specific statistical physics model
train;,|)(
),(
)|,(
),|( RR
yHyP
rp
yrp
ryp
Solution through Transfer
Learning Framework25 June 2015 15Whispers of Speckles [Debdoot Sheet] - WMLMIA
16. HOW TO DEAL WITH THIS AS A
MACHINE LEARNING CHALLENGE?
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 16
17. Learning?
A computer program is said to learn from
experience E with respect to some class of
tasks T and performance measure P, if its
performance at tasks in T, as measured by
P, improves with experience E
-Tom Mitchell
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 17
18. Demystifying Learning
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 18
Man 1 Man 2 Man 3Man 4
Great Wall logo
Great Wall tower
Kim Jung
WangDebdoot
Experience (E)
Performance(P)
Debdoot, Kim, Jung and Wang are standing near the
Great Wall logo and the Great Wall tower is behind them.
19. How was it Learning?
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 19
Man 1 Man 2 Man 3Man 4
Great Wall logo
Great Wall tower
Kim Jung
WangDebdoot
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Debdoot, Kim, Jung and Wang are standing near the
Great Wall logo and the Great Wall tower is behind them.
Recognize
humans
20. GETTING MACHINES TO LEARN
TISSUE – ENERGY INTERACTION
FOR IN SITU HISTOLOGY
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 20
21. IVUS Tissue Characterization
21
Background
Lipidic
Fibrotic
Calcified
Necrosis
Iterative self-organizing
atherosclerotic tissue labeling in
intravascular ultrasound images and
comparison with virtual histology,
IEEE TBME, 59(11), 2012
Hunting for necrosis in the shadows
of intravascular ultrasound, CMIG,
38(2), 2014
Joint learning of ultrasonic
backscattering statistical physics and
signal confidence primal for
characterizing atherosclerotic plaques
using intravascular ultrasound, Med.
Image Anal,18(1), 2014
Nakagami parameter
and signal
confidence estimate
Random forest
learning
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA
22. Ultrasound Signal Confidence
• An ultrasonic pulse as well as backscattered
echo travel along the same path through a
heterogeneous media.
• They are subjected to the same attenuation.
• Confidence of the received signal is a
reflection of fidelity of samples received by
the transducer.
• It can be estimated by treating its
propagation as a random walk along an
ultrasonic scan-line.
• A random walker starting at a point on the
scan-line reaches the virtual transducer
element placed at the origin of each scan-
line.
• This random walk is solved using the electric
network equivalent and solving it in the
paradigm of graph theory.
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 22
A. Karamalis, W. Wein, T. Klein, N. Navab (2012) Ultrasound confidence maps
using random walks, Medical Image Analysis, 16:1101–1112.
23. Transfer Learning
Framework
23
Ultrasound RF data
(i) Signal confidence
(ii) Speckle statistics
Tissue labels
f
Learnt random forest
Learning
phase
(offline) Tissue labels
Prediction
(online)
f
Whispers of Speckles [Debdoot Sheet] - WMLMIA25 June 2015
24. Random Forests for Learning
25 June 2015 24Whispers of Speckles [Debdoot Sheet] - WMLMIA
A. Criminisi and J. Shotton, Decision Forests for Computer
Vision and Medical Image Analysis, Springer, 2013.
25. Experiment Design
• Data Collection:
– Columbia University, New York City, NY, USA
– Interventional Cardiologist: Dr. Stephane G. Carlier
– Cardiovascular Histopathologist: Dr. Renu Virmani,
CV Path Institute, Gaithersburg, USA
– Cases # 13
– Tissue Sections # 53
– Atlantis, 40 MHz IVUS, Boston Scientific,CA, USA
– Sampling freq: 400 MHz
– Sampling geometry: 256 scan lines per rotation,
2048 samples per scan line
• Learning
– Source task: {Ω,m} estimated at 28 scales +
Ultrasonic Confidence (A. Karamalis, et al. (2012))
– Target task: Random forest 50 decision trees
• Cross validation
– 53 fold cross validation
– Learn with 52, test on the remaining
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 25
26. Ultrasonic Histology of
Atherosclerotic Plaques
• Characterization based on ultrasonic statistical physics.
• Superior machine learning algorithm.
• Reliability measure for estimation of tissues.
Probability of Calcified
tissues
Probability of Fibrotic
tissues
Probability of Lipidic
tissues
Probability of Necrotic
tissues
Calcified
Lipidic
Fibrotic
Necrotic
25 June 2015 26Whispers of Speckles [Debdoot Sheet] - WMLMIA
38. Take home message
• Different types of soft tissues have characteristic response
when interacting with acoustic energy.
• Heterogeneous tissues can be identified by learning of
tissue specific energy interaction response using statistical
physics models.
• Transfer Learning can be employed for efficiently solving
tissue characterization problems modeled as tissue-energy
interaction problems.
– CPU/GPU handshaking can be used for fast implementation of such
tasks
• Explore possibility of Functional Histopathology In situ
25 June 2015 38Whispers of Speckles [Debdoot Sheet] - WMLMIA
39. Whispers of Speckle
Part II: Enlightenment from Shallow to
Complex Reasoning with Deep Learning
Dr. Debdoot Sheet
Assistant Professor
Department of Electrical Engineering
Indian Institute of Technology Kharagpur
www.facweb.iitkgp.ernet.in/~debdoot/
40. DOES THIS METHOD OF TRANSFER
LEARNING APPLY ONLY TO
ULTRASONIC IMAGING?
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 40
41. Skin• Skin forms the general covering of the
body protecting us from external
influences.
• Functions
– Thermoregulation
– Sweat secretion
– Tactile, pressure, temperature sensing
• Stratified organization
– Epidermis
– Papillary dermis
– Dermis
– Adipose tissue
• Wound
– Major pathological injury
– Skin is torn, cut, punctured
• Clinical challenge in management
– Healing in person specific
– Patient specific intervention
– In situ investigation of healing is challenge
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 41
42. Skin
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 42
• In situ investigation
– Optical Coherence Tomography (OCT)
• Cobb et al., (2006)
• Barui et al., (2011)
– Optical photography
• Cross-sectional information about healing
wound is not available
– NIR imaging
• Cross-section histological information not
present
• In situ Histology with OCT
– G. van Soest et al., (2010) – Cardiovascular
OCT
– A. Barui et al., (2011) – Cutaneous wound
beds.
• Challenges
– Identify co-located tissue heterogeneity
– Identify and discriminate Inter-digitated
structures
43. Tissue Photon Interaction
Whispers of Speckles [Debdoot Sheet] - WMLMIA 43
Incident
radiation
Regular
reflection Diffuse
reflection
Scattering
Absorption Multispectral optical imageOCT
B. Saleh, Introduction to Subsurface Imaging, Cambridge, 2011.
0.5 mm
0.5 mm
25 June 2015
44. Optical Coherence Tomography
Whispers of Speckles [Debdoot Sheet] - WMLMIA 44
Low time-coherence
light source
Depth scan mirror
Sample
Detector
Source beam
Reference beam
Sample beam
Detector beam
x
z
z
OCT Image
Michelson
interferometer
25 June 2015
45. TPI in Swept Source OCT
Whispers of Speckles [Debdoot Sheet] - WMLMIA 45
Source
Ballistic
backscattering
Non-ballistic
backscattering
Reference
Detector
A. F. Fercher, et al, Optical coherence tomography — principles and applications, Rep. Prog. Phys.
66 (2003) 239–303
Epithelium
Papillary dermis
Dermis
Adipose
Speckle intensity
Probability
density
25 June 2015
S
S
S
S
I
Ip
exp
1
47. Framework
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 47
Learn TPI Model
Training Image Ground Truth Labels
Test Image
Learn TPI Model
Characterized tissue
train;,| II, xH
48. Computational Histology of Skin
• Solution through a transfer learning
approach
• Performance benchmark (Accuracy)
– Epithelium = 99%
– Papilary dermis = 95%
– Dermis = 99%
– Adipose = 98%
• D. Sheet, et al, “In situ histology of mice
skin through transfer learning of tissue
energy interaction in optical coherence
tomography”, J. Biomed. Optics, 18(9),
2013.
25 June 2015 48
Multi-scale
modeling of
OCT speckles
Training
image
set Ground
truth
Random forest
learning
Multi-scale
modeling of
OCT speckles
Test image
Labeled
tissue
Whispers of Speckles [Debdoot Sheet] - WMLMIA
49. In situ Histology
of Skin
OCT
Histo
Epithelium
Epithelium
Papillary dermis
Dermis
Adipose tissue
25 June 2015 49
Papillary dermisDermisAdipose tissueAll tissues
In situ histology of mice skin through
transfer learning of tissue energy
interaction in optical coherence
tomography, J. Biomed. Optics,
18(9), 2013
Whispers of Speckles [Debdoot Sheet] - WMLMIA
50. In vitro
validation
towards
In vivo
translation
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 50
Transfer Learning of Tissue
Photon Interaction in Optical
Coherence Tomography towards
In vivo Histology of the Oral
Mucosa, Proc. ISBI, 2014.
51. Computational Histology of Retina
• Transfer learning approach
– Retinal OCT tissue labeling
• Performance benchmark (Accuracy)
– Anterior coat > 98%
– RPE > 92%
– Posterior coat > 99%
• SPK Karri and D. Sheet, et al.,
“Computational Histology of Retina
through Transfer Learning of Tissue
Photon Interaction in Optical
Coherence Tomography”, Proc. Int.
Symp. Biomedical Imaging (ISBI), 2014.
25 June 2015 51
Multi-scale
modeling of
OCT speckles
Training
image
set
Ground
truth
Random forest
learning
Multi-scale
modeling of
OCT speckles
Test image
Labeled
tissue
Whispers of Speckles [Debdoot Sheet] - WMLMIA
53. State of the Art
• In situ Histology with OCT
– G. van Soest et al., (2010), G.
J. Ughi et al., (2013) –
Cardiovascular OCT
– D. Sheet et al., (2013, 2014) –
Cutaneous wounds, oral
• Challenges
– Heuristic features
• Texture
• Intensity statistics
– Heuristic computational
models
• Transfer learning of speckle
occurrence models
– Incomplete representation
dictionary
Whispers of Speckles [Debdoot Sheet] - WMLMIA 53
Multi-scale
modeling of
OCT speckles
Training
image
set Ground
truth
Random forest
learning
Multi-scale
modeling of
OCT speckles
Test image
Labeled
tissue
25 June 2015
54. Heuristics in State of Art
Whispers of Speckles [Debdoot Sheet] - WMLMIA 5425 June 2015
55. (RE)EXPLORING THE CONCEPTS OF
HIERARCHY IN LEARNING
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 55
56. How was it Learning?
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 56
Man 1 Man 2 Man 3Man 4
Great Wall logo
Great Wall tower
Kim Jung
WangDebdoot
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Debdoot, Kim, Jung and Wang are standing near the
Great Wall logo and the Great Wall tower is behind them.
Recognize
humans
57. Challenges
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 57
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Recognize
humans
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Recognize
humans
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Recognize
humans
Salient Segments
Describe Scene
58. Challenges
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 58
Salient Segments
Objectify
Recognize
inanimate
Describe Scene
Recognize
humans
LBP
Wavelets
HoG
Body part
recognition
Human
appearance
Chroma
clustering
Posture
realign Silhouette
matching
Recognize
human
Detect
humans
59. FROM SHALLOW TO COMPLEX
REASONING
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 59
60. Heuristics in State of Art
Whispers of Speckles [Debdoot Sheet] - WMLMIA 6025 June 2015
61. The Solution
Whispers of Speckles [Debdoot Sheet] - WMLMIA 61
DenoisingAutoEncoder
DenoisingAutoEncoder
LogisticReg.
25 June 2015
62. Using a Deep Network
Whispers of Speckles [Debdoot Sheet] - WMLMIA 6225 June 2015
63. COMPLEX REASONING AND
ITS DEEP LEARNING
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 63
64. Challenges
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 64
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Recognize
humans
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Recognize
humans
Salient Segments
Objectify
Detect
humans
Recognize
inanimate
Describe Scene
Recognize
humans
Salient Segments
Describe Scene
65. Challenges
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 65
Salient Segments
Objectify
Recognize
inanimate
Describe Scene
Recognize
humans
LBP
Wavelets
HoG
Body part
recognition
Human
appearance
Chroma
clustering
Posture
realign Silhouette
matching
Recognize
human
Detect
humans
66. How to tackle this dilemma?
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 66
Great Wall
behind
Great Wall
logo beside
Debdoot, Kim,
Jung, Wang
67. Multilayer Perceptron (MLP)
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 67
Hiddenlayers
Hiddenlayers
Hiddenlayers
68. MLP Learning, troubles thereof
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 68
P
T1
T2
69. MLP Learning troubles, why so?
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 69
P
T1
T2
LBP
Wavelets
HoG
Body part
recognition
Human
appearance
Chroma
clustering
Posture
realign Silhouette
matching
Recognize
human?
70. HOW TO DEEP LEARN?
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 70
71. Deep Learning, origin and growth
• Around 1950 – NN age
– Neural Nets (McCulloch and Pitts,
1943)
– Unsupervised Learn. (Hebb, 1949)
– Supervised Learn. (Rosenblatt, 1958)
– Associative Memory (Palm, 1980;
Hopfield, 1982)
• 1960
– Discovery of visual sensory cells that
respond to Edges (Hubel and Wiesel,
1962)
– Feed Forward Multi Layer Perceptron
(FF-MLP) (Ivakhnenko, 1968)
• 1980 – Neocognition
– Convolution + WeightReplication +
Subsampling (Fukushima, 1980)
– Max Pooling
– Back-propagation (Werbos, 1981;
LeCunn, 1985, 1988)
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 71
72. Deep Learning, origin and growth
• 1980-2000 – Search for simple,
low-complexity, problem-solvers
– Recurrent Neural Network (RNN)
(Hochreiter and Schmidhuber, 1996)
– Local learning Feed forward NN
(Dayan and Hinton, 1996)
– Advanced gradient descent
(Schaback and Werner, 1992)
– Sequential Network Construction
(Honavar and Uhr, 1988)
– Unsupervised Pre-training (Ritter
and Kohonen, 1989)
– Auto-Encoder (Hinton et al., 1989)
– Back Propagating Convolutional
Neural Networks (LeCun et al., 1989,
1990a, 1998)
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 72
73. Deep Learning, origin and growth
• 2000 – Era of Deep Learning
– NIPS 2003 Feature Selection
Challenge (Neal and Zhang, 2006)
– MNIST digit recognition (LeCun et
al., 1989)
– Deep Belief Network (DBN) /
Restricted Boltzmann Machines
(Hinton et al., 2006)
– Auto Encoders (Bengio, 2009)
• 2006
– GPU based CNN (Chellapilla et al.,
2006)
• 2009
– GPU DBN (Raina et al., 2009)
• 2011
– Max-Pooling CNN on the GPU
(Ciresan et al., 2011)
• 2012
– Image Net (Krizhevsky et al., 2012)
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 73
74. DEEP LEARNING OF COMPLEX
REASONING FOR OCT TISSUE
CHARACTERIZATION
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 74
75. Exploring Deep Architecture
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 75
Multi-scale
modeling of
OCT speckles
Training
image
set
Ground
truth
Random forest
learning
Multi-scale
modeling of
OCT speckles
Test image
Labeled
tissue
Stacked Auto-
Encoders,
Logistic
Regression
Random
Forest
Training
image
set
Ground
truth
http://www.facweb.iitkgp.ernet.in/~debdoot/current.html
76. Auto Encoder for Deep Learning
Whispers of Speckles [Debdoot Sheet] - WMLMIA 7625 June 2015
77. Results in Wounds
Whispers of Speckles [Debdoot Sheet] - WMLMIA 77
(a) OCT image of wound (b) Ground truth (c) In situ histology
Epithelium, Papillary
dermis, Dermis, Adipose
Epithelium, Papillary
dermis, Dermis, Adipose
25 June 2015
78. Experiment Design
• Data Collection
– School of Medical Science
and Technology, Indian
Institute of Technology
Kharagpur
– 1300 nm (HPBW 100 nm)
Swept Source OCT System
• OCS 1300 SS, ThorLabs, NJ,
USA
• 8 bit bitmap images
– Histology for ground truth
• HE stained
• Samples
– Mus musculus (small mice)
– 16 healthy skin
– 2 wounds on skin
• DNN architecture
– Patch size – 36 × 36 px
– DAE1 – 400 nodes
– DAE2 – 100 nodes
– Target – Logistic Reg.
• 5 outputs
– Sparsity – 20%
– Mini-batch training
• In situ Histology
Performance
– Epithelium – 96%
– Papillary dermis – 93%
– Dermis – 99%
– Adipose tissue – 98%
Whispers of Speckles [Debdoot Sheet] - WMLMIA 7825 June 2015
79. Learning of Representations
Whispers of Speckles [Debdoot Sheet] - WMLMIA 79
Representation of speckle
appearance models learned by DAE1
Sparsity of representations learned by
DAE2
25 June 2015
80. END NOTE
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 80
81. Messages for Human Learning
• Photons interact characteristically with different tissues.
– Stochastic similarity exists in speckle appearance.
– Such representations are hard to heuristically encode.
• Deep learning and auto-encoders for computational imaging
– Speckle imaging application viz. OCT tissue characterization
– Hierarchical learning
• Locally embedded representations.
• Sparsity is in learned (auto-encoded) representations.
Whispers of Speckles [Debdoot Sheet] - WMLMIA 81
Queries: Debdoot Sheet (debdoot@ee.iitkgp.ernet.in)
25 June 2015
82. About Deep Learning
“It’s like in quantum physics at the beginning of the
20th century” Trishul Chilimbi (MSR, DNN, Adam)
“The experimentalists and practitioners were ahead
of the theoreticians. They couldn’t explain the
results. We appear to be at a similar stage with
DNNs. We’re realizing the power and the
capabilities, but we still don’t understand the
fundamentals of exactly how they work.”
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 82
83. Take home message
“We’ve humanized the scientist;
now we must scientize the
humanist. We didn’t try to cover
physics... we uncovered it.”
- Robert Resnick
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 83
84. Take home message
• Challenges
– Architectures
• Neural Nets vs. Others
– Implementation
• CPU vs. GPU vs. Cloud
– GPU (VLSI) architectures
• Hierarchical Temporal
Memory
• Potential Causal Connection
• Toolboxes
– Theano (Python/SciPy)
– Pylearn2
– Torch
– Caffe
– Matlab (Rasmus Berg Palm)
• More information
– www.deeplearning.net
– Schmidhuber (2014). Deep
Learning in Neural
Networks: An Overview
(arXiv:1404.7828)
– Bengio (2009). Learning
Deep Architectures for AI.
– Deng and Yu (2013). Deep
Learning: Methods and
Applications.
• Conferences
– Int. Conf. Learning
Representations (ICLR)
25 June 2015 Whispers of Speckles [Debdoot Sheet] - WMLMIA 84