Basics of RNNs and its applications with following papers:
- Generating Sequences With Recurrent Neural Networks, 2013
- Show and Tell: A Neural Image Caption Generator, 2014
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention, 2015
- DenseCap: Fully Convolutional Localization Networks for Dense Captioning, 2015
- Deep Tracking- Seeing Beyond Seeing Using Recurrent Neural Networks, 2016
- Robust Modeling and Prediction in Dynamic Environments Using Recurrent Flow Networks, 2016
- Social LSTM- Human Trajectory Prediction in Crowded Spaces, 2016
- DESIRE- Distant Future Prediction in Dynamic Scenes with Interacting Agents, 2017
- Predictive State Recurrent Neural Networks, 2017
Basics of RNNs and its applications with following papers:
- Generating Sequences With Recurrent Neural Networks, 2013
- Show and Tell: A Neural Image Caption Generator, 2014
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention, 2015
- DenseCap: Fully Convolutional Localization Networks for Dense Captioning, 2015
- Deep Tracking- Seeing Beyond Seeing Using Recurrent Neural Networks, 2016
- Robust Modeling and Prediction in Dynamic Environments Using Recurrent Flow Networks, 2016
- Social LSTM- Human Trajectory Prediction in Crowded Spaces, 2016
- DESIRE- Distant Future Prediction in Dynamic Scenes with Interacting Agents, 2017
- Predictive State Recurrent Neural Networks, 2017
K-Means clustering uses an iterative procedure which is very much sensitive and dependent upon the initial centroids. The initial centroids in the k-means clustering are chosen randomly, and hence the clustering also changes with respect to the initial centroids. This paper tries to overcome this problem of random selection of centroids and hence change of clusters with a premeditated selection of initial centroids. We have used the iris, abalone and wine data sets to demonstrate that the proposed method of finding the initial centroids and using the centroids in k-means algorithm improves the clustering performance. The clustering also remains the same in every run as the initial centroids are not randomly selected but through premeditated method.
Current approaches to exploring materials and manufacturing (or processing) design spaces in pursuit of new/improved engineered structural materials continue to rely heavily on extensive experimentation, which typically demand inordinate investments in both time and effort. Although tremendous progress has been made in the development and validation of a wide range of simulation toolsets capturing the multiscale phenomena controlling the material properties and performance characteristics of interest to advanced technologies, their systematic insertion into the materials innovation efforts has encountered several hurdles. The most common of these are related to (i) the lack of a generalized (applicable to a wide variety of materials classes and phenomena) mathematical framework that allows objective extraction and synergistic integration of the high value materials knowledge (defined from the perspective of producing reliable process-structure-property (PSP) linkages) from all available datasets (including a variety of multiscale experiments and simulations), while accounting for the inherent uncertainty associated with each dataset, (ii) the lack of formal approaches that identify objectively where to invest the next effort (could be a new experiment or a new simulation) for maximizing the likelihood of success (i.e., meeting or exceeding the designer-specified combinations of materials properties) at any step of the innovation effort, and (iii) the lack of experimental techniques that are specifically designed to provide the quality and quantity of information needed to calibrate the large number of material parameters present in most multiscale materials models. This talk will describe ongoing efforts in my research group aimed at addressing the gaps identified above.
https://telecombcn-dl.github.io/dlmm-2017-dcu/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...Francisco Zamora-Martinez
Artificial neural networks have proved to be good at time-series forecasting
problems, being widely studied at literature. Traditionally, shallow
architectures were used due to convergence problems when dealing with deep
models. Recent research findings enable deep architectures training, opening a
new interesting research area called deep learning. This paper presents a study
of deep learning techniques applied to time-series forecasting in a real indoor
temperature forecasting task, studying performance due to different
hyper-parameter configurations. When using deep models, better generalization
performance at test set and an over-fitting reduction has been observed.
This is the slides of my master defense; 17 april 2003
subject: "High capacity neural network optimization problems: study & solutions exploration"
In recent machine learning community, there is a trend of constructing a linear logarithm version of
nonlinear version through the ‘kernel method’ for example kernel principal component analysis, kernel
fisher discriminant analysis, support Vector Machines (SVMs), and the current kernel clustering
algorithms. Typically, in unsupervised methods of clustering algorithms utilizing kernel method, a
nonlinear mapping is operated initially in order to map the data into a much higher space feature, and then
clustering is executed. A hitch of these kernel clustering algorithms is that the clustering prototype resides
in increased features specs of dimensions and therefore lack intuitive and clear descriptions without
utilizing added approximation of projection from the specs to the data as executed in the literature
presented. This paper aims to utilize the ‘kernel method’, a novel clustering algorithm, founded on the
conventional fuzzy clustering algorithm (FCM) is anticipated and known as kernel fuzzy c-means algorithm
(KFCM). This method embraces a novel kernel-induced metric in the space of data in order to interchange
the novel Euclidean matric norm in cluster prototype and fuzzy clustering algorithm still reside in the space
of data so that the results of clustering could be interpreted and reformulated in the spaces which are
original. This property is used for clustering incomplete data. Execution on supposed data illustrate that
KFCM has improved performance of clustering and stout as compare to other transformations of FCM for
clustering incomplete data.
Slides by Albert Jiménez about the paper:
Yang, Jianwei, Devi Parikh, and Dhruv Batra. "Joint Unsupervised Learning of Deep Representations and Image Clusters." CVPR 2016.
- POSTECH EECE695J, "딥러닝 기초 및 철강공정에의 활용", 2017-11-10
- Contents: introduction to reccurent neural networks, LSTM, variants of RNN, implementation of RNN, case studies
- Video: https://youtu.be/pgqiEPb4pV8
A NOVEL ANT COLONY ALGORITHM FOR MULTICAST ROUTING IN WIRELESS AD HOC NETWORKS cscpconf
The Steiner tree is the underlying model for multicast communication. This paper presents a
novel ant colony algorithm guided by problem relaxation for unconstrained Steiner tree in static
wireless ad hoc networks. The framework of the proposed algorithm is based on ant colony
system (ACS). In the first step, the ants probabilistically construct the path from the source tothe terminal nodes. These paths are then merged together to generate a Steiner tree rooted at the source. The problem is relaxed to incorporate the structural information into the heuristic value for the selection of nodes. The effectiveness of the algorithm is tested on the benchmark problems of the OR-library. Simulation results show that our algorithm can find optimal Steiner tree with high success rate.
K-Means clustering uses an iterative procedure which is very much sensitive and dependent upon the initial centroids. The initial centroids in the k-means clustering are chosen randomly, and hence the clustering also changes with respect to the initial centroids. This paper tries to overcome this problem of random selection of centroids and hence change of clusters with a premeditated selection of initial centroids. We have used the iris, abalone and wine data sets to demonstrate that the proposed method of finding the initial centroids and using the centroids in k-means algorithm improves the clustering performance. The clustering also remains the same in every run as the initial centroids are not randomly selected but through premeditated method.
Current approaches to exploring materials and manufacturing (or processing) design spaces in pursuit of new/improved engineered structural materials continue to rely heavily on extensive experimentation, which typically demand inordinate investments in both time and effort. Although tremendous progress has been made in the development and validation of a wide range of simulation toolsets capturing the multiscale phenomena controlling the material properties and performance characteristics of interest to advanced technologies, their systematic insertion into the materials innovation efforts has encountered several hurdles. The most common of these are related to (i) the lack of a generalized (applicable to a wide variety of materials classes and phenomena) mathematical framework that allows objective extraction and synergistic integration of the high value materials knowledge (defined from the perspective of producing reliable process-structure-property (PSP) linkages) from all available datasets (including a variety of multiscale experiments and simulations), while accounting for the inherent uncertainty associated with each dataset, (ii) the lack of formal approaches that identify objectively where to invest the next effort (could be a new experiment or a new simulation) for maximizing the likelihood of success (i.e., meeting or exceeding the designer-specified combinations of materials properties) at any step of the innovation effort, and (iii) the lack of experimental techniques that are specifically designed to provide the quality and quantity of information needed to calibrate the large number of material parameters present in most multiscale materials models. This talk will describe ongoing efforts in my research group aimed at addressing the gaps identified above.
https://telecombcn-dl.github.io/dlmm-2017-dcu/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...Francisco Zamora-Martinez
Artificial neural networks have proved to be good at time-series forecasting
problems, being widely studied at literature. Traditionally, shallow
architectures were used due to convergence problems when dealing with deep
models. Recent research findings enable deep architectures training, opening a
new interesting research area called deep learning. This paper presents a study
of deep learning techniques applied to time-series forecasting in a real indoor
temperature forecasting task, studying performance due to different
hyper-parameter configurations. When using deep models, better generalization
performance at test set and an over-fitting reduction has been observed.
This is the slides of my master defense; 17 april 2003
subject: "High capacity neural network optimization problems: study & solutions exploration"
In recent machine learning community, there is a trend of constructing a linear logarithm version of
nonlinear version through the ‘kernel method’ for example kernel principal component analysis, kernel
fisher discriminant analysis, support Vector Machines (SVMs), and the current kernel clustering
algorithms. Typically, in unsupervised methods of clustering algorithms utilizing kernel method, a
nonlinear mapping is operated initially in order to map the data into a much higher space feature, and then
clustering is executed. A hitch of these kernel clustering algorithms is that the clustering prototype resides
in increased features specs of dimensions and therefore lack intuitive and clear descriptions without
utilizing added approximation of projection from the specs to the data as executed in the literature
presented. This paper aims to utilize the ‘kernel method’, a novel clustering algorithm, founded on the
conventional fuzzy clustering algorithm (FCM) is anticipated and known as kernel fuzzy c-means algorithm
(KFCM). This method embraces a novel kernel-induced metric in the space of data in order to interchange
the novel Euclidean matric norm in cluster prototype and fuzzy clustering algorithm still reside in the space
of data so that the results of clustering could be interpreted and reformulated in the spaces which are
original. This property is used for clustering incomplete data. Execution on supposed data illustrate that
KFCM has improved performance of clustering and stout as compare to other transformations of FCM for
clustering incomplete data.
Slides by Albert Jiménez about the paper:
Yang, Jianwei, Devi Parikh, and Dhruv Batra. "Joint Unsupervised Learning of Deep Representations and Image Clusters." CVPR 2016.
- POSTECH EECE695J, "딥러닝 기초 및 철강공정에의 활용", 2017-11-10
- Contents: introduction to reccurent neural networks, LSTM, variants of RNN, implementation of RNN, case studies
- Video: https://youtu.be/pgqiEPb4pV8
A NOVEL ANT COLONY ALGORITHM FOR MULTICAST ROUTING IN WIRELESS AD HOC NETWORKS cscpconf
The Steiner tree is the underlying model for multicast communication. This paper presents a
novel ant colony algorithm guided by problem relaxation for unconstrained Steiner tree in static
wireless ad hoc networks. The framework of the proposed algorithm is based on ant colony
system (ACS). In the first step, the ants probabilistically construct the path from the source tothe terminal nodes. These paths are then merged together to generate a Steiner tree rooted at the source. The problem is relaxed to incorporate the structural information into the heuristic value for the selection of nodes. The effectiveness of the algorithm is tested on the benchmark problems of the OR-library. Simulation results show that our algorithm can find optimal Steiner tree with high success rate.
Characterization of Subsurface Heterogeneity: Integration of Soft and Hard In...Amro Elfeki
Park, E., Elfeki, A. M. M., Dekking, F.M. (2003). Characterization of subsurface heterogeneity: Integration of soft and hard information using multi-dimensional Coupled Markov chain approach. Underground Injection Science and Technology Symposium, Lawrence Berkeley National Lab., October 22-25, 2003. p.49. Eds. Tsang, Chin.-Fu and Apps, John A.
http://www.lbl.gov/Conferences/UIST/index.html#topics
Segway and the Graphical Models Toolkit: a framework for probabilistic genomi...Michael Hoffman
Segway is a widely-used method for performing automated genome annotation. Researchers usually use Segway to discover recurring patterns across multiple epigenomic datasets. Then they use Segway to annotate the genome with labels for these patterns, often called chromatin states. For example, one might learn labels for promoters, enhancers, or quiescent genomic regions from histone modification and open chromatin data.
Segway is implemented using the Graphical Models Toolkit, a flexible system for hidden Markov model and dynamic Bayesian network inference. While the prevailing use of Segway remains unsupervised annotation from epigenome data, it can also perform training, posterior probability estimation, and Viterbi decoding for a wide range of probabilistic models with a recurrent structure on a genomic axis. It can accept a wide variety of models that relate hidden and observed random variables at each genomic position with each other and neighboring positions and perform necessary inference without much additional programming.
We have developed a more configurable interface to Segway to allow its use for more diverse classes of problems, models, and algorithms. We will describe several of the extensions over a simple hidden Markov model, such as semi-Markov state durations, a transcriptome model with a reversed copy of the model for stranded data, a graph-based regularization method for incorporating long-range chromatin interaction data, a semi-supervised training approach, locus-specific prior knowledge, and modeling observations with arbitrary mixtures of Gaussians. We will use some of these examples to describe how you might develop your own models.
Similar to Gaussian Process Latent Variable Models & applications in single-cell genomics (20)
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
Gaussian Process Latent Variable Models & applications in single-cell genomics
1. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Gaussian Process Latent Variable Models &
applications in single-cell genomics
Kieran Campbell
University of Oxford
November 19, 2015
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
2. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Introduction to Gaussian Processes
Gaussian Process Latent Variable Models
Applications in single-cell genomics
References
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
3. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Introduction
In (Bayesian) supervised learning some (non-)linear function
f (x; w) parametrized by w is assumed to generate data {xn, yn}.
f may take any parametric form, e.g. linear f (x) = w0 + w1x
Posterior inference can be performed on
p(w|y, X) =
p(y|w, X)p(w)
p(y|X)
(1)
Predictions of a new point {y∗, x∗} can be made by
marginalising over w:
p(y∗|y, X, x∗) = dwp(y∗|w, X, x∗)p(w|y, X) (2)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
4. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Gaussian Process Regression
Gaussian Processes place a non-parametric prior over the functions
f (x)
f always indexed by ‘input variable’ x
Any subset of functions {fi }N
i=1 are jointly drawn from a
multivariate Gaussian distribution with zero mean and
covariance matrix K:
p(f1, . . . , fN) = N(0, K) (3)
In other words, entirely defined by second-order statistics K
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
5. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Choice of Kernel
Behaviour of the GP defined by choice of kernel & parameters
Kernel function K(x, x ) becomes covariance matrix once set
of points ‘realised’
Typical choice is double exponential
K(x, x ) = exp(−λ x − x 2
) (4)
Intuition is if x and x are similar, covariance will be larger and
so f and f will - on average - be closer together
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
6. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
GPs with noisy observations
So far assumed observations of f are noise free - GP becomes
function interpolator
Instead observations y(x) corrupted by noise so
y ∼ N(f (x), σ2)
Because everything is Gaussian, can marginalise over (latent)
functions f and find
p(y1, . . . , yN) ∼ N(0, K + σ2
I) (5)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
7. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Predictions with noisy observations
To make predictions with GPs only need covariance between ‘old’
inputs X and ‘new’ input x∗:
Let k∗ = K(X, x∗) and k∗∗ = K(x∗, x∗)
Then
p(f∗|x∗, X, y) = N(f∗|kT
∗ K−1
, k∗∗ − kT
∗ K−1
k∗) (6)
This highlights the major disadvantage of GPs - to make
predictions we need to invert an n × n matrix - O(n3)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
8. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Effect of RBF kernel parameters
Kernel
κ(xp, xq) =
σ2
f exp − 1
2l2 (xp − xq)2
+ σ2
y δqp
Parameters
l controls horizontal length scale
σf controls vertical length scale
σy noise variance
In figure (l, σf , σy ) have values
(a) (1, 1, 0.1)
(b) (0.3, 1.08, 0.00005)
(c) (3.0, 1.16, 0.89)
Figure: Rasmussen and Williams
2006
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
9. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Dimensionality reduction & unsupervised learning
Dimensionality reduction
Want to reduce some observed data Y ∈ RN×D to a set of latent
variables X ∈ RN×Q where Q D.
Methods
Linear: PCA, ICA
Non-linear: Laplacian eigenmaps, MDS, etc.
Probabilistic: PPCA, GPLVM
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
10. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Probabilistic PCA (Tipping and Bishop, 1999)
Recall Y observed data matrix, X latent matrix. Then assume
yn = Wxn + ηn
where
W linear relationship between latent space and data space
ηn Gaussian noise mean 0 precision β
Then marginalise out X to find
p(yn|W , β) = N(yn|0, WW T
+ β−1
I)
Analytic solution when W spans principal subspace - probabilistic
PCA.
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
11. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
GPLVM (Lawrence 2005)
Alternative representation (dual probabilistic PCA)
Instead of marginalising latent factors X, marginalise mapping W . Let
p(W ) = i N(wi , |0, I) then
p(y:,d |X, β) = N(y:,d |0, XXT
+ β−1
I)
GPLVM
Lawrence’s breakthrough was to realise that the covariance matrix
K = XXT
+ β−1
I
can be replaced by any similarity (kernel) matrix S as in the GP
framework.
GP-LVM define a mapping from the latent space to the observed space.
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
12. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
GPLVM example - oil flow data
Figure: PCA (left) and GPLVM (right) on multi-phase oil flow data
(Lawrence 2006)
GPLVM shows better separation between oil flow class (shape) compared
to PCA
GPLVM gives uncertainty in the data space. Since this is shared across all
feautures, can visualise in latent space (pixel intensity)
If we want true uncertainty in latent need Bayesian approach to find
p(latent|data)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
13. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Bayesian GPLVM
Ideally we want to know the uncertainty in the latent factors
p(latent|data). Approaches to inference:
Metropolis-hastings - requires lots of tweaking but
‘guaranteed’ for any model
HMC with Stan - fast, requires less tweaking but less support
for arbitrary priors
Variational inference1
1
Titsias, M., & Lawrence, N. (2010). Bayesian Gaussian Process Latent
Variable Model. Artificial Intelligence, 9, 844-851.
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
14. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Buettner 2012
Introduce ‘structure preserving’ GPLVM for clustering of single-cell qPCR from
zygote to blastocyst development
Includes a ‘prior’ that preserves local structure by modifying likelihood
(previously studied2
)
Find modified GPLVM gives better separation between different
developmental stages)
2
Maaten, L. Van Der. (2005). Preserving Local Structure in Gaussian
Process Latent Variable Models
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
15. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Buettner 2015
Use (GP?)-LVM to construct low-rank cell-to-cell covariance based on expression of
specific gene pathway
Model
yg ∼ N(µg , XXT
+ σ2
ν CCT
+ ν2
g I)
where
X hidden factor such as cell cycle
C observed covariate
Can then assess gene-gene correlation controlling for hidden factors
Non-linear PCA of genes not
annotated as cell-cycle. Left:
before scLVM, right: after.
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
16. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Bayesian Gaussian Process Latent Variable Models for
pseudotime inference
Pseudotime Artificial measure of a cells progression through some
process (differentiation, apoptosis, cell cycle)
Cell ordering problem Order high-dimensional transcriptomes
through process
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
17. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Current approaches
Monocle
ICA for dimensionality
reduction, longest path
through minimum spanning
tree to assign pseudotime
Uses cubic smoothing splines &
likelihood ratio test for
differential expression analysis
Standard analysis is to examine differential expression across
pseudotime
Questions What is the uncertainty in pseudotime? How does this
impact the false discovery rate of differential expression analysis?
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
18. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Bayesian GPLVM for pseudotime inference
1. Reduce dimensionality of gene expression data (LE, t-SNE,
PCA or all at once!)
2. Fit Bayesian GPLVM in reduced space (this is essentially a
probabilistic curve)
3. Quantify posterior samples, uncertainty etc
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
19. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Model
γ ∼ Gamma(γα, γβ)
λj ∼ Exp(γ)
σj ∼ InvGamma(α, β)
ti ∼ πt, i = 1, . . . , N,
Σ = diag(σ2
1, . . . , σ2
P)
K(j)
(t, t ) = exp(−λj (t − t )2
)
µj ∼ GP(0, K(j)
), j = 1, . . . , P,
xi ∼ N(µ(ti ), Σ), i = 1, . . . , N.
(7)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
20. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Prior issues
How do we define the prior on t, πt?
Typically want t = (t1, . . . , tn) to sit uniformly on [0, 1]
t only appears in the likelihood via λj (t − t )2
Means we can arbitrarily rescale λ → λ
and t →
√
t and get
same likelihood
t equivalent on any subset of [0, 1] - ill-defined problem
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
21. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Solutions
Corp prior
Want t to ‘fill out’ over [0, 1]
Introduce repulsive prior
πt(t) ∝
N
i=1
N
j=i+1
sin (π|ti − tj |) (8)
Non conjugate & difficult to evaluate gradient - need Metropolis
Hastings
Constrained random walk inference
If we constrain t to be on [0, 1] and use random walk sampling
(MH, HMC), pseudotimes naturally ‘wander’ towards the boundary
Once there, covariance structure settles them into a steady state
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
22. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Applications to biological datasets
Applied Bayesian GPLVM to three datasets:
1. Monocle Differentiating human myoblasts (time series) - 155
cells once contamination removed
2. Ear Differentiating cells from mouse cochlear & utricular
sensory epithelia. Pseudotime shows supporting cells (SC)
differentiating into hair cells (HC)
3. Waterfall Adult neurogenesis (PCA captures pseudotime
variation)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
23. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Sampling posterior curves
A Monocle dataset, laplacian eigenmaps representation
B Ear dataset, laplacian eigenmaps representation
C Waterfall dataset, PCA representation
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
24. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
What does the posterior uncertainty look like? (I)
95% HPD credible interval typically spans ∼ 1
4 of pseudotime
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
25. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
What does the posterior uncertainty look like? (II)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
26. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Effect of hyperparameters (Monocle dataset)
Recall
K(t, t ) ∝ exp −λj (t − t )2
λj ∼ Exp(γ)
γ ∼ Gamma(γα, γβ)
|λ| roughly corresponds to arc-length. So what are the effects of
changing γα, γβ?
E[γ] = γα
γβ
, Var[γ] = γα
γ2
β
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
27. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Approximate false discovery rate
How to approximate false discovery rate?
Refit differential expression
for each gene across
posterior samples of
pseudotime
Compute p- and q- values
for each sample for each
gene
Statistic is proportion
significant at 5% FDR
Differential gene expression
is false positive if
proportion significant
< 0.95 and q-value < 0.05
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
28. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Approximate false discovery rates
Approximate false discovery rate can be very high (∼ 3× larger
than it should be) but is also variable
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
29. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Integrating multiple dimensionality reduction algorithms
Can very easily integrate multiple source of data from different
dimensionality reduction algorithms:
p(t, {X}) ∝ πt(t)p(XLE|t)p(XPCA|t)p(XtSNE|t) (9)
Natural extension to integrate multiple heterogeneous source of
data, e.g.
p(t, {X}) ∝ πt(t)p(imaging|t)p(ATAC|t)p(transcriptomics|t)
(10)
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
30. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Example: Monocle with LE, PCA & t-SNE
Learning curves for each representation separately:
Joint learning of all representations:
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
31. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
FDR from multiple representation learning
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
32. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Some good references (I)
Gaussian Processes
Rasmussen, Carl Edward. ”Gaussian processes for machine learning.” (2006).
GPLVM
Lawrence, Neil D. ”Gaussian process latent variable models for visualisation of high
dimensional data.” Advances in neural information processing systems 16.3 (2004):
329-336.
Titsias, Michalis K., and Neil D. Lawrence. ”Bayesian Gaussian process latent variable
model.” International Conference on Artificial Intelligence and Statistics. 2010.
van der Maaten, Laurens. ”Preserving local structure in Gaussian process latent variable
models.” Proceedings of the 18th Annual Belgian-Dutch Conference on Machine Learning.
2009.
Wang, Ye, and David B. Dunson. ”Probabilistic Curve Learning: Coulomb Repulsion and
the Electrostatic Gaussian Process.” arXiv preprint arXiv:1506.03768 (2015).
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics
33. Introduction to Gaussian Processes Gaussian Process Latent Variable Models Applications in single-cell genomics References
Some good references (II)
Latent variable models in single-cell genomics
Buettner, Florian, and Fabian J. Theis. ”A novel approach for resolving differences in
single-cell gene expression patterns from zygote to blastocyst.” Bioinformatics 28.18
(2012): i626-i632.
Buettner, Florian, et al. ”Computational analysis of cell-to-cell heterogeneity in single-cell
RNA-sequencing data reveals hidden subpopulations of cells.” Nature biotechnology 33.2
(2015): 155-160.
Pseudotime
Trapnell, Cole, et al. ”The dynamics and regulators of cell fate decisions are revealed by
pseudotemporal ordering of single cells.” Nature biotechnology 32.4 (2014): 381-386.
Bendall, Sean C., et al. ”Single-cell trajectory detection uncovers progression and regulatory
coordination in human B cell development.” Cell 157.3 (2014): 714-725.
Marco, Eugenio, et al. ”Bifurcation analysis of single-cell gene expression data reveals
epigenetic landscape.” Proceedings of the National Academy of Sciences 111.52 (2014):
E5643-E5650.
Shin, Jaehoon, et al. ”Single-Cell RNA-Seq with Waterfall Reveals Molecular Cascades
underlying Adult Neurogenesis.” Cell stem cell 17.3 (2015): 360-372.
Leng, Ning, et al. ”Oscope identifies oscillatory genes in unsynchronized single-cell
RNA-seq experiments.” Nature methods 12.10 (2015): 947-950.
Kieran Campbell University of Oxford
Gaussian Process Latent Variable Models & applications in single-cell genomics