The document discusses probability distributions and their natural parameters. It provides examples of several common distributions including the Bernoulli, multinomial, Gaussian, and gamma distributions. For each distribution, it derives the natural parameter representation and shows how to write the distribution in the form p(x|η) = h(x)g(η)exp{η^T μ(x)}. Maximum likelihood estimation for these distributions is also briefly discussed.
The document outlines linear dynamical systems (LDS), which allow modeling of systems that evolve over time in a linear fashion. LDS can be used for inference to estimate unobserved states from observations, and learning to estimate the underlying linear system from example data. Key aspects of LDS include modeling systems with both continuous and discrete states or observations, using maximum likelihood methods for inference and learning, and estimating hidden states given single or multiple observations over time.
The document discusses probability distributions and their natural parameters. It provides examples of several common distributions including the Bernoulli, multinomial, Gaussian, and gamma distributions. For each distribution, it derives the natural parameter representation and shows how to write the distribution in the form p(x|η) = h(x)g(η)exp{η^T μ(x)}. Maximum likelihood estimation for these distributions is also briefly discussed.
The document outlines linear dynamical systems (LDS), which allow modeling of systems that evolve over time in a linear fashion. LDS can be used for inference to estimate unobserved states from observations, and learning to estimate the underlying linear system from example data. Key aspects of LDS include modeling systems with both continuous and discrete states or observations, using maximum likelihood methods for inference and learning, and estimating hidden states given single or multiple observations over time.
The document discusses probabilistic principal component analysis (PPCA). It provides motivations for using PPCA instead of traditional PCA, including that PPCA allows for modeling latent variables, use of the EM algorithm, and Bayesian treatment. The key advantages of PPCA mentioned are that it can be used as a generative model, efficiently handle missing data, and be extended to mixture models.
The document discusses variational inference methods, including local variational inference and variational logistic regression. It introduces variational inference as directly approximating the posterior distribution over all variables. Variational logistic regression is described as having a variational posterior distribution and optimizing variational parameters to find the best approximation to the true posterior. Hyperparameter inference is also briefly mentioned.
The document discusses algorithms for probabilistic graphical models including:
1) The sum-product algorithm for efficiently computing marginal distributions by avoiding redundant calculations when computing multiple marginals.
2) The max-sum algorithm.
3) Exact inference on general graphs.
4) Loopy belief propagation.
5) Learning graph structures.
The document discusses convolutional neural networks (CNNs). It explains that CNNs have convolutional layers and pooling layers, as well as fully connected layers. It describes three key aspects of CNNs: local receptive fields, subsampling, and shared weights. Local receptive fields allow a neuron to only be influenced by a small region of the input. Subsampling reduces the spatial resolution but increases the number of features. Shared weights enable the same pattern to be detected across the input. The document provides an overview of how CNNs work, from input to convolutional and pooling layers to fully connected output layers.