Dirichlet Process
•
2 likes•689 views
This document discusses Bayesian nonparametric modeling and summarizes the seminal 1973 paper by Ferguson that proposed the Dirichlet process. The Dirichlet process satisfies the requirements of having a large support for the prior while maintaining a tractable posterior. The paper outlines several formulations of the Dirichlet process, including using the Dirichlet distribution, Gamma process, Pólya urn scheme, and stick-breaking construction. It also notes that the paper suggests applications of the Dirichlet process using Bayesian rules.
Report
Share
Report
Share
Download to read offline
Recommended
Chinese Restaurant Process
The Chinese Restaurant Process is a non-parametric Bayesian process for clustering data. It models data points as customers sitting at tables in a restaurant with infinite seats. The first data point sits at the first table. Each subsequent point sits at a new table with a probability proportional to the number of occupants of existing tables, or a new empty table with some fixed probability. This allows the number of clusters to be learned from the data. The Chinese Restaurant Process can be extended to hierarchical models to allow sharing of clusters between groups.
Algorithm analysis (All in one)
Algorithm analysis (All in one): Algorithm, Algoritthm Analysis, data structure, algorithm, algorithm Complexity, Floyd Algorithm, Kruskel Algorithm, Pakistan, Sorting Complexity, Big O notations, Time Complexity, Computational Complexity, Bubble Sort Complexity, Best case, Worst Case, Inversion in Algorithm, Traveling Salesperson Problem.
Inductive and deductive reasoning
The document discusses inductive and deductive reasoning and arguments. It defines the key terms and provides examples to illustrate the differences between deductive and inductive arguments. Specifically, it notes that deductive arguments provide logically conclusive support for conclusions based on premises, while inductive arguments provide probable support based on observations or experiences. It then examines various types of deductive and inductive arguments as well as common logical fallacies.
Chapter 1 Logic of Compound Statements
This document introduces basic concepts in propositional logic and discrete mathematics including:
- Statements and their truth values
- Logical connectives such as negation, conjunction, disjunction, implication, biconditional
- Compound statements formed using logical connectives
- Truth tables to determine the truth values of compound statements
- Tautologies, contradictions and contingencies
- Negation, contrapositive, converse and inverse of conditional statements
- De Morgan's laws of negation for conjunction and disjunction
Examples are provided to illustrate key concepts and definitions throughout.
Numerical Solution of Ordinary Differential Equations
1. The document discusses numerical methods for solving ordinary differential equations, including power series approximations, Taylor series, Euler's method, and the Runge-Kutta method.
2. It provides examples of using each of these methods to solve sample differential equations and compares the numerical solutions to exact solutions.
3. Truncation errors are defined as errors that result from using an approximation instead of an exact mathematical procedure.
Multi Objective Optimization
This document discusses multi-objective optimization. It begins by defining multi-objective optimization as involving more than one objective function to optimize simultaneously. Objectives are often conflicting. The notion of an optimum must be redefined as a set of Pareto optimal solutions. Weighted sum methods are commonly used to generate Pareto optimal solutions by converting the multi-objective problem into single objective problems, but this has limitations such as an inability to find non-convex portions of the Pareto front. Evolutionary algorithms are now often used for multi-objective optimization.
Infinite series
Infinite series are useful in mathematics and fields like physics, chemistry, biology, and engineering. They allow complicated functions to be expressed as the sum of infinitely many terms, which can then be directly solved. Fourier analysis breaks functions into infinite trigonometric series, allowing the study of wave phenomena. The area inside the Koch snowflake, a fractal shape generated by infinite recursion, can be found by summing an infinite geometric series. However, its perimeter is infinite because the length grows without bound at each step of recursion. Infinite series commonly arise in solving differential equations and representing functions and are applied in fields such as image compression, sound analysis, and electrical engineering.
Design and Analysis of Algorithms
The document discusses the analysis of algorithms. It begins by defining an algorithm and describing different types. It then covers analyzing algorithms in terms of correctness, time efficiency, space efficiency, and optimality through theoretical and empirical analysis. The document discusses analyzing time efficiency by determining the number of repetitions of basic operations as a function of input size. It provides examples of input size, basic operations, and formulas for counting operations. It also covers analyzing best, worst, and average cases and establishes asymptotic efficiency classes. The document then analyzes several examples of non-recursive and recursive algorithms.
Recommended
Chinese Restaurant Process
The Chinese Restaurant Process is a non-parametric Bayesian process for clustering data. It models data points as customers sitting at tables in a restaurant with infinite seats. The first data point sits at the first table. Each subsequent point sits at a new table with a probability proportional to the number of occupants of existing tables, or a new empty table with some fixed probability. This allows the number of clusters to be learned from the data. The Chinese Restaurant Process can be extended to hierarchical models to allow sharing of clusters between groups.
Algorithm analysis (All in one)
Algorithm analysis (All in one): Algorithm, Algoritthm Analysis, data structure, algorithm, algorithm Complexity, Floyd Algorithm, Kruskel Algorithm, Pakistan, Sorting Complexity, Big O notations, Time Complexity, Computational Complexity, Bubble Sort Complexity, Best case, Worst Case, Inversion in Algorithm, Traveling Salesperson Problem.
Inductive and deductive reasoning
The document discusses inductive and deductive reasoning and arguments. It defines the key terms and provides examples to illustrate the differences between deductive and inductive arguments. Specifically, it notes that deductive arguments provide logically conclusive support for conclusions based on premises, while inductive arguments provide probable support based on observations or experiences. It then examines various types of deductive and inductive arguments as well as common logical fallacies.
Chapter 1 Logic of Compound Statements
This document introduces basic concepts in propositional logic and discrete mathematics including:
- Statements and their truth values
- Logical connectives such as negation, conjunction, disjunction, implication, biconditional
- Compound statements formed using logical connectives
- Truth tables to determine the truth values of compound statements
- Tautologies, contradictions and contingencies
- Negation, contrapositive, converse and inverse of conditional statements
- De Morgan's laws of negation for conjunction and disjunction
Examples are provided to illustrate key concepts and definitions throughout.
Numerical Solution of Ordinary Differential Equations
1. The document discusses numerical methods for solving ordinary differential equations, including power series approximations, Taylor series, Euler's method, and the Runge-Kutta method.
2. It provides examples of using each of these methods to solve sample differential equations and compares the numerical solutions to exact solutions.
3. Truncation errors are defined as errors that result from using an approximation instead of an exact mathematical procedure.
Multi Objective Optimization
This document discusses multi-objective optimization. It begins by defining multi-objective optimization as involving more than one objective function to optimize simultaneously. Objectives are often conflicting. The notion of an optimum must be redefined as a set of Pareto optimal solutions. Weighted sum methods are commonly used to generate Pareto optimal solutions by converting the multi-objective problem into single objective problems, but this has limitations such as an inability to find non-convex portions of the Pareto front. Evolutionary algorithms are now often used for multi-objective optimization.
Infinite series
Infinite series are useful in mathematics and fields like physics, chemistry, biology, and engineering. They allow complicated functions to be expressed as the sum of infinitely many terms, which can then be directly solved. Fourier analysis breaks functions into infinite trigonometric series, allowing the study of wave phenomena. The area inside the Koch snowflake, a fractal shape generated by infinite recursion, can be found by summing an infinite geometric series. However, its perimeter is infinite because the length grows without bound at each step of recursion. Infinite series commonly arise in solving differential equations and representing functions and are applied in fields such as image compression, sound analysis, and electrical engineering.
Design and Analysis of Algorithms
The document discusses the analysis of algorithms. It begins by defining an algorithm and describing different types. It then covers analyzing algorithms in terms of correctness, time efficiency, space efficiency, and optimality through theoretical and empirical analysis. The document discusses analyzing time efficiency by determining the number of repetitions of basic operations as a function of input size. It provides examples of input size, basic operations, and formulas for counting operations. It also covers analyzing best, worst, and average cases and establishes asymptotic efficiency classes. The document then analyzes several examples of non-recursive and recursive algorithms.
Brief History of Visual Representation Learning
The document summarizes the history of visual representation learning in 3 eras: (1) 2012-2015 saw the evolution of deep learning architectures like AlexNet and ResNet; (2) 2016-2019 brought diverse learning paradigms for tasks like few-shot learning and self-supervised learning; (3) 2020-present focuses on scaling laws and foundation models through larger models, data and compute as well as self-supervised methods like MAE and multimodal models like CLIP. The field is now exploring how to scale up vision transformers to match natural language models and better combine self-supervision and generative models.
Learning Visual Representations from Uncurated Data
Slide about the defense of my Ph.D. dissertation: "Learning Visual Representations from Uncurated Data"
It includes four papers about
- Learning from multi-object images for contrastive learning [1] and Vision Transformer (ViT) [2]
- Learning with limited labels (semi-sup) for image classification [3] and vision-language [4] models
[1] Mo*, Kang* et al. Object-aware Contrastive Learning for Debiased Scene Representation. NeurIPS’21.
[2] Kang*, Mo* et al. OAMixer: Object-aware Mixing Layer for Vision Transformers. CVPRW’22.
[3] Mo et al. RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data. ICLR’23.
[4] Mo et al. S-CLIP: Semi-supervised Vision-Language Pre-training using Few Specialist Captions. Under Review.
Hyperbolic Deep Reinforcement Learning
This document proposes using hyperbolic space to embed hierarchical tree structures, like those that can represent sequences of events in reinforcement learning problems. Specifically, it suggests a method called S-RYM that applies spectral normalization to regularize gradients when training deep reinforcement learning agents with hyperbolic embeddings. This stabilization technique allows naive hyperbolic embeddings to outperform standard Euclidean embeddings. It works by reducing gradient norm explosions during training, allowing the entropy loss to converge properly. The document provides technical details on spectral normalization, hyperbolic space representations, and how S-RYM trains deep reinforcement learning agents with stabilized hyperbolic embeddings.
A Unified Framework for Computer Vision Tasks: (Conditional) Generative Model...
Lab seminar introduces Ting Chen's recent 3 works:
- Pix2seq: A Language Modeling Framework for Object Detection (ICLR’22)
- A Unified Sequence Interface for Vision Tasks (NeurIPS’22)
- A Generalist Framework for Panoptic Segmentation of Images and Videos (submitted to ICLR’23)
Self-supervised Learning Lecture Note
This document is a slide presentation on recent advances in deep learning. It discusses self-supervised learning, which involves using unlabeled data to learn representations by predicting structural information within the data. The presentation covers pretext tasks, invariance-based approaches, and generation-based approaches for self-supervised learning in computer vision and natural language processing. It provides examples of specific self-supervised methods like predicting image rotations, clustering representations to generate pseudo-labels, and masked language modeling.
Deep Learning Theory Seminar (Chap 3, part 2)
This document summarizes key points from a lecture on deep learning theory:
1) It discusses the Maurey sampling technique, which shows that a finite sample approximation X^ of a random variable X converges to X as the number of samples k goes to infinity.
2) It proposes extending this technique to sample finite-width neural networks by converting the weight distribution of an infinite network to a probability measure through normalization.
3) The approximation error between outputs of the infinite and finite networks is bounded using Maurey sampling, with the bound converging to zero as the number of samples increases.
Deep Learning Theory Seminar (Chap 1-2, part 1)
1. The document discusses the approximation capabilities of deep neural networks. It outlines topics that will be covered, including approximation, optimization, and generalization.
2. For approximation, it shows that a neural network can approximate any smooth function over a compact domain to any desired accuracy by bounding the function norm. Specifically, it presents constructive proofs that a univariate function can be approximated by a 2-layer network and a multivariate function by a 3-layer network.
3. The chapter will prove approximation capabilities of finite-width neural networks, including constructive proofs for specific activations and universal approximation for general activations. It will discuss approximating indicators with ReLU activations.
Introduction to Diffusion Models
The document provides an introduction to diffusion models. It discusses that diffusion models have achieved state-of-the-art performance in image generation, density estimation, and image editing. Specifically, it covers the Denoising Diffusion Probabilistic Model (DDPM) which reparametrizes the reverse distributions of diffusion models to be more efficient. It also discusses the Denoising Diffusion Implicit Model (DDIM) which generates rough sketches of images and then refines them, significantly reducing the number of sampling steps needed compared to DDPM. In summary, diffusion models have emerged as a highly effective approach for generative modeling tasks.
Object-Region Video Transformers
1) The document discusses object-region video transformers (ORViT) for video recognition. ORViT applies attention at both the patch and object levels.
2) ORViT considers three aspects of objects: the objects themselves, interactions between objects, and object dynamics over time.
3) Experimental results show ORViT outperforms baseline models on action recognition, compositional action recognition, and spatio-temporal action detection tasks. ORViT better captures object-level information and dynamics compared to patch-level attention alone.
Deep Implicit Layers: Learning Structured Problems with Neural Networks
Deep implicit layers allow neural networks to solve structured problems by following algorithmic rules. They include layers for convex optimization, discrete optimization, differential equations, and more. The forward pass runs an algorithm, while the backward pass computes gradients using algorithmic properties like KKT conditions. This enables problems like structured prediction, meta-learning, and time series modeling to be solved reliably with neural networks by respecting their underlying structure.
Learning Theory 101 ...and Towards Learning the Flat Minima
The document discusses recent theories on why deep neural networks generalize well despite being highly overparameterized. Classic learning theory, which assumes restricting the hypothesis space is necessary for generalization, fails to explain modern neural networks. Recent studies suggest neural networks generalize because 1) their complexity is underestimated and 2) SGD regularization finds flat minima. Sharpness-aware minimization (SAM) directly optimizes for flat minima and consistently improves generalization, especially for vision transformers which have sharper loss landscapes than ResNets. SAM produces more interpretable attention maps and significantly boosts performance of vision transformers and MLP-Mixers on in-domain and out-of-domain tasks.
Sharpness-aware minimization (SAM)
Lab seminar on
- Sharpness-Aware Minimization for Efficiently Improving Generalization (ICLR 2021)
- When Vision Transformers Outperform ResNets without Pretraining or Strong Data Augmentations (under review)
Explicit Density Models
This document summarizes recent advances in deep generative models with explicit density estimation. It discusses variational autoencoders (VAEs), including techniques to improve VAEs such as importance weighting, semi-amortized inference, and mitigating posterior collapse. It also covers energy-based models, autoregressive models, flow-based models, vector-quantized VAEs, hierarchical VAEs, and diffusion probabilistic models. The document provides an overview of these generative models with a focus on density estimation and generation quality.
Score-Based Generative Modeling through Stochastic Differential Equations
Lab seminar on "Score-Based Generative Modeling through Stochastic Differential Equations", ICLR 2021 under review
Self-Attention with Linear Complexity
This document summarizes research on reducing the computational complexity of self-attention in Transformer models from O(L2) to O(L log L) or O(L). It describes the Reformer model which uses locality-sensitive hashing to achieve O(L log L) complexity, the Linformer model which uses low-rank approximations and random projections to achieve O(L) complexity, and the Synthesizer model which replaces self-attention with dense or random attention. It also briefly discusses the expressive power of sparse Transformer models.
Meta-Learning with Implicit Gradients
This document summarizes two meta-learning papers:
1) "Meta-Learning with Implicit Gradients" which introduces Implicit Model-Agnostic Meta-Learning (iMAML), an efficient alternative to MAML that computes meta-gradients without differentiating through the inner loop.
2) "Modular Meta-Learning with Shrinkage" which proposes learning a separate set of parameters for each module with different levels of shrinkage, optimized in an alternating manner to avoid collapse.
Challenging Common Assumptions in the Unsupervised Learning of Disentangled R...
Lab seminar about "Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations" (ICML 2019 best paper).
Generative Models for General Audiences
Introduction (application) of generative models for general audiences. Many figures are borrowed from https://lilianweng.github.io.
Bayesian Model-Agnostic Meta-Learning
Lab seminar slide for Bayesian Model-Agnostic Meta-Learning (NeurIPS 2018 spotlight). Sorry for not citing figures.
Deep Learning for Natural Language Processing
This document summarizes a lecture on recent advances in deep learning for natural language processing. It discusses improvements to network architectures like attention mechanisms and self-attention, which help models learn long-term dependencies and attend to relevant parts of the input. It also discusses improved training methods to reduce exposure bias and the loss-evaluation mismatch. Newer models presented include the Transformer, which uses only self-attention, and BERT, which introduces a pretrained bidirectional transformer encoder that achieves state-of-the-art results on many NLP tasks.
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
More Related Content
More from Sangwoo Mo
Brief History of Visual Representation Learning
The document summarizes the history of visual representation learning in 3 eras: (1) 2012-2015 saw the evolution of deep learning architectures like AlexNet and ResNet; (2) 2016-2019 brought diverse learning paradigms for tasks like few-shot learning and self-supervised learning; (3) 2020-present focuses on scaling laws and foundation models through larger models, data and compute as well as self-supervised methods like MAE and multimodal models like CLIP. The field is now exploring how to scale up vision transformers to match natural language models and better combine self-supervision and generative models.
Learning Visual Representations from Uncurated Data
Slide about the defense of my Ph.D. dissertation: "Learning Visual Representations from Uncurated Data"
It includes four papers about
- Learning from multi-object images for contrastive learning [1] and Vision Transformer (ViT) [2]
- Learning with limited labels (semi-sup) for image classification [3] and vision-language [4] models
[1] Mo*, Kang* et al. Object-aware Contrastive Learning for Debiased Scene Representation. NeurIPS’21.
[2] Kang*, Mo* et al. OAMixer: Object-aware Mixing Layer for Vision Transformers. CVPRW’22.
[3] Mo et al. RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data. ICLR’23.
[4] Mo et al. S-CLIP: Semi-supervised Vision-Language Pre-training using Few Specialist Captions. Under Review.
Hyperbolic Deep Reinforcement Learning
This document proposes using hyperbolic space to embed hierarchical tree structures, like those that can represent sequences of events in reinforcement learning problems. Specifically, it suggests a method called S-RYM that applies spectral normalization to regularize gradients when training deep reinforcement learning agents with hyperbolic embeddings. This stabilization technique allows naive hyperbolic embeddings to outperform standard Euclidean embeddings. It works by reducing gradient norm explosions during training, allowing the entropy loss to converge properly. The document provides technical details on spectral normalization, hyperbolic space representations, and how S-RYM trains deep reinforcement learning agents with stabilized hyperbolic embeddings.
A Unified Framework for Computer Vision Tasks: (Conditional) Generative Model...
Lab seminar introduces Ting Chen's recent 3 works:
- Pix2seq: A Language Modeling Framework for Object Detection (ICLR’22)
- A Unified Sequence Interface for Vision Tasks (NeurIPS’22)
- A Generalist Framework for Panoptic Segmentation of Images and Videos (submitted to ICLR’23)
Self-supervised Learning Lecture Note
This document is a slide presentation on recent advances in deep learning. It discusses self-supervised learning, which involves using unlabeled data to learn representations by predicting structural information within the data. The presentation covers pretext tasks, invariance-based approaches, and generation-based approaches for self-supervised learning in computer vision and natural language processing. It provides examples of specific self-supervised methods like predicting image rotations, clustering representations to generate pseudo-labels, and masked language modeling.
Deep Learning Theory Seminar (Chap 3, part 2)
This document summarizes key points from a lecture on deep learning theory:
1) It discusses the Maurey sampling technique, which shows that a finite sample approximation X^ of a random variable X converges to X as the number of samples k goes to infinity.
2) It proposes extending this technique to sample finite-width neural networks by converting the weight distribution of an infinite network to a probability measure through normalization.
3) The approximation error between outputs of the infinite and finite networks is bounded using Maurey sampling, with the bound converging to zero as the number of samples increases.
Deep Learning Theory Seminar (Chap 1-2, part 1)
1. The document discusses the approximation capabilities of deep neural networks. It outlines topics that will be covered, including approximation, optimization, and generalization.
2. For approximation, it shows that a neural network can approximate any smooth function over a compact domain to any desired accuracy by bounding the function norm. Specifically, it presents constructive proofs that a univariate function can be approximated by a 2-layer network and a multivariate function by a 3-layer network.
3. The chapter will prove approximation capabilities of finite-width neural networks, including constructive proofs for specific activations and universal approximation for general activations. It will discuss approximating indicators with ReLU activations.
Introduction to Diffusion Models
The document provides an introduction to diffusion models. It discusses that diffusion models have achieved state-of-the-art performance in image generation, density estimation, and image editing. Specifically, it covers the Denoising Diffusion Probabilistic Model (DDPM) which reparametrizes the reverse distributions of diffusion models to be more efficient. It also discusses the Denoising Diffusion Implicit Model (DDIM) which generates rough sketches of images and then refines them, significantly reducing the number of sampling steps needed compared to DDPM. In summary, diffusion models have emerged as a highly effective approach for generative modeling tasks.
Object-Region Video Transformers
1) The document discusses object-region video transformers (ORViT) for video recognition. ORViT applies attention at both the patch and object levels.
2) ORViT considers three aspects of objects: the objects themselves, interactions between objects, and object dynamics over time.
3) Experimental results show ORViT outperforms baseline models on action recognition, compositional action recognition, and spatio-temporal action detection tasks. ORViT better captures object-level information and dynamics compared to patch-level attention alone.
Deep Implicit Layers: Learning Structured Problems with Neural Networks
Deep implicit layers allow neural networks to solve structured problems by following algorithmic rules. They include layers for convex optimization, discrete optimization, differential equations, and more. The forward pass runs an algorithm, while the backward pass computes gradients using algorithmic properties like KKT conditions. This enables problems like structured prediction, meta-learning, and time series modeling to be solved reliably with neural networks by respecting their underlying structure.
Learning Theory 101 ...and Towards Learning the Flat Minima
The document discusses recent theories on why deep neural networks generalize well despite being highly overparameterized. Classic learning theory, which assumes restricting the hypothesis space is necessary for generalization, fails to explain modern neural networks. Recent studies suggest neural networks generalize because 1) their complexity is underestimated and 2) SGD regularization finds flat minima. Sharpness-aware minimization (SAM) directly optimizes for flat minima and consistently improves generalization, especially for vision transformers which have sharper loss landscapes than ResNets. SAM produces more interpretable attention maps and significantly boosts performance of vision transformers and MLP-Mixers on in-domain and out-of-domain tasks.
Sharpness-aware minimization (SAM)
Lab seminar on
- Sharpness-Aware Minimization for Efficiently Improving Generalization (ICLR 2021)
- When Vision Transformers Outperform ResNets without Pretraining or Strong Data Augmentations (under review)
Explicit Density Models
This document summarizes recent advances in deep generative models with explicit density estimation. It discusses variational autoencoders (VAEs), including techniques to improve VAEs such as importance weighting, semi-amortized inference, and mitigating posterior collapse. It also covers energy-based models, autoregressive models, flow-based models, vector-quantized VAEs, hierarchical VAEs, and diffusion probabilistic models. The document provides an overview of these generative models with a focus on density estimation and generation quality.
Score-Based Generative Modeling through Stochastic Differential Equations
Lab seminar on "Score-Based Generative Modeling through Stochastic Differential Equations", ICLR 2021 under review
Self-Attention with Linear Complexity
This document summarizes research on reducing the computational complexity of self-attention in Transformer models from O(L2) to O(L log L) or O(L). It describes the Reformer model which uses locality-sensitive hashing to achieve O(L log L) complexity, the Linformer model which uses low-rank approximations and random projections to achieve O(L) complexity, and the Synthesizer model which replaces self-attention with dense or random attention. It also briefly discusses the expressive power of sparse Transformer models.
Meta-Learning with Implicit Gradients
This document summarizes two meta-learning papers:
1) "Meta-Learning with Implicit Gradients" which introduces Implicit Model-Agnostic Meta-Learning (iMAML), an efficient alternative to MAML that computes meta-gradients without differentiating through the inner loop.
2) "Modular Meta-Learning with Shrinkage" which proposes learning a separate set of parameters for each module with different levels of shrinkage, optimized in an alternating manner to avoid collapse.
Challenging Common Assumptions in the Unsupervised Learning of Disentangled R...
Lab seminar about "Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations" (ICML 2019 best paper).
Generative Models for General Audiences
Introduction (application) of generative models for general audiences. Many figures are borrowed from https://lilianweng.github.io.
Bayesian Model-Agnostic Meta-Learning
Lab seminar slide for Bayesian Model-Agnostic Meta-Learning (NeurIPS 2018 spotlight). Sorry for not citing figures.
Deep Learning for Natural Language Processing
This document summarizes a lecture on recent advances in deep learning for natural language processing. It discusses improvements to network architectures like attention mechanisms and self-attention, which help models learn long-term dependencies and attend to relevant parts of the input. It also discusses improved training methods to reduce exposure bias and the loss-evaluation mismatch. Newer models presented include the Transformer, which uses only self-attention, and BERT, which introduces a pretrained bidirectional transformer encoder that achieves state-of-the-art results on many NLP tasks.
More from Sangwoo Mo (20)
Learning Visual Representations from Uncurated Data
Learning Visual Representations from Uncurated Data
A Unified Framework for Computer Vision Tasks: (Conditional) Generative Model...
A Unified Framework for Computer Vision Tasks: (Conditional) Generative Model...
Deep Implicit Layers: Learning Structured Problems with Neural Networks
Deep Implicit Layers: Learning Structured Problems with Neural Networks
Learning Theory 101 ...and Towards Learning the Flat Minima
Learning Theory 101 ...and Towards Learning the Flat Minima
Score-Based Generative Modeling through Stochastic Differential Equations
Score-Based Generative Modeling through Stochastic Differential Equations
Challenging Common Assumptions in the Unsupervised Learning of Disentangled R...
Challenging Common Assumptions in the Unsupervised Learning of Disentangled R...
Recently uploaded
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Dandelion Hashtable: beyond billion requests per second on a commodity server
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
AWS Cloud Cost Optimization Presentation.pptx
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
This case study explores designing a scalable e-commerce platform, covering key requirements, system components, and best practices.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
dbms calicut university B. sc Cs 4th sem.pdf
Its a seminar ppt on database management system using sql
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
JavaLand 2024: Application Development Green Masterplan
My presentation slides I used at JavaLand 2024
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
Predictive maintenance is a proactive approach that anticipates equipment failures before they happen. At the forefront of this innovative strategy is Artificial Intelligence (AI), which brings unprecedented precision and efficiency. AI in predictive maintenance is transforming industries by reducing downtime, minimizing costs, and enhancing productivity.
5th LF Energy Power Grid Model Meet-up Slides
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
A Comprehensive Guide to DeFi Development Services in 2024
DeFi represents a paradigm shift in the financial industry. Instead of relying on traditional, centralized institutions like banks, DeFi leverages blockchain technology to create a decentralized network of financial services. This means that financial transactions can occur directly between parties, without intermediaries, using smart contracts on platforms like Ethereum.
In 2024, we are witnessing an explosion of new DeFi projects and protocols, each pushing the boundaries of what’s possible in finance.
In summary, DeFi in 2024 is not just a trend; it’s a revolution that democratizes finance, enhances security and transparency, and fosters continuous innovation. As we proceed through this presentation, we'll explore the various components and services of DeFi in detail, shedding light on how they are transforming the financial landscape.
At Intelisync, we specialize in providing comprehensive DeFi development services tailored to meet the unique needs of our clients. From smart contract development to dApp creation and security audits, we ensure that your DeFi project is built with innovation, security, and scalability in mind. Trust Intelisync to guide you through the intricate landscape of decentralized finance and unlock the full potential of blockchain technology.
Ready to take your DeFi project to the next level? Partner with Intelisync for expert DeFi development services today!
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Taking AI to the Next Level in Manufacturing.pdf
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.Recently uploaded (20)
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Dirichlet Process
- 2. Table of Contents • Brief Introduction • Dirichlet Process • Applications
- 3. Table of Contents • Brief Introduction • Dirichlet Process • Applications
- 4. Brief Introduction • In Bayesian Nonparametrics, we need a prior such that 1. support of prior is large 2. posterior is manageable • (Ferguson, 1973) is the first paper proposed Dirichlet Process that satisfies two conditions above • Also, it suggests some applications using Bayes rule
- 5. Table of Contents • Brief Introduction • Dirichlet Process • Applications
- 32. Table of Contents • Brief Introduction • Dirichlet Process • Applications
- 48. End of Document