This document summarizes key concepts in nonparametric econometrics and kernel density estimation. It discusses bandwidth selection methods like cross-validation and plug-in approaches. It also covers multivariate density estimation, noting the trade-off between bias and variance. The document analyzes a real example from DiNardo and Tobias on estimating the density of female wages.
On estimating the integrated co volatility usingkkislas
This document proposes a method to estimate the integrated co-volatility of two asset prices using high-frequency data that contains both microstructure noise and jumps.
It considers two cases - when the jump processes of the two assets are independent, and when they are dependent. For the independent case, it proposes an estimator that is robust to jumps. For the dependent case, it proposes a threshold estimator that combines pre-averaging to remove noise with a threshold method to reduce the effect of jumps. It proves the estimators are consistent and establishes their central limit theorems. Simulation results are also presented to illustrate the performance of the proposed methods.
This document discusses decision theory and its applications in machine learning. It describes how decision theory uses probability to make optimal decisions given input and target data. It also discusses how to minimize expected error and loss when making predictions. Finally, it explains how inference and decision problems can be broken into two stages and different models like generative, discriminative, and discriminant functions can be used.
Numerical smoothing and hierarchical approximations for efficient option pric...Chiheb Ben Hammouda
1. The document presents a numerical smoothing technique to improve the efficiency of option pricing and density estimation when analytic smoothing is not possible.
2. The technique involves numerically determining discontinuities in the integrand and computing the integral only over the smooth regions. It also uses hierarchical representations and Brownian bridges to reduce the effective dimension of the problem.
3. The numerical smoothing approach outperforms Monte Carlo methods for high dimensional cases and improves the complexity of multilevel Monte Carlo from O(TOL^-2.5) to O(TOL^-2 log(TOL)^2).
CVPR2010: Advanced ITinCVPR in a Nutshell: part 4: additional slideszukun
This document discusses probability density function estimation using isocontours and its applications to image registration and filtering. It proposes estimating densities from image intensities using the areas enclosed by isocontours rather than histograms. This density estimation technique is applied to mutual information-based image registration and anisotropic neighborhood filtering.
Dependent processes in Bayesian NonparametricsJulyan Arbel
This document summarizes dependent processes in Bayesian nonparametrics. It motivates the need for dependent random probability measures to accommodate temporal dependence structures beyond the exchangeability assumption. It describes modeling collections of random probability measures indexed by time as either discrete-time or continuous-time processes. The diffusive Dirichlet process is introduced as a dependent Dirichlet process with Dirichlet marginal distributions at each time point and continuous sample paths. Simulation and estimation methods are discussed for this model.
Bayesian inversion of deterministic dynamic causal modelskhbrodersen
1. The document discusses various methods for Bayesian inference and model comparison in dynamic causal models, including variational Laplace approximation, sampling methods, and computing model evidence.
2. Variational Laplace approximation involves factorizing the posterior distribution and iteratively optimizing a lower bound on the model evidence called the negative free energy.
3. Sampling methods like Markov chain Monte Carlo generate stochastic approximations to the posterior by constructing a Markov chain with the target distribution as its equilibrium distribution.
On estimating the integrated co volatility usingkkislas
This document proposes a method to estimate the integrated co-volatility of two asset prices using high-frequency data that contains both microstructure noise and jumps.
It considers two cases - when the jump processes of the two assets are independent, and when they are dependent. For the independent case, it proposes an estimator that is robust to jumps. For the dependent case, it proposes a threshold estimator that combines pre-averaging to remove noise with a threshold method to reduce the effect of jumps. It proves the estimators are consistent and establishes their central limit theorems. Simulation results are also presented to illustrate the performance of the proposed methods.
This document discusses decision theory and its applications in machine learning. It describes how decision theory uses probability to make optimal decisions given input and target data. It also discusses how to minimize expected error and loss when making predictions. Finally, it explains how inference and decision problems can be broken into two stages and different models like generative, discriminative, and discriminant functions can be used.
Numerical smoothing and hierarchical approximations for efficient option pric...Chiheb Ben Hammouda
1. The document presents a numerical smoothing technique to improve the efficiency of option pricing and density estimation when analytic smoothing is not possible.
2. The technique involves numerically determining discontinuities in the integrand and computing the integral only over the smooth regions. It also uses hierarchical representations and Brownian bridges to reduce the effective dimension of the problem.
3. The numerical smoothing approach outperforms Monte Carlo methods for high dimensional cases and improves the complexity of multilevel Monte Carlo from O(TOL^-2.5) to O(TOL^-2 log(TOL)^2).
CVPR2010: Advanced ITinCVPR in a Nutshell: part 4: additional slideszukun
This document discusses probability density function estimation using isocontours and its applications to image registration and filtering. It proposes estimating densities from image intensities using the areas enclosed by isocontours rather than histograms. This density estimation technique is applied to mutual information-based image registration and anisotropic neighborhood filtering.
Dependent processes in Bayesian NonparametricsJulyan Arbel
This document summarizes dependent processes in Bayesian nonparametrics. It motivates the need for dependent random probability measures to accommodate temporal dependence structures beyond the exchangeability assumption. It describes modeling collections of random probability measures indexed by time as either discrete-time or continuous-time processes. The diffusive Dirichlet process is introduced as a dependent Dirichlet process with Dirichlet marginal distributions at each time point and continuous sample paths. Simulation and estimation methods are discussed for this model.
Bayesian inversion of deterministic dynamic causal modelskhbrodersen
1. The document discusses various methods for Bayesian inference and model comparison in dynamic causal models, including variational Laplace approximation, sampling methods, and computing model evidence.
2. Variational Laplace approximation involves factorizing the posterior distribution and iteratively optimizing a lower bound on the model evidence called the negative free energy.
3. Sampling methods like Markov chain Monte Carlo generate stochastic approximations to the posterior by constructing a Markov chain with the target distribution as its equilibrium distribution.
This document describes a new algorithm for dual tree kernel conditional density estimation (KCDE) that provides fast and accurate density predictions. The algorithm extends previous work on univariate KCDE to allow for multivariate labels (Y) and conditioning variables (X). It applies Gray's dual tree approach separately to the numerator and denominator of the KCDE formula, and uses error bounds to ensure the quotient estimates have bounded relative error. This new algorithm provides the fastest known method for kernel conditional density estimation for prediction tasks.
This document summarizes a presentation on distributed subgradient methods for saddle-point problems. It begins with an overview of distributed convex optimization and consensus-based algorithms. It then discusses using Lagrangian decomposition to distribute constraints, which allows agents to agree on Lagrange multipliers through communication. A more general saddle-point framework is presented, along with an algorithm using projected subgradients and Laplacian averaging. The algorithm is proven to converge to a saddle-point evaluation error of O(1/√t). Applications to distributed constrained optimization and low-rank matrix completion are discussed.
The document discusses Approximate Bayesian Computation (ABC). ABC allows inference for statistical models where the likelihood function is not available in closed form. ABC works by simulating data under different parameter values and comparing simulated to observed data. ABC has been used for model choice by comparing evidence for different models. Consistency of ABC for model choice depends on the criterion used and asymptotic identifiability of the parameters.
This paper proves a Caccioppoli-Kannan type fixed point theorem in generalized metric spaces (g.m.s.) that includes previous theorems by Kannan, Saha, and Mihet. It shows that if a mapping T on a T-orbitally complete g.m.s. satisfies d(T^n x, T^n y) ≤ a_n[d(x,Tx) + d(y,Ty)] where a_n converges and a_1 < 1, then T has a unique fixed point. This generalizes Caccioppoli's theorem to g.m.s. and recovers Kannan's fixed point theorem in g.
The document discusses cumulative distribution functions (CDFs) and probability density functions (PDFs) for continuous random variables. It provides definitions and properties of CDFs and PDFs. For CDFs, it describes how they give the probability that a random variable is less than or equal to a value. For PDFs, it explains how they provide the probability of a random variable taking on a particular value. The document also gives examples of CDFs and PDFs for exponential and uniform random variables.
This document discusses techniques for analyzing and summarizing medical images using language modeling. It presents several technical contributions, including multi-scale texture description using wavelet and Riesz transforms, developing a visual grammar from bags of visual words, and detecting regions of interest using geodesic operations. Experiments are described applying these methods to analyze texture in 2D and 3D medical images from datasets of lung CT and brain MRI scans.
Delayed acceptance for Metropolis-Hastings algorithmsChristian Robert
The document proposes a delayed acceptance method for accelerating Metropolis-Hastings algorithms. It begins with a motivating example of non-informative inference for mixture models where computing the prior density is costly. It then introduces the delayed acceptance approach which splits the acceptance probability into pieces that are evaluated sequentially, avoiding computing the full acceptance ratio each time. It validates that the delayed acceptance chain is reversible and provides bounds on its spectral gap and asymptotic variance compared to the original chain. Finally, it discusses optimizing the delayed acceptance approach by considering the expected square jump distance and cost per iteration to maximize efficiency.
Gentle Introduction to Dirichlet ProcessesYap Wooi Hen
This document provides an introduction to Dirichlet processes. It begins by motivating the need for nonparametric clustering when the number of clusters in the data is unknown. It then provides an overview of Dirichlet processes and discusses them from multiple perspectives, including samples from a Dirichlet process, the Chinese restaurant process representation, stick breaking construction, and formal definition. It also covers Dirichlet process mixtures and common inference techniques like Markov chain Monte Carlo and variational inference.
This document provides an overview of several topics in industrial engineering including:
- Linear programming and how to formulate it as a minimization or maximization problem subject to constraints.
- Statistical process control methods like X-bar and R charts to monitor quality.
- Process capability analysis to determine if a process meets specifications.
- Queueing models and their fundamental relationships to model waiting times in systems.
- Simulation techniques like random number generation and the inverse transform method.
- Forecasting methods such as moving averages and exponentially weighted moving averages.
- Linear regression to model relationships between variables and determine coefficients.
- Experimental design topics including randomized block design and analysis of variance calculations.
Geometric and viscosity solutions for the Cauchy problem of first orderJuliho Castillo
This document summarizes a doctoral dissertation on geometric and viscosity solutions to first order Cauchy problems. It introduces two types of solutions - viscosity solutions and minimax solutions - which are generally different. The aim is to show that iterating the minimax procedure over shorter time intervals approaches the viscosity solution. This extends previous work relating geometric and viscosity solutions in the symplectic case. The document outlines characteristics methods, generating families, Clarke calculus tools, and a proof constructing generating families to relate iterated minimax solutions to viscosity solutions.
A Gentle Introduction to Bayesian NonparametricsJulyan Arbel
The document provides an introduction to Bayesian nonparametrics and the Dirichlet process. It explains that Bayesian nonparametrics aims to fit models that can adapt their complexity based on the data, without strictly imposing a fixed structure. The Dirichlet process is described as a prior distribution on the space of all probability distributions, allowing the model to utilize an infinite number of parameters. Nonparametric mixture models using the Dirichlet process provide a flexible approach to density estimation and clustering.
Study of the impact of dielectric constant perturbation on electromagneticAlexander Decker
This document presents a study using MathCAD to numerically solve the scalar wave equation for electromagnetic wave propagation through an inhomogeneous material medium with dielectric constant perturbation. The wave equation was transformed into a form suitable for numerical solution in MathCAD. Solutions were obtained for three values of dielectric constant perturbation representing different absorption levels, within the ultraviolet, optical, and near-infrared regions of the electromagnetic spectrum. The results show the correlation between the optical field profile and propagation distance increases with increasing dielectric perturbation. MathCAD provided greater correlation between the field profile and propagation distance compared to theoretical solutions.
Resource theory of asymmetric distinguishabilityMark Wilde
We systematically develop the resource-theoretic perspective on distinguishability. The theory is a resource theory of asymmetric distinguishability, given that approximation is allowed for the first quantum state in general transformation tasks. We introduce bits of asymmetric distinguishability as the basic currency in this resource theory, and we prove that it is a reversible resource theory in the asymptotic limit, with the quantum relative entropy being the fundamental rate of resource interconversion. We formally define the distillation and dilution tasks, and we find that the exact one-shot distillable distinguishability is equal to the min-relative entropy, the exact one-shot distinguishability cost is equal to the max-relative entropy, the approximate one-shot distillable distinguishability is equal to the smooth min-relative entropy, and the approximate one-shot distinguishability cost is equal to the smooth max-relative entropy. We also develop the resource theory of asymmetric distinguishability for quantum channels. For this setting, we prove that the exact distinguishability cost is equal to channel max-relative entropy and the distillable distinguishability is equal to the amortized channel relative entropy.
This document discusses various methods for estimating normalizing constants that arise when evaluating integrals numerically. It begins by noting there are many computational methods for approximating normalizing constants across different communities. It then lists the topics that will be covered in the upcoming workshop, including discussions on estimating constants using Monte Carlo methods and Bayesian versus frequentist approaches. The document provides examples of estimating normalizing constants using Monte Carlo integration, reverse logistic regression, and Xiao-Li Meng's maximum likelihood estimation approach. It concludes by discussing some of the challenges in bringing a statistical framework to constant estimation problems.
Beating the Spread: Time-Optimal Point MeshingDon Sheehy
We present NetMesh, a new algorithm that produces a conforming Delaunay mesh for point sets in any fixed dimension with guaranteed optimal mesh size and quality.
Our comparison based algorithm runs in time $O(n\log n + m)$, where $n$ is the input size and $m$ is the output size, and with constants depending only on the dimension and the desired element quality bounds.
It can terminate early in $O(n\log n)$ time returning a $O(n)$ size Voronoi diagram of a superset of $P$ with a relaxed quality bound, which again matches the known lower bounds.
The previous best results in the comparison model depended on the log of the <b>spread</b> of the input, the ratio of the largest to smallest pairwise distance among input points.
We reduce this dependence to $O(\log n)$ by using a sequence of $\epsilon$-nets to determine input insertion order in an incremental Voronoi diagram.
We generate a hierarchy of well-spaced meshes and use these to show that the complexity of the Voronoi diagram stays linear in the number of points throughout the construction.
This document discusses quantifying measurement uncertainty. There are two main sources of uncertainty: a repeatable component and a random component. The random component incorporates all factors affecting measurement precision and leads to uncertainty in measured and calculated values. There are two approaches to quantifying standard uncertainty: Type A uses statistical analysis of replicates, while Type B uses best estimates from other factors like instrument specifications. Standard uncertainty is reported with measured values to indicate the precision of the measurement.
Network Based Kernel Density Estimation for Cycling Facilities Optimal Locati...Beniamino Murgante
Network Based Kernel Density Estimation for Cycling Facilities Optimal Location Applied to Ljubljana
Nicolas Lachance-Bernard, Timothée Produit - Ecole polytechnique fédérale de Lausanne
Biba Tominc, Matej Niksic, Barbara Golicnik Marusic - Urban Planning Institute of the Republic of Slovenia
2014.7.9 detecting p2 p botnets through network behavior analysis and machine...ericsuboy
1) The document discusses detecting P2P botnets through analyzing network behavior and machine learning. It focuses on detecting bots during the command and control (C&C) phase.
2) Network traffic from the Storm and Walowdac botnets was analyzed to identify distinguishing characteristics. Non-malicious traffic was also captured for comparison.
3) The data was evaluated using machine learning techniques, with 10-fold cross-validation showing the approach can effectively classify malicious traffic from normal P2P traffic and non-P2P traffic.
This document describes a new algorithm for dual tree kernel conditional density estimation (KCDE) that provides fast and accurate density predictions. The algorithm extends previous work on univariate KCDE to allow for multivariate labels (Y) and conditioning variables (X). It applies Gray's dual tree approach separately to the numerator and denominator of the KCDE formula, and uses error bounds to ensure the quotient estimates have bounded relative error. This new algorithm provides the fastest known method for kernel conditional density estimation for prediction tasks.
This document summarizes a presentation on distributed subgradient methods for saddle-point problems. It begins with an overview of distributed convex optimization and consensus-based algorithms. It then discusses using Lagrangian decomposition to distribute constraints, which allows agents to agree on Lagrange multipliers through communication. A more general saddle-point framework is presented, along with an algorithm using projected subgradients and Laplacian averaging. The algorithm is proven to converge to a saddle-point evaluation error of O(1/√t). Applications to distributed constrained optimization and low-rank matrix completion are discussed.
The document discusses Approximate Bayesian Computation (ABC). ABC allows inference for statistical models where the likelihood function is not available in closed form. ABC works by simulating data under different parameter values and comparing simulated to observed data. ABC has been used for model choice by comparing evidence for different models. Consistency of ABC for model choice depends on the criterion used and asymptotic identifiability of the parameters.
This paper proves a Caccioppoli-Kannan type fixed point theorem in generalized metric spaces (g.m.s.) that includes previous theorems by Kannan, Saha, and Mihet. It shows that if a mapping T on a T-orbitally complete g.m.s. satisfies d(T^n x, T^n y) ≤ a_n[d(x,Tx) + d(y,Ty)] where a_n converges and a_1 < 1, then T has a unique fixed point. This generalizes Caccioppoli's theorem to g.m.s. and recovers Kannan's fixed point theorem in g.
The document discusses cumulative distribution functions (CDFs) and probability density functions (PDFs) for continuous random variables. It provides definitions and properties of CDFs and PDFs. For CDFs, it describes how they give the probability that a random variable is less than or equal to a value. For PDFs, it explains how they provide the probability of a random variable taking on a particular value. The document also gives examples of CDFs and PDFs for exponential and uniform random variables.
This document discusses techniques for analyzing and summarizing medical images using language modeling. It presents several technical contributions, including multi-scale texture description using wavelet and Riesz transforms, developing a visual grammar from bags of visual words, and detecting regions of interest using geodesic operations. Experiments are described applying these methods to analyze texture in 2D and 3D medical images from datasets of lung CT and brain MRI scans.
Delayed acceptance for Metropolis-Hastings algorithmsChristian Robert
The document proposes a delayed acceptance method for accelerating Metropolis-Hastings algorithms. It begins with a motivating example of non-informative inference for mixture models where computing the prior density is costly. It then introduces the delayed acceptance approach which splits the acceptance probability into pieces that are evaluated sequentially, avoiding computing the full acceptance ratio each time. It validates that the delayed acceptance chain is reversible and provides bounds on its spectral gap and asymptotic variance compared to the original chain. Finally, it discusses optimizing the delayed acceptance approach by considering the expected square jump distance and cost per iteration to maximize efficiency.
Gentle Introduction to Dirichlet ProcessesYap Wooi Hen
This document provides an introduction to Dirichlet processes. It begins by motivating the need for nonparametric clustering when the number of clusters in the data is unknown. It then provides an overview of Dirichlet processes and discusses them from multiple perspectives, including samples from a Dirichlet process, the Chinese restaurant process representation, stick breaking construction, and formal definition. It also covers Dirichlet process mixtures and common inference techniques like Markov chain Monte Carlo and variational inference.
This document provides an overview of several topics in industrial engineering including:
- Linear programming and how to formulate it as a minimization or maximization problem subject to constraints.
- Statistical process control methods like X-bar and R charts to monitor quality.
- Process capability analysis to determine if a process meets specifications.
- Queueing models and their fundamental relationships to model waiting times in systems.
- Simulation techniques like random number generation and the inverse transform method.
- Forecasting methods such as moving averages and exponentially weighted moving averages.
- Linear regression to model relationships between variables and determine coefficients.
- Experimental design topics including randomized block design and analysis of variance calculations.
Geometric and viscosity solutions for the Cauchy problem of first orderJuliho Castillo
This document summarizes a doctoral dissertation on geometric and viscosity solutions to first order Cauchy problems. It introduces two types of solutions - viscosity solutions and minimax solutions - which are generally different. The aim is to show that iterating the minimax procedure over shorter time intervals approaches the viscosity solution. This extends previous work relating geometric and viscosity solutions in the symplectic case. The document outlines characteristics methods, generating families, Clarke calculus tools, and a proof constructing generating families to relate iterated minimax solutions to viscosity solutions.
A Gentle Introduction to Bayesian NonparametricsJulyan Arbel
The document provides an introduction to Bayesian nonparametrics and the Dirichlet process. It explains that Bayesian nonparametrics aims to fit models that can adapt their complexity based on the data, without strictly imposing a fixed structure. The Dirichlet process is described as a prior distribution on the space of all probability distributions, allowing the model to utilize an infinite number of parameters. Nonparametric mixture models using the Dirichlet process provide a flexible approach to density estimation and clustering.
Study of the impact of dielectric constant perturbation on electromagneticAlexander Decker
This document presents a study using MathCAD to numerically solve the scalar wave equation for electromagnetic wave propagation through an inhomogeneous material medium with dielectric constant perturbation. The wave equation was transformed into a form suitable for numerical solution in MathCAD. Solutions were obtained for three values of dielectric constant perturbation representing different absorption levels, within the ultraviolet, optical, and near-infrared regions of the electromagnetic spectrum. The results show the correlation between the optical field profile and propagation distance increases with increasing dielectric perturbation. MathCAD provided greater correlation between the field profile and propagation distance compared to theoretical solutions.
Resource theory of asymmetric distinguishabilityMark Wilde
We systematically develop the resource-theoretic perspective on distinguishability. The theory is a resource theory of asymmetric distinguishability, given that approximation is allowed for the first quantum state in general transformation tasks. We introduce bits of asymmetric distinguishability as the basic currency in this resource theory, and we prove that it is a reversible resource theory in the asymptotic limit, with the quantum relative entropy being the fundamental rate of resource interconversion. We formally define the distillation and dilution tasks, and we find that the exact one-shot distillable distinguishability is equal to the min-relative entropy, the exact one-shot distinguishability cost is equal to the max-relative entropy, the approximate one-shot distillable distinguishability is equal to the smooth min-relative entropy, and the approximate one-shot distinguishability cost is equal to the smooth max-relative entropy. We also develop the resource theory of asymmetric distinguishability for quantum channels. For this setting, we prove that the exact distinguishability cost is equal to channel max-relative entropy and the distillable distinguishability is equal to the amortized channel relative entropy.
This document discusses various methods for estimating normalizing constants that arise when evaluating integrals numerically. It begins by noting there are many computational methods for approximating normalizing constants across different communities. It then lists the topics that will be covered in the upcoming workshop, including discussions on estimating constants using Monte Carlo methods and Bayesian versus frequentist approaches. The document provides examples of estimating normalizing constants using Monte Carlo integration, reverse logistic regression, and Xiao-Li Meng's maximum likelihood estimation approach. It concludes by discussing some of the challenges in bringing a statistical framework to constant estimation problems.
Beating the Spread: Time-Optimal Point MeshingDon Sheehy
We present NetMesh, a new algorithm that produces a conforming Delaunay mesh for point sets in any fixed dimension with guaranteed optimal mesh size and quality.
Our comparison based algorithm runs in time $O(n\log n + m)$, where $n$ is the input size and $m$ is the output size, and with constants depending only on the dimension and the desired element quality bounds.
It can terminate early in $O(n\log n)$ time returning a $O(n)$ size Voronoi diagram of a superset of $P$ with a relaxed quality bound, which again matches the known lower bounds.
The previous best results in the comparison model depended on the log of the <b>spread</b> of the input, the ratio of the largest to smallest pairwise distance among input points.
We reduce this dependence to $O(\log n)$ by using a sequence of $\epsilon$-nets to determine input insertion order in an incremental Voronoi diagram.
We generate a hierarchy of well-spaced meshes and use these to show that the complexity of the Voronoi diagram stays linear in the number of points throughout the construction.
This document discusses quantifying measurement uncertainty. There are two main sources of uncertainty: a repeatable component and a random component. The random component incorporates all factors affecting measurement precision and leads to uncertainty in measured and calculated values. There are two approaches to quantifying standard uncertainty: Type A uses statistical analysis of replicates, while Type B uses best estimates from other factors like instrument specifications. Standard uncertainty is reported with measured values to indicate the precision of the measurement.
Network Based Kernel Density Estimation for Cycling Facilities Optimal Locati...Beniamino Murgante
Network Based Kernel Density Estimation for Cycling Facilities Optimal Location Applied to Ljubljana
Nicolas Lachance-Bernard, Timothée Produit - Ecole polytechnique fédérale de Lausanne
Biba Tominc, Matej Niksic, Barbara Golicnik Marusic - Urban Planning Institute of the Republic of Slovenia
2014.7.9 detecting p2 p botnets through network behavior analysis and machine...ericsuboy
1) The document discusses detecting P2P botnets through analyzing network behavior and machine learning. It focuses on detecting bots during the command and control (C&C) phase.
2) Network traffic from the Storm and Walowdac botnets was analyzed to identify distinguishing characteristics. Non-malicious traffic was also captured for comparison.
3) The data was evaluated using machine learning techniques, with 10-fold cross-validation showing the approach can effectively classify malicious traffic from normal P2P traffic and non-P2P traffic.
This document presents a talk on kernel density estimation using Bernstein polynomials and for circular data. It begins with an introduction to kernel density estimation and the empirical distribution function. It then discusses an approximation lemma due to Feller that motivates the Bernstein polynomial density estimator. The talk outlines how the kernel density estimation can be extended to multivariate density estimation and adapted for circular density estimation using Bernstein polynomials. It concludes with connecting circular kernel density estimation to orthogonal polynomials on a unit circle and provides examples of applying it to turtle direction and ant movement data.
Relational learning with social status analysis
Classify social media users with content- and network-centric features.
Social status of users are leveraged to estimate utility of content from different sources, which is induced from the social network structure.
Keywords: machine learning, graph data mining, social media analytics
This document discusses kernel-based estimation methods for inequality indices and risk measures. It begins with an overview of stochastic dominance and related indices like first-order, convex, and second-order stochastic dominance. It then discusses nonparametric estimation of densities and copula densities using kernel methods. Specifically, it proposes using beta kernels and transformed kernels to improve estimation at the boundaries. The document explores combining these approaches and using mixtures of distributions like beta distributions within the kernels. It concludes by discussing applications to heavy-tailed distributions.
- The document discusses nonparametric kernel estimation methods for copula density functions.
- It proposes using a probit transformation of the data to estimate the copula density on the unit square, which improves consistency at the boundaries compared to standard kernel methods.
- Two improved probit-transformation kernel copula density estimators are presented - one using a local log-linear approximation and one using a local log-quadratic approximation.
Randomness conductors are a general framework that unifies various combinatorial objects like expanders, extractors, condensers, and universal hash functions. They can transform a probability distribution X with a certain amount of "entropy" into another distribution X' with a specified amount of entropy. The document discusses how expanders, extractors, and other objects are special cases of randomness conductors. It also describes how zigzag graph products can be used to construct explicit constant-degree randomness conductors and discusses some open problems in further studying and constructing these objects.
- Regression discontinuity (RD) designs exploit a cutoff or threshold in an assignment variable to estimate treatment effects. They provide a "quasi-experimental" design when true randomization is not possible.
- There are several R packages that can be used to implement RD analyses, including rdrobust for estimation and inference, and rddensity for manipulation testing.
- Key steps in RD analysis include choosing an appropriate estimation method (e.g. local polynomial), selecting a bandwidth, conducting manipulation checks and placebo tests, and using robust inference methods like bias correction.
This document presents a computational framework based on transported meshfree methods. It discusses using Monte Carlo integration with kernels to estimate integration errors. Two types of kernels are introduced: lattice-based kernels suited for Lebesgue measures, and transported kernels where a transport map is applied. An example shows optimal discrepancy errors can be achieved for Monte Carlo integration with a Matern kernel in various dimensions. The framework is applied to machine learning problems, showing how kernels can be used for interpolation and extrapolation of observations with error bounds.
Condition Monitoring Of Unsteadily Operating EquipmentJordan McBain
The document discusses techniques for condition monitoring of unsteadily operating equipment. It proposes a statistical parameterization approach involving segmenting vibration data based on steady speeds/loads, extracting statistical parameters from segments, and using novelty detection with support vectors to classify patterns as normal or faulted while accounting for changing operating conditions. Experimental results on gearbox data demonstrated superior fault detection performance compared to alternative approaches.
This document provides an overview of optimization techniques. It defines optimization as identifying variable values that minimize or maximize an objective function subject to constraints. It then discusses various applications of optimization in finance, engineering, and data modeling. The document outlines different types of optimization problems and algorithms. It provides examples of unconstrained optimization algorithms like gradient descent, conjugate gradient, Newton's method, and BFGS. It also discusses the Nelder-Mead simplex algorithm for constrained optimization and compares the performance of these algorithms on sample problems.
Monte Carlo methods rely on repeated random sampling to compute results. They generate random samples from a population according to a probability distribution and use them to obtain numerical results. The founders of the Monte Carlo method were J. von Neumann and S. Ulam during the Manhattan Project in the 1940s. Monte Carlo methods can be used to solve multidimensional integrals and have better convergence than classical numerical integration methods for dimensions greater than 4. The variance of Monte Carlo estimates decreases as 1/N, where N is the number of samples, resulting in slow convergence. Variance reduction techniques can improve the convergence rate.
We examine the effectiveness of randomized quasi Monte Carlo (RQMC) to improve the convergence rate of the mean integrated square error, compared with crude Monte Carlo (MC), when estimating the density of a random variable X defined as a function over the s-dimensional unit cube (0,1)^s. We consider histograms and kernel density estimators. We show both theoretically and empirically that RQMC estimators can achieve faster convergence rates in
some situations.
This is joint work with Amal Ben Abdellah, Art B. Owen, and Florian Puchhammer.
A generalized class of normalized distance functions called Q-Metrics is described in this presentation. The Q-Metrics approach relies on a unique functional, using a single bounded parameter (Lambda), which characterizes the conventional distance functions in a normalized per-unit metric space. In addition to this coverage property, a distinguishing and extremely attractive characteristic of the Q-Metric function is its low computational complexity. Q-Metrics satisfy the standard metric axioms. Novel networks for classification and regression tasks are defined and constructed using Q-Metrics. These new networks are shown to outperform conventional feed forward back propagation networks with the same size when tested on real data sets.
A generalized class of normalized distance functions called Q-Metrics is described in this presentation. The Q-Metrics approach relies on a unique functional, using a single bounded parameter Lambda, which characterizes the conventional distance functions in a normalized per-unit metric space. In addition to this coverage property, a distinguishing and extremely attractive characteristic of the Q-Metric function is its low computational complexity. Q-Metrics satisfy the standard metric axioms. Novel networks for classification and regression tasks are defined and constructed using Q-Metrics. These new networks are shown to outperform conventional feed forward back propagation networks with the same size when tested on real data sets.
AACIMP 2010 Summer School lecture by Leonidas Sakalauskas. "Applied Mathematics" stream. "Stochastic Programming and Applications" course. Part 3.
More info at http://summerschool.ssa.org.ua
Stochastic reaction networks (SRNs) are a particular class of continuous-time Markov chains used to model a wide range of phenomena, including biological/chemical reactions, epidemics, risk theory, queuing, and supply chain/social/multi-agents networks. In this context, we explore the efficient estimation of statistical quantities, particularly rare event probabilities, and propose two alternative importance sampling (IS) approaches [1,2] to improve the Monte Carlo (MC) estimator efficiency. The key challenge in the IS framework is to choose an appropriate change of probability measure to achieve substantial variance reduction, which often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection between finding optimal IS parameters and solving a variance minimization problem via a stochastic optimal control formulation. We pursue two alternative approaches to mitigate the curse of dimensionality when solving the resulting dynamic programming problem. In the first approach [1], we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. As an alternative, we present in [2] a dimension reduction method, based on mapping the problem to a significantly lower dimensional space via the Markovian projection (MP) idea. The output of this model reduction technique is a low dimensional SRN (potentially one dimension) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained via a discrete $L^2$ regression. By solving a resulting projected Hamilton-Jacobi-Bellman (HJB) equation for the reduced-dimensional SRN, we get projected IS parameters, which are then mapped back to the original full-dimensional SRN system, and result in an efficient IS-MC estimator of the full-dimensional SRN. Our analysis and numerical experiments verify that both proposed IS (learning based and MP-HJB-IS) approaches substantially reduce the MC estimator’s variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators. [1] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. Learning-based importance sampling via stochastic optimal control for stochastic reaction net-works. Statistics and Computing 33, no. 3 (2023): 58. [2] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. (2023). Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach. To appear soon.
This document outlines the key concepts that will be covered in Lecture 2 on Bayesian modeling. It introduces the likelihood function and how it can be used to determine the most likely parameter values given observed data. It provides examples of applying Bayesian modeling to proportions, normal distributions, linear regression with one predictor, and linear regression with multiple predictors. The lecture aims to give students a basic understanding of how Bayesian analysis works and prepare them for fitting linear mixed models.
Mapping Ash Tree Colonization in an Agricultural Moutain Landscape_ Investiga...grssieee
This document summarizes a study that used hyperspectral imagery to map ash tree colonization in an agricultural mountain landscape. Researchers were able to accurately differentiate ash trees from other tree species using support vector machines with kernel alignment on very high resolution hyperspectral images. Field data was collected on tree species and biophysical parameters for analysis. Experimental results showed 94% overall accuracy and 89.9% producer accuracy for identifying ash trees. The study concluded that hyperspectral imagery enables accurate ash tree mapping and has potential for estimating biophysical parameters, with perspectives on spatial regularization.
This document summarizes research on computing stochastic partial differential equations (SPDEs) using an adaptive multi-element polynomial chaos method (MEPCM) with discrete measures. Key points include:
1) MEPCM uses polynomial chaos expansions and numerical integration to compute SPDEs with parametric uncertainty.
2) Orthogonal polynomials are generated for discrete measures using various methods like Vandermonde, Stieltjes, and Lanczos.
3) Numerical integration is tested on discrete measures using Genz functions in 1D and sparse grids in higher dimensions.
4) The method is demonstrated on the KdV equation with random initial conditions. Future work includes applying these techniques to SPDEs driven
In this talk, I address two new ideas in sampling geometric objects. The first is a new take on adaptive sampling with respect to the local feature size, i.e., the distance to the medial axis. We recently proved that such samples acn be viewed as uniform samples with respect to an alternative metric on the Euclidean space. The second is a generalization of Voronoi refinement sampling. There, one also achieves an adaptive sample while simultaneously "discovering" the underlying sizing function. We show how to construct such samples that are spaced uniformly with respect to the $k$th nearest neighbor distance function.
A common fixed point theorem in cone metric spacesAlexander Decker
This academic article summarizes a common fixed point theorem for continuous and asymptotically regular self-mappings on complete cone metric spaces. The theorem extends previous results to cone metric spaces, which generalize metric spaces by replacing real numbers with an ordered Banach space. It proves that under certain contractive conditions, the self-mapping has a unique fixed point. The proof constructs a Cauchy sequence that converges to the fixed point.
AACIMP 2010 Summer School lecture by Leonidas Sakalauskas. "Applied Mathematics" stream. "Stochastic Programming and Applications" course. Part 5.
More info at http://summerschool.ssa.org.ua
Slides: A glance at information-geometric signal processingFrank Nielsen
This document discusses information geometry and its applications in statistical signal processing. It introduces several key concepts:
1) Statistical signal processing models data with probability distributions like Gaussians and histograms. Information geometry provides a geometric framework for intuitive reasoning about these statistical models.
2) Exponential family mixture models generalize Gaussian and Rayleigh mixtures and are algorithmically useful in dually flat spaces.
3) Distances between statistical models, like Kullback-Leibler divergence and Bregman divergences, can be interpreted geometrically in terms of convex conjugates and Legendre transformations.
In topological inference, the goal is to extract information about a shape, given only a sample of points from it. There are many approaches to this problem, but the one we focus on is persistent homology. We get a view of the data at different scales by imagining the points are balls and consider different radii. The shape information we want comes in the form of a persistence diagram, which describes the components, cycles, bubbles, etc in the space that persist over a range of different scales.
To actually compute a persistence diagram in the geometric setting, previous work required complexes of size n^O(d). We reduce this complexity to O(n) (hiding some large constants depending on d) by using ideas from mesh generation.
This talk will not assume any knowledge of topology. This is joint work with Gary Miller, Benoit Hudson, and Steve Oudot.
1. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Nonparametric Econometrics
Kernel Methods for Density Estimation
James Nordlund
April 21, 2011
Nordlund Nonparametric Econometrics
2. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Example Problem
Nordlund Nonparametric Econometrics
3. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Example Problem
Nordlund Nonparametric Econometrics
4. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
How useful are kernel density estimates?
How many sample observations should we have?
Are kernel functions always reliable or did I just provide
one lucky example?
Nordlund Nonparametric Econometrics
5. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Modes of Convergence
Convergence in rth Mean
Big O notation
Nordlund Nonparametric Econometrics
6. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Definitions
Definition (Convergence in rth Mean)
We say that xn converges to X in the rth mean, if for some
r > 0,
lim E[||xn − X||r ] = 0
n→∞
rth
We write this as xn → X
Nordlund Nonparametric Econometrics
7. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Definitions
Definition (Order: Big O)
For a positive integer n, we write an = O(1) if, as n → ∞, an
remains bounded, i.e., |an | ≤ C for some constant C and for all
large values of n (an is a bounded sequence). Similarly, we write
an = O(bn ) if an /bn = O(1), or equivalently an ≤ Cbn , for some
constant C and for all n sufficiently large.
Nordlund Nonparametric Econometrics
8. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Main Theorem
Theorem
Let X1 , X2 , ..., Xn denote independent, identically distributed
observations with a twice differentiable p.d.f., f (x), and let
f (s) (x) denote the sth order derivative of f (x)(s = 1, 2). Let x
be an interior point in the support of X, and let
−x
f (x) = nh n k Xih . Assume that the kernel function, k(∗)
ˆ 1
i=1
is bounded and has µ2 < ∞. Assume that
supξ∈S(X) |f (l) (ξ)| < ∞ for l = 0, 1, 2 where S(X) denotes the
support of X. Assume that |u3 k(u)|du < ∞. Also, as n → ∞,
h → 0, and nh → ∞, then
ˆ 1
M SE(f (x)) = O h4 +
nh
Nordlund Nonparametric Econometrics
9. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Inside the Proof
Recall that
ˆ ˆ ˆ ˆ
M SE(f (x) = E [f (x) − f (x)]2 = V ar(f (x)) + Bias(f (x))2 .
Along the proof, we obtain
ˆ h2 (2)
Bias(f (x)) = f (x) u2 k(u)du + O(h3 )
2
and
ˆ 1
V ar(f (x)) = f (x) k(u)j du + O(h)
nh
Notice that there is a trade off between minimizing variance
and bias
Nordlund Nonparametric Econometrics
10. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
How do we balance variance and bias?
Nordlund Nonparametric Econometrics
11. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Important Tools
2
ISE(h) = ˆ
f (x) − f (x) dx
2
M ISE(h) = E ˆ
f (x) − f (x) dx
1 1 2
AM ISE(h) = k(x)2 dx + h4 f (2) (x)2 dx x2 k(x)dx
nh 4
Nordlund Nonparametric Econometrics
12. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Optimal h
1
k(x)2 dx 5
hopt,AM ISE = 2
n f (2) (x)2 dx x2 k(x)dx
Nordlund Nonparametric Econometrics
13. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Rule of Thumb Methods
A popular method is to assume the unknown function f has a
ˆ
normal distribution. Then we know we know what S(α) should
look like. This gives
hROT ≈ 1.06ˆ n−1/5
σ
Of course, if we knew what f looked like, we’d stick to
parametric estimation techniques.
Importantly, hROT is close to optimal for symmetric, unimodal
densities
In this case, we call hROT the normal reference rule of thumb
Nordlund Nonparametric Econometrics
14. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Plug-In Methods
The plug-in method is a two step process
Find hROT (usually just by taking the normal reference
rule of thumb)
Use hROT to estimate f (2) (x)2 dx in
1
k(x)2 dx 5
2
n f (2) (x)2 dx x2 k(x)dx
Nordlund Nonparametric Econometrics
15. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Plug-In Methods
This improves the asymptotic rate of convergence for the kernel
function
Higher iterations do not add any usefulness
Nordlund Nonparametric Econometrics
16. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
To be fair, we still make some assumption on the form of f
using the rule of thumb method
However, the assumption has less influence on our estimation,
and in applied settings the plug-in method does fairly well
More data-driven methods exist (e.g. Least Squares
Cross-Validation), but these can have a very slow rate of
convergence
Nordlund Nonparametric Econometrics
17. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
What about multivariate density?
Nordlund Nonparametric Econometrics
18. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Multivariate Kernel
Univariate:
n
ˆ 1 Xi − x
f (x) = k
nh h
i=1
Multivariate:
n
ˆ 1 Xi − x
f (x) = K
nh1 h2 · · · hq h
i=1
Nordlund Nonparametric Econometrics
19. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Multivariate Properties
Univariate:
ˆ 1
M SE(f (x)) = O h4 +
nh
Multivariate:
q
ˆ
M SE(f (x)) = O h2
s + (nh1 · · · hq )−1
s=1
Same trade-off between minimizing bias and variance
Nordlund Nonparametric Econometrics
20. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Real Example
DiNardo and Tobias (2001) - growth in female wage inequality
Parametric methods missed sharp lower bound from minimum
wage in 1979
Nordlund Nonparametric Econometrics
21. Review
Measuring Quality
Bandwidth Selection
Multivariate Density Estimation
Dr. Olofsson and Trinity Mathematics
H¨rdle, W. and Linton, O. (1994). Applied Nonparametric
a
Methods. Handbook of Econometrics. 2297-2339.
Jones, M., Marron, J., Sheather, S. (1996). A Brief Survey
of Bandwidth Selection for Density Estimation. Journal of
the American Statistical Association. Vol 91. No. 433.
401-407.
Li, Q., Racine, J. (2007). Nonparametric Econometrics.
Nordlund Nonparametric Econometrics