SlideShare a Scribd company logo
Lecture 5
Barna Saha
AT&T-Labs Research
September 19, 2013
Outline
Approximate Near Neighbor Search
Near Neighbor Problem
Given a set of points V , a distance metric d and a query point
q, is there any point x close to query point q: d(x, q) ≤ R.
Easy in low dimension. Complexity increases exponentially in
dimension.
Approximate Near Neighbor Problem
Given a set of points V , a distance metric d and a query
point q, the (c, R)-approximate near neighbor problem
requires if there exists a point x such that d(x, q) ≤ R, then
one must find a point x such that d(x , q) ≤ cR with
probability > (1 − δ) for a given δ > 0.
The technique that we will be using to solve it is Locality Sensitive
Hashing
Locality Sensitive Hashing
A family of hash functions H that is said to be
(c, R, p1, p2)-sensitive for a distance metric d, when:
1. Prh∼H[h(x) = h(y)] ≥ p1 for all x and y such that
d(x, y) ≤ R
2. Prh∼H[h(x) = h(y)] ≤ p2 for all x and y such that
d(x, y) > cR
For H to be LSH p1 > p2.
Locality Sensitive Hashing
Example
Let V ⊆ [0, 1]n and d(x, y) = Hamming distance between x and y.
Let R << n and cR << n, define H = {h1, h2, ..., hn} such that
hi (x) = xi . p1 ≥ 1 − R
n and p2 ≤ 1 − cR
n .
Locality Sensitive Hashing for solving (c,R)-NN problem
LSH H : (c, R, p1, p2)-sensitive
hi,j ∼ H, i ∈ [1, K], j ∈ [1, L]
define gj = h1,j , h2,j , ..., hK,j for all j ∈ [1, L]
Locality Sensitive Hashing for solving (c,R)-NN problem
Preprocessing For all x ∈ V and for all j ∈ [L], add x to
bucketj (gj (x)).
Time=O(NLK)
Query(q)
for j=1,2,..,L
for all x ∈ bucketj (gj (q))
if d(x, q) ≤ cR then return x
return none
Time=O(KL + NLF) where F is the probability for any given j
that a point x is hashed to the same bucket by gj as q but
d(x, q) > cR.
Locality Sensitive Hashing for solving (c,R)-NN problem
How much is F ?
Given x and y with d(x, y) > cr,
F = Pr gj (x) = gj (y) | d(x, y) > cR
K
j=1
Pr hi,j (x) = hi,j (y) | d(x, y) > cR ≤ pk
2
Hence query time O(KL + NLpk
2 ).
Locality Sensitive Hashing for solving (c,R)-NN problem
Success Probability
Pr ∃j s.t.gj (x) = gj (q)|d(x, q) < cR
≥ Pr ∃j s.t.gj (x) = gj (q)|d(x, q) < cR
≥ 1 − (1 − pK
1 )L
Locality Sensitive Hashing for solving (c,R)-NN problem
How to choose K and L
set L = 1
pK
1
. Success probability becomes 1 − 1
e . If
δ = 1
e -happy!
To minimize query cost : O(L): NpK
2 = 1
We have
N =
1
pk
2
=
1
p1
k
log 1/p2
log 1/p1
= L
log 1/p2
log 1/p1
We have L = Nρ, ρ = log 1/p1
log 1/p2
Example
p1 = 0.1, p2 = 0.01 leads to ρ = 0.5, L =
√
N, K = O(log N).
Preprocessing time=O(N
√
N log N), Query time=O(
√
N log N).

More Related Content

What's hot

QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...
QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...
QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...
The Statistical and Applied Mathematical Sciences Institute
 
Low-rank tensor approximation (Introduction)
Low-rank tensor approximation (Introduction)Low-rank tensor approximation (Introduction)
Low-rank tensor approximation (Introduction)
Alexander Litvinenko
 
Testing for mixtures by seeking components
Testing for mixtures by seeking componentsTesting for mixtures by seeking components
Testing for mixtures by seeking components
Christian Robert
 
Bidimensionality
BidimensionalityBidimensionality
BidimensionalityASPAK2014
 
Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)
IJERD Editor
 
Important Cuts and (p,q)-clustering
Important Cuts and (p,q)-clusteringImportant Cuts and (p,q)-clustering
Important Cuts and (p,q)-clusteringASPAK2014
 
A Signature Scheme as Secure as the Diffie Hellman Problem
A Signature Scheme as Secure as the Diffie Hellman ProblemA Signature Scheme as Secure as the Diffie Hellman Problem
A Signature Scheme as Secure as the Diffie Hellman Problem
vsubhashini
 
Information in the Weights
Information in the WeightsInformation in the Weights
Information in the Weights
Mark Chang
 
Paths and Polynomials
Paths and PolynomialsPaths and Polynomials
Paths and PolynomialsASPAK2014
 
Processing Reachability Queries with Realistic Constraints on Massive Network...
Processing Reachability Queries with Realistic Constraints on Massive Network...Processing Reachability Queries with Realistic Constraints on Massive Network...
Processing Reachability Queries with Realistic Constraints on Massive Network...
BigMine
 
Kolev skalna2018 article-exact_solutiontoa_parametricline
Kolev skalna2018 article-exact_solutiontoa_parametriclineKolev skalna2018 article-exact_solutiontoa_parametricline
Kolev skalna2018 article-exact_solutiontoa_parametricline
Alina Barbulescu
 
Loss Calibrated Variational Inference
Loss Calibrated Variational InferenceLoss Calibrated Variational Inference
Loss Calibrated Variational Inference
Tomasz Kusmierczyk
 
Iterative Compression
Iterative CompressionIterative Compression
Iterative CompressionASPAK2014
 
Information in the Weights
Information in the WeightsInformation in the Weights
Information in the Weights
Mark Chang
 
Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...
Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...
Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...Tomonari Masada
 
Algorithm Design and Complexity - Course 10
Algorithm Design and Complexity - Course 10Algorithm Design and Complexity - Course 10
Algorithm Design and Complexity - Course 10Traian Rebedea
 
Achieving Spatial Adaptivity while Searching for Approximate Nearest Neighbors
Achieving Spatial Adaptivity while Searching for Approximate Nearest NeighborsAchieving Spatial Adaptivity while Searching for Approximate Nearest Neighbors
Achieving Spatial Adaptivity while Searching for Approximate Nearest NeighborsDon Sheehy
 

What's hot (20)

QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...
QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...
QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop,...
 
Low-rank tensor approximation (Introduction)
Low-rank tensor approximation (Introduction)Low-rank tensor approximation (Introduction)
Low-rank tensor approximation (Introduction)
 
Final
Final Final
Final
 
1404.1503
1404.15031404.1503
1404.1503
 
Testing for mixtures by seeking components
Testing for mixtures by seeking componentsTesting for mixtures by seeking components
Testing for mixtures by seeking components
 
Bidimensionality
BidimensionalityBidimensionality
Bidimensionality
 
Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)
 
Important Cuts and (p,q)-clustering
Important Cuts and (p,q)-clusteringImportant Cuts and (p,q)-clustering
Important Cuts and (p,q)-clustering
 
A Signature Scheme as Secure as the Diffie Hellman Problem
A Signature Scheme as Secure as the Diffie Hellman ProblemA Signature Scheme as Secure as the Diffie Hellman Problem
A Signature Scheme as Secure as the Diffie Hellman Problem
 
Information in the Weights
Information in the WeightsInformation in the Weights
Information in the Weights
 
Paths and Polynomials
Paths and PolynomialsPaths and Polynomials
Paths and Polynomials
 
Processing Reachability Queries with Realistic Constraints on Massive Network...
Processing Reachability Queries with Realistic Constraints on Massive Network...Processing Reachability Queries with Realistic Constraints on Massive Network...
Processing Reachability Queries with Realistic Constraints on Massive Network...
 
Kolev skalna2018 article-exact_solutiontoa_parametricline
Kolev skalna2018 article-exact_solutiontoa_parametriclineKolev skalna2018 article-exact_solutiontoa_parametricline
Kolev skalna2018 article-exact_solutiontoa_parametricline
 
Loss Calibrated Variational Inference
Loss Calibrated Variational InferenceLoss Calibrated Variational Inference
Loss Calibrated Variational Inference
 
Iterative Compression
Iterative CompressionIterative Compression
Iterative Compression
 
Information in the Weights
Information in the WeightsInformation in the Weights
Information in the Weights
 
Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...
Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...
Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Al...
 
Algorithm Design and Complexity - Course 10
Algorithm Design and Complexity - Course 10Algorithm Design and Complexity - Course 10
Algorithm Design and Complexity - Course 10
 
Rand final
Rand finalRand final
Rand final
 
Achieving Spatial Adaptivity while Searching for Approximate Nearest Neighbors
Achieving Spatial Adaptivity while Searching for Approximate Nearest NeighborsAchieving Spatial Adaptivity while Searching for Approximate Nearest Neighbors
Achieving Spatial Adaptivity while Searching for Approximate Nearest Neighbors
 

Viewers also liked

!Web index report 201404 all rus
!Web index report 201404 all rus!Web index report 201404 all rus
!Web index report 201404 all rusAtner Yegorov
 
программа жрс презентация 16 02 14
программа жрс   презентация 16 02 14программа жрс   презентация 16 02 14
программа жрс презентация 16 02 14Atner Yegorov
 
Biermann AstroPhysic
Biermann AstroPhysicBiermann AstroPhysic
Biermann AstroPhysic
Atner Yegorov
 
Biermann feedback
Biermann feedbackBiermann feedback
Biermann feedback
Atner Yegorov
 
Prezentazia ryazanskiy
Prezentazia ryazanskiyPrezentazia ryazanskiy
Prezentazia ryazanskiyAtner Yegorov
 
Указ Президента Чувашии от 10 октября 2007 г. №87
Указ Президента Чувашии от 10 октября 2007 г. №87Указ Президента Чувашии от 10 октября 2007 г. №87
Указ Президента Чувашии от 10 октября 2007 г. №87
Atner Yegorov
 
Prezentazia nikola lenivez
Prezentazia nikola lenivezPrezentazia nikola lenivez
Prezentazia nikola lenivezAtner Yegorov
 
лизин (презентация для тпп) январь 2012
лизин (презентация для тпп) январь 2012лизин (презентация для тпп) январь 2012
лизин (презентация для тпп) январь 2012
Atner Yegorov
 
О выполнении работ на автодорогах Чувашии
О выполнении работ на автодорогах ЧувашииО выполнении работ на автодорогах Чувашии
О выполнении работ на автодорогах Чувашии
Atner Yegorov
 
гзн васильева т.в
гзн васильева т.вгзн васильева т.в
гзн васильева т.вAtner Yegorov
 
Prezentazia zadonchina
Prezentazia zadonchinaPrezentazia zadonchina
Prezentazia zadonchinaAtner Yegorov
 
химпром
химпромхимпром
химпром
Atner Yegorov
 
презентация 2013 для пудовой и.н.
презентация 2013 для пудовой и.н.презентация 2013 для пудовой и.н.
презентация 2013 для пудовой и.н.Atner Yegorov
 
технотрон
технотронтехнотрон
технотрон
Atner Yegorov
 

Viewers also liked (18)

!Web index report 201404 all rus
!Web index report 201404 all rus!Web index report 201404 all rus
!Web index report 201404 all rus
 
программа жрс презентация 16 02 14
программа жрс   презентация 16 02 14программа жрс   презентация 16 02 14
программа жрс презентация 16 02 14
 
Biermann AstroPhysic
Biermann AstroPhysicBiermann AstroPhysic
Biermann AstroPhysic
 
Biermann feedback
Biermann feedbackBiermann feedback
Biermann feedback
 
Prezentazia ryazanskiy
Prezentazia ryazanskiyPrezentazia ryazanskiy
Prezentazia ryazanskiy
 
Lec2 slides
Lec2 slidesLec2 slides
Lec2 slides
 
Biermann clusters
Biermann clustersBiermann clusters
Biermann clusters
 
Указ Президента Чувашии от 10 октября 2007 г. №87
Указ Президента Чувашии от 10 октября 2007 г. №87Указ Президента Чувашии от 10 октября 2007 г. №87
Указ Президента Чувашии от 10 октября 2007 г. №87
 
Prezentazia nikola lenivez
Prezentazia nikola lenivezPrezentazia nikola lenivez
Prezentazia nikola lenivez
 
Itogi2012
Itogi2012Itogi2012
Itogi2012
 
лизин (презентация для тпп) январь 2012
лизин (презентация для тпп) январь 2012лизин (презентация для тпп) январь 2012
лизин (презентация для тпп) январь 2012
 
О выполнении работ на автодорогах Чувашии
О выполнении работ на автодорогах ЧувашииО выполнении работ на автодорогах Чувашии
О выполнении работ на автодорогах Чувашии
 
гзн васильева т.в
гзн васильева т.вгзн васильева т.в
гзн васильева т.в
 
Prezentazia zadonchina
Prezentazia zadonchinaPrezentazia zadonchina
Prezentazia zadonchina
 
химпром
химпромхимпром
химпром
 
Itogi pr
Itogi prItogi pr
Itogi pr
 
презентация 2013 для пудовой и.н.
презентация 2013 для пудовой и.н.презентация 2013 для пудовой и.н.
презентация 2013 для пудовой и.н.
 
технотрон
технотронтехнотрон
технотрон
 

Similar to Lec 5-nn-slides

QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...
QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...
QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...
The Statistical and Applied Mathematical Sciences Institute
 
Approximate Nearest Neighbour in Higher Dimensions
Approximate Nearest Neighbour in Higher DimensionsApproximate Nearest Neighbour in Higher Dimensions
Approximate Nearest Neighbour in Higher Dimensions
Shrey Verma
 
IVR - Chapter 1 - Introduction
IVR - Chapter 1 - IntroductionIVR - Chapter 1 - Introduction
IVR - Chapter 1 - Introduction
Charles Deledalle
 
Q-Metrics in Theory And Practice
Q-Metrics in Theory And PracticeQ-Metrics in Theory And Practice
Q-Metrics in Theory And Practice
guest3550292
 
Q-Metrics in Theory and Practice
Q-Metrics in Theory and PracticeQ-Metrics in Theory and Practice
Q-Metrics in Theory and Practice
Magdi Mohamed
 
Delayed acceptance for Metropolis-Hastings algorithms
Delayed acceptance for Metropolis-Hastings algorithmsDelayed acceptance for Metropolis-Hastings algorithms
Delayed acceptance for Metropolis-Hastings algorithms
Christian Robert
 
Bird’s-eye view of Gaussian harmonic analysis
Bird’s-eye view of Gaussian harmonic analysisBird’s-eye view of Gaussian harmonic analysis
Bird’s-eye view of Gaussian harmonic analysis
Radboud University Medical Center
 
A Generalized Metric Space and Related Fixed Point Theorems
A Generalized Metric Space and Related Fixed Point TheoremsA Generalized Metric Space and Related Fixed Point Theorems
A Generalized Metric Space and Related Fixed Point Theorems
IRJET Journal
 
6-Nfa & equivalence with RE.pdf
6-Nfa & equivalence with RE.pdf6-Nfa & equivalence with RE.pdf
6-Nfa & equivalence with RE.pdf
shruti533256
 
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...
Leo Asselborn
 
Sensors and Samples: A Homological Approach
Sensors and Samples:  A Homological ApproachSensors and Samples:  A Homological Approach
Sensors and Samples: A Homological Approach
Don Sheehy
 
Lossy Kernelization
Lossy KernelizationLossy Kernelization
Lossy Kernelization
msramanujan
 
Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...
Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...
Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...
The Statistical and Applied Mathematical Sciences Institute
 
CMSC 56 | Lecture 3: Predicates & Quantifiers
CMSC 56 | Lecture 3: Predicates & QuantifiersCMSC 56 | Lecture 3: Predicates & Quantifiers
CMSC 56 | Lecture 3: Predicates & Quantifiers
allyn joy calcaben
 
Some Thoughts on Sampling
Some Thoughts on SamplingSome Thoughts on Sampling
Some Thoughts on Sampling
Don Sheehy
 
How to find a cheap surrogate to approximate Bayesian Update Formula and to a...
How to find a cheap surrogate to approximate Bayesian Update Formula and to a...How to find a cheap surrogate to approximate Bayesian Update Formula and to a...
How to find a cheap surrogate to approximate Bayesian Update Formula and to a...
Alexander Litvinenko
 
Andreas Eberle
Andreas EberleAndreas Eberle
Andreas Eberle
BigMC
 
Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...
Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...
Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...
Leo Asselborn
 
Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli...
 Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli... Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli...
Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli...
The Statistical and Applied Mathematical Sciences Institute
 

Similar to Lec 5-nn-slides (20)

Lecture5
Lecture5Lecture5
Lecture5
 
QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...
QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...
QMC: Transition Workshop - Probabilistic Integrators for Deterministic Differ...
 
Approximate Nearest Neighbour in Higher Dimensions
Approximate Nearest Neighbour in Higher DimensionsApproximate Nearest Neighbour in Higher Dimensions
Approximate Nearest Neighbour in Higher Dimensions
 
IVR - Chapter 1 - Introduction
IVR - Chapter 1 - IntroductionIVR - Chapter 1 - Introduction
IVR - Chapter 1 - Introduction
 
Q-Metrics in Theory And Practice
Q-Metrics in Theory And PracticeQ-Metrics in Theory And Practice
Q-Metrics in Theory And Practice
 
Q-Metrics in Theory and Practice
Q-Metrics in Theory and PracticeQ-Metrics in Theory and Practice
Q-Metrics in Theory and Practice
 
Delayed acceptance for Metropolis-Hastings algorithms
Delayed acceptance for Metropolis-Hastings algorithmsDelayed acceptance for Metropolis-Hastings algorithms
Delayed acceptance for Metropolis-Hastings algorithms
 
Bird’s-eye view of Gaussian harmonic analysis
Bird’s-eye view of Gaussian harmonic analysisBird’s-eye view of Gaussian harmonic analysis
Bird’s-eye view of Gaussian harmonic analysis
 
A Generalized Metric Space and Related Fixed Point Theorems
A Generalized Metric Space and Related Fixed Point TheoremsA Generalized Metric Space and Related Fixed Point Theorems
A Generalized Metric Space and Related Fixed Point Theorems
 
6-Nfa & equivalence with RE.pdf
6-Nfa & equivalence with RE.pdf6-Nfa & equivalence with RE.pdf
6-Nfa & equivalence with RE.pdf
 
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reach...
 
Sensors and Samples: A Homological Approach
Sensors and Samples:  A Homological ApproachSensors and Samples:  A Homological Approach
Sensors and Samples: A Homological Approach
 
Lossy Kernelization
Lossy KernelizationLossy Kernelization
Lossy Kernelization
 
Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...
Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...
Deep Learning Opening Workshop - ProxSARAH Algorithms for Stochastic Composit...
 
CMSC 56 | Lecture 3: Predicates & Quantifiers
CMSC 56 | Lecture 3: Predicates & QuantifiersCMSC 56 | Lecture 3: Predicates & Quantifiers
CMSC 56 | Lecture 3: Predicates & Quantifiers
 
Some Thoughts on Sampling
Some Thoughts on SamplingSome Thoughts on Sampling
Some Thoughts on Sampling
 
How to find a cheap surrogate to approximate Bayesian Update Formula and to a...
How to find a cheap surrogate to approximate Bayesian Update Formula and to a...How to find a cheap surrogate to approximate Bayesian Update Formula and to a...
How to find a cheap surrogate to approximate Bayesian Update Formula and to a...
 
Andreas Eberle
Andreas EberleAndreas Eberle
Andreas Eberle
 
Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...
Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...
Control of Discrete-Time Piecewise Affine Probabilistic Systems using Reachab...
 
Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli...
 Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli... Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli...
Program on Quasi-Monte Carlo and High-Dimensional Sampling Methods for Appli...
 

More from Atner Yegorov

Здравоохранение Чувашии 2015
Здравоохранение Чувашии 2015Здравоохранение Чувашии 2015
Здравоохранение Чувашии 2015
Atner Yegorov
 
Чувашия. Итоги за 5 лет. 2015
Чувашия. Итоги за 5 лет. 2015Чувашия. Итоги за 5 лет. 2015
Чувашия. Итоги за 5 лет. 2015
Atner Yegorov
 
Инвестиционная политика Чувашской Республики 2015
Инвестиционная политика Чувашской Республики 2015Инвестиционная политика Чувашской Республики 2015
Инвестиционная политика Чувашской Республики 2015
Atner Yegorov
 
Господдержка малого и среднего бизнеса в Чувашии 2015
Господдержка малого и среднего бизнеса в Чувашии 2015Господдержка малого и среднего бизнеса в Чувашии 2015
Господдержка малого и среднего бизнеса в Чувашии 2015
Atner Yegorov
 
ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015
ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015
ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015
Atner Yegorov
 
Брендинг Чувашии
Брендинг ЧувашииБрендинг Чувашии
Брендинг Чувашии
Atner Yegorov
 
VIII Экономический форум Чебоксары
VIII Экономический форум ЧебоксарыVIII Экономический форум Чебоксары
VIII Экономический форум Чебоксары
Atner Yegorov
 
Рейтин инвестклимата в регионах 2015
Рейтин инвестклимата в регионах 2015Рейтин инвестклимата в регионах 2015
Рейтин инвестклимата в регионах 2015
Atner Yegorov
 
Аттракционы тематического парка, Татарстан
Аттракционы тематического парка, ТатарстанАттракционы тематического парка, Татарстан
Аттракционы тематического парка, Татарстан
Atner Yegorov
 
Тематический парк, Татарстан
Тематический парк, ТатарстанТематический парк, Татарстан
Тематический парк, Татарстан
Atner Yegorov
 
Будущее журналистики
Будущее журналистикиБудущее журналистики
Будущее журналистики
Atner Yegorov
 
Будущее нишевых СМИ
Будущее нишевых СМИБудущее нишевых СМИ
Будущее нишевых СМИ
Atner Yegorov
 
Relap
RelapRelap
Будущее онлайн СМИ в регионах
Будущее онлайн СМИ в регионах Будущее онлайн СМИ в регионах
Будущее онлайн СМИ в регионах
Atner Yegorov
 
Война форматов. Как люди читают новости
Война форматов. Как люди читают новости Война форматов. Как люди читают новости
Война форматов. Как люди читают новости
Atner Yegorov
 
Национальная технологическая инициатива
Национальная технологическая инициативаНациональная технологическая инициатива
Национальная технологическая инициатива
Atner Yegorov
 
О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ
О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ  ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ  ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ
О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ
Atner Yegorov
 
Участники заседания по модернизации
Участники заседания по модернизацииУчастники заседания по модернизации
Участники заседания по модернизации
Atner Yegorov
 
Город Innopolis
Город InnopolisГород Innopolis
Город Innopolis
Atner Yegorov
 
презентация1
презентация1презентация1
презентация1Atner Yegorov
 

More from Atner Yegorov (20)

Здравоохранение Чувашии 2015
Здравоохранение Чувашии 2015Здравоохранение Чувашии 2015
Здравоохранение Чувашии 2015
 
Чувашия. Итоги за 5 лет. 2015
Чувашия. Итоги за 5 лет. 2015Чувашия. Итоги за 5 лет. 2015
Чувашия. Итоги за 5 лет. 2015
 
Инвестиционная политика Чувашской Республики 2015
Инвестиционная политика Чувашской Республики 2015Инвестиционная политика Чувашской Республики 2015
Инвестиционная политика Чувашской Республики 2015
 
Господдержка малого и среднего бизнеса в Чувашии 2015
Господдержка малого и среднего бизнеса в Чувашии 2015Господдержка малого и среднего бизнеса в Чувашии 2015
Господдержка малого и среднего бизнеса в Чувашии 2015
 
ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015
ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015
ИННОВАЦИИ В ПРОМЫШЛЕННОСТИ ЧУВАШИИ. 2015
 
Брендинг Чувашии
Брендинг ЧувашииБрендинг Чувашии
Брендинг Чувашии
 
VIII Экономический форум Чебоксары
VIII Экономический форум ЧебоксарыVIII Экономический форум Чебоксары
VIII Экономический форум Чебоксары
 
Рейтин инвестклимата в регионах 2015
Рейтин инвестклимата в регионах 2015Рейтин инвестклимата в регионах 2015
Рейтин инвестклимата в регионах 2015
 
Аттракционы тематического парка, Татарстан
Аттракционы тематического парка, ТатарстанАттракционы тематического парка, Татарстан
Аттракционы тематического парка, Татарстан
 
Тематический парк, Татарстан
Тематический парк, ТатарстанТематический парк, Татарстан
Тематический парк, Татарстан
 
Будущее журналистики
Будущее журналистикиБудущее журналистики
Будущее журналистики
 
Будущее нишевых СМИ
Будущее нишевых СМИБудущее нишевых СМИ
Будущее нишевых СМИ
 
Relap
RelapRelap
Relap
 
Будущее онлайн СМИ в регионах
Будущее онлайн СМИ в регионах Будущее онлайн СМИ в регионах
Будущее онлайн СМИ в регионах
 
Война форматов. Как люди читают новости
Война форматов. Как люди читают новости Война форматов. Как люди читают новости
Война форматов. Как люди читают новости
 
Национальная технологическая инициатива
Национальная технологическая инициативаНациональная технологическая инициатива
Национальная технологическая инициатива
 
О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ
О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ  ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ  ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ
О РАЗРАБОТКЕ И РЕАЛИЗАЦИИ НАЦИОНАЛЬНОЙ ТЕХНОЛОГИЧЕСКОЙ ИНИЦИАТИВЫ
 
Участники заседания по модернизации
Участники заседания по модернизацииУчастники заседания по модернизации
Участники заседания по модернизации
 
Город Innopolis
Город InnopolisГород Innopolis
Город Innopolis
 
презентация1
презентация1презентация1
презентация1
 

Recently uploaded

To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 

Recently uploaded (20)

To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 

Lec 5-nn-slides

  • 1. Lecture 5 Barna Saha AT&T-Labs Research September 19, 2013
  • 3. Near Neighbor Problem Given a set of points V , a distance metric d and a query point q, is there any point x close to query point q: d(x, q) ≤ R. Easy in low dimension. Complexity increases exponentially in dimension.
  • 4. Approximate Near Neighbor Problem Given a set of points V , a distance metric d and a query point q, the (c, R)-approximate near neighbor problem requires if there exists a point x such that d(x, q) ≤ R, then one must find a point x such that d(x , q) ≤ cR with probability > (1 − δ) for a given δ > 0. The technique that we will be using to solve it is Locality Sensitive Hashing
  • 5. Locality Sensitive Hashing A family of hash functions H that is said to be (c, R, p1, p2)-sensitive for a distance metric d, when: 1. Prh∼H[h(x) = h(y)] ≥ p1 for all x and y such that d(x, y) ≤ R 2. Prh∼H[h(x) = h(y)] ≤ p2 for all x and y such that d(x, y) > cR For H to be LSH p1 > p2.
  • 6. Locality Sensitive Hashing Example Let V ⊆ [0, 1]n and d(x, y) = Hamming distance between x and y. Let R << n and cR << n, define H = {h1, h2, ..., hn} such that hi (x) = xi . p1 ≥ 1 − R n and p2 ≤ 1 − cR n .
  • 7. Locality Sensitive Hashing for solving (c,R)-NN problem LSH H : (c, R, p1, p2)-sensitive hi,j ∼ H, i ∈ [1, K], j ∈ [1, L] define gj = h1,j , h2,j , ..., hK,j for all j ∈ [1, L]
  • 8. Locality Sensitive Hashing for solving (c,R)-NN problem Preprocessing For all x ∈ V and for all j ∈ [L], add x to bucketj (gj (x)). Time=O(NLK) Query(q) for j=1,2,..,L for all x ∈ bucketj (gj (q)) if d(x, q) ≤ cR then return x return none Time=O(KL + NLF) where F is the probability for any given j that a point x is hashed to the same bucket by gj as q but d(x, q) > cR.
  • 9. Locality Sensitive Hashing for solving (c,R)-NN problem How much is F ? Given x and y with d(x, y) > cr, F = Pr gj (x) = gj (y) | d(x, y) > cR K j=1 Pr hi,j (x) = hi,j (y) | d(x, y) > cR ≤ pk 2 Hence query time O(KL + NLpk 2 ).
  • 10. Locality Sensitive Hashing for solving (c,R)-NN problem Success Probability Pr ∃j s.t.gj (x) = gj (q)|d(x, q) < cR ≥ Pr ∃j s.t.gj (x) = gj (q)|d(x, q) < cR ≥ 1 − (1 − pK 1 )L
  • 11. Locality Sensitive Hashing for solving (c,R)-NN problem How to choose K and L set L = 1 pK 1 . Success probability becomes 1 − 1 e . If δ = 1 e -happy! To minimize query cost : O(L): NpK 2 = 1 We have N = 1 pk 2 = 1 p1 k log 1/p2 log 1/p1 = L log 1/p2 log 1/p1 We have L = Nρ, ρ = log 1/p1 log 1/p2 Example p1 = 0.1, p2 = 0.01 leads to ρ = 0.5, L = √ N, K = O(log N). Preprocessing time=O(N √ N log N), Query time=O( √ N log N).