• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
MLconf NYC Corinna Cortes
 

MLconf NYC Corinna Cortes

on

  • 668 views

 

Statistics

Views

Total Views
668
Views on SlideShare
146
Embed Views
522

Actions

Likes
0
Downloads
3
Comments
0

4 Embeds 522

http://mlconf.com 454
http://geeksessions.wordpress.com 43
http://mlconf.wordpress.com 23
http://geeksessions.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    MLconf NYC Corinna Cortes MLconf NYC Corinna Cortes Presentation Transcript

    • Searching for Similar Items in Diverse Universes Large-scale machine learning at Google Corinna Cortes Google Research corinna@google.com 1
    • pageMachine Learning at Google 2014 Browse for Fashion 2
    • pageMachine Learning at Google 2014 Browse forVideos 3
    • pageMachine Learning at Google 2014 Outline Metric setting • Using similarities (efficiency) • Learning similarities (quality) Graph-based setting • Generating similarities 4
    • pageMachine Learning at Google 2014 Image Browsing Image browsing relies on some measure of similarity. 5 ⎛ ⎜ ⎜ ⎜ ⎝ x11 x12 ... x1N ⎞ ⎟ ⎟ ⎟ ⎠ ⎛ ⎜ ⎜ ⎜ ⎝ x21 x22 ... x2N ⎞ ⎟ ⎟ ⎟ ⎠ Sim(x1, x2) = ⎛ ⎜ ⎜ ⎜ ⎝ x11 x12 ... x1N ⎞ ⎟ ⎟ ⎟ ⎠ · ⎛ ⎜ ⎜ ⎜ ⎝ x21 x22 ... x2N ⎞ ⎟ ⎟ ⎟ ⎠
    • pageMachine Learning at Google 2014 Clustering of Images Compute all the pair-wise distances between related images and form clusters: 6
    • pageMachine Learning at Google 2014 Similar Images - version 0.0 7 Demo
    • pageMachine Learning at Google 2014 Helsinki, some cluster 8
    • pageMachine Learning at Google 2014 “Machine Learning” comes up empty 9
    • pageMachine Learning at Google 2014 The Web is Huge Image Swirl was precomputed and restricted to top 20K queries. People search on billions of different queries. Cannot precompute billions of distances we need to do something smarter. 10
    • pageMachine Learning at Google 2014 Approximate Nearest Neighbors Preprocessing: • Represent images by short vectors, kernel-PCA; • Grow tree top-down based on ‘random’ projections. Spill a bit for robustness; • Save the node ID(s) with each image. At query time: • propagate down tree to find node ID; • retrieve other image with same ID; • rank according to similarity with the short vector and other meta data. 11
    • pageMachine Learning at Google 2014 Similar Images,Version 1.0 Live Search, Similar Images Inspect the single clusters • cluster 700 • cluster 700 + machine learning • cluster 1000 • cluster 400 12
    • pageMachine Learning at Google 2014 Cluster 700 13
    • pageMachine Learning at Google 2014 Cluster 700 + Machine Learning 14
    • pageMachine Learning at Google 2014 Cluster 1000 15
    • pageMachine Learning at Google 2014 Cluster 1000 + Machine Learning 16
    • pageMachine Learning at Google 2014 Cluster 400 17
    • pageMachine Learning at Google 2014 Cluster 400 + Machine Learning 18
    • pageMachine Learning at Google 2014 Finding Similar Time Series 19
    • pageMachine Learning at Google 2014 Flu Trends 20 http://www.google.org/flutrends/us/#US
    • pageMachine Learning at Google 2014 Flu Trends 21
    • pageMachine Learning at Google 2014 Split time series into N/k chunks: Represent each chunk with a set of cluster centers (256) using k-means. Save the coordinates of the centers, (ID, coordinates). Save each series as a set of closest IDs, hashcode. Asymmetric Hashing, Indexing 22
    • pageMachine Learning at Google 2014 For given input u, divide it into its N/k chunks, uj: • Compute the N/k * 256 distances to all centers. • Compute the distances to all hash codes: MN/k additions needed. The “Asymmetric” in “Asymmetric Hashing” refers to the fact that we hash the database vectors but not the search vector. Asymmetric Hashing, Searching 23 d2 (u, vi ) = N/k j=1 d2 (uj, c(vi j))
    • pageMachine Learning at Google 2014 Google Correlate http://www.google.com/trends/correlate/ 24
    • pageMachine Learning at Google 2014 Outline Metric setting • Using similarities (efficiency) • Learning similarities (quality) Graph-based setting • Generating similarities 25
    • pageMachine Learning at Google 2014Machine Learning at Google 2014 Table Search 26 http://webdatacommons.org/webtables/ March 5, 2014: 11 billion HTML tables reduced to 147 million quasi-relational Web tables.
    • Standard Learning with Kernels The user is burdened with choosing an appropriate kernel. User Algorithm m points (x, y) 27
    • Learning Kernels Demands less commitment from the user: instead of a specific kernel, only requires the definition of a family of kernels. User Algorithm m points 28
    • pageMachine Learning at Google 2014pageMachine Learning at Google Two stages: Outperforms uniform baseline and previous algorithms. Centered alignment is key: different from notion used by (Cristiannini et al., 2001). 2014 Centered Alignment-Based LK K h 29 [C. Cortes, M. Mohri, and A. Rostamizadeh:Two-stage learning kernel methods, ICML 2010]
    • pageMachine Learning at Google 2014Machine Learning at Google 2014 Centered Alignment Centered kernels: Centered alignment: Choose kernel to maximize alignment with the target kernel: where expectation is over pairs of points. KY (xi, xj) = yiyj.
    • pageMachine Learning at Google 2014pageMachine Learning at Google 2014 Alignment-Based Kernel Learning Theoretical Results: • Concentration bound. • Existence of good predictors. Alignment algorithms: • Simple, highly scalable algorithm • Quadratic Program based algorithm. 31
    • pageMachine Learning at Google 2014Machine Learning at Google 2014 Table Search http://research.google.com/tables • Machine Learning Conferences • Marathons USA Research Tool in Docs 32
    • pageMachine Learning at Google 2014 Outline Metric setting • Using similarities (efficiency) • Learning similarities (quality) Graph-based setting • Generating similarities 33
    • pageMachine Learning at Google 2014 Graph-based setting Personalized Page Rank • PPR used to densify the graph • Single-Linkage Clustering used to find clusters in the graph 34
    • pageMachine Learning at Google 2014 Example 35
    • pageMachine Learning at Google 2014 PPR Clustering Personalized PageRank (PPR) of : • Probability of visiting in a random walk starting at : • With probability , go to a neighbor uniformly at random. • With probability , go back to ; Resulting graph: single hops enriched with multiple hops, resulting in a denser graph.Threshold graph at appropriate level. 36
    • pageMachine Learning at Google 2014 2 Pass Map-Reduce • Each push operation takes a single vertex u, moves an fraction of the probability from r(u) onto p(u), and then spreads the remaining (1 ) fraction within r, as if a single step of the lazy random walk were applied only to the vertex u. • Initialization: p = ~0 and r = indicator function at u Algorithm 37
    • pageMachine Learning at Google 2014 “Single-Linkage” Clustering Examples Can be parallelized; Can be repeated hierarchically. 38
    • pageMachine Learning at Google 2014 Demo, Phileas, Landmark The goal of this project is to develop a system to automatically recognize touristic landmarks and popular location in images and videos. Applications include automatic geo-tagging and annotation of photos or videos. Current clients of the service include Personal Photos Search, and Google Goggles, Map Explorer 39
    • pageMachine Learning at Google 2014pageMachine Learning at Google 2014 Summary Challenging very large-scale problems: • efficient and effective similarity measures algorithms. Still more to do: • learning similarities for graph based algorithms; • similarity measures vs discrepancy measures: • match time series on spikes • match shoes with shoes, highlighting differences. 40