1.
Presented by: Mike Limcaco
Contributions: Lexi Flynn & Bryan Rutkowski
2022 August 13
DataCon LA 2022
Trends To Recommendations
Using Google trends data to build product recommendations

2.
https://www.themoviedb.org/

3.
https://developer.apple.com/design/human-interface-guidelines/components/system-experiences/top-shelf/
Featured Content

4.
How can we tap into what’s trending &
continuously promote new, interesting and
relevant content?
1. Aggregate trending search interest
2. Turn these into smart searches against our product catalog(s)
3. Filter, sort and rank to further refine results

5.
Leverage Google-Unique Data & APIs
To Drive Personalized Discovery
Data Signals
Aggregated search trend data across
geographic focus areas
Retail Search
Power your ecommerce site or
application with Google-quality
search capabilities
Vertex AI Matching Engine
Use innovative semantic matching
features to improve search results

6.
Data Signals
Aggregated search trend data across
geographic focus areas
Retail Search
Power your ecommerce site or
application with Google-quality
search capabilities
Vertex AI Matching Engine
Use innovative semantic matching
features to improve search results
Leverage Google-Unique Data & APIs
To Drive Personalized Discovery

7.
What is BigQuery Data Signals?
Google Trends
Data Signals
BigQuery Public
Datasets
Program
Insights for Everyone
Marketing
How do I optimize my paid media spend
when it comes to search term bidding?
Merchandising & Demand Forecasting
How can I understand demand trends to
make better assortment planning decisions?
Executives
How is the business performing across all of
our markets? Against competitors?
Supply Chain
How do I manage supply chain logistics and
distribute inventory better?

8.
https://datasignals.looker.com/embed/dashboards/11

9.
Data Signals
Aggregated search trend data across
geographic focus areas
Retail Search
Power your ecommerce site or
application with Google-quality
search capabilities
Vertex AI Matching Engine
Use innovative semantic matching
features to improve search results
Leverage Google-Unique Data & APIs
To Drive Personalized Discovery

10.
Assistant
Text search
Google Cloud Retail Search
High-Quality Product Search Results as a Service
Typeahead
Product Catalog
User Events, Logs
Custom LTR Models
Query Understanding
Web + Structured Index
Pricing Updates
Business / Config Rules
Your
Team
Retail
Search

11.
Data Signals
Aggregated search trend data across
geographic focus areas
Retail Search
Power your ecommerce site or
application with Google-quality
search capabilities
Vertex AI Matching Engine
Use innovative semantic matching
features to improve search results
Leverage Google-Unique Data & APIs
To Drive Personalized Discovery

12.
Google Cloud Vertex AI: Matching Engine
Scalable & cost-effective Vector similarity matching service
Low Latency
Find nearest neighbors in milliseconds
Scalable
Scales to billions of vectors
Cost Effective
Requires fewer VMs to serve the same workload
● 1/4th the CPU consumption of faiss
● 1/3rd the memory consumption of nmslib Google’s technology (labelled ScaNN) compared
with popular ANN services
Scalable Nearest Neighbors
https://ai.googleblog.com/2020/07/announcing-scann-efficient-vector.html

13.
1. Training Data
Labeled Pairs

14.
Query:
“Shakespeare
Tragedy”

15.
Putting it all together
Signal
Gather trending search
data (Trends) via Google
Cloud BigQuery
Enrich
Leverage AI / NLP
services to enrich the
term (entity processing)
Search
Search your product
catalog with enriched
terms
Refine
Refine results by scoring
against additional
semantic relevance

16.
Putting it all together
Signal
Gather trending search
data (Trends) via Google
Cloud BigQuery
Enrich
Leverage AI / NLP
services to enrich the
term (entity processing)
Search
Search your product
catalog with enriched
terms
Refine
Refine results by scoring
against additional
semantic relevance
BigQuery Vertex AI / Retail Search Vertex AI
Natural Language

17.
Google Data Signals (BQ)
Search Term 1 | Location | DateTime
Search Term 2 | Location | DateTime
Search Term ...
Extract entities
Salience
Wikipedia / KG
BigQuery
Filter &
Enrichment
Refine
End User or
App Experience
Product Catalog
1B 3
Catalog-ID 1 | Title | Description
Catalog-ID 2 | Title | Description
. ..
Terms
Enriched
Terms
Enriched Search (Example Only):
select Catalog-ID from Catalog where
Metadata like ‘<Search Term>’ and
Genre = “<Genre>”
Semantically Relevant
Video Watch List
Search
Retail AI
Search
2
Natural
Language API
Vertex AI
Vector Similarity
1A
Putting it all together

18.
Example

19.
Example: Man United
Wikipedia: Manchester United Football Club, commonly referred to as Man United, or
simply United, is a professional football club based in the Old Trafford area of
Manchester, England. The club competes in the Premier League, the top division in
the English football league system.
https://trends.google.com/trends/explore?geo=US-CA-803&q=man%20united
https://en.wikipedia.org/wiki/Manchester_United_F.C.

20.
Trending in Los Angeles
“Manchester United”
Retail
Ticketing & Live Events
Video / OTT Streaming
You might like …

21.
Video / OTT: Trends Influencing Top Shelf
1. Respond to community search trends / interest in “Man United”
○ Sports team touring and visiting Los Angeles
2. Generate search candidates against sample Catalog
(movielens.org)
○ Return Movies & Products involving Manchester United FC
3. Create a tailored “Recommended / Hot” Shelf on App Landing page
○ Filter to include Products that are thematically related (sports,
football / soccer, documentary, action etc)

22.
Enrich Search
Signal Refine

23.
Enrich Search
Signal Refine

24.
SELECT
*
FROM
`bigquery-public-data.google_trends.top_terms`
WHERE
refresh_date = DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) and
dma_name = 'Los Angeles CA'
ORDER BY
week DESC,
rank ASC
https://cloud.google.com/bigquery/docs/sandbox
https://console.cloud.google.com/marketplace/product/bigquery-public-datasets/google-search-trends

25.
Enrich Search
Signal Refine

26.
Search
Signal Refine
Enrich

27.
Optional: Enrich the Catalog Search
$ gcloud ml language analyze-entities --content="Man United vs. Liverpool"
{
"entities": [
{
…
"metadata": {
"mid": "/m/050fh",
"wikipedia_url": "https://en.wikipedia.org/wiki/Manchester_United_F.C."
},
"name": "Man United",
"salience": 0.8100557,
"type": "ORGANIZATION"
}...
"metadata": {
"mid": "/m/04ltf",
"wikipedia_url": "https://en.wikipedia.org/wiki/Liverpool_F.C."
}..
https://cloud.google.com/sdk/gcloud/reference/ml

28.
Search
Signal Refine
Enrich

29.
Enrich
Signal Refine
Search

30.
{
"id": "103661",
"title": "United",
"description": "United is based on the true story of Manchester United's
legendary Busby Babes, the youngest side ever to win the
Football League and the 1958 Munich Air Crash that claimed
eight of the their number. …."
"tags": [... "football (soccer)", "manchester united",
"historical event", "sports documentary"]
. . ..
}
Example catalog (extract)
https://movielens.org/
https://www.themoviedb.org/

31.
https://console.cloud.google.com/ai/retail
my-bucket/movies/catalog/data.json

32.
https://console.cloud.google.com/ai/retail

33.
“Man United”

34.
https://console.cloud.google.com/ai/retail

35.
”Man United” related video
content & movies where core
metadata matches (token
match) but there is content
here that is potentially
thematically “distant”
https://console.cloud.google.com/ai/retail

36.
Enrich
Signal Refine
Search

37.
Enrich Search
Signal Refine

38.
"United is based on the true story of Manchester
United's legendary Busby Babes”
"Iron Man and Captain America battle to keep the
Red Skull and his triggerman, Taskmaster, from
unleashing an army”
"The Invincible Iron Man and the Incredible Hulk
must join forces to save the Earth from its
greatest threat yet”
Encoder
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
[-2.70270444e-02, -3.13250013e-02, 6.68615196e-03 …]
[8.240270444e-03, -5.11250213e-02, 7.68615196e-03 …]
[-7.60240444e-01, 4.28360013e-03, 7.68615196e-03 …]

39.
"United is based on the true story of Manchester
United's legendary Busby Babes”
"Iron Man and Captain America battle to keep the
Red Skull and his triggerman, Taskmaster, from
unleashing an army”
"The Invincible Iron Man and the Incredible Hulk
must join forces to save the Earth from its
greatest threat yet”
Encoder
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
Input: Catalog Text Fragments (60K)
● Movie Title
● Movie Description
● Genres
● Tags
1

40.
"United is based on the true story of Manchester
United's legendary Busby Babes”
"Iron Man and Captain America battle to keep the
Red Skull and his triggerman, Taskmaster, from
unleashing an army”
"The Invincible Iron Man and the Incredible Hulk
must join forces to save the Earth from its
greatest threat yet”
Encoder
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
Input: Catalog Text Fragments (60K)
● Movie Title
● Movie Description
● Genres
● Tags
[-2.70270444e-02, -3.13250013e-02, 6.68615196e-03 …]
[8.240270444e-03, -5.11250213e-02, 7.68615196e-03 …]
[-7.60240444e-01, 4.28360013e-03, 7.68615196e-03 …]
Output: Embeddings (Vectors)
● Example: Leverage sentence-t5-base
● 60K Vectors @ 768 Dimensions
NLP Model (Transformer)
Prebuilt (HuggingFace, TFHub)
Custom (Two Tower, Swivel)
1 2

41.
"United is based on the true story of Manchester
United's legendary Busby Babes”
"Iron Man and Captain America battle to keep the
Red Skull and his triggerman, Taskmaster, from
unleashing an army”
"The Invincible Iron Man and the Incredible Hulk
must join forces to save the Earth from its
greatest threat yet”
Encoder
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
Input: Catalog Text Fragments (60K)
● Movie Title
● Movie Description
● Genres
● Tags
[-2.70270444e-02, -3.13250013e-02, 6.68615196e-03 …]
[8.240270444e-03, -5.11250213e-02, 7.68615196e-03 …]
[-7.60240444e-01, 4.28360013e-03, 7.68615196e-03 …]
Output: Embeddings (Vectors)
● Example: Leverage sentence-t5-base
● 60K Vectors @ 768 Dimensions
NLP Model (Transformer)
Prebuilt (HuggingFace, TFHub)
Custom (Two Tower, Swivel)
1 2
3
Vertex AI
Matching
Engine
Index & Deploy!

42.
from google.cloud import aiplatform
aiplatform.init()
my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(index_endpoint_name=<ID Here>)
from sentence_transformers import SentenceTransformer
NLP_MODEL = 'sentence-transformers/sentence-t5-base'
model = SentenceTransformer(NLP_MODEL)
search_term = 'Man United' # OPTIONAL: add text data from Enrichment Phase
query_sentences = [search_term]
embedding = model.encode(query_sentences)
DEPLOYED_INDEX_ID = "my_movielens_ndx_001"
response = my_index_endpoint.match(deployed_index_id=DEPLOYED_INDEX_ID,
queries=embedding,
num_neighbors=10)
print(response)
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
Python >= 3.6

43.
from google.cloud import aiplatform
aiplatform.init()
my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(index_endpoint_name=<ID Here>)
from sentence_transformers import SentenceTransformer
NLP_MODEL = 'sentence-transformers/sentence-t5-base'
model = SentenceTransformer(NLP_MODEL)
search_term = 'Man United' # OPTIONAL: add text data from Enrichment Phase
query_sentences = [search_term]
embedding = model.encode(query_sentences)
DEPLOYED_INDEX_ID = "my_movielens_ndx_001"
response = my_index_endpoint.match(deployed_index_id=DEPLOYED_INDEX_ID,
queries=embedding,
num_neighbors=10)
print(response)
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
Python >= 3.6

44.
from google.cloud import aiplatform
aiplatform.init()
my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(index_endpoint_name=<ID Here>)
from sentence_transformers import SentenceTransformer
NLP_MODEL = 'sentence-transformers/sentence-t5-base'
model = SentenceTransformer(NLP_MODEL)
search_term = 'Man United' # OPTIONAL: add text data from Enrichment Phase
query_sentences = [search_term]
embedding = model.encode(query_sentences)
DEPLOYED_INDEX_ID = "my_movielens_ndx_001"
response = my_index_endpoint.match(deployed_index_id=DEPLOYED_INDEX_ID,
queries=embedding,
num_neighbors=10)
print(response)
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
Python >= 3.6

45.
from google.cloud import aiplatform
aiplatform.init()
my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(index_endpoint_name=<ID Here>)
from sentence_transformers import SentenceTransformer
NLP_MODEL = 'sentence-transformers/sentence-t5-base'
model = SentenceTransformer(NLP_MODEL)
search_term = 'Man United' # OPTIONAL: add text data from Enrichment Phase
query_sentences = [search_term]
embedding = model.encode(query_sentences)
DEPLOYED_INDEX_ID = "my_movielens_ndx_001"
response = my_index_endpoint.match(deployed_index_id=DEPLOYED_INDEX_ID,
queries=embedding,
num_neighbors=10)
print(response)
https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine
Python >= 3.6

46.
[[MatchNeighbor(id='movielens-68194', distance=0.16755491495132446),
MatchNeighbor(id='movielens-185867', distance=0.16838324069976807),
MatchNeighbor(id='movielens-195337', distance=0.1685701608657837),
MatchNeighbor(id='movielens-145921', distance=0.17106527090072632),
MatchNeighbor(id='movielens-81676', distance=0.17208701372146606),
MatchNeighbor(id='movielens-103661', distance=0.17541170120239258),
MatchNeighbor(id='movielens-183153', distance=0.18686127662658691),
MatchNeighbor(id='movielens-147436', distance=0.1904624104499817),
MatchNeighbor(id='movielens-176405', distance=0.1911325454711914),
MatchNeighbor(id='movielens-6618', distance=0.1932288408279419)]]
Cosine Distance = 1 — Cosine Similarity
Man United
Iron Men

47.
Search: Gather Candidates

48.
Refine: Semantic Match & Add
Add Candidate Title
● Semantic Match = YES
Delete Candidate Title
● Semantic Match = NO

49.
Discussion

50.
Commentary & Summary
1. Overall Strategy
a. Trending Data Signals (Search) to “seed” Content Recommendations
b. Search Engine (Google Cloud Retail Search AI) to gather candidates
c. Vector Similarity (Google Cloud Vertex AI) to rank sort candidates
2. Part of larger solution!
a. Ex: Use collaborative filtering (or other) to gauge user interest / engagement
b. Good for cold-start or content-heavy approach
3. More details available
a. https://console.cloud.google.com/marketplace/product/bigquery-public-datasets/google-search-trends
b. https://cloud.google.com/solutions/retail-product-discovery
c. https://cloud.google.com/vertex-ai/docs/matching-engine/overview

51.
Thank You
https://cloud.google.com/contact