This webinar is based on the Dzone Refcard (http://refcardz.dzone.com/refcardz/search-patterns) and provides patterns for integrating cloud-based search with a variety of applications. Examples of these patterns are demonstrated using Amazon CloudSearch to abstract away the complexities of deploying and administering your own search servers, but the principles apply to other search systems as well.

1. Search Patterns
Jon Handler, Amazon CloudSearch Solution Architect
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

2. Agenda
!
!
!
!
!
!
!
Amazon CloudSearch Basics
Searching in the Cloud
Ranking
Location-Based Search
Faceting
Mixed Data Sources
Performance
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

3. Patterns
!
!
!
!
Title-Body Search
Social Search Patterns
Mobile Search Patterns
eCommerce Patterns
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

4. © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

5. AMAZON CLOUDSEARCH
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

6. Search, In The Cloud
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

7. © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

8. The Cloud is Elastic
DATA
Document Quantity and Size
SEARCH INSTANCE
SEARCH INSTANCE
SEARCH INSTANCE
SEARCH INSTANCE
SEARCH INSTANCE
SEARCH INSTANCE
SEARCH INSTANCE
SEARCH INSTANCE
SEARCH INSTANCE
Index Partition 1
Copy 1
Index Partition 2
Copy 1
Index Partition n
Copy 1
TRAFFIC
Search
Request
Volume and
Complexity
Index Partition 1
Copy 2
Index Partition 1
Copy n
Index Partition 2
Copy 2
Index Partition 2
Copy n
Index Partition n
Copy 2
Index Partition n
Copy n
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

9. SEARCHING IN THE CLOUD
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

10. CloudSearch Batches
{ "type": "add",!
"id": "tt0076759",!
"fields": { !
"title": "Star Wars",!
! "director": "Lucas, George",!
! "year": 1977,!
! "genre": ["Action","Adventure","Fantasy","Sci-Fi"],!
! "actor": ["Ford, Harrison","Fisher, Carrie","Hamill,!
!
Mark","Jones, James Earl","Guinness, !
! ! ! !
Alec",...] } },!
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

11. Bootstrapping Data
Amazon
CloudSearch
Amazon EC2
Amazon SQS
Source
System
Processing
Script
Amazon EC2
Queuing Batching
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

12. Configuring for Search
! Text fields for individual word search
• User-generated and external text – titles, descriptions
! Literal fields for exact matches
• Application-generated text like facets
! Integer fields for range searching and ranking
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

13. Sending Queries
http(s)://<endpoint>/2011-02-01/search?
! Simple searches
• q=<text>
! Filtering
• bq= (and title:'iron man' genre:'Action')
! Filtering with integer ranges
• bq=(and 'iron man' year:..2010)
! Geo filtering
• bq=(and 'iron man' latitude:12700..12900 longitude:5700..5800)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

14. Search Results
{ "rank": "-text_relevance",!
"match-expr": "(label 'star wars')",!
"hits": { "found": 7, "start": 0,!
"hit": [{"id": "tt1185834"},!
{"id": "tt0076759"},!
{"id": "tt0086190"},!
{"id": "tt0120915"},!
{"id": "tt0121765"},!
{"id": "tt0080684"},!
{"id": "tt0121766"} ]!
} ...!
}!
!
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

15. Updating CloudSearch
Update Processor
Web Server
Users
Amazon EC2
Amazon SQS
Amazon EC2
DynamoDB
Amazon RDS
Amazon
CloudSearch
Amazon S3
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

16. BASIC RANKING
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

17. © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

18. Customizing Ranking
! text_relevance and cs.text_relevance
! Rank expressions
• Compute a score for each document
• &rank=<function>
! Defined in the console
! Defined at query-time
• &q='iron-man'&rank-recency=text_relevance + year
&rank=recency
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

19. Document Structure
Movie
title
description
user_rating
likes
release_date
latitude
longitude
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

20. Field Weighting
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

21. Field Weighting
! Adjust relative importance of fields
! &rank-title_boost=
cs.text_relevance({"weights":{"title":4.0},
"default_weight":1})
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

22. Popularity
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

23. Popularity
! Convert floating point to integer
! Weight by the number of ranks
! rank-pop=
(user-rating - 2) * log10(number-user-ranks) * 10 +
metascore * 3
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

24. Freshness
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

25. Freshness
! Exponential decay function
r = ce
− λt
! &rank-decay=
200*Math.exp(-0.1*days_ago)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

26. Rank Expressions: Combined
! &rank-combined=1.0 * title + 0.5 * popularity + 0.3 *
freshness
! &rank=combined
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

27. © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

28. LOCATION-BASED SEARCH
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

29. Mobile Experience
Cancel
Iron Man!
Iron Man
Done
Iron Man 3 (2013)!
When Tony Stark's world is torn apart by a
formidable terrorist called the Mandarin, he
starts an odyssey of rebuilding and retribution. !
Iron Man 2 (2010)!
Tony Stark has declared himself Iron Man and
installed world peace... or so he thinks. He soon
realizes that not only is there a mad man...!
Iron Man (2008)!
!
When wealthy industrialist Tony Stark is forced
to build an armored suit after a life-threatening
incident, he ultimately decides to use its
technology to fight against evil. !
The Man With The Iron Fists (2012) !
On the hunt for a fabled treasure of gold, a band
of warriors, assassins, and a rogue British soldier
descend upon a village in feudal China, where a
humble blacksmith...!
Movies
Search
Social
Nearby
Account
Movies
Search
Social
Nearby
Account
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

30. Encoding Location
Movie
title
! Latitude and longitude expressed as
integers
description
user_rating
likes
release_date
latitude
longitude
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

31. Bounding Box Search
! Latitude min/max
! Longitude min/max
bq=(and 'theater'
latitude:12700..12900
longitude:5700..5800)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

32. Location Sort
! Cartesian distance function
(lat − latuser )2 + (lon − lonuser )2
! &rank-geo=sqrt(pow(latitude - lat, 2) +
pow(longitude - lon, 2)
! &rank=-geo
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

33. FACETING
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

34. Facets
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

35. Facets
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

36. Simple Faceting: Document
Movie
title
description
genre
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

37. Simple Faceting: Configuration
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

38. Simple Faceting: Query
q=iron+man&facet=genre
{"rank":
"-­‐text_relevance",
"match-­‐expr":
"(label
'star
wars')",
"hits":
{"found":
7,
"start":
0,
"hit":
[]
},
"facets":
{
"genre":
{
"constraints":
[
{"value":
"Family",
"count":
62},
{"value":
"Action/Adventure",
"count":
21},
{"value":
"Drama",
"count":
5
},
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

39. Simple Faceting: UI
<div
class='facet'>
<ul
class='facet_list'>
<?php
$genres
=
$resultsObj-­‐>facets-­‐>genre-­‐>constraints;
for
($i
=
0;
$i
<
count($genres);
$i++)
{
$curGenre
=
$genres[$i];
$curCount
=
$thisGenre-­‐>count;
?>
<li
class='facet_item'>
<div
class='facet_name'><?=$curGenre?></div>
<div
class='facet_count'><?=$curCount?></div>
</li>
<?php
}
?>
</ul>
</div>
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

40. Facets
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

41. Document
Movie
title
description
oscar1
oscar2
oscar3
!
!
!
!
!
title: Lincoln
description: ...
oscar1: Awards
oscar2: Awards/Best Actor
oscar3: Awards/Best Actor/Daniel Day
Lewis
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

42. Query
&q=lincoln&facet=oscar1,oscar2,oscar3
{"rank":
"-­‐text_relevance",
"hits":{...},
"facets":
{
"oscar1":
{
"constraints":
[
{"value":
"Awards",
"count":
23},
{"value":
"Nominations",
"count":
124}]},
"oscar2":
{
"constraints":
[
{"value":
"Awards/Best
Actor",
"count":
6},
{"value":
"Awards/Best
Actress",
"count":
3}...]},
"oscar3":
{
"constraints":
[
{"value":
"Awards/Best
Actor/Daniel
Day
Lewis",
"count":
1},
{"value":
"Awards/Best
Actor/Denzel
Washington",
"count":
2}...]},
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

43. Drilldown
!
!
!
!
bq=oscar1:'Awards'
bq=oscar2:'Awards/Best Actor'
bq=oscar3:'Awards/Best Actor/Daniel Day Lewis'
bq=(and 'star' oscar2:'Awards/Best Actor')
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

44. MIXED DATA SOURCES
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

45. Document
Movie
Showtime
Review
type
type
type
title
title
title
description
theater_name
movie_name
user_rating
city
author
likes
latitude
url
release_date
longitude
body
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

46. Heterogeneous Data
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

47. Multi Domain
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

48. Trade-offs
! Multiple domain
• Independent configuration
• Independent scale
! Single domain
• Simpler
• Lower cost
• bq=(and 'iron man' type:'movie')
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

49. TUNING
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

50. What to Track
!
!
!
!
User queries
Responses
Response times
Click positions
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

51. Tuning Relevance
! Return relevance values
! Check no-result queries
! Check most common results
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

52. Tuning Performance
!
!
!
!
Identify consistent slow queries
Tend towards text matching
Cache slow queries when possible
Benchmark with JMeter or Siege
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

53. Q&A
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

54. Resources
! Amazon CloudSearch Overview Page
http://aws.amazon.com/cloudsearch/
• Developer Guide
• FAQs, Articles
• Community Forum
• Tutorial
! Free 30-day trial
! Contact: handler@amazon.com
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.