The document discusses using ElasticSearch to enable fast and scalable search of reviews. It describes how ElasticSearch allows for tokenization, stemming, stop words removal and faceting to improve search performance compared to a basic SQL search. An example query and response show how ElasticSearch returns search results and highlights matching text. The document also briefly outlines how data could be indexed in ElasticSearch through a queueing system and how shards and replicas can provide replication and scalability. It closes by noting some potential performance issues to be aware of with ElasticSearch.

1. Using ElasticSearch to scale near real-time search
John Billings
2013-11-20

2. Problem: Finding reviews

3. Problem: Finding reviews

4. Where are reviews stored?

5. Searching using SQL
select * from reviews where
content like ‘%chicken tikka masala%’
and id = 123456

6. And we’re done

7. Not so fast...

8. What did we forget?
● Analysis
○ Tokenization
○ Stemming (‘curries’ vs ‘curry’)
○ Stop words (‘the’, ‘and’, ‘a’, …)
● Performance
● Highlighting / snippetting
● Faceting

9. Version 2

10. Query
curl -w'n' -XGET 'host:14900/reviews/_search' -d '{
"fields" : [],
"highlight": {"fields": {"review_comment" : {}}},
"query": {
"bool": {
"must": [
{"match": {"review_comment": "chicken tikka masala"}},
{"match": {"business_id": "123456"}}
]
}
}
}'

11. Response
{
"took" : 10,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 341,
"max_score" : 2.7088916,
"hits" : [
// Hit objects
]
}
}

12. Response
{
"_index" : "reviews",
"_type" : "review",
"_id" : "123456",
"_score" : 2.0553625,
"highlight" : {
"review_comment" : [ "
I've found. We usually order saag paneer,
<em>chicken</em> <em>tikka</em> <em>masala</em>, bengan bharta would recommend them all" ]
}
}

13. Aside: Clients

14. Indexing
Updates
JSON docs
Gearman
Indexer
modules

15. Indexing
class IndexerEvent:
table: String
id: int
action: [‘insert’, ‘update’, ‘delete’]
timestamp: float
class IndexRequest:
index_name: String
doc_type: String
doc_id: int
field_values: Map<String, String>
class Indexer:
tables_to_watch: List<String>
handle_event: IndexerEvent -> List<IndexRequest>

16. Replication and sharding
curl -w'n' -XPUT 'host:14900/reviews/' -d '{
“settings”: {
“index”: {
“number_of_shards”: 3,
“number_of_replicas”: 2,
}
}
}’

17. Using ElasticSearch to discover local businesses

18. And we’re done

19. Not so fast…

20. Performance can be unpredictable

21. Problem: Disks are slow

22. Problem: Memory usage is unpredictable

23. Problem: Tenants can be noisy

24. Any questions?