This document discusses sorting and relevance in ElasticSearch. It provides examples of sorting search results by date or score. It also covers multilevel sorting, sorting on multivalue fields, and sorting on string fields after analyzing or not analyzing text. The document explains what determines relevance in ElasticSearch, including term frequency, inverse document frequency, and field length norm. It shows how to get explain plans and failure messages for queries. Finally, it provides a brief introduction to doc values in ElasticSearch and references a book for further information.

1. ElasticSearch
Sorting and Relevance
http://elastic.openthinklabs.com/

2. Sorting
GET /_search
{
"query" : {
"bool" : {
"filter" : {
"term" : {
"user_id" : 1
}
}
}
}
}
GET /_search
{
"query" : {
"constant_score" : {
"filter" : {
"term" : {
"user_id" : 1
}
}
}
}
}

3. Sorting
Sorting by Field Values
GET /_search
{
"query" : {
"bool" : {
"filter" : { "term" : { "user_id" : 1 }}
}
},
"sort": { "date": { "order": "desc" }}
}
"hits" : {
"total" : 6,
"max_score" : null,
"hits" : [ {
"_index" : "us",
"_type" : "tweet",
"_id" : "14",
"_score" : null,
"_source" : {
"date": "2014-09-24",
...
},
"sort" : [ 1411516800000 ]
},
...
}

4. Sorting
Multilevel Sorting
GET /_search
{
"query" : {
"bool" : {
"must": { "match": { "tweet": "manage text
search" }},
"filter" : { "term" : { "user_id" : 2 }}
}
},
"sort": [
{ "date": { "order": "desc" }},
{ "_score": { "order": "desc" }}
]
}

5. Sorting
Multilevel Sorting
GET /_search
{
"query" : {
"bool" : {
"must": { "match": { "tweet": "manage text
search" }},
"filter" : { "term" : { "user_id" : 2 }}
}
},
"sort": [
{ "date": { "order": "desc" }},
{ "_score": { "order": "desc" }}
]
}

6. Sorting
Sorting on Multivalue Fields
"sort": {
"dates": {
"order": "asc",
"mode": "min"
}
}

7. String Sorting and Multifields
"tweet": {
"type":
"string",
"analyzer":
"english"
}
"tweet": {
"type": "string",
"analyzer":
"english",
"fields": {
"raw": {
"type":
"string",
"index":
"not_analyzed"
}
}
}
GET /_search
{
"query": {
"match": {
"tweet":
"elasticsearch"
}
},
"sort": "tweet.raw"
}

8. What Is Relevance?
● The standard similarity algorithm used in Elasticsearch :
● Term frequency : How often does the term appear in the
field? The more often, the more relevant. A field containing
five mentions of the same term is more likely to be relevant
than a field containing just one mention.
● Inverse document frequency : How often does each term
appear in the index? The more often, the less relevant.
Terms that appear in many documents have a lower weight
than more-uncommon terms.
● Field-length norm : How long is the field? The longer it is,
the less likely it is that words in the field will be relevant. A
term appearing in a short title field carries more weight than
the same term appearing in a long content field

9. What Is Relevance?
Understanding the Score
GET /_search?explain
{
"query" : { "match" : { "tweet" :
"honeymoon" }}
}

10. What Is Relevance?
Understanding Why a Document Matched
GET /us/tweet/12/_explain
{
"query" : {
"bool" : {
"filter" : { "term" : { "user_id" : 2 }},
"must" : { "match" : { "tweet" : "honeymoon" }}
}
}
}
"failure to match filter: cache(user_id:[2 TO 2])"

11. Doc Values Intro
● Doc values are used in several places in
Elasticsearch:
● Sorting on a field
● Aggregations on a field
● Certain filters (for example, geolocation filters)
● Scripts that refer to fields

12. Referensi
● ElasticSearch, The Definitive Guide, A Distrib
uted Real-Time Search and Analytics Engine, Cl
inton Gormely & Zachary Tong, O’Reilly