This document provides an overview of Elasticsearch including: - Elasticsearch is a database server that is document-based and schema-free, built on Apache Lucene. - It allows for full-text search across fields in documents and supports CRUD operations via a RESTful API. - Common operations demonstrated include indexing, updating, retrieving, and deleting documents. - Advanced search capabilities include filtering, phrase matching, and highlighting search terms. - Analytics features like aggregations allow grouping and metrics like average age to be computed across document fields and buckets.

1. Elasticsearch
Tony Messias
MaceioDevMeetup #5

2. who am I?
➔ Tony Messias ~ @tony0x01
➔ Building web stuff since ~2010

4. search

5. what is “search”?
“A search is the organized pursuit of
information (...) that you want to find, but
you have no idea where it is.”
(NASA).

6. how do we add search to our apps?
➔ SQL queries (not just LIKE queries);
➔ Elasticsearch;
➔ Lucene;
➔ Solr;
➔ …

7. how do we add search to our apps?
➔ SQL queries (not just LIKE queries);
➔ Elasticsearch;
➔ Lucene;
➔ Solr;
➔ …

8. who uses it?

10. what is Elasticsearch?
➔ database server;
➔ document based;
➔ based on Apache Lucene;
➔ schema-free*;

11. why Elasticsearch?
➔ speaks HTTP/JSON;
➔ easy to setup;
➔ data replication;
➔ easily scalable;

12. what can I use it for?

14. but it can do more
than that…

15. some analogy *
Relational ES
database index
table type
row document
column field
schema mapping
index everything
SQL QueryDSL

16. REST(ish)
$ curl -XPOST http://es-server.com/index/type/id
-d '{}'

17. REST(ish)
$ curl -XPOST http://es-server.com/index/type/id
-d '{}'

18. REST(ish)
$ curl -XPOST http://es-server.com/index/type/id
-d '{}'

19. REST(ish)
$ curl -XPOST http://es-server.com/index/type/id
-d '{}'

20. REST(ish)
$ curl -XPOST http://es-server.com/index/type/id
-d '{}'

21. enough with the
talking… let’s do
some searches!

22. but first, let’s talk
about CRUD

24. indexing

25. {
"email": "john@smith.com",
"first_name": "John",
"last_name": "Smith",
"info": {
"bio": "Eco-warrior and defender of the weak",
"age": 25,
"interests": [ "dolphins", "whales" ]
},
"join_date": "2014/05/01"
}

26. $ curl -XPOST localhost:9200/my-app/users -d ‘{
"email": "john@smith.com",
"first_name": "John",
"last_name": "Smith",
"info": {
"bio": "Eco-warrior and defender of the weak",
"age": 25,
"interests": [ "dolphins", "whales" ]
},
"join_date": "2014/05/01"
}’

27. {
"_index": "my-app",
"_type": "users",
"_id": "AUxOBeJbue-jeR3jNcHj",
"_version": 1,
"created": true
}

28. $ curl -XPUT localhost:9200/my-app/users/AUxOBeJbue-jeR3jNcHj -d
‘{
"email": "john@smith.com",
"first_name": "John"
}’

29. {
"_index": "my-app",
"_type": "users",
"_id": "AUxOBeJbue-jeR3jNcHj",
"_version": 2,
"created": false
}

30. $ curl -XGET localhost:9200/my-app/users/AUxOBeJbue-jeR3jNcHj
{
"_index": "my-app",
"_type": "users",
"_id": "AUxOJe-4ue-jeR3jNcHo",
"_version": 2,
"found": true,
"_source": {
"email": "john@smith.com",
"first_name": "John"
}
}

31. $ curl -XDELETE localhost:9200/my-app/users/AUxOBeJbue-jeR3jNcHj
{
"found": true,
"_index": "my-app",
"_type": "users",
"_id": "AUxOBeJbue-jeR3jNcHj",
"_version": 3
}

32. searching

33. $ curl -XPOST localhost:9200/my-app/users/_search -d ‘{
"query": {
"match": {
"first_name": "John"
}
}
}’

34. {
...,
"hits": {
"total": 4,
"max_score": 0.30685282,
"hits": [
{
"_index": "my-app",
"_type": "users",
"_id": "AUxOJWlCue-jeR3jNcHn",
"_score": 0.30685282,
"_source": {
"email": "john@smith.com",
"first_name": "John",
"last_name": "Doe",
"info": {
"bio": "Like rock music",
"age": 35,
"interests": [
"dolphins",
"whales"
]
},
"join_date": "2014/05/01"
}
},...

35. $ curl -XPOST localhost:9200/my-app/users/_search -d ‘{
"query": {
"filtered": {
"query": {
"match": {
"first_name": "John"
}
},
"filter": {
"range": {
"info.age": {
"gt": 34
}
}
}
}
}
}’

36. {
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.30685282,
"hits": [
{
"_index": "my-app",
"_type": "users",
"_id": "AUxOJWlCue-jeR3jNcHn",
"_score": 0.30685282,
"_source": {
"email": "john@smith.com",
"first_name": "John",
"last_name": "Doe",
"info": {
"bio": "Like rock music",
"age": 35,
"interests": ["dolphins","whales"]
},
"join_date": "2014/05/01"
}
}
]
}
}

37. queries vs. filters

38. queries vs. filters

39. full-text search

40. $ curl -XPOST localhost:9200/my-app/users/_search -d ‘{
"query": {
"match": {
"info.bio": "rock climbing"
}
}
}’

41. {
"took": 9,
"timed_out": false,
"_shards": {...},
"hits": {
"total": 2,
"max_score": 0.2169777,
"hits": [
{
"_index": "my-app",
"_type": "users",
"_id": "AUxOJe-4ue-jeR3jNcHo",
"_score": 0.2169777,
"_source": {
"email": "john@smith.com",
"first_name": "John",
"last_name": "Kennedy",
"info": {
"bio": "I love rock climbing!",
"age": 29,
"interests": ["dolphins","whales"]
},
"join_date": "2014/05/01"
}
},
{
"_index": "my-app",
"_type": "users",
"_id": "AUxOJWlCue-jeR3jNcHn",
"_score": 0.02250402,
"_source": {
"email": "john@smith.com",
"first_name": "John",
"last_name": "Doe",
"info": {
"bio": "Like rock music",
"age": 35,
"interests": ["dolphins","whales"]
},
"join_date": "2014/05/01"
}
}
]
}
}

42. $ curl -XPOST localhost:9200/my-app/users/_search -d ‘{
"query": {
"match_phrase": {
"info.bio": "rock climbing"
}
}
}’

43. {…,
"hits": {
"total": 1,
"max_score": 0.30685282,
"hits": [
{
"_index": "my-app",
"_type": "users",
"_id": "AUxOJe-4ue-jeR3jNcHo",
"_score": 0.30685282,
"_source": {
"email": "john@smith.com",
"first_name": "John",
"last_name": "Kennedy",
"info": {
"bio": "I love rock climbing!",
"age": 29,
"interests": ["dolphins","whales"]
},
"join_date": "2014/05/01"
}
}
]
}
}

44. $ curl -XPOST localhost:9200/my-app/users/_search -d ‘{
"query": {
"match_phrase": {
"info.bio": "rock climbing"
}
},
"highlight": {
"fields": {
"info.bio": {}
}
}
}’

45. ...
{
"_index": "my-app",
"_type": "users",
"_id": "AUxOJe-4ue-jeR3jNcHo",
"_score": 0.30685282,
"_source": {
"email": "john@smith.com",
"first_name": "John",
"last_name": "Kennedy",
"info": {
"bio": "I love rock climbing!",
"age": 29,
"interests": ["dolphins","whales"]
},
"join_date": "2014/05/01"
},
"highlight": {
"info.bio": ["I love <em>rock</em> <em>climbing</em>!"]
}
}
]
}
}

46. is this it?

49. analytics

50. aggregations

51. buckets metrics

52. $ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{
"aggs": {
"all_interests": {
"terms": {
"field": "info.interests"
}
}
}
}'

53. { ...,
"hits": {"total": 4,"max_score": 0,"hits": []},
"aggregations": {
"all_interests": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "whales",
"doc_count": 4
},
{
"key": "dolphins",
"doc_count": 3
}
]
}
}}

54. $ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{
"aggs": {
"all_interests": {
"terms": {
"field": "info.interests"
}
}
}
}'

55. $ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{
"aggs": {
"all_interests": {
"terms": {
"field": "info.interests"
},
"aggs": {
"avg_age": {
"avg": {
"field": "info.age"
}
}
}
}
}
}'

56. {
…,
"aggregations": {
"all_interests": {
…,
"buckets": [
{
"key": "whales",
"doc_count": 4,
"avg_age": { "value": 31.5 }
},
{
"key": "dolphins",
"doc_count": 3,
"avg_age": { "value": 32.333333333333336 }
}
]
}}}

57. $ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{
"query": {
"match_phrase": {
"info.bio": "rock climbing"
}
},
"aggs": {
"all_interests": {
"terms": { "field": "info.interests" },
"aggs": {
"avg_age": {
"avg": { "field": "info.age" }
}
}
}
}}'

58. {
…,
"aggregations": {
"all_interests": {
…,
"buckets": [
{
"key": "whales",
"doc_count": 1,
"avg_age": {
"value": 29
}
}
]
}}}

60. questions?

61. Resources
➔ http://www.elastic.
co/guide/en/elasticsearch/guide/current/index.
html
➔ https://leanpub.com/elasticsearch-quick-start
➔ https://www.elastic.co/use-cases
➔ https://speakerdeck.com/asm89/elasticsearch

62. Resources
➔ https://speakerdeck.
com/johnbeynon/elasticsearch-101
➔ http://blog.madewithlove.be/post/integrating-
elasticsearch-with-your-laravel-app/
➔ http://blog.madewithlove.be/post/elasticsearch-
aggregations/