Scale with confidence. From deploying a small dev cluster for app search to managing a prod deployment of hundreds of nodes, our Elastic experts have seen it all and they’re sharing everything you need to know.

1. !1
Dave Erickson
September 6, 2018
Scale with confidence

2. !2
From your laptop to
lots of nodes, and
multiple use cases.
So let's go...

3. !3
840 Pages of Elasticsearch Documentation

4. !4
30 minute presentation

5. !5
2.142857143 seconds per page

6. !6
Cluster Size
3
nodes
10
nodes
100’s
nodes

7. !7
Mappings
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

8. !8
Base template structure
Elasticsearch Template
curl –XPUT localhost:9200/_template/template_1
{
"index_patterns" : ["windows-*"],
"order" : 0,
"settings" : {},
"mappings" : {}
}

9. !9
Add Dynamic setting
Elasticsearch Template
curl –XPUT localhost:9200/_template/template_1
{
"index_patterns" : ["windows-*"],
"order" : 0,
"settings" : {...},
"mappings" : {
"docs" : {
"dynamic": true, // default value
"properties": {...}
}
}
}

10. !10
Dynamic template settings
Elasticsearch Template
• "dynamic": true
‒ Newly detected fields are added to the mapping. (default)
https://www.elastic.co/guide/en/elasticsearch/reference/master/dynamic.html

11. !11
Dynamic template settings
Elasticsearch Template
• "dynamic": true
‒ Newly detected fields are added to the mapping. (default)
• "dynamic": false
‒ Newly detected fields are ignored.
https://www.elastic.co/guide/en/elasticsearch/reference/master/dynamic.html

12. !12
Dynamic template settings
Elasticsearch Template
• "dynamic": true
‒ Newly detected fields are added to the mapping. (default)
• "dynamic": false
‒ Newly detected fields are ignored.
• "dynamic": strict
‒ If new fields are detected, an exception is thrown and the document is rejected. New fields
must be explicitly added to the mapping.
https://www.elastic.co/guide/en/elasticsearch/reference/master/dynamic.html

13. !13
Cluster Size
3
nodes
10
nodes
100’s
nodes

14. !14
Add Shard & Replica setting
Elasticsearch Template
curl –XPUT localhost:9200/_template/template_1
{
"index_patterns" : ["windows-*"],
"order" : 0,
"settings" : {
"number_of_shards" : 1,
"number_of_replicas" : 1
},
"mappings" : {}
}

15. !15
Shard Size
https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster

16. !16
15 to 50GB’ish per Shards

17. !17
But why 15 to 50GB’ish?

18. !18
Dedicated Nodes
https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-node.html

19. !19
To Dedicate or not to Dedicate?
• Master Nodes
• Ingest Nodes
• Coordinator Nodes (a.k.a. Client Nodes)
• Machine Learning Nodes
• Data Nodes - Hot / Warm / Cold
• Data Nodes - Workload
• Cross-Cluster Search

20. !20
Cluster Size
3
nodes
10
nodes
100’s
nodes

21. !21
Specific Cluster Sizing
https://github.com/elastic/rally
https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right
https://www.elastic.co/blog/announcing-rally-benchmarking-for-elasticsearch

22. !22

23. !23
• Logging:
‒ Rollover API - https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html
‒ Shrink API - https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-shrink-index.html
‒ Split API - https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-split-index.html
• Search:
‒ Query Profiler - https://www.elastic.co/guide/en/elasticsearch/reference/master/_profiling_queries.html
‒ Force Merge - https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-forcemerge.html
And there is much more...

24. !24
"Here, Hold My Beer."
Customer Stories to
Learn from as Told by
Support
https://www.elastic.co/elasticon/conf/2018/sf/here-hold-my-beer-customer-stories-to-learn-from-as-told-by-support

25. !25
Thank you...
Come and find us at the
AMA Booth