This document discusses the evolution of Elasticsearch from versions 5.0 to 8.0. It covers topics like strict bootstrap checks, rolling upgrades, floodstage watermarks, sequence numbers, mapping types removal, automatic queue resizing, adaptive replica selection, shard shrinking and splitting. The document provides demos of some of these features and recommends benchmarks and meetups for further learning.

1. (R)Evolution
Philipp Krenn @xeraa

2. Revolution

3. https://db-engines.com/en/ranking

5. https://www.reddit.com/r/SQL/comments/7i828i/ms_sql_query_is_too_complex/

6. Who uses Elasticsearch?

7. Infrastructure | Developer Advocate

8. http://thedudeabides.com/articles/
the_birth_of_compass

10. Terms
Cluster, Node, Index, Shard, Document, ID

11. $ curl http://localhost:9200
{
"name": "elasticsearch1",
"cluster_name": "docker-cluster",
"cluster_uuid": "l6wfwv8XSniiI_Fx6qqrcw",
"version": {
"number": "5.6.9",
"build_hash": "877a590",
"build_date": "2018-04-12T16:25:14.838Z",
"build_snapshot": false,
"lucene_version": "6.6.1"
},
"tagline": "You Know, for Search"
}

12. $ curl http://localhost:9200
{
"name": "elasticsearch1",
"cluster_name": "docker-cluster",
"cluster_uuid": "l6wfwv8XSniiI_Fx6qqrcw",
"version": {
"number": "5.6.9",
"build_hash": "877a590",
"build_date": "2018-04-12T16:25:14.838Z",
"build_snapshot": false,
"lucene_version": "6.6.1"
},
"tagline": "You Know, for Search"
}

21. Evolution

22. Questions: https://sli.do/xeraa
Answers: https://twitter.com/xeraa

24. Strictness*
5.0
*
Demo

26. Bootstrap Checks
https://www.elastic.co/guide/en/elasticsearch/
reference/current/bootstrap-checks.html

27. Parameters & Configs

28. Rolling Upgrades*
6.0
*
Demo

30. Floodstage Watermark*
6.0
*
Demo

32. Low 85%
High 90%
Floodstage 95%

33. Sequence Numbers*
6.0
*
Demo

36. 63 bits ought to be
enough for anyone.

37. Tradeoff
index.translog.retention.size: 512MB
index.translog.retention.age: 12h

38. Cross Datacenter Replication
6.x or 7.x

39. Types*
5.6 to 8.0
*
Demo

41. Why
Data types
Sparsity
Scoring

42. How
5.6 opt-in single type
6.x single type
7.x type optional in API
8.x no more types
https://www.elastic.co/guide/en/elasticsearch/reference/current/
removal-of-types.html#_schedule_for_removal_of_mapping_types

43. Automatic Queue Resizing
6.0

44. Reject and Retry Instead of Long Queues
thread_pool.search.target_response_rate: 2s
Serving 50 requests/s
Queue size:

45. Adaptive Replica Selection
6.1 (enabled by default in 7.0)

47. Pick Best Shard
Exponentially Weighted Moving Average
(EWMA)
Piggyback on responses to coordinating node

49. Shrink & Split*
5.0 6.1
*
Demo

51. Shrink
Combine shards by a factor

53. Split
Split into a factor of number_of_routing_shards
Not required in 7.0+

54. Shards
7.0

56. Default: 1 shard per index
Oversharding

57. Benchmarks
∞

60. Rally
https://elasticsearch-benchmarks.elastic.co

61. Conclusion

62. Strictness
Rolling Upgrades
Floodstage Watermark
Sequence Numbers
Types

63. Automatic Queue Resizing
Adaptive Replica Selection
Shrink & Split
Default Shard Number

64. https://www.meetup.com/Greece-Elastic/
Monday 19:00, The Cube Athens

65. Questions?
Philipp Krenn @xeraa