1
ElasticON Global
The Elastic clients:
Recent developments
2
This presentation and the accompanying oral presentation contain forward-looking statements, including statements
concerning plans for future offerings; the expected strength, performance or benefits of our offerings; and our future
operations and expected performance. These forward-looking statements are subject to the safe harbor provisions
under the Private Securities Litigation Reform Act of 1995. Our expectations and beliefs in light of currently
available information regarding these matters may not materialize. Actual outcomes and results may differ materially
from those contemplated by these forward-looking statements due to uncertainties, risks, and changes in
circumstances, including, but not limited to those related to: the impact of the COVID-19 pandemic on our business
and our customers and partners; our ability to continue to deliver and improve our offerings and successfully
develop new offerings, including security-related product offerings and SaaS offerings; customer acceptance and
purchase of our existing offerings and new offerings, including the expansion and adoption of our SaaS offerings;
our ability to realize value from investments in the business, including R&D investments; our ability to maintain and
expand our user and customer base; our international expansion strategy; our ability to successfully execute our
go-to-market strategy and expand in our existing markets and into new markets, and our ability to forecast customer
retention and expansion; and general market, political, economic and business conditions.
Additional risks and uncertainties that could cause actual outcomes and results to differ materially are included in
our filings with the Securities and Exchange Commission (the “SEC”), including our Annual Report on Form 10-K for
the most recent fiscal year, our quarterly report on Form 10-Q for the most recent fiscal quarter, and any
subsequent reports filed with the SEC. SEC filings are available on the Investor Relations section of Elastic’s
website at ir.elastic.co and the SEC’s website at www.sec.gov.
Any features or functions of services or products referenced in this presentation, or in any presentations, press
releases or public statements, which are not currently available or not currently available as a general availability
release, may not be delivered on time or at all. The development, release, and timing of any features or functionality
described for our products remains at our sole discretion. Customers who purchase our products and services
should make the purchase decisions based upon services and product features and functions that are currently
available.
All statements are made only as of the date of the presentation, and Elastic assumes no obligation to, and does not
currently intend to, update any forward-looking statements or statements relating to features or functions of services
or products, except as required by law.
Forward-Looking Statements
3
Elasticsearch clients
Tomas Della Vedova
Senior Software Engineer, Clients Team
4
Interact with Elasticsearch is simple
cURL
5
Interact with Elasticsearch is simple
HTTP request from your favourite language
6
Why should you use
the official clients?
Which problems the clients are solving for you?
7
How you see it
Your Application
ES Client
8
How it really is
Master Nodes
Ingest Nodes
Data Nodes - Hot
Data Nodes - Warm
Your Application
ES Client
9
Distributed search
Master Node
Ingest Node
Data Nodes
Elasticsearch is distributed and scales horizontally
Master Nodes
Ingest Nodes
Data Nodes - Hot
Data Nodes - Warm
10
System Topology
Automatic node sniffing
? ? ?
11
Load Balancing
Avoid overloading a single node
✓✖
12
Unreachable Nodes and Network issues
Fail fast and automatic retry
13
Maintenance
• Dedicated team that maintains all the
official clients.
• Always in sync with Elasticsearch
• Extensive testing suite
and support
14
Supported language
• Java
• JavaScript
• .NET
• PHP
• Perl
• Python
• Ruby
• Go
• Rust
15
Versioning
major.minor.patch
16
Versioning
semver.org
A breaking change
is coming
A new feature is
here!
We have fixed
a nasty bug
MAJOR MINOR PATCH
17
Compatibility
Which version of the client should I use?
Σ cm
⊆
em+1
m = 0
k
18
Developer Experience
Low level, high level and DSL
19
Always code as if the guy who ends up
maintaining your code will be a violent
psychopath who knows where you live.
Martin Golding
20
Low
Level
High
Level
• Implements the full API
• Binary stream support
• Extremely configurable
• Generated from a spec
• Consistent across
languages
• Implements the full API
• Better developer
experience
• Language specific
implementations
• Fluent API
Developer Experience
low-level vs high-level
DSL
Helpers
• Implements a smaller
portion of the API
• Better developer
experience
• Language specific
implementations
• Offers a custom query
language
21
Consistency across languages
RubyJavaScript
client.index({
index: ‘my_index’,
id: ‘1’,
body: {
title: ‘Hello world’
}
})
client.index index: ‘my_index’,
id: ‘1’,
body: {
title: ‘Hello world’
}
22
PythonRuby
client.index index: ‘my_index’,
id: ‘1’,
body: {
title: ‘Hello world’
}
client.index(
index=‘my_index’,
id=‘1’,
body={
‘title’: ‘Hello world’
}
)
Consistency across languages
23
PerlPython
client.index(
index=‘my_index’,
id=‘1’,
body={
‘title’: ‘Hello world’
}
)
$client->index(
index => ‘my_index’,
id => ‘1’,
body => {
title => ‘Hello world’
}
)
Consistency across languages
24
PHPPerl
$client->index(
index => ‘my_index’,
id => ‘1’,
body => {
title => ‘Hello world’
}
)
$client->index([
‘index’ => ‘my_index’,
‘id’ => ‘1’,
‘body’ => [
‘title’ => ‘Hello world’
]
])
Consistency across languages
25
JavaScriptPHP
$client->index([
‘index’ => ‘my_index’,
‘id’ => ‘1’,
‘body’ => [
‘title’ => ‘Hello world’
]
])
client.index({
index: ‘my_index’,
id: ‘1’,
body: {
title: ‘Hello world’
}
})
Consistency across languages
26
Developer Experience
helpers - Python
27
Developer Experience
helpers - JavaScript
28
Developer Experience
The biggest problem of the “low level”
clients, is that the query language can be
quite complex to use, especially with BIG
queries.
Query language
29
Developer Experience
elastic/elasticsearch-dsl-py
30
Developer Experience
elastic/elasticsearch-net (NEST
NEST is the High-Level client for .NET
31
Developer Experience
XOpaque-Id
Deprecations
logs
X-Opaque-Id
Audit logs
Search slow
logs
X-Opaque-IdX-Opaque-Id X-Opaque-Id
32
Client Internals
A deep dive
33
Clients internals
Component architecture
Client
APIs
Transport
Serializer
Parser
Sniffer
Connection HTTP client
Keep-Alive
Agent
Connection Pool
Selector Resurrect Update
ConnectionConnection
ConnectionConnection
ConnectionConnection
ConnectionConnection
34
Extending the client
Clients internals
35
To retry or not to retry?
In case of Connection or Timeout Error
In case of 502/3/4 status code
In case of a 4xx/5xx status code
unless specified otherwise
In case of a (de)serialization error
Clients internals
36
Connection Pooling
Clients internals
Connection Alive pool
Connection
Connection
Connection
Connection
Selector
Dead
pool
Mark Alive
Connection
Connection
url: ‘http://cloud.elastic.co’,
id: ‘unique-id’
status: ‘alive’,
roles: {
MASTER: false,
DATA: true,
INGEST: false,
ML: false
}
Mark Dead
Mark Alive
37
25K230
Supported
APIs
All generated from
the JSON spec
Integration
Test
Generated from the
YAML test suite
Some numbers
Clients internals
38
Enterprise Search
clients
Fernando Briano
Senior Software Engineer, Clients Team
Elastic Enterprise Search
Workplace Search App Search Site Search
40
Enterprise Search Client
Your Application
ES Client
41
Enterprise Search Client
Elasticsearch
clients HTTP
transport layer
Enterprise Search API
Workplace Search API
App Search API
Code Demo
Enterprise Search Client
Workplace Search Client
App Search Client
Release
• Python and Ruby beta clients - available on 7.10.0 Stack release
• More languages to come!
• Check out the free Quick Start guides:
https://ela.st/getting-started-enterprise
48
Get the best out of
Elastic Cloud
Martijn Laarman
Tech Lead, Clients Team
49
The cloud is a single node … (to the client)
Get the best out of cloud
Do the clients carry too much
complexity?
ES Client
Connection
Alive
pool
Connection
Connection
Connection
Connection
Selector
Mark Alive
Dead
pool
Mark Alive
Mark Alive
Connection
Connection
url:
‘http://cloud.elastic.co’,
id: ‘unique-id’
status: ‘alive’,
roles: {
MASTER: false,
DATA: true,
INGEST: false,
ML: false
}
50
Clients optimize for single node interactions
Get the best out of cloud
ES Client
ES Client
ES Client
Connection
Alive
pool
Connection
Connection
Connection
Connection
Selector
Mark Alive
Dead
pool
Mark Alive
Mark Alive
Connection
Connection
url:
‘http://cloud.elastic.co’,
id: ‘unique-id’
status: ‘alive’,
roles: {
MASTER: false,
DATA: true,
INGEST: false,
ML: false
}
51
Cloud ! Elasticsearch Node
Get the best out of cloud
● Compression enabled by default
● Less aggressive DNS defaults
● Optimize for workloads against cloud’s proxy
● Compression disabled by default
● DNS cached
● Optimized for workloads against
Elasticsearch directly
ES Client
ES Client
52
Connecting to either is easy as pie!
Get the best out of cloud
ES Client
ES Client
53
Get started today!
Get the best out of cloud
1
2
54
Thank You!

The Elastic clients: Recent developments

  • 1.
    1 ElasticON Global The Elasticclients: Recent developments
  • 2.
    2 This presentation andthe accompanying oral presentation contain forward-looking statements, including statements concerning plans for future offerings; the expected strength, performance or benefits of our offerings; and our future operations and expected performance. These forward-looking statements are subject to the safe harbor provisions under the Private Securities Litigation Reform Act of 1995. Our expectations and beliefs in light of currently available information regarding these matters may not materialize. Actual outcomes and results may differ materially from those contemplated by these forward-looking statements due to uncertainties, risks, and changes in circumstances, including, but not limited to those related to: the impact of the COVID-19 pandemic on our business and our customers and partners; our ability to continue to deliver and improve our offerings and successfully develop new offerings, including security-related product offerings and SaaS offerings; customer acceptance and purchase of our existing offerings and new offerings, including the expansion and adoption of our SaaS offerings; our ability to realize value from investments in the business, including R&D investments; our ability to maintain and expand our user and customer base; our international expansion strategy; our ability to successfully execute our go-to-market strategy and expand in our existing markets and into new markets, and our ability to forecast customer retention and expansion; and general market, political, economic and business conditions. Additional risks and uncertainties that could cause actual outcomes and results to differ materially are included in our filings with the Securities and Exchange Commission (the “SEC”), including our Annual Report on Form 10-K for the most recent fiscal year, our quarterly report on Form 10-Q for the most recent fiscal quarter, and any subsequent reports filed with the SEC. SEC filings are available on the Investor Relations section of Elastic’s website at ir.elastic.co and the SEC’s website at www.sec.gov. Any features or functions of services or products referenced in this presentation, or in any presentations, press releases or public statements, which are not currently available or not currently available as a general availability release, may not be delivered on time or at all. The development, release, and timing of any features or functionality described for our products remains at our sole discretion. Customers who purchase our products and services should make the purchase decisions based upon services and product features and functions that are currently available. All statements are made only as of the date of the presentation, and Elastic assumes no obligation to, and does not currently intend to, update any forward-looking statements or statements relating to features or functions of services or products, except as required by law. Forward-Looking Statements
  • 3.
    3 Elasticsearch clients Tomas DellaVedova Senior Software Engineer, Clients Team
  • 4.
  • 5.
    5 Interact with Elasticsearchis simple HTTP request from your favourite language
  • 6.
    6 Why should youuse the official clients? Which problems the clients are solving for you?
  • 7.
    7 How you seeit Your Application ES Client
  • 8.
    8 How it reallyis Master Nodes Ingest Nodes Data Nodes - Hot Data Nodes - Warm Your Application ES Client
  • 9.
    9 Distributed search Master Node IngestNode Data Nodes Elasticsearch is distributed and scales horizontally Master Nodes Ingest Nodes Data Nodes - Hot Data Nodes - Warm
  • 10.
  • 11.
  • 12.
    12 Unreachable Nodes andNetwork issues Fail fast and automatic retry
  • 13.
    13 Maintenance • Dedicated teamthat maintains all the official clients. • Always in sync with Elasticsearch • Extensive testing suite and support
  • 14.
    14 Supported language • Java •JavaScript • .NET • PHP • Perl • Python • Ruby • Go • Rust
  • 15.
  • 16.
    16 Versioning semver.org A breaking change iscoming A new feature is here! We have fixed a nasty bug MAJOR MINOR PATCH
  • 17.
    17 Compatibility Which version ofthe client should I use? Σ cm ⊆ em+1 m = 0 k
  • 18.
  • 19.
    19 Always code asif the guy who ends up maintaining your code will be a violent psychopath who knows where you live. Martin Golding
  • 20.
    20 Low Level High Level • Implements thefull API • Binary stream support • Extremely configurable • Generated from a spec • Consistent across languages • Implements the full API • Better developer experience • Language specific implementations • Fluent API Developer Experience low-level vs high-level DSL Helpers • Implements a smaller portion of the API • Better developer experience • Language specific implementations • Offers a custom query language
  • 21.
    21 Consistency across languages RubyJavaScript client.index({ index:‘my_index’, id: ‘1’, body: { title: ‘Hello world’ } }) client.index index: ‘my_index’, id: ‘1’, body: { title: ‘Hello world’ }
  • 22.
    22 PythonRuby client.index index: ‘my_index’, id:‘1’, body: { title: ‘Hello world’ } client.index( index=‘my_index’, id=‘1’, body={ ‘title’: ‘Hello world’ } ) Consistency across languages
  • 23.
    23 PerlPython client.index( index=‘my_index’, id=‘1’, body={ ‘title’: ‘Hello world’ } ) $client->index( index=> ‘my_index’, id => ‘1’, body => { title => ‘Hello world’ } ) Consistency across languages
  • 24.
    24 PHPPerl $client->index( index => ‘my_index’, id=> ‘1’, body => { title => ‘Hello world’ } ) $client->index([ ‘index’ => ‘my_index’, ‘id’ => ‘1’, ‘body’ => [ ‘title’ => ‘Hello world’ ] ]) Consistency across languages
  • 25.
    25 JavaScriptPHP $client->index([ ‘index’ => ‘my_index’, ‘id’=> ‘1’, ‘body’ => [ ‘title’ => ‘Hello world’ ] ]) client.index({ index: ‘my_index’, id: ‘1’, body: { title: ‘Hello world’ } }) Consistency across languages
  • 26.
  • 27.
  • 28.
    28 Developer Experience The biggestproblem of the “low level” clients, is that the query language can be quite complex to use, especially with BIG queries. Query language
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
    33 Clients internals Component architecture Client APIs Transport Serializer Parser Sniffer ConnectionHTTP client Keep-Alive Agent Connection Pool Selector Resurrect Update ConnectionConnection ConnectionConnection ConnectionConnection ConnectionConnection
  • 34.
  • 35.
    35 To retry ornot to retry? In case of Connection or Timeout Error In case of 502/3/4 status code In case of a 4xx/5xx status code unless specified otherwise In case of a (de)serialization error Clients internals
  • 36.
    36 Connection Pooling Clients internals ConnectionAlive pool Connection Connection Connection Connection Selector Dead pool Mark Alive Connection Connection url: ‘http://cloud.elastic.co’, id: ‘unique-id’ status: ‘alive’, roles: { MASTER: false, DATA: true, INGEST: false, ML: false } Mark Dead Mark Alive
  • 37.
    37 25K230 Supported APIs All generated from theJSON spec Integration Test Generated from the YAML test suite Some numbers Clients internals
  • 38.
  • 39.
    Elastic Enterprise Search WorkplaceSearch App Search Site Search
  • 40.
    40 Enterprise Search Client YourApplication ES Client
  • 41.
    41 Enterprise Search Client Elasticsearch clientsHTTP transport layer Enterprise Search API Workplace Search API App Search API
  • 42.
  • 43.
  • 44.
  • 46.
  • 47.
    Release • Python andRuby beta clients - available on 7.10.0 Stack release • More languages to come! • Check out the free Quick Start guides: https://ela.st/getting-started-enterprise
  • 48.
    48 Get the bestout of Elastic Cloud Martijn Laarman Tech Lead, Clients Team
  • 49.
    49 The cloud isa single node … (to the client) Get the best out of cloud Do the clients carry too much complexity? ES Client Connection Alive pool Connection Connection Connection Connection Selector Mark Alive Dead pool Mark Alive Mark Alive Connection Connection url: ‘http://cloud.elastic.co’, id: ‘unique-id’ status: ‘alive’, roles: { MASTER: false, DATA: true, INGEST: false, ML: false }
  • 50.
    50 Clients optimize forsingle node interactions Get the best out of cloud ES Client ES Client ES Client Connection Alive pool Connection Connection Connection Connection Selector Mark Alive Dead pool Mark Alive Mark Alive Connection Connection url: ‘http://cloud.elastic.co’, id: ‘unique-id’ status: ‘alive’, roles: { MASTER: false, DATA: true, INGEST: false, ML: false }
  • 51.
    51 Cloud ! ElasticsearchNode Get the best out of cloud ● Compression enabled by default ● Less aggressive DNS defaults ● Optimize for workloads against cloud’s proxy ● Compression disabled by default ● DNS cached ● Optimized for workloads against Elasticsearch directly ES Client ES Client
  • 52.
    52 Connecting to eitheris easy as pie! Get the best out of cloud ES Client ES Client
  • 53.
    53 Get started today! Getthe best out of cloud 1 2
  • 54.