1
Autoscaling
From zero to production
seamlessly
Yoel Spotts and Henning Andersen
Software Engineer, Cloud & Elasticsearch
2
This presentation and the accompanying oral presentation contain forward-looking statements, including statements
concerning plans for future offerings; the expected strength, performance or benefits of our offerings; and our future
operations and expected performance. These forward-looking statements are subject to the safe harbor provisions
under the Private Securities Litigation Reform Act of 1995. Our expectations and beliefs in light of currently
available information regarding these matters may not materialize. Actual outcomes and results may differ materially
from those contemplated by these forward-looking statements due to uncertainties, risks, and changes in
circumstances, including, but not limited to those related to: the impact of the COVID-19 pandemic on our business
and our customers and partners; our ability to continue to deliver and improve our offerings and successfully
develop new offerings, including security-related product offerings and SaaS offerings; customer acceptance and
purchase of our existing offerings and new offerings, including the expansion and adoption of our SaaS offerings;
our ability to realize value from investments in the business, including R&D investments; our ability to maintain and
expand our user and customer base; our international expansion strategy; our ability to successfully execute our
go-to-market strategy and expand in our existing markets and into new markets, and our ability to forecast customer
retention and expansion; and general market, political, economic and business conditions.
Additional risks and uncertainties that could cause actual outcomes and results to differ materially are included in
our filings with the Securities and Exchange Commission (the “SEC”), including our Annual Report on Form 10-K for
the most recent fiscal year, our quarterly report on Form 10-Q for the most recent fiscal quarter, and any
subsequent reports filed with the SEC. SEC filings are available on the Investor Relations section of Elastic’s
website at ir.elastic.co and the SEC’s website at www.sec.gov.
Any features or functions of services or products referenced in this presentation, or in any presentations, press
releases or public statements, which are not currently available or not currently available as a general availability
release, may not be delivered on time or at all. The development, release, and timing of any features or functionality
described for our products remains at our sole discretion. Customers who purchase our products and services
should make the purchase decisions based upon services and product features and functions that are currently
available.
All statements are made only as of the date of the presentation, and Elastic assumes no obligation to, and does not
currently intend to, update any forward-looking statements or statements relating to features or functions of services
or products, except as required by law.
Forward-Looking Statements
3
Autoscaling
The feature level
4
Fast time to value Operational Simplicity Secure from the start
Agility Development Simplicity Reduce Risk
SECURITY & RISK
REDUCTIONSPEED SIMPLICITY
Elasticsearch Service on Elastic Cloud: Benefits
5
6
7
8
9
Elasticsearch
Cluster
2 GB
8 GB
16 GB
64GB
64GB
Analytics
ML
16GB
Search
10
Autoscaling
Under the Hood
11
From the Region
Down to the Storage
12
AWS
Google Cloud
Azure
Regions
13
AWS
Google Cloud
Azure
Regions
REGION
64 GB 64 GB 8 GB
ZooKeeper
Coordination
Service)
Constructor
Orchestration
Service)
16 GB
Allocator (container host)
14
AWS
Google Cloud
Azure
Regions
REGION
AVAILABILITY ZONE 1
64 GB 64 GB 8 GB
16 GB
Allocator (container host)
AVAILABILITY ZONE 2
64 GB 64 GB 8 GB
16 GB
Allocator (container host)
AVAILABILITY ZONE 3
64 GB 64 GB 8 GB
16 GB
Allocator (container host)
15
REGION
DeploymentCluster
64 GB 64 GB 8 GB
ZooKeeper
Coordination
Service)
Constructor
Orchestration
Service)
16 GB
16
Cluster
64 GB 64 GB
logs-1 logs-2
Buffer
Free
Storage logs-3 logs-4
number_of_shards=1
number_of_replicas=0
17
Cluster
64 GB 64 GB
logs-2logs-1Storage logs-3 logs-4
18
Cluster
64 GB 64 GB
logs-2Storage logs-3 logs-4 logs-1
19
Cluster
64 GB 64 GB
logs-1logs-2Storage logs-3 logs-4
20
Storage Autoscaling
Explained
21
Cluster
64 GB 64 GB 8 GB
ZooKeeper Constructor
H
GET decision
H H
Harvester
22
Cluster
logs-1logs-2
64 GB 64 GB
Storage
GET decision
logs-3 logs-4
H
4 TB storage
23
ZooKeeper Constructor
Cluster
64 GB 64 GB 8 GB
H
4 TB storage
Cluster: 0xabcd
4 TB storage
1.9 TB
storage * 2
H H
24
Cluster
64 GB 64 GB 8 GB
ZooKeeper Constructor
H
64 GB
H H
25
logs-1logs-2
64 GB 64 GB
Cluster
Storage logs-3 logs-4
64 GB
logs-1
26
Scaling Down
27
Cluster
64 GB 64 GB 8 GB
ZooKeeper Constructor
H2 TB storage
Cluster: 0xabcd
2 TB storage
1.9 TB
storage * 3
64 GB
H H
GET decision
28
Wait, there is more...
29
Solutions and Stack, all autoscaled
• Stack
– Search
– Ingest
– Storage
• Hot tier forecasting
– Machine Learning
– Data tier specific (e.g.,
Hot/Warm)
– Kibana
• Observability
– APM
• Security
• Enterprise Search
Data tier sessions:
“The path to data tiers:
from fast SSDs to
spinning disks to
low-cost S3 buckets”
“How we built this: data
tiering, snapshots, and
asynchronous search”
30
Thank You!

Autoscaling: From zero to production seamlessly

  • 1.
    1 Autoscaling From zero toproduction seamlessly Yoel Spotts and Henning Andersen Software Engineer, Cloud & Elasticsearch
  • 2.
    2 This presentation andthe accompanying oral presentation contain forward-looking statements, including statements concerning plans for future offerings; the expected strength, performance or benefits of our offerings; and our future operations and expected performance. These forward-looking statements are subject to the safe harbor provisions under the Private Securities Litigation Reform Act of 1995. Our expectations and beliefs in light of currently available information regarding these matters may not materialize. Actual outcomes and results may differ materially from those contemplated by these forward-looking statements due to uncertainties, risks, and changes in circumstances, including, but not limited to those related to: the impact of the COVID-19 pandemic on our business and our customers and partners; our ability to continue to deliver and improve our offerings and successfully develop new offerings, including security-related product offerings and SaaS offerings; customer acceptance and purchase of our existing offerings and new offerings, including the expansion and adoption of our SaaS offerings; our ability to realize value from investments in the business, including R&D investments; our ability to maintain and expand our user and customer base; our international expansion strategy; our ability to successfully execute our go-to-market strategy and expand in our existing markets and into new markets, and our ability to forecast customer retention and expansion; and general market, political, economic and business conditions. Additional risks and uncertainties that could cause actual outcomes and results to differ materially are included in our filings with the Securities and Exchange Commission (the “SEC”), including our Annual Report on Form 10-K for the most recent fiscal year, our quarterly report on Form 10-Q for the most recent fiscal quarter, and any subsequent reports filed with the SEC. SEC filings are available on the Investor Relations section of Elastic’s website at ir.elastic.co and the SEC’s website at www.sec.gov. Any features or functions of services or products referenced in this presentation, or in any presentations, press releases or public statements, which are not currently available or not currently available as a general availability release, may not be delivered on time or at all. The development, release, and timing of any features or functionality described for our products remains at our sole discretion. Customers who purchase our products and services should make the purchase decisions based upon services and product features and functions that are currently available. All statements are made only as of the date of the presentation, and Elastic assumes no obligation to, and does not currently intend to, update any forward-looking statements or statements relating to features or functions of services or products, except as required by law. Forward-Looking Statements
  • 3.
  • 4.
    4 Fast time tovalue Operational Simplicity Secure from the start Agility Development Simplicity Reduce Risk SECURITY & RISK REDUCTIONSPEED SIMPLICITY Elasticsearch Service on Elastic Cloud: Benefits
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
    9 Elasticsearch Cluster 2 GB 8 GB 16GB 64GB 64GB Analytics ML 16GB Search
  • 10.
  • 11.
  • 12.
  • 13.
    13 AWS Google Cloud Azure Regions REGION 64 GB64 GB 8 GB ZooKeeper Coordination Service) Constructor Orchestration Service) 16 GB Allocator (container host)
  • 14.
    14 AWS Google Cloud Azure Regions REGION AVAILABILITY ZONE1 64 GB 64 GB 8 GB 16 GB Allocator (container host) AVAILABILITY ZONE 2 64 GB 64 GB 8 GB 16 GB Allocator (container host) AVAILABILITY ZONE 3 64 GB 64 GB 8 GB 16 GB Allocator (container host)
  • 15.
    15 REGION DeploymentCluster 64 GB 64GB 8 GB ZooKeeper Coordination Service) Constructor Orchestration Service) 16 GB
  • 16.
    16 Cluster 64 GB 64GB logs-1 logs-2 Buffer Free Storage logs-3 logs-4 number_of_shards=1 number_of_replicas=0
  • 17.
    17 Cluster 64 GB 64GB logs-2logs-1Storage logs-3 logs-4
  • 18.
    18 Cluster 64 GB 64GB logs-2Storage logs-3 logs-4 logs-1
  • 19.
    19 Cluster 64 GB 64GB logs-1logs-2Storage logs-3 logs-4
  • 20.
  • 21.
    21 Cluster 64 GB 64GB 8 GB ZooKeeper Constructor H GET decision H H Harvester
  • 22.
    22 Cluster logs-1logs-2 64 GB 64GB Storage GET decision logs-3 logs-4 H 4 TB storage
  • 23.
    23 ZooKeeper Constructor Cluster 64 GB64 GB 8 GB H 4 TB storage Cluster: 0xabcd 4 TB storage 1.9 TB storage * 2 H H
  • 24.
    24 Cluster 64 GB 64GB 8 GB ZooKeeper Constructor H 64 GB H H
  • 25.
    25 logs-1logs-2 64 GB 64GB Cluster Storage logs-3 logs-4 64 GB logs-1
  • 26.
  • 27.
    27 Cluster 64 GB 64GB 8 GB ZooKeeper Constructor H2 TB storage Cluster: 0xabcd 2 TB storage 1.9 TB storage * 3 64 GB H H GET decision
  • 28.
  • 29.
    29 Solutions and Stack,all autoscaled • Stack – Search – Ingest – Storage • Hot tier forecasting – Machine Learning – Data tier specific (e.g., Hot/Warm) – Kibana • Observability – APM • Security • Enterprise Search Data tier sessions: “The path to data tiers: from fast SSDs to spinning disks to low-cost S3 buckets” “How we built this: data tiering, snapshots, and asynchronous search”
  • 30.