SlideShare a Scribd company logo
1 of 17
Download to read offline
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Introducing Open Distro for
Elasticsearch
Kunal Batra
Senior technical evangelist
Amazon Web Services
A D B 2 0 1
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What is Elasticsearch?
Sometimes referred to as the
“ELK Stack”—Elasticsearch, Logstash, &
Kibana
Distributed search and analytics
engine build on Apache Lucene
Easy ingestion and visualization
Other partner solutions: Splunk, Sumo
Logic, Logz.io, and Loggly
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine data driving Elasticsearch growth
IT & DevOps: databases,
servers, storage, and
networking
Increase in IoT and mobile
devices: gaming, sensors, and
web content
Cloud-based
architectures
Machine-generated data is growing vs. business data … logs, logs, and more logs
Source: insideBigData—The Exponential Growth of Data, February 16, 2017
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How we think about OSS licensing
and distribution?
Base OSS
free of
proprietary code
Keep commercial
software on top
of OSS separate
Allow anyone
to innovate
on OSS
Don’t change licensing
or distribution
midstream
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
An Apache 2.0-licensed distribution
of Elasticsearch enhanced with
enterprise-grade security, alerting,
SQL, and more
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Enterprise-grade
Delivering security
and advanced capabilities such
as alerting, SQL,
and cluster diagnostics
100% open source Providing
you the
freedoms, so you can
freely view, use, change, and
distribute the code
Community-driven
Providing individuals
and organizations the freedom
to easily contribute changes to
the distro
Benefits of Open Distro for Elasticsearch
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Open Distro for Elasticsearch—features
Achieve encryption in-flight, role-
based access control, audit
logging, and compliance
Monitor your data and send
automatic alerts on any changes
in your data
Easily interact with your
Elasticsearch cluster and
extract insights using the familiar
SQL query syntax
SQL
Get deep visibility into
system bottlenecks even when your
Elasticsearch cluster is under duress
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Security
Keep your data secure
Encryption
Keep your data secure when in transit
Authentication
Leverage your existing authentication infrastructure
RBAC
Granular access control to control the user actions on your
cluster
Audit logging
Track and record all user actions and meet HIPAA, PCI
compliance
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Alerting
Receive alerts on your data
Create monitors
Query the data you want to and receive alerts on it
Customize alert conditions
Define alerting threshold and severity for multiple trigger
conditions
Get notifications
Built-in integrations for webhook and Slack to get
notified on the channels you use
View alerts
All alert executions are indexed for easy tracking and
visualization
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
SQL Support
Query data with SQL
Comprehensive SQL support
Supports over 40 functions, data types, and
commands including join support
Translate SQL to JSON
Create JSON using SQL to configure sophisticated
access control policies
Use existing tools
Provides a JDBC driver so you can use a variety of
business intelligence, analytics, and ETL tools
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Performance Analyzer
Get deep diagnostic insights into your cluster
Identify bottlenecks across the stack
Provides a powerful REST API for querying Elasticsearch
metrics to diagnose issues across stack
Runs independent of your cluster
Perform diagnostics even if the cluster is under duress
Analyze hundreds of data points
Supports over 60 metrics across 10 dimensions for
instrumentation of your cluster health
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
PerfTop CLI
• Provides pre-configured
dashboards for analyzing
cluster, node, and shard
performance
• Custom JSON templates to
create the dashboards to
diagnose your cluster
performance
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Flexible deployment options
• Docker
• RPM
• Debian
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Simple to get started
Visit the
website
Download the
Elasticsearch and
Kibana packages
Load and query
data
1 2 3
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Community and contributions
• Open Distro for Elasticsearch success is driven by the community’s
participation, contributions, and innovation to the project.
• You can follow project discussions, engage with fellow community
members, contribute PRs, file bugs or request a feature at:
• Discussion forums
https://discuss.opendistrocommunity.dev/
• Community
https://github.com/opendistro-for-elasticsearch/community/issues
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Useful links
• Project website and technical documentation
https://opendistro.github.io/for-elasticsearch/
• Source Code
https://github.com/opendistro-for-elasticsearch
Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kunal Batra
@Kunal732
kbatra@amazon.com

More Related Content

What's hot

Introducing-AWS-Hong-Kong-Region
Introducing-AWS-Hong-Kong-RegionIntroducing-AWS-Hong-Kong-Region
Introducing-AWS-Hong-Kong-RegionAmazon Web Services
 
IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...
IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...
IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...Amazon Web Services
 
Best practices for queue processing in serverless applications - MAD313 - Chi...
Best practices for queue processing in serverless applications - MAD313 - Chi...Best practices for queue processing in serverless applications - MAD313 - Chi...
Best practices for queue processing in serverless applications - MAD313 - Chi...Amazon Web Services
 
AWS IoT Greengrass Workshop - SVC303 - Anaheim AWS Summit
AWS IoT Greengrass Workshop - SVC303 - Anaheim AWS SummitAWS IoT Greengrass Workshop - SVC303 - Anaheim AWS Summit
AWS IoT Greengrass Workshop - SVC303 - Anaheim AWS SummitAmazon Web Services
 
Running Amazon EC2 workloads at scale - CMP301 - New York AWS Summit
Running Amazon EC2 workloads at scale - CMP301 - New York AWS SummitRunning Amazon EC2 workloads at scale - CMP301 - New York AWS Summit
Running Amazon EC2 workloads at scale - CMP301 - New York AWS SummitAmazon Web Services
 
Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...
Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...
Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...Amazon Web Services
 
Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...
Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...
Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...Amazon Web Services
 
Driving performance & security across your industrial facility with AWS - SVC...
Driving performance & security across your industrial facility with AWS - SVC...Driving performance & security across your industrial facility with AWS - SVC...
Driving performance & security across your industrial facility with AWS - SVC...Amazon Web Services
 
Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...
Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...
Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...Amazon Web Services
 
Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...
Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...
Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...Amazon Web Services
 
Introducing Open Distro for Elasticsearch - ADB201 - New York AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - New York AWS SummitIntroducing Open Distro for Elasticsearch - ADB201 - New York AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - New York AWS SummitAmazon Web Services
 
Threat Detection using artificial intelligence
Threat Detection using artificial intelligenceThreat Detection using artificial intelligence
Threat Detection using artificial intelligenceAmazon Web Services
 
Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...
Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...
Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...Amazon Web Services
 
Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...
Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...
Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...Amazon Web Services
 
Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...
Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...
Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...Amazon Web Services
 
AWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 Barcelona
AWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 BarcelonaAWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 Barcelona
AWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 BarcelonaAmazon Web Services
 
Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...
Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...
Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...Amazon Web Services
 
Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...
Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...
Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...Amazon Web Services
 
Fundamentals of AWS networking - SVC303 - Atlanta AWS Summit
Fundamentals of AWS networking - SVC303 - Atlanta AWS SummitFundamentals of AWS networking - SVC303 - Atlanta AWS Summit
Fundamentals of AWS networking - SVC303 - Atlanta AWS SummitAmazon Web Services
 
Migrate a relational database to Aurora - ADB302 - Atlanta AWS Summit
Migrate a relational database to Aurora - ADB302 - Atlanta AWS SummitMigrate a relational database to Aurora - ADB302 - Atlanta AWS Summit
Migrate a relational database to Aurora - ADB302 - Atlanta AWS SummitAmazon Web Services
 

What's hot (20)

Introducing-AWS-Hong-Kong-Region
Introducing-AWS-Hong-Kong-RegionIntroducing-AWS-Hong-Kong-Region
Introducing-AWS-Hong-Kong-Region
 
IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...
IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...
IoT at scale - Monitor and manage devices with AWS IoT Device Management - SV...
 
Best practices for queue processing in serverless applications - MAD313 - Chi...
Best practices for queue processing in serverless applications - MAD313 - Chi...Best practices for queue processing in serverless applications - MAD313 - Chi...
Best practices for queue processing in serverless applications - MAD313 - Chi...
 
AWS IoT Greengrass Workshop - SVC303 - Anaheim AWS Summit
AWS IoT Greengrass Workshop - SVC303 - Anaheim AWS SummitAWS IoT Greengrass Workshop - SVC303 - Anaheim AWS Summit
AWS IoT Greengrass Workshop - SVC303 - Anaheim AWS Summit
 
Running Amazon EC2 workloads at scale - CMP301 - New York AWS Summit
Running Amazon EC2 workloads at scale - CMP301 - New York AWS SummitRunning Amazon EC2 workloads at scale - CMP301 - New York AWS Summit
Running Amazon EC2 workloads at scale - CMP301 - New York AWS Summit
 
Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...
Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...
Accelerate your cloud strategy with AWS Training and Certification - ANC201 -...
 
Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...
Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...
Optimize your workloads with Amazon EC2 and AMD EPYC - DEM01-R - Atlanta AWS ...
 
Driving performance & security across your industrial facility with AWS - SVC...
Driving performance & security across your industrial facility with AWS - SVC...Driving performance & security across your industrial facility with AWS - SVC...
Driving performance & security across your industrial facility with AWS - SVC...
 
Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...
Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...
Next generation intelligent data lakes, powered by GraphQL & AWS AppSync - MA...
 
Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...
Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...
Deploy and scale your first cloud application with Amazon Lightsail - CMP208 ...
 
Introducing Open Distro for Elasticsearch - ADB201 - New York AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - New York AWS SummitIntroducing Open Distro for Elasticsearch - ADB201 - New York AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - New York AWS Summit
 
Threat Detection using artificial intelligence
Threat Detection using artificial intelligenceThreat Detection using artificial intelligence
Threat Detection using artificial intelligence
 
Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...
Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...
Fulfilling_a_Billion_Requests_from_a_Global_SaaS_Company_Insights_into_AfterS...
 
Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...
Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...
Architecting Digital Media Archive Migrations with AWS - STG301 - Anaheim AWS...
 
Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...
Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...
Build secure, offline, real-time-enabled mobile apps - MAD304 - Atlanta AWS S...
 
AWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 Barcelona
AWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 BarcelonaAWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 Barcelona
AWS App Mesh (Service Mesh Magic)- AWS Container Day 2019 Barcelona
 
Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...
Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...
Safeguard the Integrity of Your Code for Fast and Secure Deployments - SVC206...
 
Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...
Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...
Driving Overall Equipment Effectiveness with AWS IoT SiteWise - SVC213 - Chic...
 
Fundamentals of AWS networking - SVC303 - Atlanta AWS Summit
Fundamentals of AWS networking - SVC303 - Atlanta AWS SummitFundamentals of AWS networking - SVC303 - Atlanta AWS Summit
Fundamentals of AWS networking - SVC303 - Atlanta AWS Summit
 
Migrate a relational database to Aurora - ADB302 - Atlanta AWS Summit
Migrate a relational database to Aurora - ADB302 - Atlanta AWS SummitMigrate a relational database to Aurora - ADB302 - Atlanta AWS Summit
Migrate a relational database to Aurora - ADB302 - Atlanta AWS Summit
 

Similar to Introducing Open Distro for Elasticsearch - ADB201 - Atlanta AWS Summit

Introducing Open Distro for Elasticsearch - ADB201 - Chicago AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - Chicago AWS SummitIntroducing Open Distro for Elasticsearch - ADB201 - Chicago AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - Chicago AWS SummitAmazon Web Services
 
Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...
Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...
Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...javier ramirez
 
OpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOps
OpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOpsOpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOps
OpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOpsjavier ramirez
 
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...Amazon Web Services
 
Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...
Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...
Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...Amazon Web Services
 
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per DayCyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per DayAmazon Web Services
 
Analyzing your web and application logs with the Amazon Elasticsearch Service...
Analyzing your web and application logs with the Amazon Elasticsearch Service...Analyzing your web and application logs with the Amazon Elasticsearch Service...
Analyzing your web and application logs with the Amazon Elasticsearch Service...javier ramirez
 
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...Amazon Web Services
 
Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018
Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018
Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018Amazon Web Services
 
Scalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS Summit
Scalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS SummitScalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS Summit
Scalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS SummitAmazon Web Services
 
Stream processing and managing real-time data
Stream processing and managing real-time dataStream processing and managing real-time data
Stream processing and managing real-time dataAmazon Web Services
 
Building Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS Summit
Building Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS SummitBuilding Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS Summit
Building Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS SummitAmazon Web Services
 
利用 Fargate - 無伺服器的容器環境建置高可用的系統
利用 Fargate - 無伺服器的容器環境建置高可用的系統利用 Fargate - 無伺服器的容器環境建置高可用的系統
利用 Fargate - 無伺服器的容器環境建置高可用的系統Amazon Web Services
 
Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...
Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...
Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...Amazon Web Services
 
Building Data Lakes & Analytics on AWS
Building Data Lakes & Analytics on AWSBuilding Data Lakes & Analytics on AWS
Building Data Lakes & Analytics on AWSAWS Summits
 
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWSAWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWSSteven Hsieh
 
Build your own log analytics solution on AWS - ADB301 - Atlanta AWS Summit
Build your own log analytics solution on AWS - ADB301 - Atlanta AWS SummitBuild your own log analytics solution on AWS - ADB301 - Atlanta AWS Summit
Build your own log analytics solution on AWS - ADB301 - Atlanta AWS SummitAmazon Web Services
 
Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019
Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019
Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019javier ramirez
 
AWS Learning Series: Harnessing the Power of Data | An Introduction to IoT
AWS Learning Series: Harnessing the Power of Data | An Introduction to IoTAWS Learning Series: Harnessing the Power of Data | An Introduction to IoT
AWS Learning Series: Harnessing the Power of Data | An Introduction to IoTAmazon Web Services
 

Similar to Introducing Open Distro for Elasticsearch - ADB201 - Atlanta AWS Summit (20)

Introducing Open Distro for Elasticsearch - ADB201 - Chicago AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - Chicago AWS SummitIntroducing Open Distro for Elasticsearch - ADB201 - Chicago AWS Summit
Introducing Open Distro for Elasticsearch - ADB201 - Chicago AWS Summit
 
Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...
Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...
Open Distro for ElasticSearch and how Grimoire is using it. Madrid DevOps Oct...
 
OpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOps
OpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOpsOpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOps
OpenDistro for Elasticsearch and how Bitergia is using it.Madrid DevOps
 
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
 
Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...
Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...
Searching for patterns: Log analytics using Amazon ES - ADB205 - New York AWS...
 
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per DayCyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
 
Analyzing your web and application logs with the Amazon Elasticsearch Service...
Analyzing your web and application logs with the Amazon Elasticsearch Service...Analyzing your web and application logs with the Amazon Elasticsearch Service...
Analyzing your web and application logs with the Amazon Elasticsearch Service...
 
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
 
Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018
Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018
Have Your Front End and Monitor It, Too (ANT303) - AWS re:Invent 2018
 
Scalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS Summit
Scalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS SummitScalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS Summit
Scalable, secure log analytics with Amazon ES - ADB302 - Chicago AWS Summit
 
Stream processing and managing real-time data
Stream processing and managing real-time dataStream processing and managing real-time data
Stream processing and managing real-time data
 
Building Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS Summit
Building Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS SummitBuilding Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS Summit
Building Data Lakes for Analytics on AWS - ADB201 - Anaheim AWS Summit
 
利用 Fargate - 無伺服器的容器環境建置高可用的系統
利用 Fargate - 無伺服器的容器環境建置高可用的系統利用 Fargate - 無伺服器的容器環境建置高可用的系統
利用 Fargate - 無伺服器的容器環境建置高可用的系統
 
Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...
Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...
Easy and Scalable Log Analytics with Amazon Elasticsearch Service - ABD326 - ...
 
Building Data Lakes & Analytics on AWS
Building Data Lakes & Analytics on AWSBuilding Data Lakes & Analytics on AWS
Building Data Lakes & Analytics on AWS
 
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWSAWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
 
Build your own log analytics solution on AWS - ADB301 - Atlanta AWS Summit
Build your own log analytics solution on AWS - ADB301 - Atlanta AWS SummitBuild your own log analytics solution on AWS - ADB301 - Atlanta AWS Summit
Build your own log analytics solution on AWS - ADB301 - Atlanta AWS Summit
 
Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019
Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019
Building a Modern Data Platform on AWS. Public Sector Summit Brussels 2019
 
AWS Espressif Amazon FreeRTOS
AWS Espressif Amazon FreeRTOSAWS Espressif Amazon FreeRTOS
AWS Espressif Amazon FreeRTOS
 
AWS Learning Series: Harnessing the Power of Data | An Introduction to IoT
AWS Learning Series: Harnessing the Power of Data | An Introduction to IoTAWS Learning Series: Harnessing the Power of Data | An Introduction to IoT
AWS Learning Series: Harnessing the Power of Data | An Introduction to IoT
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Introducing Open Distro for Elasticsearch - ADB201 - Atlanta AWS Summit

  • 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Introducing Open Distro for Elasticsearch Kunal Batra Senior technical evangelist Amazon Web Services A D B 2 0 1
  • 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What is Elasticsearch? Sometimes referred to as the “ELK Stack”—Elasticsearch, Logstash, & Kibana Distributed search and analytics engine build on Apache Lucene Easy ingestion and visualization Other partner solutions: Splunk, Sumo Logic, Logz.io, and Loggly
  • 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Machine data driving Elasticsearch growth IT & DevOps: databases, servers, storage, and networking Increase in IoT and mobile devices: gaming, sensors, and web content Cloud-based architectures Machine-generated data is growing vs. business data … logs, logs, and more logs Source: insideBigData—The Exponential Growth of Data, February 16, 2017
  • 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T How we think about OSS licensing and distribution? Base OSS free of proprietary code Keep commercial software on top of OSS separate Allow anyone to innovate on OSS Don’t change licensing or distribution midstream
  • 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T An Apache 2.0-licensed distribution of Elasticsearch enhanced with enterprise-grade security, alerting, SQL, and more
  • 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Enterprise-grade Delivering security and advanced capabilities such as alerting, SQL, and cluster diagnostics 100% open source Providing you the freedoms, so you can freely view, use, change, and distribute the code Community-driven Providing individuals and organizations the freedom to easily contribute changes to the distro Benefits of Open Distro for Elasticsearch
  • 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Open Distro for Elasticsearch—features Achieve encryption in-flight, role- based access control, audit logging, and compliance Monitor your data and send automatic alerts on any changes in your data Easily interact with your Elasticsearch cluster and extract insights using the familiar SQL query syntax SQL Get deep visibility into system bottlenecks even when your Elasticsearch cluster is under duress
  • 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Security Keep your data secure Encryption Keep your data secure when in transit Authentication Leverage your existing authentication infrastructure RBAC Granular access control to control the user actions on your cluster Audit logging Track and record all user actions and meet HIPAA, PCI compliance
  • 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Alerting Receive alerts on your data Create monitors Query the data you want to and receive alerts on it Customize alert conditions Define alerting threshold and severity for multiple trigger conditions Get notifications Built-in integrations for webhook and Slack to get notified on the channels you use View alerts All alert executions are indexed for easy tracking and visualization
  • 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T SQL Support Query data with SQL Comprehensive SQL support Supports over 40 functions, data types, and commands including join support Translate SQL to JSON Create JSON using SQL to configure sophisticated access control policies Use existing tools Provides a JDBC driver so you can use a variety of business intelligence, analytics, and ETL tools
  • 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Performance Analyzer Get deep diagnostic insights into your cluster Identify bottlenecks across the stack Provides a powerful REST API for querying Elasticsearch metrics to diagnose issues across stack Runs independent of your cluster Perform diagnostics even if the cluster is under duress Analyze hundreds of data points Supports over 60 metrics across 10 dimensions for instrumentation of your cluster health
  • 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T PerfTop CLI • Provides pre-configured dashboards for analyzing cluster, node, and shard performance • Custom JSON templates to create the dashboards to diagnose your cluster performance
  • 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Flexible deployment options • Docker • RPM • Debian
  • 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Simple to get started Visit the website Download the Elasticsearch and Kibana packages Load and query data 1 2 3
  • 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Community and contributions • Open Distro for Elasticsearch success is driven by the community’s participation, contributions, and innovation to the project. • You can follow project discussions, engage with fellow community members, contribute PRs, file bugs or request a feature at: • Discussion forums https://discuss.opendistrocommunity.dev/ • Community https://github.com/opendistro-for-elasticsearch/community/issues
  • 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Useful links • Project website and technical documentation https://opendistro.github.io/for-elasticsearch/ • Source Code https://github.com/opendistro-for-elasticsearch
  • 17. Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Kunal Batra @Kunal732 kbatra@amazon.com