OSMC 2021 | Introduction into OpenSearch

AN INTRODUCTION

TO OPENSEARCH
© 2021 floragunn GmbH - All Rights Reserved

AND ITS IMPACT ON FOSS


ABOUT ME
Jochen Kressin, Co-Founder of floragunn GmbH

Makers of Search Guard

Enterprise Security and Alerting Suite for the Elastic Stack and OpenSearch

Founded 2012

Main office: Berlin, Germany

Partner offices: Seattle, New York, Miami, Bordeaux, Mexico

Disclaimer: I am neither employed by AWS or Elastic

01.


MY HISTORY WITH SEARCH AND SECURITY
Lucene-based projects since 2004

Scaling and concurrency always an issue

SOLR

Solved many problems out of the box

Elasticsearch

Huge adoption rate, great ecosystem

OpenSearch

True FOSS search and analytics engine
02.


And what it means for FOSS


WHAT IS OPENSEARCH?
03.
“OpenSearch is a community-driven, open source search and analytics suite
derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It consists
of a search engine daemon, OpenSearch, and a visualization and user interface,
OpenSearch Dashboards. […] With OpenSearch people benefit from having an
open source product they can use, modify, extend, monetize, and resell how they
want.”
Source: https://opensearch.org/


STATUS
First announcement April 2021

OpenSearch 1.0 initial release July 2021

OpenSearch 1.1 released October 2021

ASLv2 licensed features of Elasticsearch 7.10.2

Features from Open Distro for Elasticsearch by AWS

Roadmap

https://github.com/orgs/opensearch-project/projects/1
04.
Source: https://opensearch.org/versions/opensearch-1-0-1.html


OVERVIEW
Elasticsearch

7.10.2

OSS / ASLv2
Com. Plugins

7.10.2

Elastic License
OpenSearch
Open Distro for

Elasticsearch

OSS / ASLv2
OSS / ASLv2
05.


OVERVIEW
OpenSearch
OpenSearch

Dashboards
PlugIns
Security
Alerting
SQL
Index

Management
KNN
Anomaly

Detection
Async

Search
Performance

Analyzer
Replication
Ingest /

Data Prepper
06.


RESOURCES
Project website

https://opensearch.org/

Forum

https://discuss.opendistrocommunity.dev/

Source Code

https://github.com/opensearch-project/

Community Meetings

https://www.meetup.com/Open-Distro-for-Elasticsearch-Meetup-Group
07.


FIRST QUESTIONS
Does that mean OpenSearch is a fork of Elasticsearch?

Yes, OpenSearch was forked from Elasticsearch OSS v7.10.2

Adds features from “Open Distro for Elasticsearch” (AWS)

What license does OpenSearch use?

OpenSearch is licensed under ALSv2

Does OpenSearch include any commercial features of Elasticsearch?

No, only code that was originally licensed ASLv2 by Elastic

Who is behind OpenSearch?

OpenSearch is a community project initiated by AWS
08.


STRUCTURE
Initiated by AWS

“Amazon is the primary steward and maintainer of OpenSearch today”

Source: https://aws.amazon.com/blogs/opensource/introducing-opensearch/

Designed to be a community driven project

Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/

Supported by other big players in the industry

RedHat, SAP, Capital One, logz.io …

Source: https://aws.amazon.com/blogs/opensource/introducing-opensearch/

Looking for partners and maintainers

Source: https://opensearch.org/blog/community/2021/08/community-clients/
10.


Why, pt 1


LICENSE CHANGES
Elasticsearch licensing model - until January 2021

Free, ASLv2 licensed version with basic features

Commercial add-ons

https://www.elastic.co/blog/doubling-down-on-open

License change - January 2021

Replaced ASLv2 with a dual license model: SSPL & Elastic License v1

https://www.elastic.co/blog/license-change-clarification

License change - February 2021

Introduced Elastic License v2 which replaced Elastic License v1

Permissive license, but not approved by the OSI

https://www.elastic.co/blog/elastic-license-v2
11.


OVERVIEW
Elasticsearch

OSS / ASLv2
Plugins

Elastic License
Elasticsearch

+

Plugins

SSPL or

Elastic License
Source: https://www.elastic.co/blog/licensing-change
Elasticsearch

+

Plugins

SSPL or

Elastic License v2
12.


LICENSE CHANGE: EFFECTS ON FOSS
Elasticsearch is not Open Source anymore

As per definition of the Open Source Initiative (OSI)

Source: https://opensourceconnections.com/blog/2021/01/15/is-elasticsearch-no-longer-open-source-software/

Source: https://opensource.org/node/1099

Elasticsearch cannot be offered as a managed service anymore

“You may not provide the products to others as a managed service”

Source: https://www.elastic.co/pricing/faq/licensing#what-are-the-key-changes-being-made-to-the-elastic-license

Contributions only possible under non-OSS licenses

“You can […] contribute […], whether that code is under the Elastic License or SSPL”

Permissive license, but not approved by Open Source Initiative as OSS license

Source: https://www.elastic.co/pricing/faq/licensing#what-are-the-key-changes-being-made-to-the-elastic-license
13.


Why, pt. II

The Elephant in the Room OR

Two sides of the story


BACKGROUND
Elastic published portions of their code under ASLv2 since the beginning

OSS project, typical benefits from community contributions

Also offered commercial, paid features

AWS offered a (paid) managed service based on that code

“Amazon Elasticsearch Service” (AESS), launched 2015

Source: https://aws.amazon.com/blogs/aws/new-amazon-elasticsearch-service/

Elastic began offering their own managed service

“Elastic Cloud”, 2015

“Elastic Cloud Enterprise” (ECE), launched 2017

Source: https://www.elastic.co/about/history-of-elasticsearch
14.


ELASTIC
“So why the change? AWS and Amazon Elasticsearch Service. They have been
doing things that we think are just NOT OK”

Source: https://www.elastic.co/what-is/opensearch

“Our license change is aimed at preventing companies from taking our […]
products and providing them directly as a service without collaborating with us”


“We think that Amazon’s behavior is inconsistent with the norms and values that
are especially important in the open source ecosystem.”


“The change from the Apache License to the Elastic License v2 was a response to
the continued non-collaborative engagement AWS pursued with Elasticsearch”

15.


AWS
“Developers embrace open source software for many reasons, perhaps the most
important being the freedom to use that software where and how they wish.”


“[…] we followed the recommended open source development practice of
“upstream first.” All changes to Elasticsearch were sent as upstream pull requests”


“[…] to ensure open source versions […] remain available […] AWS will create and
maintain a ALv2-licensed fork of open source Elasticsearch and Kibana”


“They believe that restricting their license will lock others out of offering managed
Elasticsearch services, which will let Elastic build a bigger business.”

16.


COMMUNITY
“[…] if you choose […] FOSS, that means something, and you have the moral
obligation to uphold”

Source: https://drewdevault.com/2021/01/19/Elasticsearch-does-not-belong-to-Elastic.html

“The whole point of open source is that amazon has a right to do this”

Source: https://www.reddit.com/r/programming/comments/l0m56s/amazon_not_ok_why_we_had_to_change_elastic/

“Amazon went too far, and now the pendulum is swinging back.”

Source: https://www.reddit.com/r/programming/comments/l0m56s/amazon_not_ok_why_we_had_to_change_elastic/

“I can't say I morally disagree with this move.”

https://www.reddit.com/r/linux/comments/kxa4ri/elasticsearch_and_kibana_changing_license_from/

“It's corporate vs corporate.”

https://www.reddit.com/r/linux/comments/kxa4ri/elasticsearch_and_kibana_changing_license_from/
17.


Do we need to rethink FOSS?


“FOSS SPIRIT” - Theory
In essence

Here’s my software, do whatever you want with it

Including: Make money with it (if you find a way)

Would be great if you contribute

The goal

Get developers on board to contribute (mostly for free)

Drive development

Make the project known in the FOSS space

Increase adoption rate (because it’s free)
21.


“FOSS SPIRIT” - Practice
Most projects remain “hobby” projects

Never gain enough traction to “take off”

Many projects struggle

“One person projects”, developed by FTEs in their spare time

Lack of funding, lack of developers, lack of time

Successful projects often backed by companies or organizations

Apache Software Foundation, RedHat etc.

Example: hapi.js

https://github.com/hapijs/hapi/issues/4111
22.


HAPI.JS
23.


MY PERSONAL OPINION
Choose your weapons and license wisely

Fully understand what the license allows and disallows

It’s either FOSS, or it is not

You can’t enjoy the advantages and avoid the disadvantages at the same time

FOSS might not be the best option for your project

Again, understand the advantages and disadvantages of your license choice

Stick to your guns and license

If you have chosen a license, stick with it

What if Lucene would change its license?
25.


MY PERSONAL OPINION
Wide adoption often only possible with a permissive license

Apache web server, nginx, mySQL, K8s, Lucene etc.

Managed services explicitely allowed

FOSS is not a business model!

FOSS is a philosophy backed by ethics, norms and values

It’s not a means to generate revenue

Can not be the (only) foundation of a company

no FOSS no Future
26.


Learnings


JUST FORKING IS NOT ENOUGH
Forking is easy, developing is challenging, maintaining is hard

It’s an ecosystem, not a single product

Logstash, Beats, Client Libraries etc.

Keeping development over all components aligned is a challenge

Testing / QA is a challenge

Many different setups and usecases

Many different plugins (Alerting, SQL, Security etc.)
26.


COMMUNITY & ENTERPRISE
Community building requires effort, dedication and steadiness

Community support and engagement

Enterprises require trust and stability

Production readiness

Enterprise support including SLAs

Professional services
27.


Q&A

SEARCH GUARD
info@search-guard.com


send us a message:
31


floragunn GmbH

Tempelhofer Ufer 16

10963 Berlin

Germany

 
E-Mail: info@search-guard.com

Web: search-guard.com
Managing Directors: Claudia Kressin, Jochen Kressin
 
Registergericht: Amtsgericht Charlottenburg
 
Registernummer: HRB 147010 B

E-Mail: info@floragunn.com

Search Guard is a trademark of floragunn GmbH, registered in the U.S. and in other countries.

Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries.

OpenSearch is a trademark of AWS

floragunn GmbH is not affiliated with Elasticsearch BV. or AWS

OSMC 2021 | Introduction into OpenSearch

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to OSMC 2021 | Introduction into OpenSearch

Similar to OSMC 2021 | Introduction into OpenSearch (20)

Recently uploaded

Recently uploaded (20)

OSMC 2021 | Introduction into OpenSearch