The OpenMetadata Community Meeting was held on Dec 14th, 2023. In this meeting, we discussed and demoed the upcoming features in the 1.3.0 Release. You can refer the meeting slides here and watch the video on Youtube - https://www.youtube.com/@OpenMetadataChannel/ About OpenMetadata: OpenMetadata is a unified platform for discovery, observability, and governance, powered by a central metadata repository, in-depth lineage, and seamless team collaboration. Based on Open Metadata Standards and APIs, supporting connectors to a wide range of data services, OpenMetadata enables end-to-end metadata management, giving you the freedom to unlock the value of your data assets. OpenMetadata (https://open-metadata.org/) is an open-source project supported by Collate, Inc. (https://www.getcollate.io/). Collate also supports a *SaaS version* (https://cloud.getcollate.io/). Join us on Meetup to get notified about OpenMetadata Webinars: https://www.meetup.com/openmetadata-meetup-group/

1. OpenMetadata
Community Meeting
December 2023 🚀

2. 3193
GitHub Stars
Open Source
Developers
204
Community
Members
4467
Community
1957
+14
+264
+379
+113
Community Stats

3. 👤 Anatoliy Shulika
👤 Andrey Skibunov
👤 Andro Wijaya
👤 Carl Kristensen
👤 Chaitanya
👤 Chyueyi
👤 Dmitrii Volkov
👤 Dominik Choma
👤 Dương Phan Nam Hưng
👤 Gábor Hermann
👤 Harpreet Singh
21 Community Contributions 💖
👤 Iaroslav Frolikov
👤 Keagan O'Donoghue
👤 Lucas Garcia
👤 Magamet Koloev
👤 Mariusz Górski
👤 Playdev
👤 Preet Shah
👤 Shin-ichi Hashiba
👤 Tom Bushell
👤 Vinícius Mello

4. 👤 Chyueyi - Metadata & Profiler!
Apache Doris 🎉

5. 👤 Anatoliy Shulika - Improved API Docs UX
RapidDoc 🎉

6. ● 406 in the last 4 weeks
Community Metrics

7. How-to Guides
● No Code; Steps using the UI
● Feature-based Documentation
● Quick Start for Admins, and Data Users
● Advanced Guides
● Step-by-Step Guides with Images
● Webinar Recordings
● Short Video Tutorials
● Tips, Use Cases, and Best Practices

9. Release 1.3 Sneak Peek Jan 2024
+380 Commits so far…
● GCS Storage Connector
● Iceberg Connector
● Doris Connector
● Lineage UI Revamp
● Alerts - Data Observability
● Ingestion Performance
● MetaPilot - Gen AI
● Spark Lineage Agent
● Knowledge Center Improvements
● DQ Incidents Manager
● Custom Metrics

10. Data Quality - What we shipped so far
● 30+ Native Tests
● Database & Data Lake support
● Time Series Visualization of Failures
● Resolution Status
● 3rd Party Integration

11. Data Quality - What’s coming
● Actively manage quality failures with OpenMetadata Incident Manager

12. Data Quality - What’s coming

13. Profiler - What we shipped so far
● 20+ metrics
● Database & Data Lake support
● Fine grain column level profiling
● Sample, partition, filter out metrics

14. Profiler - What’s coming
● Define your own metrics with custom SQL

15. Knowledge Center
● Knowledge Page Hierarchy
● Associate Data Assets With Knowledge Page
● QuickLink Improvements
● Search Support for Knowledge Page

16. Lineage - What we shipped so far
Lineage Specification and API

17. Lineage - What we shipped so far
Manual Lineage Editor

18. Lineage - What we shipped so far
State of the Art - SQL Query Processor
● Contributions to SQLFluff to improve the parsing of different
SQL dialects
● Major contributions to SQL Lineage python package to make
pluggable SQL parsers
● Performance improvements in SQL Lineage
● Stored Procedure support

19. Lineage - What we shipped so far
Lineage UI
● We shipped huge improvements to the Lineage UI
● Expand all to see the column level lineage
● Ability to load 1000s of lineage nodes by using pagination
innovation
● Preview panel to look at the data assets details without leaving
the Lineage UI
● Users can expand the Lineage view to full screen and many more.

20. Lineage - What’s Coming
● Ability to draw lineage for larger depth and larger height for
100s of nodes and more
● Search for a column and find its lineage E2E, even when its
renamed
● Drive root cause analysis through Lineage
● Notify Users and Consumers of upstream failures
● Propagate tags/descriptions through Lineage

21. Lineage - What’s Coming in 1.3.0
● All the Lineage Graph is published to
ElasticSearch/OpenSearch
● API additions to do Graph like queries to Search Indexes
● Lineage UI built with new APIs
● Brings in additional information such as service details of a data
asset
● Showcasing DQ results and Pipeline status right with-in lineage
view
● Ability to search and filter based on service, ownership, tags
etc., right with-in lineage

22. Data Observability & Notifications
What OpenMetadata Provides
● Metadata Change Events: Any changes from your sources and
with-in OpenMetadata are versioned and a Change Event is
generated
● Webhook, Slack, GChat, Email Integrations
● Simple UI to configure alerts using various conditions on
Change Events
● Enables users to build automation based on Change Events.

23. Data Observability & Notifications
What we are shipping in 1.3.0
● Add Durable Queue based off of Database - ISSUE-12764

24. Data Observability & Notifications
What we are shipping in 1.3.0
● Separation of Data Observability and Notifications
● Notifications
○ Data Assets updates - Ownership, Tags, description etc..
○ Task assignment/resolution
● Data Observability Alerts
○ Schema Change - Tables/Topics/Containers
○ DQ Test Case
○ Pipeline Status
○ Data Asset Deletion

25. Data Observability & Notifications
What we are shipping in 1.3.0
● Improvements to the Slack/GChat and activity feed rendering of
messages
● At-least once guarantees of delivering of these notifications

26. Star us on GitHub
https://github.com/open-metadata/OpenMetadata
Join our Slack
https://slack.open-metadata.org/
Follow us on X
@open_metadata
Discover Collate SaaS
https://www.getcollate.io/