Tutorial Workgroup - Model versioning and collaboration

•

0 likes•29 views

Hackolade Studio has native integration with Git repositories to provide state-of-the-art collaboration, versioning, branching, conflict resolution, peer review workflows, change tracking and traceability. Mostly, it allows to co-locate data models and schemas with application code, and further integrate with DevOps CI/CD pipelines as part of our vision for Metadata-as-Code. Co-located application code and data models provide the single source-of-truth for business and technical stakeholders.

Technology

Hackolade Tutorial
Model versioning and collaboration
Copyright © 2016-2023 Hackolade 1

DevOps infinity loop illustrates continuous flow of dev
2
Dev Ops

Design
Data Model
Generate
Schema
Catalog
I
n
t
e
g
r
a
t
e
/
M
i
g
r
a
t
e
Publish
Use
Evaluate
S
p
e
c
i
f
y
Data
Modeler
Data
Architect
Subject
Matter
Expert
Data
Engineer
Data
Analyst
Business
Analyst
Data model lifecycle unified with DevOps applications
Copyright © 2016-2023 Hackolade
Application
Metadata Dev Ops

Role of Data Modeling in an Agile/DevOps context
Copyright © 2016-2022 Hackolade 4
Domain-Driven
Design
Physical Data
Modeling
Iterative
Evolution
Fake data
Generation
Optimization
Governance
& Compliance
Maintenance &
Performance,
Documentation
& Facilitation

We call this
Metadata-as-Code
Copyright © 2016-2022 Hackolade 5

Leveraging GIT in Hackolade Studio
• Free, open source, *distributed* version control system
• Very different from traditional *centralized* version control
systems:
• made for Agile teams
• Centralized systems depend on “locking” mechanisms:
decentralised systems no longer use check-in/check-out
• Vocabulary is different: push / pull / pull requests are new
concepts for many data modelers
• The command-line interface to git can be daunting for many data
modelers
Copyright © 2016-2023 Hackolade 6

Leveraging GIT in Hackolade Studio
• Perfect fit for Hackolade Studio
• Our JSON based file storage format is perfect for Git
• The approach is completely in tune with our love for Agile,
Domain-Driven Data Modeling, and NoSQL
• Plays well with our vision of “Reconciling Business and IT
through a shared understanding of the meaning and
context of data”
• In Hackolade Studio, it is *EASY* to interact with Git
• With Hackolade Studio, business and IT can use the same tool
• With Hackolade Studio, business and IT can leverage the same
single source-of-truth
Copyright © 2016-2023 Hackolade 7

Data models
Schema contracts
Application code
Git repository DevOps CI/CD
pipeline
Application
Database
Copyright © 2016-2023 Hackolade 8
Co-locate data models and schemas with application code
Environments:
• Dev
• Test
• Integration
• Production
ALTER scripts

… …
Consumers
Subject area models
Project models
Models
Local
Repo
Local
Repo
Local
Repo
Clone
Push
Pull
Merge …
Copyright © 2016-2022 Hackolade 9
Hackolade Studio to Schema Repo workflow
CLI
Schemas
DDL files
OAS files
JSON Schema files
Avro Schema files
…

Copyright © 2016-2022 Hackolade
Hackolade’s Metadata-as-Code orchestrates Metadata Pipelines to keep in sync
technical data structures with business-facing data dictionaries
Repository
Human-readable
Models
A
c
t
o
r
s
Design
Models
Data
modelers,
architects,
developers
Technical side
Machine-readable
Schemas
Databases
APIs
Registries
Hubs
Code Classes
Technology
Command-Line
Interface Repository
Schemas
DevOps
CI/CD
Business side
Data
citizens
Metadata
management Catalog
Reverse-engineering
Requirements
Wiki

Copyright © 2016-2022 Hackolade 11
Collaboration, version control, change tracking, peer reviews
Remote
Repository
Local
Repository
Pull
Push
Commit
Local
Repository
Commit
Push
Pull
Clone
Pull = Fetch + Merge

4 scenarios to illustrate
Copyright © 2016-2023 Hackolade 12

Scenario 1
Small conflicting change to the data model cosmetic and auto-resolved
Copyright © 2016-2023 Hackolade 13

Scenario 2
Small non-conflicting / non-cosmetic change to the data model with
git-based resolution
Copyright © 2016-2023 Hackolade 14

Scenario 3
Small conflicting change to the data model with git-based resolution
Copyright © 2016-2023 Hackolade 15

Scenario 4a
Large branch-based changes (minor-fixes and new-features) to the data
model using Pull Requests
Copyright © 2016-2023 Hackolade 16

Scenario 4b
Large (feature) branch-based change to the data model using Pull
Requests
Copyright © 2016-2023 Hackolade 17

Reading material
• See Hackolade online documentation
• The Hackolade Blog
• This excellent new book:
MongoDB Data Modeling & Schema Design
• Many of the principles in the book are related to query
driven modeling based on access patterns
• Hackolade’s on social media: LinkedIn page, Twitter page
• Download Hackolade Studio for free
Copyright © 2016-2023 Hackolade 18

Questions?
Answers!
Copyright © 2016-2023 Hackolade 19

Co-locate data models and their schema artifacts with application code, so they can follow the lifecycle of application changes. Provides a single source-of-truth for business and technical stakeholders as the data model in the Git repo is the source of the technical schemas used by applications, databases, and APIs at the same time as it is the source for the business-facing data dictionaries. This architecture contributes to the shared understanding of the context and meaning of data by all the stakeholders.

Hackolade Tutorial - part 9 - Export or forward-engineer.pdf

PascalDesmarets1

Tutorial Getting Started part 1 - Overview

PascalDesmarets1

Enabling the digital thread using open OSLC standards

Axel Reichwein

Achieving the Digital Thread through PLM and ALM Integration using OSLC

Koneksys

Achieving the digital thread through PLM and ALM integration using oslc

Axel Reichwein

With the advent of cloud computing platforms, many companies are studying the migration of legacy applications to the cloud. The main difficulty in dealing with such system is the obsolescence, either due to the dependency on an obsolete platform, incomplete/incorrect documentation or using an inappropriate architecture for the cloud. The FP7 project REMICS (Reuse and Migration of legacy applications to Interoperable Cloud Services) intends to provide a model-driven approach to extract valuable information from existing code and automating the refactoring of old code into cloud enabled architectures. In order to do so, REMICS proposes a process based on three steps: Recovery, Migration and Deployment. The work to be performed during each step is partially automated by the tools developed in the project. This presentation is going to focus on the description of the process and its associated tools and on our experience in applying the process in an industrial case study.

The Enterprise Guide to Building a Data Mesh - Introducing SpecMesh

IanFurlong4

For organisations to successfully adopt data mesh, setting up and maintaining infrastructure needs to be easy. We believe the best way to achieve this is to leverage the learnings from building a ‘central nervous system‘, commonly used in modern data-streaming ecosystems. This approach formalises and automates of the manual parts of building a data mesh. This presentation introduces SpecMesh; a methodology and supporting developer toolkit to enable business to build the foundations of their data mesh.

APEX – jak vytvořit jednoduše aplikaci

MarketingArrowECS_CZ

Vital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and Spark

Vital.AI

Gimel and PayPal Notebooks @ TDWI Leadership Summit Orlando

Romit Mehta

Extending open source and hybrid cloud to drive OT transformation - Future Oi...

John Archer

The path to an hybrid open source paradigm

Jonathan Challener

Open Source project failure often stems from not setting clear objectives or having a shared vision from the start. That said there are many success stories, including two well known Statistical examples: Demetra; and Eurostat SDMX tools (SDMX-RI). However, in all these examples there was at first a founding organisation/entity that created the right environment for its successful path into a new paradigm. In the context of my presentation this being the Statistical Information System Collaboration Community (SIS-CC / http://siscc.oecd.org). Presented at the International Marketing and Output DataBase Conference, Gozd Martuljek, September 18 - 22, 2016.

Serverless machine learning architectures at Helixa

Data Science Milan

A Software Factory Integrating Rational & WebSphere Tools

ghodgkinson

ML-Ops:Philosophy, Best-Practices and Tools

Jorge Davila-Chacon

Enterprise guide to building a Data Mesh

Sion Smith

Sparkflows.io

sparkflows

Introduction to Open Services for Lifecycle Collaboration (OSLC)

Axel Reichwein

Quick Introduction on OSLC APIs For Digital Engineering Information Exchange (DEIX) Community explaining fundamental ideas behind OSLC including: - Distributed Link Creation Strategy from within ANY application - Link creation decoupled from link persistence/analysis/visualization - Embeddable search dialogs to support user-friendly link target discovery in any application - Standardized API discovery resources (Hypermedia API)

DevOps Digital Transformation: A real life use case enabled by Alien4Cloud

Cloudify Community

DICE & Cloudify – Quality Big Data Made Easy

Cloudify Community

This presentation was part of the Cloudify and XLAB Research Webinar about DevOps for Data Intensive Applications. In this webinar we discussed how to leverage automation for your big data applications, using DICE tools based on the Cloudify Open Source Orchestration. We want to make sure that developers use the time to develop their big data applications and not have to worry about deployment and operations, and have the shortest time to delivery possible. We also cover using the DICE deployment tools for automated deployment of Spark, Storm, Cassandra or Hadoop.

OSLC & The Future of Interoperability

Koneksys

Software engineering practices for the data science and machine learning life...

DataWorks Summit

With the advent of newer frameworks and toolkits, data scientists are now more productive than ever and starting to prove indispensable to enterprises. Typical organizations have large teams of data scientists who build out key analytics assets that are used on a daily basis and an integral part of live transactions. However, there is also quite a lot of chaos and complexities that get introduced because of the state of the industry. Many packages used by data scientists are from open source, and even if they are well curated, there is a growing tendency to pick out the cutting-edge or unstable packages and frameworks to accelerate analytics. Different data scientists may use different versions of runtimes, different Python or R versions, or even different versions of the same packages. Predominantly data scientists work on their laptops and it becomes difficult to reproduce their environments for use by others. Since data science is now a team sport across multiple personas, involving non-practitioners, traditional application developers, execs, and IT operators, how does an enterprise create a platform for productive cross-role collaboration? Enterprises need a very reliable and repeatable process, especially when it results in something that affects their production environments. They also require a well managed approach that enables the graduation of an asset from development through a testing and staging process to production. Given the pace of businesses nowadays, the process needs to be quite agile and flexible too—even enabling an easy path to reversing a change. Compliance and audit processes require clear lineage and history as well as approval chains. In the traditional software engineering world, this lifecycle has been well understood and best practices have been followed for ages. But what does it mean when you have non-programmers or users who are not really trained in software engineering philosophies or who perceive all of this as "big process" roadblocks in their daily work ? How do you we engage them in a productive manner and yet support enterprise requirements for reliability, tracking, and a clear continuous integration and delivery practice? The presenters, in this session, will bring up interesting techniques based on their user research, real life customer interviews, and productized best practices. The presenters also invite the audience to share their stories and best practices to make this a lively conversation. Speaker Sriram Srinivasan, Senior Technical Staff Member, Analytics Platform Architect, IBM

Tutorial Workgroup - Working with Forks

PascalDesmarets1

With the fork & pull strategy, contributors create their own (remote) copy of the original repository and push changes to that copy, which is referred to as a fork. Once a contribution is ready, it can be submitted through a pull request: the contributor basically requests the maintainer(s) of the original repository to pull into that repository the work that has been prepared in a fork. Note that nothing prevents multiple contributors from working together in the same fork prior to submitting their changes to the original repository. The main advantage of the fork & pull strategy is that the contributors do not need to be explicitly granted push access rights to the original repository. That's why that strategy is typically used for large-scale open source projects where most contributors are not known by the maintainers of the repository. Another advantage of the fork & pull strategy is that it prevents the original repository from being polluted with branches that are created and then abandoned but never deleted by their author.

Tutorial Advanced How-To - Oracle 23c Duality views

PascalDesmarets1

Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf

PascalDesmarets1

Hackolade Tutorial - part 4 - Create your first data model

PascalDesmarets1

Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...

PascalDesmarets1

Data modeling for relational databases is performed according to the rules of normalization, so larger tables are divided into smaller ones linked together with relationships. The purpose is to eliminate redundant or duplicate data. But NoSQL databases are completely different. They require a mindshift in schema design in order to leverage their capabilities, as well as a data modeling tool built specially for this new breed of state-of-the art technology.

Hackolade Tutorial - part 2 - Overview of JSON and JSON schema

PascalDesmarets1

Hackolade Tutorial - part 1 - What is a data model

PascalDesmarets1

More from PascalDesmarets1 (19)

Tutorial Workgroup - Working with Forks

Tutorial Advanced How-To - Oracle 23c Duality views

Tutorial Getting Started part 4 - Domain-Driven Data Modeling

Tutorial Getting Started part 2 - Polyglot Data Modeling

Tutorial Expert How-To - Create a model for Avro schemas

Tutorial Expert How-To - Verify Data Model

Tutorial Expert How-To - Naming Conventions

Tutorial Expert How-To - Export-Import with Excel template

Tutorial Expert How-To - Compare and Merge

Tutorial Expert How-To - Custom properties

Tutorial Expert How-To - Add reusable Definitions

Hackolade Tutorial - part 13 - Leverage a Polyglot data model

Hackolade Tutorial - part 12 - Create a REST API model

Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf

Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf

Hackolade Tutorial - part 4 - Create your first data model

Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...

Hackolade Tutorial - part 2 - Overview of JSON and JSON schema

Hackolade Tutorial - part 1 - What is a data model

Recently uploaded

Climate Impact of Software Testing at Nordic Testing Days

Kari Kakkonen

My slides at Nordic Testing Days 6.6.2024 Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

The Future of Platform Engineering

Jemma Hussein Allen

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

Assure Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

Free Complete Python - A step towards Data Science

RinaMondal9

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

PHP Frameworks: I want to break free (IPC Berlin 2024)

Ralf Eggert

In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development. This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

Elizabeth Buie - Older adults: Are we really designing for our future selves?

Nexer Digital

Elevating Tactical DDD Patterns Through Object Calisthenics

Dorra BARTAGUIZ

After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!

Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™

UiPathCommunity

In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni. 📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath: Autopilot per Studio Web Autopilot per Studio Autopilot per Apps Clipboard AI GenAI applicata alla Document Understanding 👨‍🏫👨‍💻 Speakers: Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath Andrei Tasca, RPA Solutions Team Lead @NTT Data

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance

Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx

nkrafacyberclub

UiPath Community Day Dubai: AI at Work..

UiPathCommunity

Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking. 📕 Curious on our agenda? Wait no more! 10:00 Welcome note - UiPath Community in Dubai Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank 10:20 A UiPath cross-region MEA overview Ashraf El Zarka, VP and Managing Director MEA, UiPath 10:35: Customer Success Journey Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank 11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more Boris Krumrey, Global VP, Automation Innovation, UiPath 12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services. Brendan Lingam, Director of Sales and Business Development, Marc Ellis

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

Recently uploaded (20)

Climate Impact of Software Testing at Nordic Testing Days

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

The Future of Platform Engineering

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Epistemic Interaction - tuning interfaces to provide information for AI support

Assure Contact Center Experiences for Your Customers With ThousandEyes

GraphRAG is All You need? LLM & Knowledge Graph

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

Free Complete Python - A step towards Data Science

FIDO Alliance Osaka Seminar: Overview.pdf

PHP Frameworks: I want to break free (IPC Berlin 2024)

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

Elizabeth Buie - Older adults: Are we really designing for our future selves?

Elevating Tactical DDD Patterns Through Object Calisthenics

Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx

UiPath Community Day Dubai: AI at Work..

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

Tutorial Workgroup - Model versioning and collaboration

2. DevOps infinity loop illustrates continuous flow of dev 2 Dev Ops

3. Design Data Model Generate Schema Catalog I n t e g r a t e / M i g r a t e Publish Use Evaluate S p e c i f y Data Modeler Data Architect Subject Matter Expert Data Engineer Data Analyst Business Analyst Data model lifecycle unified with DevOps applications Copyright © 2016-2023 Hackolade Application Metadata Dev Ops

4. Role of Data Modeling in an Agile/DevOps context Copyright © 2016-2022 Hackolade 4 Domain-Driven Design Physical Data Modeling Iterative Evolution Fake data Generation Optimization Governance & Compliance Maintenance & Performance, Documentation & Facilitation

6. Leveraging GIT in Hackolade Studio • Free, open source, *distributed* version control system • Very different from traditional *centralized* version control systems: • made for Agile teams • Centralized systems depend on “locking” mechanisms: decentralised systems no longer use check-in/check-out • Vocabulary is different: push / pull / pull requests are new concepts for many data modelers • The command-line interface to git can be daunting for many data modelers Copyright © 2016-2023 Hackolade 6

7. Leveraging GIT in Hackolade Studio • Perfect fit for Hackolade Studio • Our JSON based file storage format is perfect for Git • The approach is completely in tune with our love for Agile, Domain-Driven Data Modeling, and NoSQL • Plays well with our vision of “Reconciling Business and IT through a shared understanding of the meaning and context of data” • In Hackolade Studio, it is *EASY* to interact with Git • With Hackolade Studio, business and IT can use the same tool • With Hackolade Studio, business and IT can leverage the same single source-of-truth Copyright © 2016-2023 Hackolade 7

8. Data models Schema contracts Application code Git repository DevOps CI/CD pipeline Application Database Copyright © 2016-2023 Hackolade 8 Co-locate data models and schemas with application code Environments: • Dev • Test • Integration • Production ALTER scripts

9. … … Consumers Subject area models Project models Models Local Repo Local Repo Local Repo Clone Push Pull Merge … Copyright © 2016-2022 Hackolade 9 Hackolade Studio to Schema Repo workflow CLI Schemas DDL files OAS files JSON Schema files Avro Schema files …

10. Copyright © 2016-2022 Hackolade Hackolade’s Metadata-as-Code orchestrates Metadata Pipelines to keep in sync technical data structures with business-facing data dictionaries Repository Human-readable Models A c t o r s Design Models Data modelers, architects, developers Technical side Machine-readable Schemas Databases APIs Registries Hubs Code Classes Technology Command-Line Interface Repository Schemas DevOps CI/CD Business side Data citizens Metadata management Catalog Reverse-engineering Requirements Wiki

11. Copyright © 2016-2022 Hackolade 11 Collaboration, version control, change tracking, peer reviews Remote Repository Local Repository Pull Push Commit Local Repository Commit Push Pull Clone Pull = Fetch + Merge

18. Reading material • See Hackolade online documentation • The Hackolade Blog • This excellent new book: MongoDB Data Modeling & Schema Design • Many of the principles in the book are related to query driven modeling based on access patterns • Hackolade’s on social media: LinkedIn page, Twitter page • Download Hackolade Studio for free Copyright © 2016-2023 Hackolade 18

Tutorial Workgroup - Model versioning and collaboration

Recommended

Recommended

More Related Content

Similar to Tutorial Workgroup - Model versioning and collaboration

Similar to Tutorial Workgroup - Model versioning and collaboration (20)

More from PascalDesmarets1

More from PascalDesmarets1 (19)

Recently uploaded

Recently uploaded (20)

Tutorial Workgroup - Model versioning and collaboration