Hackolade Tutorial
Tutorial – Vision & Getting Started
Copyright © 2016-2023 Hackolade 1
Hackolade Vision
Copyright © 2016-2023 Hackolade 2
At the highest level
Hackolade wants to
Reconcile
Business and IT
through a
shared understanding of the
context and meaning of data
Copyright © 2016-2023 Hackolade 3
Copyright © 2016-2023 Hackolade 4
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries , single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Copyright © 2016-2023 Hackolade 5
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries , single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Copyright © 2016-2023 Hackolade 6
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries, single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Copyright © 2016-2023 Hackolade 7
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries , single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Hackolade Overview
Copyright © 2016-2023 Hackolade 8
1. Data modeling for the 21st century
• Hackolade Studio provides
developers, architects and
data modelers with the core
functionality that they need –
without trying to offer all
solutions to everyone
• Hackolade Studio has been
built from the ground up for
the polyglot, agile world
• Simplicity and productivity
are core design principles that
have been adopted from the
start
2. Polyglot data modeling
• One, comprehensive data model that covers both SQL/NoSQL
backends, data pipelines, as well as streaming services and APIs
• Full forward- and reverse-engineering capabilities, with automation
and customizability built in
3. Enabling Agile with Metadata-as-Code
• Providing the same level of governance over our schemas, data
models and metadata as we do for our code, will require us to
manage these artefacts in a similar way
• Git (or its platform providers) are excellent repositories to provide
control, versioning, collaboration, and automation
• Using this, Hackolade orchestrates Metadata pipelines to keep in sync
the technical data structures with business-facing data dictionaries
• Provide a Single Source-of-Truth for both business and technical
stakeholders
Design
Data Model
Generate
Schema
Catalog
Publish
Use
Evaluate
Data
Modeler
Data
Architect
Subject
Matter
Expert
Data
Engineer
Data
Analyst
Business
Analyst
Metadata lifecycle unified with DevOps applications
Copyright © 2016-2023 Hackolade
Application
Metadata Dev Ops
What sets Hackolade Studio apart
• Patented technology: unique way of representing database schema
models for database systems and REST APIs
• Runs not only on Windows, but also Mac and Linux
• Simple to install and get started
• Unique collection of target-specific plugins to enable reverse and
forward engineering of data models for relational and non-relational
data stores and storage formats
• True technology-agnostic polyglot data modeling
• Unique integration with Git repositories, enabling co-location of data
models with other business and technical artefacts
• Unique integration with data dictionaries (eg. Collibra)
Copyright © 2016-2023 Hackolade 14
Polyglot Data Modeling
See article
Copyright © 2016-2023 Hackolade 15
Logical or Polyglot?
Technology agnostic?
Copyright © 2016-2023 Hackolade 16
Data modeling for polyglot persistence
One single application in the 21st century
Copyright © 2016-2023 Hackolade 17
Every organization has polyglot data pipelines
Build pipelines so they don’t break when the schema evolves!
Copyright © 2016-2023 Hackolade 18
Polyglot data models
• Sits over the previous boundary between logical and
physical data models.
• Polyglot data mode is a “Logical model on steroids”:
• allows denormalization, if desired, given access patterns;
• allows complex data types;
• Represents a common physical model that generates
schemas for a variety of technologies, with automatic
mapping to the specific data types of the respective
target technologies.
• built so you could create a library of canonical objects for
your domains, and use them consistently across physical
data models for different target technologies.
Copyright © 2016-2023 Hackolade 19
Additional benefit of Polyglot Data Models
A modular, interdependent, consistent data model that allows your
enterprise to leverage the same building blocks across all of their
different models
Copyright © 2016-2023 Hackolade 20
“Business user”-friendly representation of concepts
A higher level representation of the entities and relationships in a
model, by levering a Graph Diagram representation of the model
Copyright © 2016-2023 Hackolade 21
Metadata-as-code
See article
Copyright © 2016-2023 Hackolade 22
Metadata-as-Code: WHAT is it?
• data models are co-located with application code thanks to a tight
integration with Git repositories and DevOps CI/CD workflows.
• This repository becomes the Single Source-of-Truth for all stakeholders
• data models and their schema artefacts closely follow the lifecycle of
application development and deployment to operations.
• technical structures are published in data catalogs and kept in-sync to
ensure a shared understanding of meaning and context with business
users.
Copyright © 2016-2023 Hackolade 23
Metadata-as-Code answers key questions
• Does everyone interpret a column name in a report the same
way?
• Does the column name represent exactly the nuances of what
has been measured?
• Does the tech side of the organization share the same
understanding as the business side?
• How about when applications evolve quickly and new columns
are added at a rapid pace?
• Is the metadata in the catalogs of the business kept up-to-date
with the changes in data structures?
Copyright © 2016-2023 Hackolade 24
Data models
Schema contracts
Application code
Git repository DevOps CI/CD
pipeline
Application
Database
Copyright © 2016-2023 Hackolade 25
Hackolade + Git: Co-locate metadata with code
Environments:
• Dev
• Test
• Integration
• Production
ALTER scripts
Command line interface
To automate conversions and operations
Copyright © 2016-2023 Hackolade 26
Publishing to business community of data citizens
Orchestrate metadata pipelines to keep in sync technical data
structures with business facing data dictionaries
Copyright © 2016-2023 Hackolade 27
Domain-Driven data
modeling
See article
Copyright © 2016-2023 Hackolade 28
Delivering Domain-Driven Data Modeling
Copyright © 2016-2023 Hackolade 29
Getting Started with
Hackolade Studio
Copyright © 2016-2023 Hackolade 30
Three easy steps!
Copyright © 2016-2023 Hackolade 31
Design high-performing data structures
Design your tables, collections, and graphs, attributes and their data types, descriptions and constraints,
using a visual representation of nested structures.
Polyglot data modeling
Optimize with dynamic schema evolution during agile cycles
Link entities logically. Keep track of implicit relationships and denormalization. For each database
or protocol technology, add the required metadata. Or reverse-engineer from dev or prod instances.
Schema design features
Publish your schemas and documentation
Leverage GitOps to achieve metadata-as-code and synchronize technical schemas with business data
dictionaries. Generate MongoDB validator, CQL, HQL, DDLs for relational, REST APIs, JSON Schema, Avro,
Parquet, etc… plus human-readable documentation in HTML, Markdown, or PDF.
Metadata-as-Code
Hackolade Editions
• Differences outlined on this page.
• Professional Edition
• no limit on the size of models,
• produces documentation in different flexible formats.
• Key features include
• forward- and reverse-engineering functions tightly integrated with all different target technologies
• a powerful Command-Line Interface
• ability to compare different versions of your model, and optionally merge selected parts
• naming conventions, library of reusable definitions, domain/subject area ERD views,
inference of PK & FK relationships, generation of mock data for testing, model-driven API
generation, model verification, denormalization of SQL schemas, …
• Workgroup Edition
• includes all of the features of the Professional edition, plus
• a native integration with Git repositories for your data models: this enables versioning, branching,
change tracking, collaboration, peer review, and other related capabilities.
• more information in this section of our online documentation.
Copyright © 2016-2023 Hackolade 32
Installing Hackolade
• Download the latest version of the software from our website
• Support for Windows, Mac and Linux
• Check the installation instructions:
• For Windows
• For Mac
• For Linux
• For Docker containers
• Check the notes on Software registration
• Different behaviour for Individual or Concurrent licenses: see article
• Specific instructions for Network proxies
• Specific instructions for Virtual Machines / VDI environments
Copyright © 2016-2023 Hackolade 33
Getting help
• eLearning platform with self-paced online tutorials
• Read the online manual
• Watch our videos
• Send an email (support@hackolade.com)
Copyright © 2016-2023 Hackolade 34
Reading material
• See Hackolade online documentation
• The Hackolade Blog
• This excellent new book:
MongoDB Data Modeling & Schema Design
• Many of the principles in the book are related to query
driven modeling based on access patterns
• Hackolade’s on social media: LinkedIn page, Twitter page
• Download Hackolade Studio for free
Copyright © 2016-2023 Hackolade 35
Order on Amazon
Copyright © 2016-2023 Hackolade 36
Pre-order
Upcoming
Copyright © 2016-2023 Hackolade 37
Questions?
Answers!
Copyright © 2016-2023 Hackolade 38

Tutorial Getting Started part 1 - Overview

  • 1.
    Hackolade Tutorial Tutorial –Vision & Getting Started Copyright © 2016-2023 Hackolade 1
  • 2.
    Hackolade Vision Copyright ©2016-2023 Hackolade 2
  • 3.
    At the highestlevel Hackolade wants to Reconcile Business and IT through a shared understanding of the context and meaning of data Copyright © 2016-2023 Hackolade 3
  • 4.
    Copyright © 2016-2023Hackolade 4 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries , single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 5.
    Copyright © 2016-2023Hackolade 5 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries , single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 6.
    Copyright © 2016-2023Hackolade 6 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries, single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 7.
    Copyright © 2016-2023Hackolade 7 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries , single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 8.
    Hackolade Overview Copyright ©2016-2023 Hackolade 8
  • 9.
    1. Data modelingfor the 21st century • Hackolade Studio provides developers, architects and data modelers with the core functionality that they need – without trying to offer all solutions to everyone • Hackolade Studio has been built from the ground up for the polyglot, agile world • Simplicity and productivity are core design principles that have been adopted from the start
  • 10.
    2. Polyglot datamodeling • One, comprehensive data model that covers both SQL/NoSQL backends, data pipelines, as well as streaming services and APIs • Full forward- and reverse-engineering capabilities, with automation and customizability built in
  • 12.
    3. Enabling Agilewith Metadata-as-Code • Providing the same level of governance over our schemas, data models and metadata as we do for our code, will require us to manage these artefacts in a similar way • Git (or its platform providers) are excellent repositories to provide control, versioning, collaboration, and automation • Using this, Hackolade orchestrates Metadata pipelines to keep in sync the technical data structures with business-facing data dictionaries • Provide a Single Source-of-Truth for both business and technical stakeholders
  • 13.
  • 14.
    What sets HackoladeStudio apart • Patented technology: unique way of representing database schema models for database systems and REST APIs • Runs not only on Windows, but also Mac and Linux • Simple to install and get started • Unique collection of target-specific plugins to enable reverse and forward engineering of data models for relational and non-relational data stores and storage formats • True technology-agnostic polyglot data modeling • Unique integration with Git repositories, enabling co-location of data models with other business and technical artefacts • Unique integration with data dictionaries (eg. Collibra) Copyright © 2016-2023 Hackolade 14
  • 15.
    Polyglot Data Modeling Seearticle Copyright © 2016-2023 Hackolade 15
  • 16.
    Logical or Polyglot? Technologyagnostic? Copyright © 2016-2023 Hackolade 16
  • 17.
    Data modeling forpolyglot persistence One single application in the 21st century Copyright © 2016-2023 Hackolade 17
  • 18.
    Every organization haspolyglot data pipelines Build pipelines so they don’t break when the schema evolves! Copyright © 2016-2023 Hackolade 18
  • 19.
    Polyglot data models •Sits over the previous boundary between logical and physical data models. • Polyglot data mode is a “Logical model on steroids”: • allows denormalization, if desired, given access patterns; • allows complex data types; • Represents a common physical model that generates schemas for a variety of technologies, with automatic mapping to the specific data types of the respective target technologies. • built so you could create a library of canonical objects for your domains, and use them consistently across physical data models for different target technologies. Copyright © 2016-2023 Hackolade 19
  • 20.
    Additional benefit ofPolyglot Data Models A modular, interdependent, consistent data model that allows your enterprise to leverage the same building blocks across all of their different models Copyright © 2016-2023 Hackolade 20
  • 21.
    “Business user”-friendly representationof concepts A higher level representation of the entities and relationships in a model, by levering a Graph Diagram representation of the model Copyright © 2016-2023 Hackolade 21
  • 22.
  • 23.
    Metadata-as-Code: WHAT isit? • data models are co-located with application code thanks to a tight integration with Git repositories and DevOps CI/CD workflows. • This repository becomes the Single Source-of-Truth for all stakeholders • data models and their schema artefacts closely follow the lifecycle of application development and deployment to operations. • technical structures are published in data catalogs and kept in-sync to ensure a shared understanding of meaning and context with business users. Copyright © 2016-2023 Hackolade 23
  • 24.
    Metadata-as-Code answers keyquestions • Does everyone interpret a column name in a report the same way? • Does the column name represent exactly the nuances of what has been measured? • Does the tech side of the organization share the same understanding as the business side? • How about when applications evolve quickly and new columns are added at a rapid pace? • Is the metadata in the catalogs of the business kept up-to-date with the changes in data structures? Copyright © 2016-2023 Hackolade 24
  • 25.
    Data models Schema contracts Applicationcode Git repository DevOps CI/CD pipeline Application Database Copyright © 2016-2023 Hackolade 25 Hackolade + Git: Co-locate metadata with code Environments: • Dev • Test • Integration • Production ALTER scripts
  • 26.
    Command line interface Toautomate conversions and operations Copyright © 2016-2023 Hackolade 26
  • 27.
    Publishing to businesscommunity of data citizens Orchestrate metadata pipelines to keep in sync technical data structures with business facing data dictionaries Copyright © 2016-2023 Hackolade 27
  • 28.
  • 29.
    Delivering Domain-Driven DataModeling Copyright © 2016-2023 Hackolade 29
  • 30.
    Getting Started with HackoladeStudio Copyright © 2016-2023 Hackolade 30
  • 31.
    Three easy steps! Copyright© 2016-2023 Hackolade 31 Design high-performing data structures Design your tables, collections, and graphs, attributes and their data types, descriptions and constraints, using a visual representation of nested structures. Polyglot data modeling Optimize with dynamic schema evolution during agile cycles Link entities logically. Keep track of implicit relationships and denormalization. For each database or protocol technology, add the required metadata. Or reverse-engineer from dev or prod instances. Schema design features Publish your schemas and documentation Leverage GitOps to achieve metadata-as-code and synchronize technical schemas with business data dictionaries. Generate MongoDB validator, CQL, HQL, DDLs for relational, REST APIs, JSON Schema, Avro, Parquet, etc… plus human-readable documentation in HTML, Markdown, or PDF. Metadata-as-Code
  • 32.
    Hackolade Editions • Differencesoutlined on this page. • Professional Edition • no limit on the size of models, • produces documentation in different flexible formats. • Key features include • forward- and reverse-engineering functions tightly integrated with all different target technologies • a powerful Command-Line Interface • ability to compare different versions of your model, and optionally merge selected parts • naming conventions, library of reusable definitions, domain/subject area ERD views, inference of PK & FK relationships, generation of mock data for testing, model-driven API generation, model verification, denormalization of SQL schemas, … • Workgroup Edition • includes all of the features of the Professional edition, plus • a native integration with Git repositories for your data models: this enables versioning, branching, change tracking, collaboration, peer review, and other related capabilities. • more information in this section of our online documentation. Copyright © 2016-2023 Hackolade 32
  • 33.
    Installing Hackolade • Downloadthe latest version of the software from our website • Support for Windows, Mac and Linux • Check the installation instructions: • For Windows • For Mac • For Linux • For Docker containers • Check the notes on Software registration • Different behaviour for Individual or Concurrent licenses: see article • Specific instructions for Network proxies • Specific instructions for Virtual Machines / VDI environments Copyright © 2016-2023 Hackolade 33
  • 34.
    Getting help • eLearningplatform with self-paced online tutorials • Read the online manual • Watch our videos • Send an email (support@hackolade.com) Copyright © 2016-2023 Hackolade 34
  • 35.
    Reading material • SeeHackolade online documentation • The Hackolade Blog • This excellent new book: MongoDB Data Modeling & Schema Design • Many of the principles in the book are related to query driven modeling based on access patterns • Hackolade’s on social media: LinkedIn page, Twitter page • Download Hackolade Studio for free Copyright © 2016-2023 Hackolade 35
  • 36.
    Order on Amazon Copyright© 2016-2023 Hackolade 36 Pre-order
  • 37.
  • 38.