Data modeling has traditionally focused on relational database systems. But in the age of the internet, technologies such as XML and JSON have evolved to provide structure and definition to “data in motion”. Have data modeling technologies evolved to support these technologies? Can we use traditional approaches to model data in XML and JSON? Or are new tools and methodologies required? Join this webinar to discuss:
- XML & JSON vs. Relational Database Modeling
- Techniques & Tools for Data Modeling for XML
- Techniques & Tools for Data Modeling for JSON
- Use Cases & Opportunities for XML and JSON Data Modeling
Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks
Data Mesh is an innovative concept addressing many data challenges from an architectural, cultural, and organizational perspective. But is the world ready to implement Data Mesh?
In this session, we will review the importance of core Data Mesh principles, what they can offer, and when it is a good idea to try a Data Mesh architecture. We will discuss common challenges with implementation of Data Mesh systems and focus on the role of open-source projects for it. Projects like Apache Spark can play a key part in standardized infrastructure platform implementation of Data Mesh. We will examine the landscape of useful data engineering open-source projects to utilize in several areas of a Data Mesh system in practice, along with an architectural example. We will touch on what work (culture, tools, mindset) needs to be done to ensure Data Mesh is more accessible for engineers in the industry.
The audience will leave with a good understanding of the benefits of Data Mesh architecture, common challenges, and the role of Apache Spark and other open-source projects for its implementation in real systems.
This session is targeted for architects, decision-makers, data-engineers, and system designers.
The Data Driven University - Automating Data Governance and Stewardship in Au...Pieter De Leenheer
The document discusses implementing data governance and stewardship programs at universities. It provides examples of programs at Stanford University, George Washington University, and in the Flanders region of Belgium. The key aspects covered are:
- Establishing a data governance framework with roles, processes, asset definitions. and oversight council.
- Implementing data stewardship activities like data quality management, metadata development, and reference data management.
- Stanford's program established foundations for institutional research through data quality and context definitions.
- George Washington runs a centralized program managed by the IT governance office.
- The Flanders program provides research information and services across universities through consistent definitions, roles and collaborative workflows.
Data Architecture Strategies: Data Architecture for Digital TransformationDATAVERSITY
MDM, data quality, data architecture, and more. At the same time, combining these foundational data management approaches with other innovative techniques can help drive organizational change as well as technological transformation. This webinar will provide practical steps for creating a data foundation for effective digital transformation.
Data Warehouse or Data Lake, Which Do I Choose?DATAVERSITY
Today’s data-driven companies have a choice to make – where do we store our data? As the move to the cloud continues to be a driving factor, the choice becomes either the data warehouse (Snowflake et al) or the data lake (AWS S3 et al). There are pro’s and con’s for each approach. While the data warehouse will give you strong data management with analytics, they don’t do well with semi-structured and unstructured data with tightly coupled storage and compute, not to mention expensive vendor lock-in. On the other hand, data lakes allow you to store all kinds of data and are extremely affordable, but they’re only meant for storage and by themselves provide no direct value to an organization.
Enter the Open Data Lakehouse, the next evolution of the data stack that gives you the openness and flexibility of the data lake with the key aspects of the data warehouse like management and transaction support.
In this webinar, you’ll hear from Ali LeClerc who will discuss the data landscape and why many companies are moving to an open data lakehouse. Ali will share more perspective on how you should think about what fits best based on your use case and workloads, and how some real world customers are using Presto, a SQL query engine, to bring analytics to the data lakehouse.
Data Mesh in Azure using Cloud Scale Analytics (WAF)Nathan Bijnens
This document discusses moving from a centralized data architecture to a distributed data mesh architecture. It describes how a data mesh shifts data management responsibilities to individual business domains, with each domain acting as both a provider and consumer of data products. Key aspects of the data mesh approach discussed include domain-driven design, domain zones to organize domains, treating data as products, and using this approach to enable analytics at enterprise scale on platforms like Azure.
This document discusses data mesh, a distributed data management approach for microservices. It outlines the challenges of implementing microservice architecture including data decoupling, sharing data across domains, and data consistency. It then introduces data mesh as a solution, describing how to build the necessary infrastructure using technologies like Kubernetes and YAML to quickly deploy data pipelines and provision data across services and applications in a distributed manner. The document provides examples of how data mesh can be used to improve legacy system integration, batch processing efficiency, multi-source data aggregation, and cross-cloud/environment integration.
Data Governance Best Practices, Assessments, and RoadmapsDATAVERSITY
When starting or evaluating the present state of your Data Governance program, it is important to focus on best practices such that you don’t take a ready, fire, aim approach. Best practices need to be practical and doable to be selected for your organization, and the program must be at risk if the best practice is not achieved.
Join Bob Seiner for an important webinar focused on industry best practice around standing up formal Data Governance. Learn how to assess your organization against the practices and deliver an effective roadmap based on the results of conducting the assessment.
In this webinar, Bob will focus on:
- Criteria to select the appropriate best practices for your organization
- How to define the best practices for ultimate impact
- Assessing against selected best practices
- Focusing the recommendations on program success
- Delivering a roadmap for your Data Governance program
Presentation on Data Mesh: The paradigm shift is a new type of eco-system architecture, which is a shift left towards a modern distributed architecture in which it allows domain-specific data and views “data-as-a-product,” enabling each domain to handle its own data pipelines.
Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks
Data Mesh is an innovative concept addressing many data challenges from an architectural, cultural, and organizational perspective. But is the world ready to implement Data Mesh?
In this session, we will review the importance of core Data Mesh principles, what they can offer, and when it is a good idea to try a Data Mesh architecture. We will discuss common challenges with implementation of Data Mesh systems and focus on the role of open-source projects for it. Projects like Apache Spark can play a key part in standardized infrastructure platform implementation of Data Mesh. We will examine the landscape of useful data engineering open-source projects to utilize in several areas of a Data Mesh system in practice, along with an architectural example. We will touch on what work (culture, tools, mindset) needs to be done to ensure Data Mesh is more accessible for engineers in the industry.
The audience will leave with a good understanding of the benefits of Data Mesh architecture, common challenges, and the role of Apache Spark and other open-source projects for its implementation in real systems.
This session is targeted for architects, decision-makers, data-engineers, and system designers.
The Data Driven University - Automating Data Governance and Stewardship in Au...Pieter De Leenheer
The document discusses implementing data governance and stewardship programs at universities. It provides examples of programs at Stanford University, George Washington University, and in the Flanders region of Belgium. The key aspects covered are:
- Establishing a data governance framework with roles, processes, asset definitions. and oversight council.
- Implementing data stewardship activities like data quality management, metadata development, and reference data management.
- Stanford's program established foundations for institutional research through data quality and context definitions.
- George Washington runs a centralized program managed by the IT governance office.
- The Flanders program provides research information and services across universities through consistent definitions, roles and collaborative workflows.
Data Architecture Strategies: Data Architecture for Digital TransformationDATAVERSITY
MDM, data quality, data architecture, and more. At the same time, combining these foundational data management approaches with other innovative techniques can help drive organizational change as well as technological transformation. This webinar will provide practical steps for creating a data foundation for effective digital transformation.
Data Warehouse or Data Lake, Which Do I Choose?DATAVERSITY
Today’s data-driven companies have a choice to make – where do we store our data? As the move to the cloud continues to be a driving factor, the choice becomes either the data warehouse (Snowflake et al) or the data lake (AWS S3 et al). There are pro’s and con’s for each approach. While the data warehouse will give you strong data management with analytics, they don’t do well with semi-structured and unstructured data with tightly coupled storage and compute, not to mention expensive vendor lock-in. On the other hand, data lakes allow you to store all kinds of data and are extremely affordable, but they’re only meant for storage and by themselves provide no direct value to an organization.
Enter the Open Data Lakehouse, the next evolution of the data stack that gives you the openness and flexibility of the data lake with the key aspects of the data warehouse like management and transaction support.
In this webinar, you’ll hear from Ali LeClerc who will discuss the data landscape and why many companies are moving to an open data lakehouse. Ali will share more perspective on how you should think about what fits best based on your use case and workloads, and how some real world customers are using Presto, a SQL query engine, to bring analytics to the data lakehouse.
Data Mesh in Azure using Cloud Scale Analytics (WAF)Nathan Bijnens
This document discusses moving from a centralized data architecture to a distributed data mesh architecture. It describes how a data mesh shifts data management responsibilities to individual business domains, with each domain acting as both a provider and consumer of data products. Key aspects of the data mesh approach discussed include domain-driven design, domain zones to organize domains, treating data as products, and using this approach to enable analytics at enterprise scale on platforms like Azure.
This document discusses data mesh, a distributed data management approach for microservices. It outlines the challenges of implementing microservice architecture including data decoupling, sharing data across domains, and data consistency. It then introduces data mesh as a solution, describing how to build the necessary infrastructure using technologies like Kubernetes and YAML to quickly deploy data pipelines and provision data across services and applications in a distributed manner. The document provides examples of how data mesh can be used to improve legacy system integration, batch processing efficiency, multi-source data aggregation, and cross-cloud/environment integration.
Data Governance Best Practices, Assessments, and RoadmapsDATAVERSITY
When starting or evaluating the present state of your Data Governance program, it is important to focus on best practices such that you don’t take a ready, fire, aim approach. Best practices need to be practical and doable to be selected for your organization, and the program must be at risk if the best practice is not achieved.
Join Bob Seiner for an important webinar focused on industry best practice around standing up formal Data Governance. Learn how to assess your organization against the practices and deliver an effective roadmap based on the results of conducting the assessment.
In this webinar, Bob will focus on:
- Criteria to select the appropriate best practices for your organization
- How to define the best practices for ultimate impact
- Assessing against selected best practices
- Focusing the recommendations on program success
- Delivering a roadmap for your Data Governance program
Presentation on Data Mesh: The paradigm shift is a new type of eco-system architecture, which is a shift left towards a modern distributed architecture in which it allows domain-specific data and views “data-as-a-product,” enabling each domain to handle its own data pipelines.
Metadata is hotter than ever, according to a number of recent DATAVERSITY surveys. More and more organizations are realizing that in order to drive business value from data, robust metadata is needed to gain the necessary context and lineage around key data assets. At the same time, industry regulations are driving the need for better transparency and understanding of information.
While metadata has been managed for decades, new strategies & approaches have been developed to support the ever-evolving data landscape, and provide more innovative ways to drive business value from metadata. This webinar will provide an overview of metadata strategies & technologies available to today’s organization, and provide insights into building successful business strategies for metadata adoption & use.
Data Governance Takes a Village (So Why is Everyone Hiding?)DATAVERSITY
Data governance represents both an obstacle and opportunity for enterprises everywhere. And many individuals may hesitate to embrace the change. Yet if led well, a governance initiative has the potential to launch a data community that drives innovation and data-driven decision-making for the wider business. (And yes, it can even be fun!). So how do you build a roadmap to success?
This session will gather four governance experts, including Mary Williams, Associate Director, Enterprise Data Governance at Exact Sciences, and Bob Seiner, author of Non-Invasive Data Governance, for a roundtable discussion about the challenges and opportunities of leading a governance initiative that people embrace. Join this webinar to learn:
- How to build an internal case for data governance and a data catalog
- Tips for picking a use case that builds confidence in your program
- How to mature your program and build your data community
Enterprise Architecture vs. Data ArchitectureDATAVERSITY
Enterprise Architecture (EA) provides a visual blueprint of the organization, and shows key interrelationships between data, process, applications, and more. By abstracting these assets in a graphical view, it’s possible to see key interrelationships, particularly as they relate to data and its business impact across the organization. Join us for a discussion on how Data Architecture is a key component of an overall Enterprise Architecture for enhanced business value and success.
Metadata management is critical for organizations looking to understand the context, definition and lineage of key data assets. Data models play a key role in metadata management, as many of the key structural and business definitions are stored within the models themselves. Can data models replace traditional metadata solutions? Or should they integrate with larger metadata management tools & initiatives?
Join this webinar to discuss opportunities and challenges around:
How data modeling fits within a larger metadata management landscape
When can data modeling provide “just enough” metadata management
Key data modeling artifacts for metadata
Organization, Roles & Implementation Considerations
Glossaries, Dictionaries, and Catalogs Result in Data GovernanceDATAVERSITY
Data catalogs, business glossaries, and data dictionaries house metadata that is important to your organization’s governance of data. People in your organization need to be engaged in leveraging the tools, understanding the data that is available, who is responsible for the data, and knowing how to get their hands on the data to perform their job function. The metadata will not govern itself.
Join Bob Seiner for the webinar where he will discuss how glossaries, dictionaries, and catalogs can result in effective Data Governance. People must have confidence in the metadata associated with the data that you need them to trust. Therefore, the metadata in your data catalog, business glossary, and data dictionary must result in governed data. Learn how glossaries, dictionaries, and catalogs can result in Data Governance in this webinar.
Bob will discuss the following subjects in this webinar:
- Successful Data Governance relies on value from very important tools
- What it means to govern your data catalog, business glossary, and data dictionary
- Why governing the metadata in these tools is important
- The roles necessary to govern these tools
- Governance expected from metadata in catalogs, glossaries, and dictionaries
This practical presentation will cover the most important and impactful artifacts and deliverables needed to implement and sustain governance. Rather than speak hypothetically about what output is needed from governance, it covers and reviews artifact templates to help you re-create them in your organization.
Topics covered:
- Which artifacts are most important to get started
- Important artifacts for more mature programs
- How to ensure the artifacts are used and implemented, not just written
- How to integrate governance artifacts into operational processes
- Who should be involved in creating the deliverables
Building Lakehouses on Delta Lake with SQL Analytics PrimerDatabricks
You’ve heard the marketing buzz, maybe you have been to a workshop and worked with some Spark, Delta, SQL, Python, or R, but you still need some help putting all the pieces together? Join us as we review some common techniques to build a lakehouse using Delta Lake, use SQL Analytics to perform exploratory analysis, and build connectivity for BI applications.
Enterprise Architecture vs. Data ArchitectureDATAVERSITY
Enterprise Architecture (EA) provides a visual blueprint of the organization, and shows key interrelationships between data, process, applications, and more. By abstracting these assets in a graphical view, it’s possible to see key interrelationships, particularly as they relate to data and its business impact across the organization. Join us for a discussion on how data architecture is a key component of an overall enterprise architecture for enhanced business value and success.
Describes what Enterprise Data Architecture in a Software Development Organization should cover and does that by listing over 200 data architecture related deliverables an Enterprise Data Architect should remember to evangelize.
Data Catalog for Better Data Discovery and GovernanceDenodo
Watch full webinar here: https://buff.ly/2Vq9FR0
Data catalogs are en vogue answering critical data governance questions like “Where all does my data reside?” “What other entities are associated with my data?” “What are the definitions of the data fields?” and “Who accesses the data?” Data catalogs maintain the necessary business metadata to answer these questions and many more. But that’s not enough. For it to be useful, data catalogs need to deliver these answers to the business users right within the applications they use.
In this session, you will learn:
*How data catalogs enable enterprise-wide data governance regimes
*What key capability requirements should you expect in data catalogs
*How data virtualization combines dynamic data catalogs with delivery
Master Data Management – Aligning Data, Process, and GovernanceDATAVERSITY
Master Data Management (MDM) provides organizations with an accurate and comprehensive view of their business-critical data such as customers, products, vendors, and more. While mastering these key data areas can be a complex task, the value of doing so can be tremendous – from real-time operational integration to data warehousing and analytic reporting. This webinar will provide practical strategies for gaining value from your MDM initiative, while at the same time assuring a solid architectural and governance foundation that will ensure long-term, enterprise-wide success.
A conceptual data model (CDM) uses simple graphical images to describe core concepts and principles of an organization at a high level. A CDM facilitates communication between businesspeople and IT and integration between systems. It needs to capture enough rules and definitions to create database systems while remaining intuitive. Conceptual data models apply to both transactional and dimensional/analytics modeling. While different notations can be used, the most important thing is that a CDM effectively conveys an organization's key concepts.
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...HostedbyConfluent
Companies are increasingly becoming software-driven, requiring new approaches to software architecture and data integration. The "data mesh" architectural pattern decentralizes data management by organizing it around domain experts and treating data as products that can be accessed on-demand. This helps address issues with centralized data warehouses by evolving data modeling with business needs, avoiding bottlenecks, and giving autonomy to domain teams. Key principles of the data mesh include domain ownership of data, treating data as self-service products, and establishing federated governance to coordinate the decentralized system.
Real-World Data Governance: Data Governance ExpectationsDATAVERSITY
When starting a Data Governance program, significant time, effort and bandwidth is typically spent selling the concept of data governance and telling people in your organization what data governance will do for them. This may not be the best strategy to take. We should focus on making Data Governance THEIR idea not ours.
Shouldn’t the strategy be that we get the business people from our organization to tell US why data governance is necessary and what data governance will do for them? If only we could get them to tell us these things? Maybe we can.
Join Bob Seiner and DATAVERSITY for this informative Real-World Data Governance webinar that will focus on getting THEM to tell US where data governance will add value. Seiner will review techniques for acquiring this information and will share information of where this information will add specific value to your data governance program. Some of those places may surprise you.
Doug Bateman, a principal data engineering instructor at Databricks, presented on how to build a Lakehouse architecture. He began by introducing himself and his background. He then discussed the goals of describing key Lakehouse features, explaining how Delta Lake enables it, and developing a sample Lakehouse using Databricks. The key aspects of a Lakehouse are that it supports diverse data types and workloads while enabling using BI tools directly on source data. Delta Lake provides reliability, consistency, and performance through its ACID transactions, automatic file consolidation, and integration with Spark. Bateman concluded with a demo of creating a Lakehouse.
Building a Data Strategy – Practical Steps for Aligning with Business GoalsDATAVERSITY
Developing a Data Strategy for your organization can seem like a daunting task – but it’s worth the effort. Getting your Data Strategy right can provide significant value, as data drives many of the key initiatives in today’s marketplace – from digital transformation, to marketing, to customer centricity, to population health, and more. This webinar will help demystify Data Strategy and its relationship to Data Architecture and will provide concrete, practical ways to get started.
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data PipelinesDATAVERSITY
With the aid of any number of data management and processing tools, data flows through multiple on-prem and cloud storage locations before it’s delivered to business users. As a result, IT teams — including IT Ops, DataOps, and DevOps — are often overwhelmed by the complexity of creating a reliable data pipeline that includes the automation and observability they require.
The answer to this widespread problem is a centralized data pipeline orchestration solution.
Join Stonebranch’s Scott Davis, Global Vice President and Ravi Murugesan, Sr. Solution Engineer to learn how DataOps teams orchestrate their end-to-end data pipelines with a platform approach to managing automation.
Key Learnings:
- Discover how to orchestrate data pipelines across a hybrid IT environment (on-prem and cloud)
- Find out how DataOps teams are empowered with event-based triggers for real-time data flow
- See examples of reports, dashboards, and proactive alerts designed to help you reliably keep data flowing through your business — with the observability you require
- Discover how to replace clunky legacy approaches to streaming data in a multi-cloud environment
- See what’s possible with the Stonebranch Universal Automation Center (UAC)
Improving Data Literacy Around Data ArchitectureDATAVERSITY
Data Literacy is an increasing concern, as organizations look to become more data-driven. As the rise of the citizen data scientist and self-service data analytics becomes increasingly common, the need for business users to understand core Data Management fundamentals is more important than ever. At the same time, technical roles need a strong foundation in Data Architecture principles and best practices. Join this webinar to understand the key components of Data Literacy, and practical ways to implement a Data Literacy program in your organization.
The first step towards understanding what data assets mean for your organization is understanding what those assets mean for each other. Metadata—literally, data about data—is one of many data management disciplines inherent in good systems development, and is perhaps the most mislabeled and misunderstood out of the lot. Understanding metadata and its associated technologies as more than just straightforward technological tools can provide powerful insight, the efficiency of organizational practices, and can also enable you to combine more sophisticated data management techniques in support of larger and more complex business initiatives.
In this webinar, we will:
Illustrate how to leverage metadata in support of your business strategy
Discuss foundational metadata concepts based on the DAMA Guide to Data Management Book of Knowledge (DAMA DMBOK)
Enumerate guiding principles for and lessons previously learned from metadata and its practical uses
LDM Webinar: Data Modeling & Metadata ManagementDATAVERSITY
Metadata management is critical for organizations looking to understand the context, definition and lineage of key data assets. Data models play a key role in metadata management, as many of the key structural and business definitions are stored within the models themselves. Can data models replace traditional metadata solutions? Or should they integrate with larger metadata management tools & initiatives? Join this webinar to discuss opportunities and challenges around:
- How data modeling fits within a larger metadata management landscape
- When can data modeling provide “just enough” metadata management
- Key data modeling artifacts for metadata
- Organization, Roles & Implementation Considerations
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...DATAVERSITY
This document summarizes a presentation on self-service data analysis, data wrangling, data munging, and how they fit together with data modeling. It discusses how these techniques allow business stakeholders and data scientists to prepare and transform data for analysis without extensive technical expertise. While these tools increase flexibility, they can also decrease governance if not used properly. The document advocates finding a balance between managed data assets and exploratory analysis to maximize insights while maintaining data quality.
Metadata is hotter than ever, according to a number of recent DATAVERSITY surveys. More and more organizations are realizing that in order to drive business value from data, robust metadata is needed to gain the necessary context and lineage around key data assets. At the same time, industry regulations are driving the need for better transparency and understanding of information.
While metadata has been managed for decades, new strategies & approaches have been developed to support the ever-evolving data landscape, and provide more innovative ways to drive business value from metadata. This webinar will provide an overview of metadata strategies & technologies available to today’s organization, and provide insights into building successful business strategies for metadata adoption & use.
Data Governance Takes a Village (So Why is Everyone Hiding?)DATAVERSITY
Data governance represents both an obstacle and opportunity for enterprises everywhere. And many individuals may hesitate to embrace the change. Yet if led well, a governance initiative has the potential to launch a data community that drives innovation and data-driven decision-making for the wider business. (And yes, it can even be fun!). So how do you build a roadmap to success?
This session will gather four governance experts, including Mary Williams, Associate Director, Enterprise Data Governance at Exact Sciences, and Bob Seiner, author of Non-Invasive Data Governance, for a roundtable discussion about the challenges and opportunities of leading a governance initiative that people embrace. Join this webinar to learn:
- How to build an internal case for data governance and a data catalog
- Tips for picking a use case that builds confidence in your program
- How to mature your program and build your data community
Enterprise Architecture vs. Data ArchitectureDATAVERSITY
Enterprise Architecture (EA) provides a visual blueprint of the organization, and shows key interrelationships between data, process, applications, and more. By abstracting these assets in a graphical view, it’s possible to see key interrelationships, particularly as they relate to data and its business impact across the organization. Join us for a discussion on how Data Architecture is a key component of an overall Enterprise Architecture for enhanced business value and success.
Metadata management is critical for organizations looking to understand the context, definition and lineage of key data assets. Data models play a key role in metadata management, as many of the key structural and business definitions are stored within the models themselves. Can data models replace traditional metadata solutions? Or should they integrate with larger metadata management tools & initiatives?
Join this webinar to discuss opportunities and challenges around:
How data modeling fits within a larger metadata management landscape
When can data modeling provide “just enough” metadata management
Key data modeling artifacts for metadata
Organization, Roles & Implementation Considerations
Glossaries, Dictionaries, and Catalogs Result in Data GovernanceDATAVERSITY
Data catalogs, business glossaries, and data dictionaries house metadata that is important to your organization’s governance of data. People in your organization need to be engaged in leveraging the tools, understanding the data that is available, who is responsible for the data, and knowing how to get their hands on the data to perform their job function. The metadata will not govern itself.
Join Bob Seiner for the webinar where he will discuss how glossaries, dictionaries, and catalogs can result in effective Data Governance. People must have confidence in the metadata associated with the data that you need them to trust. Therefore, the metadata in your data catalog, business glossary, and data dictionary must result in governed data. Learn how glossaries, dictionaries, and catalogs can result in Data Governance in this webinar.
Bob will discuss the following subjects in this webinar:
- Successful Data Governance relies on value from very important tools
- What it means to govern your data catalog, business glossary, and data dictionary
- Why governing the metadata in these tools is important
- The roles necessary to govern these tools
- Governance expected from metadata in catalogs, glossaries, and dictionaries
This practical presentation will cover the most important and impactful artifacts and deliverables needed to implement and sustain governance. Rather than speak hypothetically about what output is needed from governance, it covers and reviews artifact templates to help you re-create them in your organization.
Topics covered:
- Which artifacts are most important to get started
- Important artifacts for more mature programs
- How to ensure the artifacts are used and implemented, not just written
- How to integrate governance artifacts into operational processes
- Who should be involved in creating the deliverables
Building Lakehouses on Delta Lake with SQL Analytics PrimerDatabricks
You’ve heard the marketing buzz, maybe you have been to a workshop and worked with some Spark, Delta, SQL, Python, or R, but you still need some help putting all the pieces together? Join us as we review some common techniques to build a lakehouse using Delta Lake, use SQL Analytics to perform exploratory analysis, and build connectivity for BI applications.
Enterprise Architecture vs. Data ArchitectureDATAVERSITY
Enterprise Architecture (EA) provides a visual blueprint of the organization, and shows key interrelationships between data, process, applications, and more. By abstracting these assets in a graphical view, it’s possible to see key interrelationships, particularly as they relate to data and its business impact across the organization. Join us for a discussion on how data architecture is a key component of an overall enterprise architecture for enhanced business value and success.
Describes what Enterprise Data Architecture in a Software Development Organization should cover and does that by listing over 200 data architecture related deliverables an Enterprise Data Architect should remember to evangelize.
Data Catalog for Better Data Discovery and GovernanceDenodo
Watch full webinar here: https://buff.ly/2Vq9FR0
Data catalogs are en vogue answering critical data governance questions like “Where all does my data reside?” “What other entities are associated with my data?” “What are the definitions of the data fields?” and “Who accesses the data?” Data catalogs maintain the necessary business metadata to answer these questions and many more. But that’s not enough. For it to be useful, data catalogs need to deliver these answers to the business users right within the applications they use.
In this session, you will learn:
*How data catalogs enable enterprise-wide data governance regimes
*What key capability requirements should you expect in data catalogs
*How data virtualization combines dynamic data catalogs with delivery
Master Data Management – Aligning Data, Process, and GovernanceDATAVERSITY
Master Data Management (MDM) provides organizations with an accurate and comprehensive view of their business-critical data such as customers, products, vendors, and more. While mastering these key data areas can be a complex task, the value of doing so can be tremendous – from real-time operational integration to data warehousing and analytic reporting. This webinar will provide practical strategies for gaining value from your MDM initiative, while at the same time assuring a solid architectural and governance foundation that will ensure long-term, enterprise-wide success.
A conceptual data model (CDM) uses simple graphical images to describe core concepts and principles of an organization at a high level. A CDM facilitates communication between businesspeople and IT and integration between systems. It needs to capture enough rules and definitions to create database systems while remaining intuitive. Conceptual data models apply to both transactional and dimensional/analytics modeling. While different notations can be used, the most important thing is that a CDM effectively conveys an organization's key concepts.
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...HostedbyConfluent
Companies are increasingly becoming software-driven, requiring new approaches to software architecture and data integration. The "data mesh" architectural pattern decentralizes data management by organizing it around domain experts and treating data as products that can be accessed on-demand. This helps address issues with centralized data warehouses by evolving data modeling with business needs, avoiding bottlenecks, and giving autonomy to domain teams. Key principles of the data mesh include domain ownership of data, treating data as self-service products, and establishing federated governance to coordinate the decentralized system.
Real-World Data Governance: Data Governance ExpectationsDATAVERSITY
When starting a Data Governance program, significant time, effort and bandwidth is typically spent selling the concept of data governance and telling people in your organization what data governance will do for them. This may not be the best strategy to take. We should focus on making Data Governance THEIR idea not ours.
Shouldn’t the strategy be that we get the business people from our organization to tell US why data governance is necessary and what data governance will do for them? If only we could get them to tell us these things? Maybe we can.
Join Bob Seiner and DATAVERSITY for this informative Real-World Data Governance webinar that will focus on getting THEM to tell US where data governance will add value. Seiner will review techniques for acquiring this information and will share information of where this information will add specific value to your data governance program. Some of those places may surprise you.
Doug Bateman, a principal data engineering instructor at Databricks, presented on how to build a Lakehouse architecture. He began by introducing himself and his background. He then discussed the goals of describing key Lakehouse features, explaining how Delta Lake enables it, and developing a sample Lakehouse using Databricks. The key aspects of a Lakehouse are that it supports diverse data types and workloads while enabling using BI tools directly on source data. Delta Lake provides reliability, consistency, and performance through its ACID transactions, automatic file consolidation, and integration with Spark. Bateman concluded with a demo of creating a Lakehouse.
Building a Data Strategy – Practical Steps for Aligning with Business GoalsDATAVERSITY
Developing a Data Strategy for your organization can seem like a daunting task – but it’s worth the effort. Getting your Data Strategy right can provide significant value, as data drives many of the key initiatives in today’s marketplace – from digital transformation, to marketing, to customer centricity, to population health, and more. This webinar will help demystify Data Strategy and its relationship to Data Architecture and will provide concrete, practical ways to get started.
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data PipelinesDATAVERSITY
With the aid of any number of data management and processing tools, data flows through multiple on-prem and cloud storage locations before it’s delivered to business users. As a result, IT teams — including IT Ops, DataOps, and DevOps — are often overwhelmed by the complexity of creating a reliable data pipeline that includes the automation and observability they require.
The answer to this widespread problem is a centralized data pipeline orchestration solution.
Join Stonebranch’s Scott Davis, Global Vice President and Ravi Murugesan, Sr. Solution Engineer to learn how DataOps teams orchestrate their end-to-end data pipelines with a platform approach to managing automation.
Key Learnings:
- Discover how to orchestrate data pipelines across a hybrid IT environment (on-prem and cloud)
- Find out how DataOps teams are empowered with event-based triggers for real-time data flow
- See examples of reports, dashboards, and proactive alerts designed to help you reliably keep data flowing through your business — with the observability you require
- Discover how to replace clunky legacy approaches to streaming data in a multi-cloud environment
- See what’s possible with the Stonebranch Universal Automation Center (UAC)
Improving Data Literacy Around Data ArchitectureDATAVERSITY
Data Literacy is an increasing concern, as organizations look to become more data-driven. As the rise of the citizen data scientist and self-service data analytics becomes increasingly common, the need for business users to understand core Data Management fundamentals is more important than ever. At the same time, technical roles need a strong foundation in Data Architecture principles and best practices. Join this webinar to understand the key components of Data Literacy, and practical ways to implement a Data Literacy program in your organization.
The first step towards understanding what data assets mean for your organization is understanding what those assets mean for each other. Metadata—literally, data about data—is one of many data management disciplines inherent in good systems development, and is perhaps the most mislabeled and misunderstood out of the lot. Understanding metadata and its associated technologies as more than just straightforward technological tools can provide powerful insight, the efficiency of organizational practices, and can also enable you to combine more sophisticated data management techniques in support of larger and more complex business initiatives.
In this webinar, we will:
Illustrate how to leverage metadata in support of your business strategy
Discuss foundational metadata concepts based on the DAMA Guide to Data Management Book of Knowledge (DAMA DMBOK)
Enumerate guiding principles for and lessons previously learned from metadata and its practical uses
LDM Webinar: Data Modeling & Metadata ManagementDATAVERSITY
Metadata management is critical for organizations looking to understand the context, definition and lineage of key data assets. Data models play a key role in metadata management, as many of the key structural and business definitions are stored within the models themselves. Can data models replace traditional metadata solutions? Or should they integrate with larger metadata management tools & initiatives? Join this webinar to discuss opportunities and challenges around:
- How data modeling fits within a larger metadata management landscape
- When can data modeling provide “just enough” metadata management
- Key data modeling artifacts for metadata
- Organization, Roles & Implementation Considerations
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...DATAVERSITY
This document summarizes a presentation on self-service data analysis, data wrangling, data munging, and how they fit together with data modeling. It discusses how these techniques allow business stakeholders and data scientists to prepare and transform data for analysis without extensive technical expertise. While these tools increase flexibility, they can also decrease governance if not used properly. The document advocates finding a balance between managed data assets and exploratory analysis to maximize insights while maintaining data quality.
The recent focus on Big Data in the data management community brings with it a paradigm shift—from the more traditional top-down, “design then build” approach to data warehousing and business intelligence, to the more bottom up, “discover and analyze” approach to analytics with Big Data. Where does data modeling fit in this new world of Big Data? Does it go away, or can it evolve to meet the emerging needs of these exciting new technologies? Join this webinar to discuss:
Big Data –A Technical & Cultural Paradigm Shift
Big Data in the Larger Information Management Landscape
Modeling & Technology Considerations
Organizational Considerations
The Role of the Data Architect in the World of Big Data
LDM Webinar: UML for Data Modeling – When Does it Make Sense?DATAVERSITY
When most data architects think of data modeling, they think of Entity-Relationship modeling. But other notations exist for data modeling, and the UML has for many years been used by application developers and enterprise architects to describe data-centric systems. Is the divide simply a cultural one, then, with the ER and UML “camps” choosing sides? Or are there key technological difference to choose one notation over the other? Join our panel of experts to discuss the following topics:
ER vs. UML: When to Use Each
UML for the Business Audience – Pros and Cons
UML for Database Design – Pros and Cons
UML in the Industry: Where It’s Been and where It’s Headed
Real World Use Cases for Data Modeling with UML
LDM Webinar: Data Modeling & Business IntelligenceDATAVERSITY
Business Intelligence (BI) is a valuable way to use information to show the overall health and performance of the organization. At its core is quality, well-structured data that allows for successful reporting and analytics. A data model helps provide both the business definitions as well as the structural optimization needed for successful BI implementations.
Join this webinar to see how a data model underpins business intelligence and analytics in today’s organization.
Lessons in Data Modeling: Data Modeling & MDMDATAVERSITY
Master Data Management (MDM) can create a 360 view of core business assets such as Customer, Product, Vendor, and more. Data modeling is a core component of MDM in both creating the technical integration between disparate systems and, perhaps more importantly, aligning business definitions & rules.
Join this webcast to learn how to effectively apply a data model in your MDM implementation.
Creating Effective Data Visualizations in Excel 2016: Some BasicsShalin Hai-Jew
One of the mainstays of a modern software toolkit is Excel 2016, from Microsoft Office 2016. By reputation, Excel is considered a beginner’s tool that self-respecting data analysts would bypass, but Excel is fairly high-powered, can take up to 1.06 million rows of data per set, contains complex statistical analysis capabilities (without the need for scripting), and enables rich data visualizations. It has a number of rich add-ons to empower different analytical and data visualization functionalities. It works as a great bridging tool to more complex types of statistical analyses.
This session walks participants through some basic built-in data visualizations in Excel 2016, including pie charts and doughnuts, bar charts, tree maps and sunburst diagrams, cluster diagrams, spider (radar) charts, scattergraphs, and others. This session will cover how data structures and desired emphases will determine the options for particular data visualizations.
In this session, participants will
review how to load a data table,
read the general data in a data table (or worksheet),
process or clean the data as needed,
use the Recommended Charts feature,
decide which built-in data visualizations to use, and
consider how to add relevant data visualization elements (including data labels, background grids, axis labels, and titles) for a coherent and effective data visualization.
Also, participants will help co-build data visualizations from open-source and other datasets.
Agile & Data Modeling – How Can They Work Together?DATAVERSITY
A tenet of the Agile Manifesto is ‘Working software over comprehensive documentation’, and many have interpreted that to mean that data models are not necessary in the agile development environment. Others have seen the value of data models for achieving the other core tenets of ‘Customer Collaboration’ and ‘Responding to Change’.
This webinar will discuss how data models are being effectively used in today’s Agile development environment and the benefits that are being achieved from this approach.
Data modeling continues to be a tried-and-true method of managing critical data aspects from both the business and technical perspective. Like any tool or methodology, there is a “right tool for the right job”, and specific model types exist for both business and technical users across operational, reporting, analytic, and other use cases. This webinar will provide an overview of the various data modeling techniques available, and how to use each for maximum value to the organization.
“Opening Pandora’s box” - Why bother data model for ERP systems?
This presentation covers :
a. Why should you bother with data modelling when you’ve got or are planning to get an ERP?
i. For requirements gathering.
ii. For Data migration / take on
iii. Master Data alignment
iv. Data lineage (particularly important with Data Lineage & SoX compliance issues)
v. For reporting (Particularly Business Intelligence & Data Warehousing)
vi. But most importantly, for integration of the ERP metadata into your overall Information Architecture.
b. But don’t you get a data model with the ERP anyway?
i. Errr not with all of them (e.g. SAP) – in fact non of them to our knowledge
ii. What can be leveraged from the vendor?
c. How can you incorporate SAP metadata into your overall model?
i. What are the requirements?
ii. How to get inside the black box
iii. Is there any technology available?
iv. What about DIY?
d. So, what are the overall benefits of doing this:
i. Ease of integration
ii. Fitness for purpose
iii. Reuse of data artefacts
iv. No nasty data surprises
v. Alignment with overall data strategy
Transforming Data Management and Time to Insight with Anzo Smart Data Lake®Cambridge Semantics
The document discusses how Anzo Smart Data Lake can help government agencies transform data management and increase time to insight. It provides an overview of Anzo and how it uses semantic knowledge graphs to link and harmonize diverse data sources for self-service data preparation, discovery, and analytics. Examples are given of how Anzo has helped organizations in intelligence and defense integrate data sources and gain better visibility into areas like contract performance. The presentation concludes by discussing how Anzo could help agencies drive business efficiency, enable more self-service for citizens using public data, and suggests next steps of proof of concept or proposal.
INTRODUCTION TO BIG DATA AND HADOOP
9
Introduction to Big Data, Types of Digital Data, Challenges of conventional systems - Web data, Evolution of analytic processes and tools, Analysis Vs reporting - Big Data Analytics, Introduction to Hadoop - Distributed Computing
Challenges - History of Hadoop, Hadoop Eco System - Use case of Hadoop – Hadoop Distributors – HDFS – Processing Data with Hadoop – Map Reduce.
MySQL JSON Document Store - A Document Store with all the benefits of a Trans...Olivier DASINI
SQL + NoSQL = MySQL
MySQL Document Store allows developers to work with SQL relational tables and schema-less JSON collections. To make that possible MySQL has created the X Dev API which puts a strong focus on CRUD by providing a fluent API allowing you to work with JSON documents in a natural way. The X Protocol is a highly extensible and is optimized for CRUD as well as SQL API operations.
A column oriented database, rather a columnar database is a DBMS (Database Management System) that stores data in columns instead of rows. A columnar database aims to efficiently write and read data to and from hard disk storage to speed up the time to execute a query. A column-store is a physical concept. Here, I primarily focus on what a columnar database is, how it works, its advantages, disadvantages and applications at current times. In due course, the top three market selling columnar databases are discussed with their features. Thus, it is seen that, columnar database is an emerging concept which has high prospect in coming future.
How to Reveal Hidden Relationships in Data and Risk AnalyticsOntotext
Imagine risk analysis manager or compliance officer who can discover easily relationships like this: Big Bucks Café out of Seattle controls My Local Café in NYC through an offshore company. Such discovery can be a game changer if My Local Café pretends to be an independent small enterprise, while recently Big Bucks experiences financial difficulties.
Data Integration is a key part of many of today’s data management challenges: from data warehousing, to MDM, to mergers & acquisitions. Issues can arise not only in trying to align technical formats from various databases and legacy systems, but in trying to achieve common business definitions and rules.
Join this webinar to see how a data model can help with both of these challenges – from ‘bottom-up’ technical integration, to the ‘top-down’ business alignment.
This document discusses different types of data and how they influence data models. It describes structured data as organized in rows and columns that can be easily processed with SQL. Semi-structured data includes XML and JSON and has attributes but no defined schema. Unstructured data, like videos and images, has grown to 90% of all data but does not fit relational databases, leading to new NoSQL database models. The growth of diverse data types directly impacts the development of new data models and database technologies.
Organize & manage master meta data centrally, built upon kong, cassandra, neo4j & elasticsearch. Managing master & meta data is a very common problem with no good opensource alternative as far as I know, so initiating this project – MasterMetaData.
BDVe Webinar Series - Designing Big Data pipelines with Toreador (Ernesto Dam...Big Data Value Association
In the Internet of Everything, huge volumes of multimedia data are generated at very high rates by heterogeneous sources in various formats, such as sensors readings, process logs, structured data from RDBMS, etc. The need of the hour is setting up efficient data pipelines that can compute advanced analytics models on data and use results to customize services, predict future needs or detect anomalies. This Webinar explores the TOREADOR conversational, service-based approach to the easy design of efficient and reusable analytics pipelines to be automatically deployed on a variety of cloud-based execution platforms.
Similar to LDM Slides: Data Modeling for XML and JSON (20)
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...DATAVERSITY
Organizations today need a broad set of enterprise data cloud services with key data functionality to modernize applications and utilize machine learning. They need a comprehensive platform designed to address multi-faceted needs by offering multi-function data management and analytics to solve the enterprise’s most pressing data and analytic challenges in a streamlined fashion.
In this research-based session, I’ll discuss what the components are in multiple modern enterprise analytics stacks (i.e., dedicated compute, storage, data integration, streaming, etc.) and focus on total cost of ownership.
A complete machine learning infrastructure cost for the first modern use case at a midsize to large enterprise will be anywhere from $3 million to $22 million. Get this data point as you take the next steps on your journey into the highest spend and return item for most companies in the next several years.
Data at the Speed of Business with Data Mastering and GovernanceDATAVERSITY
Do you ever wonder how data-driven organizations fuel analytics, improve customer experience, and accelerate business productivity? They are successful by governing and mastering data effectively so they can get trusted data to those who need it faster. Efficient data discovery, mastering and democratization is critical for swiftly linking accurate data with business consumers. When business teams can quickly and easily locate, interpret, trust, and apply data assets to support sound business judgment, it takes less time to see value.
Join data mastering and data governance experts from Informatica—plus a real-world organization empowering trusted data for analytics—for a lively panel discussion. You’ll hear more about how a single cloud-native approach can help global businesses in any economy create more value—faster, more reliably, and with more confidence—by making data management and governance easier to implement.
What is data literacy? Which organizations, and which workers in those organizations, need to be data-literate? There are seemingly hundreds of definitions of data literacy, along with almost as many opinions about how to achieve it.
In a broader perspective, companies must consider whether data literacy is an isolated goal or one component of a broader learning strategy to address skill deficits. How does data literacy compare to other types of skills or “literacy” such as business acumen?
This session will position data literacy in the context of other worker skills as a framework for understanding how and where it fits and how to advocate for its importance.
Uncover how your business can save money and find new revenue streams.
Driving profitability is a top priority for companies globally, especially in uncertain economic times. It's imperative that companies reimagine growth strategies and improve process efficiencies to help cut costs and drive revenue – but how?
By leveraging data-driven strategies layered with artificial intelligence, companies can achieve untapped potential and help their businesses save money and drive profitability.
In this webinar, you'll learn:
- How your company can leverage data and AI to reduce spending and costs
- Ways you can monetize data and AI and uncover new growth strategies
- How different companies have implemented these strategies to achieve cost optimization benefits
Data Catalogs Are the Answer – What is the Question?DATAVERSITY
Organizations with governed metadata made available through their data catalog can answer questions their people have about the organization’s data. These organizations get more value from their data, protect their data better, gain improved ROI from data-centric projects and programs, and have more confidence in their most strategic data.
Join Bob Seiner for this lively webinar where he will talk about the value of a data catalog and how to build the use of the catalog into your stewards’ daily routines. Bob will share how the tool must be positioned for success and viewed as a must-have resource that is a steppingstone and catalyst to governed data across the organization.
Data Catalogs Are the Answer – What Is the Question?DATAVERSITY
Organizations with governed metadata made available through their data catalog can answer questions their people have about the organization’s data. These organizations get more value from their data, protect their data better, gain improved ROI from data-centric projects and programs, and have more confidence in their most strategic data.
Join Bob Seiner for this lively webinar where he will talk about the value of a data catalog and how to build the use of the catalog into your stewards’ daily routines. Bob will share how the tool must be positioned for success and viewed as a must-have resource that is a steppingstone and catalyst to governed data across the organization.
In this webinar, Bob will focus on:
-Selecting the appropriate metadata to govern
-The business and technical value of a data catalog
-Building the catalog into people’s routines
-Positioning the data catalog for success
-Questions the data catalog can answer
Because every organization produces and propagates data as part of their day-to-day operations, data trends are becoming more and more important in the mainstream business world’s consciousness. For many organizations in various industries, though, comprehension of this development begins and ends with buzzwords: “Big Data,” “NoSQL,” “Data Scientist,” and so on. Few realize that all solutions to their business problems, regardless of platform or relevant technology, rely to a critical extent on the data model supporting them. As such, data modeling is not an optional task for an organization’s data effort, but rather a vital activity that facilitates the solutions driving your business. Since quality engineering/architecture work products do not happen accidentally, the more your organization depends on automation, the more important the data models driving the engineering and architecture activities of your organization. This webinar illustrates data modeling as a key activity upon which so much technology and business investment depends.
Specific learning objectives include:
- Understanding what types of challenges require data modeling to be part of the solution
- How automation requires standardization on derivable via data modeling techniques
- Why only a working partnership between data and the business can produce useful outcomes
Analytics play a critical role in supporting strategic business initiatives. Despite the obvious value to analytic professionals of providing the analytics for these initiatives, many executives question the economic return of analytics as well as data lakes, machine learning, master data management, and the like.
Technology professionals need to calculate and present business value in terms business executives can understand. Unfortunately, most IT professionals lack the knowledge required to develop comprehensive cost-benefit analyses and return on investment (ROI) measurements.
This session provides a framework to help technology professionals research, measure, and present the economic value of a proposed or existing analytics initiative, no matter the form that the business benefit arises. The session will provide practical advice about how to calculate ROI and the formulas, and how to collect the necessary information.
How a Semantic Layer Makes Data Mesh Work at ScaleDATAVERSITY
Data Mesh is a trending approach to building a decentralized data architecture by leveraging a domain-oriented, self-service design. However, the pure definition of Data Mesh lacks a center of excellence or central data team and doesn’t address the need for a common approach for sharing data products across teams. The semantic layer is emerging as a key component to supporting a Hub and Spoke style of organizing data teams by introducing data model sharing, collaboration, and distributed ownership controls.
This session will explain how data teams can define common models and definitions with a semantic layer to decentralize analytics product creation using a Hub and Spoke architecture.
Attend this session to learn about:
- The role of a Data Mesh in the modern cloud architecture.
- How a semantic layer can serve as the binding agent to support decentralization.
- How to drive self service with consistency and control.
Enterprise data literacy. A worthy objective? Certainly! A realistic goal? That remains to be seen. As companies consider investing in data literacy education, questions arise about its value and purpose. While the destination – having a data-fluent workforce – is attractive, we wonder how (and if) we can get there.
Kicking off this webinar series, we begin with a panel discussion to explore the landscape of literacy, including expert positions and results from focus groups:
- why it matters,
- what it means,
- what gets in the way,
- who needs it (and how much they need),
- what companies believe it will accomplish.
In this engaging discussion about literacy, we will set the stage for future webinars to answer specific questions and feature successful literacy efforts.
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...DATAVERSITY
Change is hard, especially in response to negative stimuli or what is perceived as negative stimuli. So organizations need to reframe how they think about data privacy, security and governance, treating them as value centers to 1) ensure enterprise data can flow where it needs to, 2) prevent – not just react – to internal and external threats, and 3) comply with data privacy and security regulations.
Working together, these roles can accelerate faster access to approved, relevant and higher quality data – and that means more successful use cases, faster speed to insights, and better business outcomes. However, both new information and tools are required to make the shift from defense to offense, reducing data drama while increasing its value.
Join us for this panel discussion with experts in these fields as they discuss:
- Recent research about where data privacy, security and governance stand
- The most valuable enterprise data use cases
- The common obstacles to data value creation
- New approaches to data privacy, security and governance
- Their advice on how to shift from a reactive to resilient mindset/culture/organization
You’ll be educated, entertained and inspired by this panel and their expertise in using the data trifecta to innovate more often, operate more efficiently, and differentiate more strategically.
Emerging Trends in Data Architecture – What’s the Next Big Thing?DATAVERSITY
With technological innovation and change occurring at an ever-increasing rate, it’s hard to keep track of what’s hype and what can provide practical value for your organization. Join this webinar to see the results of a recent DATAVERSITY survey on emerging trends in Data Architecture, along with practical commentary and advice from industry expert Donna Burbank.
Data Governance Trends - A Look Backwards and ForwardsDATAVERSITY
As DATAVERSITY’s RWDG series hurdles into our 12th year, this webinar takes a quick look behind us, evaluates the present, and predicts the future of Data Governance. Based on webinar numbers, hot Data Governance topics have evolved over the years from policies and best practices, roles and tools, data catalogs and frameworks, to supporting data mesh and fabric, artificial intelligence, virtualization, literacy, and metadata governance.
Join Bob Seiner as he reflects on the past and what has and has not worked, while sharing examples of enterprise successes and struggles. In this webinar, Bob will challenge the audience to stay a step ahead by learning from the past and blazing a new trail into the future of Data Governance.
In this webinar, Bob will focus on:
- Data Governance’s past, present, and future
- How trials and tribulations evolve to success
- Leveraging lessons learned to improve productivity
- The great Data Governance tool explosion
- The future of Data Governance
Data Governance Trends and Best Practices To Implement TodayDATAVERSITY
1) The document discusses best practices for data protection on Google Cloud, including setting data policies, governing access, classifying sensitive data, controlling access, encryption, secure collaboration, and incident response.
2) It provides examples of how to limit access to data and sensitive information, gain visibility into where sensitive data resides, encrypt data with customer-controlled keys, harden workloads, run workloads confidentially, collaborate securely with untrusted parties, and address cloud security incidents.
3) The key recommendations are to protect data at rest and in use through classification, access controls, encryption, confidential computing; securely share data through techniques like secure multi-party computation; and have an incident response plan to quickly address threats.
It is a fascinating, explosive time for enterprise analytics.
It is from the position of analytics leadership that the enterprise mission will be executed and company leadership will emerge. The data professional is absolutely sitting on the performance of the company in this information economy and has an obligation to demonstrate the possibilities and originate the architecture, data, and projects that will deliver analytics. After all, no matter what business you’re in, you’re in the business of analytics.
The coming years will be full of big changes in enterprise analytics and data architecture. William will kick off the fifth year of the Advanced Analytics series with a discussion of the trends winning organizations should build into their plans, expectations, vision, and awareness now.
Too often I hear the question “Can you help me with our data strategy?” Unfortunately, for most, this is the wrong request because it focuses on the least valuable component: the data strategy itself. A more useful request is: “Can you help me apply data strategically?” Yes, at early maturity phases the process of developing strategic thinking about data is more important than the actual product! Trying to write a good (must less perfect) data strategy on the first attempt is generally not productive –particularly given the widespread acceptance of Mike Tyson’s truism: “Everybody has a plan until they get punched in the face.” This program refocuses efforts on learning how to iteratively improve the way data is strategically applied. This will permit data-based strategy components to keep up with agile, evolving organizational strategies. It also contributes to three primary organizational data goals. Learn how to improve the following:
- Your organization’s data
- The way your people use data
- The way your people use data to achieve your organizational strategy
This will help in ways never imagined. Data are your sole non-depletable, non-degradable, durable strategic assets, and they are pervasively shared across every organizational area. Addressing existing challenges programmatically includes overcoming necessary but insufficient prerequisites and developing a disciplined, repeatable means of improving business objectives. This process (based on the theory of constraints) is where the strategic data work really occurs as organizations identify prioritized areas where better assets, literacy, and support (data strategy components) can help an organization better achieve specific strategic objectives. Then the process becomes lather, rinse, and repeat. Several complementary concepts are also covered, including:
- A cohesive argument for why data strategy is necessary for effective data governance
- An overview of prerequisites for effective strategic use of data strategy, as well as common pitfalls
- A repeatable process for identifying and removing data constraints
- The importance of balancing business operation and innovation
Who Should Own Data Governance – IT or Business?DATAVERSITY
The question is asked all the time: “What part of the organization should own your Data Governance program?” The typical answers are “the business” and “IT (information technology).” Another answer to that question is “Yes.” The program must be owned and reside somewhere in the organization. You may ask yourself if there is a correct answer to the question.
Join this new RWDG webinar with Bob Seiner where Bob will answer the question that is the title of this webinar. Determining ownership of Data Governance is a vital first step. Figuring out the appropriate part of the organization to manage the program is an important second step. This webinar will help you address these questions and more.
In this session Bob will share:
- What is meant by “the business” when it comes to owning Data Governance
- Why some people say that Data Governance in IT is destined to fail
- Examples of IT positioned Data Governance success
- Considerations for answering the question in your organization
- The final answer to the question of who should own Data Governance
This document summarizes a research study that assessed the data management practices of 175 organizations between 2000-2006. The study had both descriptive and self-improvement goals, such as understanding the range of practices and determining areas for improvement. Researchers used a structured interview process to evaluate organizations across six data management processes based on a 5-level maturity model. The results provided insights into an organization's practices and a roadmap for enhancing data management.
MLOps – Applying DevOps to Competitive AdvantageDATAVERSITY
MLOps is a practice for collaboration between Data Science and operations to manage the production machine learning (ML) lifecycles. As an amalgamation of “machine learning” and “operations,” MLOps applies DevOps principles to ML delivery, enabling the delivery of ML-based innovation at scale to result in:
Faster time to market of ML-based solutions
More rapid rate of experimentation, driving innovation
Assurance of quality, trustworthiness, and ethical AI
MLOps is essential for scaling ML. Without it, enterprises risk struggling with costly overhead and stalled progress. Several vendors have emerged with offerings to support MLOps: the major offerings are Microsoft Azure ML and Google Vertex AI. We looked at these offerings from the perspective of enterprise features and time-to-value.
Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...DATAVERSITY
This document discusses the importance of data observability for improving data quality. It begins with an introduction to data observability and how it works by continuously monitoring data to detect anomalies and issues. This is unlike traditional reactive approaches. Examples are then provided of how unexpected data values or volumes could negatively impact downstream processes but be resolved quicker with data observability alerts. The document emphasizes that data observability allows issues to be identified and addressed before they become costly problems. It promotes data observability as a way to proactively improve data integrity and ensure accurate, consistent data for confident decision making.
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
AppSec PNW: Android and iOS Application Security with MobSFAjin Abraham
Mobile Security Framework - MobSF is a free and open source automated mobile application security testing environment designed to help security engineers, researchers, developers, and penetration testers to identify security vulnerabilities, malicious behaviours and privacy concerns in mobile applications using static and dynamic analysis. It supports all the popular mobile application binaries and source code formats built for Android and iOS devices. In addition to automated security assessment, it also offers an interactive testing environment to build and execute scenario based test/fuzz cases against the application.
This talk covers:
Using MobSF for static analysis of mobile applications.
Interactive dynamic security assessment of Android and iOS applications.
Solving Mobile app CTF challenges.
Reverse engineering and runtime analysis of Mobile malware.
How to shift left and integrate MobSF/mobsfscan SAST and DAST in your build pipeline.
"Choosing proper type of scaling", Olena SyrotaFwdays
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
"Scaling RAG Applications to serve millions of users", Kevin GoedeckeFwdays
How we managed to grow and scale a RAG application from zero to thousands of users in 7 months. Lessons from technical challenges around managing high load for LLMs, RAGs and Vector databases.
Discover the Unseen: Tailored Recommendation of Unwatched ContentScyllaDB
The session shares how JioCinema approaches ""watch discounting."" This capability ensures that if a user watched a certain amount of a show/movie, the platform no longer recommends that particular content to the user. Flawless operation of this feature promotes the discover of new content, improving the overall user experience.
JioCinema is an Indian over-the-top media streaming service owned by Viacom18.
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...DanBrown980551
This LF Energy webinar took place June 20, 2024. It featured:
-Alex Thornton, LF Energy
-Hallie Cramer, Google
-Daniel Roesler, UtilityAPI
-Henry Richardson, WattTime
In response to the urgency and scale required to effectively address climate change, open source solutions offer significant potential for driving innovation and progress. Currently, there is a growing demand for standardization and interoperability in energy data and modeling. Open source standards and specifications within the energy sector can also alleviate challenges associated with data fragmentation, transparency, and accessibility. At the same time, it is crucial to consider privacy and security concerns throughout the development of open source platforms.
This webinar will delve into the motivations behind establishing LF Energy’s Carbon Data Specification Consortium. It will provide an overview of the draft specifications and the ongoing progress made by the respective working groups.
Three primary specifications will be discussed:
-Discovery and client registration, emphasizing transparent processes and secure and private access
-Customer data, centering around customer tariffs, bills, energy usage, and full consumption disclosure
-Power systems data, focusing on grid data, inclusive of transmission and distribution networks, generation, intergrid power flows, and market settlement data
Must Know Postgres Extension for DBA and Developer during MigrationMydbops
Mydbops Opensource Database Meetup 16
Topic: Must-Know PostgreSQL Extensions for Developers and DBAs During Migration
Speaker: Deepak Mahto, Founder of DataCloudGaze Consulting
Date & Time: 8th June | 10 AM - 1 PM IST
Venue: Bangalore International Centre, Bangalore
Abstract: Discover how PostgreSQL extensions can be your secret weapon! This talk explores how key extensions enhance database capabilities and streamline the migration process for users moving from other relational databases like Oracle.
Key Takeaways:
* Learn about crucial extensions like oracle_fdw, pgtt, and pg_audit that ease migration complexities.
* Gain valuable strategies for implementing these extensions in PostgreSQL to achieve license freedom.
* Discover how these key extensions can empower both developers and DBAs during the migration process.
* Don't miss this chance to gain practical knowledge from an industry expert and stay updated on the latest open-source database trends.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: info@mydbops.com
Visit: https://www.mydbops.com/
Follow us on LinkedIn: https://in.linkedin.com/company/mydbops
For more details and updates, please follow up the below links.
Meetup Page : https://www.meetup.com/mydbops-databa...
Twitter: https://twitter.com/mydbopsofficial
Blogs: https://www.mydbops.com/blog/
Facebook(Meta): https://www.facebook.com/mydbops/
This talk will cover ScyllaDB Architecture from the cluster-level view and zoom in on data distribution and internal node architecture. In the process, we will learn the secret sauce used to get ScyllaDB's high availability and superior performance. We will also touch on the upcoming changes to ScyllaDB architecture, moving to strongly consistent metadata and tablets.
Getting the Most Out of ScyllaDB Monitoring: ShareChat's TipsScyllaDB
ScyllaDB monitoring provides a lot of useful information. But sometimes it’s not easy to find the root of the problem if something is wrong or even estimate the remaining capacity by the load on the cluster. This talk shares our team's practical tips on: 1) How to find the root of the problem by metrics if ScyllaDB is slow 2) How to interpret the load and plan capacity for the future 3) Compaction strategies and how to choose the right one 4) Important metrics which aren’t available in the default monitoring setup.
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving
What began over 115 years ago as a supplier of precision gauges to the automotive industry has evolved into being an industry leader in the manufacture of product branding, automotive cockpit trim and decorative appliance trim. Value-added services include in-house Design, Engineering, Program Management, Test Lab and Tool Shops.
What is an RPA CoE? Session 2 – CoE RolesDianaGray10
In this session, we will review the players involved in the CoE and how each role impacts opportunities.
Topics covered:
• What roles are essential?
• What place in the automation journey does each role play?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
Essentials of Automations: Exploring Attributes & Automation ParametersSafe Software
Building automations in FME Flow can save time, money, and help businesses scale by eliminating data silos and providing data to stakeholders in real-time. One essential component to orchestrating complex automations is the use of attributes & automation parameters (both formerly known as “keys”). In fact, it’s unlikely you’ll ever build an Automation without using these components, but what exactly are they?
Attributes & automation parameters enable the automation author to pass data values from one automation component to the next. During this webinar, our FME Flow Specialists will cover leveraging the three types of these output attributes & parameters in FME Flow: Event, Custom, and Automation. As a bonus, they’ll also be making use of the Split-Merge Block functionality.
You’ll leave this webinar with a better understanding of how to maximize the potential of automations by making use of attributes & automation parameters, with the ultimate goal of setting your enterprise integration workflows up on autopilot.
JavaLand 2024: Application Development Green Masterplan
LDM Slides: Data Modeling for XML and JSON
1. Data Modeling for XML & JSON
Donna Burbank
Global Data Strategy Ltd.
Lessons in Data Modeling DATAVERSITY Series
Dec 6th, 2016
2. Global Data Strategy, Ltd. 2016
Donna is a recognized industry expert in
information management with over 20
years of experience in data strategy,
information management, data modeling,
metadata management, and enterprise
architecture.
She is currently the Managing Director at
Global Data Strategy, Ltd., an international
information management consulting
company that specialises in the alignment
of business drivers with data-centric
technology. In past roles, she has served in
a number of roles related to data modeling
& metadata:
• Metadata consultant (US, Europe, Asia,
Africa)
• Product Manager PLATINUM Metadata
Repository
• Director of Product Management,
ER/Studio
• VP of Product Marketing, Erwin
• Data modeling & data strategy
implementation & consulting
• Author of 2 books of data modeling &
contributor to 1 book on metadata
management, plus numerous articles
• OMG committee member of the
Information Management Metamodel
(IMM)
As an active contributor to the data
management community, she is a long
time DAMA International member and is
the President of the DAMA Rocky
Mountain chapter. She has worked with
dozens of Fortune 500 companies
worldwide in the Americas, Europe, Asia,
and Africa and speaks regularly at industry
conferences. She has co-authored two
books: Data Modeling for the
Business and Data Modeling Made Simple
with ERwin Data Modeler and is a regular
contributor to industry publications such
as DATAVERSITY, EM360, & TDAN. She can
be reached at
donna.burbank@globaldatastrategy.com
Donna is based in Boulder, Colorado, USA.
Donna Burbank
2
Follow on Twitter @donnaburbank
Today’s hashtag: #LessonsDM
3. Global Data Strategy, Ltd. 2016
Lessons in Data Modeling Series
• July 28th Why a Data Model is an Important Part of your Data Strategy
• August 25th Data Modeling for Big Data
• September 22nd UML for Data Modeling – When Does it Make Sense?
• October 27th Data Modeling & Metadata Management
• December 6th Data Modeling for XML and JSON
3
This Year’s Line Up
4. Global Data Strategy, Ltd. 2016
Agenda
• Overview of XML and JSON
• Data Modeling & Metadata for XML & JSON
• Integrating XML & JSON with Databases (Relational & NoSQL)
• RDF & the Semantic Web
• Summary & Questions
4
What we’ll cover today
5. Global Data Strategy, Ltd. 2016
Assumption
• An assumption for today is that the majority of attendees are familiar with relational databases &
Entity-Relationship (E/R) modeling.
• E.g. Data Modelers, Data Architects, SQL Developers, BI Developers, etc.
• The examples are given with that bias, i.e. a comparison with the relational database world.
5
From Data Modeling for the Business by
Hoberman, Burbank, Bradley, Technics
Publications, 2009
6. Global Data Strategy, Ltd. 2016
What is XML?
• What is XML? – (Extensible Markup Language) is used to store and transport data.
• Some design principles of XML:
• Simplicity: ease of usage, interoperability & understanding
• Modular design: do one thing well
• Extensible: Ability to easily modify the structure & content
• Self-descriptive: ease of understanding
• Machine readable
• Human readable
• Embedded descriptive tags
• XML is designed for data availability, sharing & transport.
• It requires complementary technology to do anything else. i.e. Someone must write a piece of
software to send, receive, store, or display it, for example:
• HTML: Format & presentation of the data
• Web Service: Transport of the data (e.g. SOAP)
• Database: Store & integrate with other data sources
6
7. Global Data Strategy, Ltd. 2016
XML and JSON Assist with Data Exchange
7
• XML and JSON can be used to assist with data exchange (B2B, B2C, etc.)
• Companies
• Government Agencies
• Research Organizations
• Etc.
Purchase Order
8. Global Data Strategy, Ltd. 2016
Emergence & the Growth of Data Exchange
In philosophy, systems theory, science, and art, emergence is
the way complex systems and patterns arise out of a
multiplicity of relatively simple interactions.
- Wikipedia
9. Global Data Strategy, Ltd. 2016
XML uses a Hierarchical Structure
• XML uses a hierarchical, nested tree structure
• An XML tree starts at a root element and branches from the root to child elements.
• All elements can have sub elements (child elements)
9
<?xml version="1.0"?>
<shipto>
<name>John Smith</name>
<address>123 Main ST</address>
<city>Boise</city>
<country>USA</country>
</shipto>
Root
element
Child
elements
10. Global Data Strategy, Ltd. 2016
XML is Extensible
• XML is extensible, in that element can be easily added as needed.
• If the <state> element is added below, older applications using the original version will still work.
10
<?xml version="1.0"?>
<shipto>
<name>John Smith</name>
<address>123 Main ST</address>
<city>Boise</city>
<country>USA</country>
</shipto>
<?xml version="1.0"?>
<shipto>
<name>John Smith</name>
<address>123 Main ST</address>
<city>Boise</city>
<state>ID</state>
<country>USA</country>
</shipto>
11. Global Data Strategy, Ltd. 2016
XML is Self-Describing
• XML is self-describing (sort of) with the use of element tags
• Human-readable format
• Tags describe the content of the element (sort of)
11
<?xml version="1.0"?>
<shipto>
<name>John Smith</name>
<address>123 Main ST</address>
<city>Boise</city>
<country>USA</country>
</shipto>
From reading the tags, it’s
pretty clear that we’re
talking about a “Ship To”
address that contains the
name, address, city &
country.
But it doesn’t provide full metadata, e.g.:
• What’s the data type?
• What’s the business definition?
• Is <name> a required field?
12. Global Data Strategy, Ltd. 2016
XML Metadata – the XML Schema
• Similar to DDL, an XML Schema (XSD) defines the structure & format of data
12
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="shiporder">
<xs:complexType>
<xs:sequence>
<xs:element name="orderperson" type="xs:string"/>
<xs:element name="shipto">
<xs:complexType>
<xs:sequence>
<xs:element name="name" type="xs:string"/>
<xs:element name="address" type="xs:string"/>
<xs:element name="city" type="xs:string"/>
<xs:element name="country" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="orderid" type="xs:string" use="required"/>
</xs:complexType>
</xs:element>
</xs:schema> XSD
Metadata
Ship to:
John Smith
123 Main ST
Boise
USA
………………………………………
………………………………………
Order Shipment
Data
<?xml version="1.0"?>
<shipto>
<name>John Smith</name>
<address>123 Main ST</address>
<city>Boise</city>
<country>USA</country>
</shipto>
XML
Data
13. Global Data Strategy, Ltd. 2016
Graphical Models of XML Schemas
13
• XML Schemas can be shown graphically as well as via text.
* Source: Altova
14. Global Data Strategy, Ltd. 2016
XML Metadata – the XML Schema
• Although the XML Schema does provide some physical structural metadata, full metadata
descriptions are incomplete, e.g.
• Is the name field required?
• What’s the business definition for each field?
• Are there code values and/or reference data that can be used?
• Can a complex data type be used?
• Etc.
14
15. Global Data Strategy, Ltd. 2016
Levels of Data Modeling
15
Conceptual
Logical
Physical
Purpose
Communication & Definition of
Business Terms & Rules
Clarification & Detail
of Business Rules &
Data Structures
Technical Implementation
with a Physical Database
or Structure
Audience
Business Stakeholders
Data Architecture
Business Analysts
DBAs
Developers
Business Concepts
Data Entities
Physical Tables
XML Schema defines some physical
metadata
But limited or no business metadata
16. Global Data Strategy, Ltd. 2016
Metadata & Context
From Data Modeling for the Business by Hoberman, Burbank,
Bradley, Technics Publications, 2009
Is this Customer a:
• Premier Customer
• Lapsed Customer
• High Risk Customer?
Can a Customer have
more than one Account?
Is the Ship To Address
related to the Customer
or the Account?
What are the valid state
codes for the Ship To
Address?
17. Global Data Strategy, Ltd. 2016
XML Assists with Data Exchange
17
• XML and JSON can be used to assist with data exchange (B2B, B2C, etc.)
• Remember modularity, simplicity, etc.
Purchase Order
Dude-all that other stuff
isn’t my job. I’m just
sending the PO!
18. Global Data Strategy, Ltd. 2016
Integrating XML with Relational Databases
• XML is often used in conjunction with relational databases for permanent storage and integration
with other operational, reporting, and reference data.
18
Purchase Order
Oracle SQL Server
19. Global Data Strategy, Ltd. 2016
Integrating XML with Relational Databases
• XML can be translated into relational databases, and vice-versa
19
XML Schema DDL
* Source: Altova
20. Global Data Strategy, Ltd. 2016
Integrating XML with Relational Databases
20
• XML can be translated into relational databases, and vice-versa
XML Model Diagram Relational Model Diagram
* Source: Altova
21. Global Data Strategy, Ltd. 2016
What is JSON?
• What is JSON? – (JavaScript Object Notation) is a minimal, readable format for structuring data. It
is used primarily to transmit data between a server and web application, as an alternative to XML.
• It is similar to XML in that it is:
• "self describing" & human readable
• hierarchical
• simple & interoperable
21
• It differs from XML in that it is:
• can be parsed with standard JavaScript notation
• uses arrays
• can be simpler & shorter to read & write.
{"employees":[
{"firstName":“Shannon", "lastName":“Kempe"},
{"firstName":"Anita", "lastName":“Kress"},
{"firstName":“Tony", "lastName":“Shaw"}
]}
<employees>
<employee>
<firstName>Shannon</firstName>
<lastName>Kempe</lastName>
</employee>
<employee>
<firstName>Anita</firstName>
<lastName>Kress</lastName>
</employee>
<employee>
<firstName>Tony</firstName>
<lastName>Shaw</lastName>
</employee>
</employees>
JSON XML
22. Global Data Strategy, Ltd. 2016
JSON Metadata – The JSON Schema
22
• The JSON schema offers a richer set of metadata.
{
"id": 127849,
“brand": “Super Cooler",
"price": 12.50,
"tags": [“camping", “sports"]
}
Example Product in the API
Data
• Can the ID contain letters?
• What is a brand?
• Is a price required?
• Etc.
Context Needed
(i.e. Metadata)
For example, assume we have a JSON based product catalog. This catalog has a product which has an id, a brand,
a price, and an optional set of tags.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Product",
"description": "A retail product from Acme's online catalog",
"type": "object",
"properties": {
"id": {
"description": "The unique identifier for a product",
"type": "integer"
},
“brand": {
"description": “The brand name of the product as shown in the online catalogue",
"type": "string"
},
"price": {
"type": "number",
},
"tags": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
}
},
"required": ["id", “brand", "price"]
}
JSON Schema
Metadata
23. Global Data Strategy, Ltd. 2016
Integrating JSON with Document Databases
• JSON is often used with document databases, such as MongoDB, which uses JSON documents in
order to store records
• Document databases are popular ways to store unstructured information in a flexible way (e.g.
multimedia, social media posts, etc. )
23
• Each Collection can contain numerous Documents which could all contain
different fields.
{type: “Artifact”,
medium: “Ceramic”
country: “China”,
}
{type: “Book”,
title: “Ancient China”
country: “China”,
}
24. Global Data Strategy, Ltd. 2016
The Semantic Web & RDF
• The RDF (Resource Description Framework) model from the World Wide Web Consortium (W3C) provides a
way to link resources on the web (people, places, things). It provides a common framework for applications to
share information without losing meaning.
• Search Engines
• Exchanging data between datasets
• Sharing information with applications / APIs
• Building social networks
• Etc.
• The goal is to move from a web of documents to a web of data.
• The Framework is a simple way to express relationships between resources.
• IRIs (International Resource Identifiers) (e.g. URI) identify resources
• Simple triples relate objects together in the format: <subject> <predicate> <object>
• These relationships create a connected Graph
• There are several serialization formats, with RDF XML being a common one. For example:
• Turtle is a human-friendly format
• RDF/XML
• JSON-LD
• Schemas define the vocabularies used to describe the objects
• Dublin Core and Schema.org are two common ones
24
Subject Object
Predicate
ACME
Publishing
RDF is
Easy
Is Publisher Of
25. Global Data Strategy, Ltd. 2016
Creating a Web of Data
25
@type: Place
Sheraton San Diego Hotel & Marina
1380 Harbor Island Drive
San Diego, California 92101 USA
"@context": "http://schema.org",
“location": {
"@type": "Place",
"name": "Sheraton San Diego Hotel & Marina",
"address": {
"@type": "PostalAddress",
"streetAddress": "1380 Harbor Island Drive",
"addressLocality": "San Diego",
"addressRegion": "CA",
"postalCode": "92101"
},
"telephone" : "+1-877-734-2726",
"image":
"http://edw2016.dataversity.net/uploads/ConfSiteAssets/72/im
age/sheraton.jpg",
"url":"http://edw2016.dataversity.net/travel.cfm"
},
"@context": "http://schema.org",
"location": {
"@type": "Place",
"name": "Sheraton San Diego Hotel & Marina",
"address": {
"@type": "PostalAddress",
"streetAddress": "1380 Harbor Island Drive",
"addressLocality": "San Diego",
"addressRegion": "CA",
"postalCode": "92101"
},
"telephone" : "+1-877-734-2726",
"image": “http://mysite.com/edw16photo.jpg",
"url":“http://mysite.com/myphotos"
},
* Script provided by: Eric Franzon, eric@smartdataconsultants.com
*
26. Global Data Strategy, Ltd. 2016
Dublin Core Metadata Initiative
• The Dublin Core Metadata Initiative provides a common metadata standards for resources such as
media, library books, etc.
• It defines standards for information such as:
26
http://dublincore.org
Title
Creator
Subject
Description
Publisher
Contributor
Date
Type
Format
Identifier
Source
Language
Relation
Coverage
Rights
Resources can be described using:
Text
HTML
XML
RDF XML
Sample Metadata
Format="video/mpeg; 5 minutes“
Language="en"
Publisher=“Kats Online, LLC"
Title=“My Favorite Cat Video“
Subject=“Cats“
Description=“A short video of a black cat playing with string."
27. Global Data Strategy, Ltd. 2016
Schema.org
• Schema.org is a vocabulary that webmasters can use to mark-up Web pages for the Semantic
Web, so that search engines understand what the pages are about .
• Created by a group of search providers (e.g. Google, Microsoft, Yahoo and Yandex).
• Vocabularies are developed by an open community process
• Through GitHub (https://github.com/schemaorg/schemaorg)
• Using the public-schemaorg@w3.org mailing list
• The schemas are a set of 'types', each associated with a set of properties. The types are arranged
in a hierarchy. There are currently over 570 types, including:
• Creative works
• Organization
• Person
• Place, LocalBusiness, Restaurant
• Product, Offer, AggregateOffer
• Etc.
• There are also extensions for particular industries such as:
• auto.schema.org
• health-lifesci.schema.org
27
Resources can be described using:
JSON-LD
RDFa
Etc.
28. Global Data Strategy, Ltd. 2016
There are Many Other Common Schemas & Vocabularies
• The Dublin Core and Schema.org are two popular schemas, but many more exist for particular
subject areas, industries, etc.
• The Linked Open Vocabularies site (LOV) provides a helpful listing
28http://lov.okfn.org/dataset/lov/
Dublin Core
Schema.org
Friend of a Friend
29. Global Data Strategy, Ltd. 2016
Summary
• XML and JSON are used for transport and interoperability of data
• They offer a variety of benefits
• Simplicity: ease of usage, interoperability & understanding
• Modular design: do one thing well
• Extensible: Ability to easily modify the structure & content
• Self-descriptive: ease of understanding
• Integration with Databases allows for broader enterprise sharing & storage
• Translation to Relational databases
• Storage for Document databases
• Graphical Models can be used across technologies for an intuitive way to visualize hierarchies &
relationships
• The Semantic Web is a powerful way to support the internet as a “web of data”
30. Global Data Strategy, Ltd. 2016
About Global Data Strategy, Ltd
• Global Data Strategy is an international information management consulting company that specializes
in the alignment of business drivers with data-centric technology.
• Our passion is data, and helping organizations enrich their business opportunities through data and
information.
• Our core values center around providing solutions that are:
• Business-Driven: We put the needs of your business first, before we look at any technology solution.
• Clear & Relevant: We provide clear explanations using real-world examples.
• Customized & Right-Sized: Our implementations are based on the unique needs of your organization’s
size, corporate culture, and geography.
• High Quality & Technically Precise: We pride ourselves in excellence of execution, with years of
technical expertise in the industry.
30
Data-Driven Business Transformation
Business Strategy
Aligned With
Data Strategy
Visit www.globaldatastrategy.com for more information
31. Global Data Strategy, Ltd. 2016
Contact Info
• Email: donna.burbank@globaldatastrategy.com
• Twitter: @donnaburbank
@GlobalDataStrat
• Website: www.globaldatastrategy.com
• Company Linkedin: https://www.linkedin.com/company/global-data-strategy-ltd
• Personal Linkedin: https://www.linkedin.com/in/donnaburbank
31
32. Global Data Strategy, Ltd. 2016
DATAVERSITY Training Center
• Learn the basics of Metadata Management and practical tips on how to apply metadata
management in the real world. This online course hosted by DATAVERSITY provides a series of six
courses including:
• What is Metadata
• The Business Value of Metadata
• Sources of Metadata
• Metamodels and Metadata Standards
• Metadata Architecture, Integration, and Storage
• Metadata Strategy and Implementation
• Purchase all six courses for $399 or individually at $79 each.
Register here
• Other courses available on Data Governance & Data Quality
32
Online Training Courses
New Metadata Management Course
Visit: http://training.dataversity.net/lms/
33. Global Data Strategy, Ltd. 2016
Lessons in Data Modeling Series - 2017
• January 26th How Data Modeling Fits into an Overall Enterprise Architecture
• February 23rd Data Modeling & Business Intelligence
• March 23rd Conceptual Data Models - How to Get the Attention of Business Users
(for a Technical Audience)
• April 27th The Evolving Role of the Data Architect – What Does it Mean for Your Career?
• May 25th Data Modeling & Metadata Management
• June 22nd Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –
how do they fit together?
• July 27th Data Modeling & Metadata for Graph Databases
• August 24th Data Modeling & Data Integration
• September 28th Data Modeling & MDM
• October 26th Agile & Data Modeling – How can they work together?
• December 5th Data Modeling, Data Governance, & Data Quality
33
Next Year’s Line Up