SlideShare a Scribd company logo
Let’s talk dataIntegration, Interoperability and Virtualization
Presented by:
javier@enroutesystems.com
art@enroutesystems.com
8/8/17
DII – The new kid in the
block
Data Integration and Interoperability (DII) describes processes
related to the movement and consolidation of data within
and between data stores, applications and organizations.
Why we’re geeking out for DII
1. SOA/Microservices are becoming more popular.
2. Integration of structured and unstructured data
3. Deliver value faster… avoid ROGUE users
4. Although it’s not new, DII in DMBOK provides clear
guidelines to organizations aiming to become more efficient
through IT.
Data Interoperability
Data Interoperability is the ability for multiple systems to communicate.
Monolyt
hs
SOA/Microservi
ces
Data Integration
Integration consolidates data into consistent forms, either physical or
virtual.
History
Data virtualization exists
since Bill Inmon
popularized data
warehouse in the 1990s.
But virtual models back
then were not very
popular due to the lack
of computer power
available (or
accessible).
Today, change in data
types and business
expectations on
information velocity have
made virtualization a
more popular concept.
Did you know?
Last time Bill Inmon wrote about
Data virtualization he compared
it to a frustrating whack a mole
game, where no matter how
much you hit the mole… it
keeps coming back!
http://www.b-eye-
network.com/view/9956
Click here
Let’s take a look at your future
Virtual vs Physical
Why data virtualization?
Fast and Easy
• Rapid data integration which enables a faster time to solution
• Integrations and changes are easy (No need to update Extractions, tables, DataMart's)
Integrate more!
• Opportunity to integrate structured and unstructured data
Cheaper and more secure
• Less expensive to maintain
• No need to replicate data
• Reduces overhead of management of data integration systems (Easier + Faster = Less
reqauired resources)
Agile
• Enables iterative development with quick deliverables (Note: very important one since in most
cases, users don’t know what they want… too many iterations)
• Developers are more focused on business instead of understanding the mechanics of data
manipulation (Why? Because Data virtualization tools automatically connect to many data
sources )
Use Cases
Data Warehouse augmentation
Problem
• Bringing in new data sources to a data warehouse
takes a significant amount of effort, but even more
so, if the data sources include unstructured data.
Fix
• Data virtualization can be applied to augment
existing data warehouse with virtual views that
incorporate unstructured data.
Support ETL process
Problem
• It is sometimes too complicated to access web
services data, extract it and make it part of the ETL,
specially if you need to develop access methods for
external or new types of data.
Fix
• Data virtualization tools have access methods which
can be used to easily extract data from web
services, pre-process this data and have it ready to
Data Warehouse Federation/Canonical
Problem
• Some organizations have multiple separate data
warehouses which may take too much effort to
integrate.
Fix
• Data virtualization allows to quickly generate
federated views of all these data warehouses and
integrate this data for different services. Individual
warehouses continue to operate with no
interruptions. (Same thing for DWH migrations!)
Use Cases
Data Warehouse prototyping
Problem
• Organizations are moving to agile development,
where iterations and short term sprints are key to
delivering value on a weekly ot bi weekly basis.
Fix
• When data prototypes are built fast and are
validated by users, this then generates a proven
product that can then be materialized saving time
and therefore money.
Data Mashups
Problem
• Web mashups are enabled by APIs and most
corporate data sources do not have accessible APIs
to support this mashup process.
Fix
• Data virtualization tools are enables of mashups
since they use same protocols and data delivery
formats as APIs.
Master Data on Steroids– Past, present and future
data
Problem
• Master Data Hubs traditionally only hold identity and
descriptive information, but transactional data is
usually not stored in MDHs.
Fix
• With data virtualization, you could make a canonical
layer where you would input data from the MDH and
other sources and enrich master data with
summarized transactional data. (E.g. adding value
of customer over time, purchasing forecast etc…)
So is ETL going away?
This does not mean ETL is not needed, its more around identifying when ETL is not
enough, and use virtualization to enhance Data integration! When ETL is too slow, or
data sources are difficult to access or data types are challenging.
Maybe in the future it’ll be the other way around, where we’ll look at ETL for cases
when data virtualization is not enough. For instance, when you need to perform highly
complex transformations that could impact performance in a virtual database.
Today, It is common to virtualize in development and materialize in production.
Misconceptions
1. VDB DOES NOT replace a DWH. VDB enhances DWH by:
• Combine structured and unstructured data into a single data layer

More Related Content

What's hot

DataOps - The Foundation for Your Agile Data Architecture
DataOps - The Foundation for Your Agile Data ArchitectureDataOps - The Foundation for Your Agile Data Architecture
DataOps - The Foundation for Your Agile Data Architecture
DATAVERSITY
 
Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...
Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...
Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...Leon Kappelman
 
Introduction to open data in DataOps
Introduction to open data in DataOpsIntroduction to open data in DataOps
Introduction to open data in DataOps
Dataops Ghent Meetup
 
Data-Ed Online: Data Architecture Requirements
Data-Ed Online: Data Architecture RequirementsData-Ed Online: Data Architecture Requirements
Data-Ed Online: Data Architecture Requirements
DATAVERSITY
 
Approaching Data Quality
Approaching Data QualityApproaching Data Quality
Approaching Data Quality
DATAVERSITY
 
Emerging Trends in Data Architecture – What’s the Next Big Thing
Emerging Trends in Data Architecture – What’s the Next Big ThingEmerging Trends in Data Architecture – What’s the Next Big Thing
Emerging Trends in Data Architecture – What’s the Next Big Thing
DATAVERSITY
 
Data-Ed Webinar: Data Modeling Fundamentals
Data-Ed Webinar: Data Modeling FundamentalsData-Ed Webinar: Data Modeling Fundamentals
Data-Ed Webinar: Data Modeling Fundamentals
DATAVERSITY
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?
DATAVERSITY
 
Data Architecture Process in a BI environment
Data Architecture Process in a BI environmentData Architecture Process in a BI environment
Data Architecture Process in a BI environmentSasha Citino
 
Data Strategy Best Practices
Data Strategy Best PracticesData Strategy Best Practices
Data Strategy Best Practices
DATAVERSITY
 
How Enterprises are Using NoSQL for Mission-Critical Applications
How Enterprises are Using NoSQL for Mission-Critical ApplicationsHow Enterprises are Using NoSQL for Mission-Critical Applications
How Enterprises are Using NoSQL for Mission-Critical Applications
DATAVERSITY
 
Building the Modern Data Hub
Building the Modern Data HubBuilding the Modern Data Hub
Building the Modern Data Hub
Datavail
 
Showing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectShowing ROI for Your Analytic Project
Showing ROI for Your Analytic Project
DATAVERSITY
 
Information & Data Architecture
Information & Data ArchitectureInformation & Data Architecture
Information & Data Architecture
Sammer Qader
 
Building a Collaborative Data Architecture
Building a Collaborative Data ArchitectureBuilding a Collaborative Data Architecture
Building a Collaborative Data Architecture
DATAVERSITY
 
Information Architecture Deliverables
Information Architecture DeliverablesInformation Architecture Deliverables
Information Architecture Deliverables
Dushyant Kanungo
 
Slides: How AI Makes Analytics More Human
Slides: How AI Makes Analytics More HumanSlides: How AI Makes Analytics More Human
Slides: How AI Makes Analytics More Human
DATAVERSITY
 
Fasten you seatbelt and listen to the Data Steward
Fasten you seatbelt and listen to the Data StewardFasten you seatbelt and listen to the Data Steward
Fasten you seatbelt and listen to the Data Steward
Jean-Pierre Riehl
 
DataEd Slides: Data Management + Data Strategy = Interoperability
DataEd Slides: Data Management + Data Strategy = InteroperabilityDataEd Slides: Data Management + Data Strategy = Interoperability
DataEd Slides: Data Management + Data Strategy = Interoperability
DATAVERSITY
 
DataEd Online: Unlock Business Value through Data Governance
DataEd Online: Unlock Business Value through Data GovernanceDataEd Online: Unlock Business Value through Data Governance
DataEd Online: Unlock Business Value through Data GovernanceDATAVERSITY
 

What's hot (20)

DataOps - The Foundation for Your Agile Data Architecture
DataOps - The Foundation for Your Agile Data ArchitectureDataOps - The Foundation for Your Agile Data Architecture
DataOps - The Foundation for Your Agile Data Architecture
 
Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...
Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...
Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Ho...
 
Introduction to open data in DataOps
Introduction to open data in DataOpsIntroduction to open data in DataOps
Introduction to open data in DataOps
 
Data-Ed Online: Data Architecture Requirements
Data-Ed Online: Data Architecture RequirementsData-Ed Online: Data Architecture Requirements
Data-Ed Online: Data Architecture Requirements
 
Approaching Data Quality
Approaching Data QualityApproaching Data Quality
Approaching Data Quality
 
Emerging Trends in Data Architecture – What’s the Next Big Thing
Emerging Trends in Data Architecture – What’s the Next Big ThingEmerging Trends in Data Architecture – What’s the Next Big Thing
Emerging Trends in Data Architecture – What’s the Next Big Thing
 
Data-Ed Webinar: Data Modeling Fundamentals
Data-Ed Webinar: Data Modeling FundamentalsData-Ed Webinar: Data Modeling Fundamentals
Data-Ed Webinar: Data Modeling Fundamentals
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?
 
Data Architecture Process in a BI environment
Data Architecture Process in a BI environmentData Architecture Process in a BI environment
Data Architecture Process in a BI environment
 
Data Strategy Best Practices
Data Strategy Best PracticesData Strategy Best Practices
Data Strategy Best Practices
 
How Enterprises are Using NoSQL for Mission-Critical Applications
How Enterprises are Using NoSQL for Mission-Critical ApplicationsHow Enterprises are Using NoSQL for Mission-Critical Applications
How Enterprises are Using NoSQL for Mission-Critical Applications
 
Building the Modern Data Hub
Building the Modern Data HubBuilding the Modern Data Hub
Building the Modern Data Hub
 
Showing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectShowing ROI for Your Analytic Project
Showing ROI for Your Analytic Project
 
Information & Data Architecture
Information & Data ArchitectureInformation & Data Architecture
Information & Data Architecture
 
Building a Collaborative Data Architecture
Building a Collaborative Data ArchitectureBuilding a Collaborative Data Architecture
Building a Collaborative Data Architecture
 
Information Architecture Deliverables
Information Architecture DeliverablesInformation Architecture Deliverables
Information Architecture Deliverables
 
Slides: How AI Makes Analytics More Human
Slides: How AI Makes Analytics More HumanSlides: How AI Makes Analytics More Human
Slides: How AI Makes Analytics More Human
 
Fasten you seatbelt and listen to the Data Steward
Fasten you seatbelt and listen to the Data StewardFasten you seatbelt and listen to the Data Steward
Fasten you seatbelt and listen to the Data Steward
 
DataEd Slides: Data Management + Data Strategy = Interoperability
DataEd Slides: Data Management + Data Strategy = InteroperabilityDataEd Slides: Data Management + Data Strategy = Interoperability
DataEd Slides: Data Management + Data Strategy = Interoperability
 
DataEd Online: Unlock Business Value through Data Governance
DataEd Online: Unlock Business Value through Data GovernanceDataEd Online: Unlock Business Value through Data Governance
DataEd Online: Unlock Business Value through Data Governance
 

Similar to Data Integration, Interoperability and Virtualization

Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
James Serra
 
Scope of Data Integration
Scope of Data IntegrationScope of Data Integration
Scope of Data Integration
HEXANIKA
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)
James Serra
 
Data Mesh using Microsoft Fabric
Data Mesh using Microsoft FabricData Mesh using Microsoft Fabric
Data Mesh using Microsoft Fabric
Nathan Bijnens
 
An Introduction to Data Virtualization in 2018
An Introduction to Data Virtualization in 2018An Introduction to Data Virtualization in 2018
An Introduction to Data Virtualization in 2018
Denodo
 
The technology of the business data lake
The technology of the business data lakeThe technology of the business data lake
The technology of the business data lake
Capgemini
 
Data Virtualization: An Introduction
Data Virtualization: An IntroductionData Virtualization: An Introduction
Data Virtualization: An Introduction
Denodo
 
How to Quickly and Easily Draw Value from Big Data Sources_Q3 symposia(Moa)
How to Quickly and Easily Draw Value  from Big Data Sources_Q3 symposia(Moa)How to Quickly and Easily Draw Value  from Big Data Sources_Q3 symposia(Moa)
How to Quickly and Easily Draw Value from Big Data Sources_Q3 symposia(Moa)Moacyr Passador
 
Data Virtualization: An Introduction
Data Virtualization: An IntroductionData Virtualization: An Introduction
Data Virtualization: An Introduction
Denodo
 
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonTraditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A Comparison
Capgemini
 
Building an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureBuilding an Effective Data Warehouse Architecture
Building an Effective Data Warehouse Architecture
James Serra
 
Data Mesh
Data MeshData Mesh
Modern Integrated Data Environment - Whitepaper | Qubole
Modern Integrated Data Environment - Whitepaper | QuboleModern Integrated Data Environment - Whitepaper | Qubole
Modern Integrated Data Environment - Whitepaper | Qubole
Vasu S
 
Data virtualization
Data virtualizationData virtualization
Data virtualizationHamed Hatami
 
Data Ninja Webinar Series: Realizing the Promise of Data Lakes
Data Ninja Webinar Series: Realizing the Promise of Data LakesData Ninja Webinar Series: Realizing the Promise of Data Lakes
Data Ninja Webinar Series: Realizing the Promise of Data Lakes
Denodo
 
The Shifting Landscape of Data Integration
The Shifting Landscape of Data IntegrationThe Shifting Landscape of Data Integration
The Shifting Landscape of Data Integration
DATAVERSITY
 
Fbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_servicesFbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_services
Cindy Irby
 
Current trends in dbms
Current trends in dbmsCurrent trends in dbms
Current trends in dbms
Daisy Joy
 

Similar to Data Integration, Interoperability and Virtualization (20)

Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Scope of Data Integration
Scope of Data IntegrationScope of Data Integration
Scope of Data Integration
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)
 
Data Mesh using Microsoft Fabric
Data Mesh using Microsoft FabricData Mesh using Microsoft Fabric
Data Mesh using Microsoft Fabric
 
Big data rmoug
Big data rmougBig data rmoug
Big data rmoug
 
An Introduction to Data Virtualization in 2018
An Introduction to Data Virtualization in 2018An Introduction to Data Virtualization in 2018
An Introduction to Data Virtualization in 2018
 
The technology of the business data lake
The technology of the business data lakeThe technology of the business data lake
The technology of the business data lake
 
Data Virtualization: An Introduction
Data Virtualization: An IntroductionData Virtualization: An Introduction
Data Virtualization: An Introduction
 
How to Quickly and Easily Draw Value from Big Data Sources_Q3 symposia(Moa)
How to Quickly and Easily Draw Value  from Big Data Sources_Q3 symposia(Moa)How to Quickly and Easily Draw Value  from Big Data Sources_Q3 symposia(Moa)
How to Quickly and Easily Draw Value from Big Data Sources_Q3 symposia(Moa)
 
Data Virtualization: An Introduction
Data Virtualization: An IntroductionData Virtualization: An Introduction
Data Virtualization: An Introduction
 
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonTraditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A Comparison
 
Building an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureBuilding an Effective Data Warehouse Architecture
Building an Effective Data Warehouse Architecture
 
Data Mesh
Data MeshData Mesh
Data Mesh
 
Data virtualization
Data virtualizationData virtualization
Data virtualization
 
Modern Integrated Data Environment - Whitepaper | Qubole
Modern Integrated Data Environment - Whitepaper | QuboleModern Integrated Data Environment - Whitepaper | Qubole
Modern Integrated Data Environment - Whitepaper | Qubole
 
Data virtualization
Data virtualizationData virtualization
Data virtualization
 
Data Ninja Webinar Series: Realizing the Promise of Data Lakes
Data Ninja Webinar Series: Realizing the Promise of Data LakesData Ninja Webinar Series: Realizing the Promise of Data Lakes
Data Ninja Webinar Series: Realizing the Promise of Data Lakes
 
The Shifting Landscape of Data Integration
The Shifting Landscape of Data IntegrationThe Shifting Landscape of Data Integration
The Shifting Landscape of Data Integration
 
Fbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_servicesFbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_services
 
Current trends in dbms
Current trends in dbmsCurrent trends in dbms
Current trends in dbms
 

Recently uploaded

When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using SmithyGenerating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using Smithy
g2nightmarescribd
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 

Recently uploaded (20)

When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using SmithyGenerating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using Smithy
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 

Data Integration, Interoperability and Virtualization

  • 1. Let’s talk dataIntegration, Interoperability and Virtualization Presented by: javier@enroutesystems.com art@enroutesystems.com 8/8/17
  • 2. DII – The new kid in the block Data Integration and Interoperability (DII) describes processes related to the movement and consolidation of data within and between data stores, applications and organizations. Why we’re geeking out for DII 1. SOA/Microservices are becoming more popular. 2. Integration of structured and unstructured data 3. Deliver value faster… avoid ROGUE users 4. Although it’s not new, DII in DMBOK provides clear guidelines to organizations aiming to become more efficient through IT.
  • 3. Data Interoperability Data Interoperability is the ability for multiple systems to communicate. Monolyt hs SOA/Microservi ces
  • 4. Data Integration Integration consolidates data into consistent forms, either physical or virtual.
  • 5. History Data virtualization exists since Bill Inmon popularized data warehouse in the 1990s. But virtual models back then were not very popular due to the lack of computer power available (or accessible). Today, change in data types and business expectations on information velocity have made virtualization a more popular concept. Did you know? Last time Bill Inmon wrote about Data virtualization he compared it to a frustrating whack a mole game, where no matter how much you hit the mole… it keeps coming back! http://www.b-eye- network.com/view/9956
  • 6.
  • 7. Click here Let’s take a look at your future Virtual vs Physical
  • 8. Why data virtualization? Fast and Easy • Rapid data integration which enables a faster time to solution • Integrations and changes are easy (No need to update Extractions, tables, DataMart's) Integrate more! • Opportunity to integrate structured and unstructured data Cheaper and more secure • Less expensive to maintain • No need to replicate data • Reduces overhead of management of data integration systems (Easier + Faster = Less reqauired resources) Agile • Enables iterative development with quick deliverables (Note: very important one since in most cases, users don’t know what they want… too many iterations) • Developers are more focused on business instead of understanding the mechanics of data manipulation (Why? Because Data virtualization tools automatically connect to many data sources )
  • 9.
  • 10. Use Cases Data Warehouse augmentation Problem • Bringing in new data sources to a data warehouse takes a significant amount of effort, but even more so, if the data sources include unstructured data. Fix • Data virtualization can be applied to augment existing data warehouse with virtual views that incorporate unstructured data. Support ETL process Problem • It is sometimes too complicated to access web services data, extract it and make it part of the ETL, specially if you need to develop access methods for external or new types of data. Fix • Data virtualization tools have access methods which can be used to easily extract data from web services, pre-process this data and have it ready to Data Warehouse Federation/Canonical Problem • Some organizations have multiple separate data warehouses which may take too much effort to integrate. Fix • Data virtualization allows to quickly generate federated views of all these data warehouses and integrate this data for different services. Individual warehouses continue to operate with no interruptions. (Same thing for DWH migrations!)
  • 11. Use Cases Data Warehouse prototyping Problem • Organizations are moving to agile development, where iterations and short term sprints are key to delivering value on a weekly ot bi weekly basis. Fix • When data prototypes are built fast and are validated by users, this then generates a proven product that can then be materialized saving time and therefore money. Data Mashups Problem • Web mashups are enabled by APIs and most corporate data sources do not have accessible APIs to support this mashup process. Fix • Data virtualization tools are enables of mashups since they use same protocols and data delivery formats as APIs. Master Data on Steroids– Past, present and future data Problem • Master Data Hubs traditionally only hold identity and descriptive information, but transactional data is usually not stored in MDHs. Fix • With data virtualization, you could make a canonical layer where you would input data from the MDH and other sources and enrich master data with summarized transactional data. (E.g. adding value of customer over time, purchasing forecast etc…)
  • 12. So is ETL going away? This does not mean ETL is not needed, its more around identifying when ETL is not enough, and use virtualization to enhance Data integration! When ETL is too slow, or data sources are difficult to access or data types are challenging. Maybe in the future it’ll be the other way around, where we’ll look at ETL for cases when data virtualization is not enough. For instance, when you need to perform highly complex transformations that could impact performance in a virtual database. Today, It is common to virtualize in development and materialize in production. Misconceptions 1. VDB DOES NOT replace a DWH. VDB enhances DWH by: • Combine structured and unstructured data into a single data layer