SlideShare a Scribd company logo
Bill Hayduk
CEO
State of the Market: Data Quality in 2023
a software division of
Statistics & Expert Opinions
State of the Market: Data Quality
a software division of
Study Driving Business Value from Data in the Face of
Fragmentation and Complexity
Organizations Face Data Fragmentation and Complexity
More volume in more sources, spread across more clouds makes it difficult to discover, manage, and control data.
State of the Market: Data Quality
a software division of
79% of organizations have
>100 data sources and 30%
have >1,000”
survey of global Chief Data
Officers (Dec. 2021)
Study
State of the Market: Data Quality
a software division of
Study
of organizations
have been able to
standardize across all
data management
functions
State of the Market: Data Quality
a software division of
Report:
How to Improve Your Data Quality
Poor data quality
costs organizations an
average $12.9 million,
increases the complexity of
data ecosystems and leads
to poor decision-making.
Good quality data
provides better leads,
better understanding of
customers and better
customer relationships.
Data quality is a
competitive advantage that
Data & Analytics leaders
need to improve upon
continuously.”
By the end of 2022, 70% of organizations will rigorously track data quality levels
via metrics, improving it by 60% to significantly reduce operational risks and costs.”
State of the Market: Data Quality
a software division of
State of the Market: Data Quality
World Quality Report 2022 – 2023
• from
• WQR’s the industry’s largest research study
• Provides the most comprehensive
assessment of the current state of quality
engineering practices
o from around the world
o across different industries
• Survey population includes:
o > 1,750 senior executives globally
o across multiple sectors
o from 32 countries
a software division of
89% agree that a robust data validation capability will not
only improve efficiencies in terms of time and resources but, more
importantly, will help to improve business decision-making.”
88% agree that a robust data validation capability has a
direct impact on customer satisfaction and accuracy of insights and
that it will help to boost business profitability”
Executive Survey
State of the Market: Data Quality
a software division of
World Quality Report Survey
State of the Market: Data Quality
a software division of
World Quality Report Survey
State of the Market: Data Quality
a software division of
Key Observations
• An increasing number of organizations see data validation as an
important task to be carried out for better decision-making.
• Most organizations agree that effective data validation has a direct
impact on customer experience and business outcomes.
Key Recommendation
• Invest in skills and the right level of thought leadership to get the
best results and outcomes from data analytics.
Key Observations & Recommendation
State of the Market: Data Quality
a software division of
QuerySurge Prospect Survey
Survey population includes:
• 1,090 managers & practitioners
• across 13 industries
• from 45 countries
• looking for an automated
Data Validation solution
State of the Market: Data Quality
Are you currently performing
data validation & ETL testing?
How are you currently performing it?
Which Use Case are you interested in?
a software division of
Manual Data Sampling:
Excel /
Utility:
Minus Queries:
ETL Testing solution:
DevOps for Data:
State of the Market: Data Quality
a software division of
State of the Market: Data Quality
The Key Focus of Data Quality in 2022 - 2023:
DEV OPS
DevOps for Data
& Continuous Testing
State of the Market: Data Quality
Some Descriptions:
State of the Market: Data Quality
William Blair: Equity Research
The most comprehensive report on DevOps
• Update on DevOps Market Consolidation
• Where Infrastructure and DevOps Meet
• Revisiting the CD Bottleneck: Balancing Speed and Risk With
Progressive Delivery
• Software Testing Coming Into Its Own
• Preventing the Next Sunburst Attack: Shifting Security Further
Left
• Dawn of a New Category: DevOps Management
• You Can Be a Developer Too: The Low Code/No Code Boom5
• Key Players of Scale in the DevOps Market
State of the Market: Data Quality
Source: Markets And Markets, William Blair
What % of your apps are developed & deployed using DevOps?
Most enterprises have
deployed < 50% of
their apps using
DevOps, as companies
struggle to scale their
DevOps models.
State of the Market: Data Quality
Sources: IDC U.S. DevOps Survey and William Blair Equity Research
What are the core challenges to building DevOps capabilities at
your organization?
State of the Market: Data Quality
Source: Markets And Markets, William Blair
Optimizing Test Automation should = DevOps
There are 6 primary goals of DevOps:
1) Increase speed of development and release processes
2) Make builds more reliable
3) Reduce turnaround for new features and bug fixes
4) Provide greater scalability of applications and infrastructure
5) Increase security by automating compliance practices
6) Improve collaboration throughout the development lifecycle
State of the Market: Data Quality
State of the Market: Data Quality
“The DevOps Market Remains Highly Fragmented and Relatively Immature”
• $8.5 billion DevOps market (IDC)
• Immense 28% growth rate
• IDC estimates a $28 billion market in 2024
• Continuous testing & DevOps will remove testing bottlenecks & clear path for Continuous Delivery (CD)
- Source: Agility Now 2.0 - Key Trends Shaping the DevOps Landscape, William Blair Equity Research
QuerySurge
seamlessly
integrates with all
other solutions in
the DevOps
marketplace
Key Players of Scale
now in DevOps
Market
a software division of
ETL Test Tool Vendors and DevOps
Tools in the market that are fully DevOps-enabled with a robust API
State of the Market: Data Quality
a software division of
Patterned after CMMI
Source: QuerySurge
Data Maturity Model for Validation & Testing
Data Testing & Commercial Solution
Level
4
Fully repeatable test automation, centralized reporting, training,
knowledge base, professional support, partner ecosystem.
DevOps-for-Data Testing Solution
Level
5
CI/CD pipeline test automation, RESTful API, minimum human
interaction, predictive data issues, auditable history & automated
reports. Business value is fully supported by management.
1%
of companies
9%
of companies
Data Sampling & Excel
Level
1
Using Excel spreadsheets to compare a small % of data sets
visually. Ad-hoc reporting. Doesn’t scale & isn’t repeatable.
Data Compares & Minus Queries
Level
2
Utilizing SQL editor & minus queries to test data. Ad hoc reporting.
Data Testing & Home-Grown Framework
Level
3
Built internally or by a system integrator. Expensive upfront costs to
build and maintain. No documentation, limited or custom reporting.
90%
of companies
on the
Data Maturity
Level
In most instances, QuerySurge is the only data validation & ETL testing tool mentioned…
State of the Market: Data Quality
QuerySurge’s DevOps for Data features provides
testers with the capability to dynamically
generate, execute, and update tests and data
stores utilizing API calls
• It’s the first full DevOps for Data solution
• improve their data quality at speed
QuerySurge DevOps for Data integrates with:
• Continuous integration/ETL solutions
• Automated build/release/deployment solutions
• Operations and DevOps monitoring solutions
• Test management/issue tracking solutions
• Scheduling and workload automation solutions
60+ API calls with almost 100 different properties
that users can utilize to retrieve, edit, update, or delete
information.
QuerySurge DevOps for Data
a software division of
QuerySurge provides two different ways to trigger API calls:
Command Line
RESTful – Documented with Swagger
QuerySurge DevOps for Data
a software division of
How to implement…
The QuerySurge team has expanded our REST API. Almost all UI
commands can now be executed through API request:
QuerySurge Focus for 2023
Test Suite Results
Individual Test Results
Source and Target Data
Failed Record Data
Test Suite Execution Status
Retrieve
QueryPairs
Create / Modify / Delete
Datastore Connections
Test Suites
Staging Tables
Query Snippets
Staging Queries
Design, execute & analyze
QuerySurge operations externally from the user interface
QuerySurge Server
Run Test Scenario
Kill Test Scenario
Execution
Run QueryPair
QuerySurge DevOps for Data
Unlimited Use Cases….
Tests are automatically initiated after ETL execution completes and
conditional logic is applied based on specific results
When new environments are deployed, new connections are automatically
created, and tests are duplicated & executed
Tests automatically analyze and detect changes to the data store
schemas and those changes are reflected in the tests.
QuerySurge DevOps for Data
a software division of
The QuerySurge team will be releasing new training courses
and new certifications that have a specific focus on DevOps
How we are going to help you get there…
Use discount code TESTGUILD2023
Learn. Earn. Inform.
• Learn DevOps for Data Testing
• Earn your digital badge
• Inform your social network of your new skills
• Receive access to free self-paced training
• take free certification exams
• receive Labor Market insights on jobs, locations, salaries
FREE Training & Certs
QuerySurge Focus for 2023
We’re seeing a huge culture shift…in business, people want
their code to be delivered immediately, and good-to-go.
This is why we’re seeing a move toward continuous testing,
where we’re aiming to prevent defects, rather than find them”
- The World Quality Report, 2021-2022
State of the Market: Data Quality
32
a software division of
State of the Market: Data Quality
www.QuerySurge.com

More Related Content

Similar to State of the Market - Data Quality in 2023

Implementing AI for improved performance testing – Cuneiform.pdf
Implementing AI for improved performance testing – Cuneiform.pdfImplementing AI for improved performance testing – Cuneiform.pdf
Implementing AI for improved performance testing – Cuneiform.pdf
Cuneiform Consulting Pvt Ltd.
 
How to Build High-Performing IT Teams - Including New Data on IT Performance ...
How to Build High-Performing IT Teams - Including New Data on IT Performance ...How to Build High-Performing IT Teams - Including New Data on IT Performance ...
How to Build High-Performing IT Teams - Including New Data on IT Performance ...
Puppet
 
Puppet Labs EMC DevOps Day NYC Aug-2015
Puppet Labs  EMC DevOps Day NYC Aug-2015Puppet Labs  EMC DevOps Day NYC Aug-2015
Puppet Labs EMC DevOps Day NYC Aug-2015
Bob Sokol
 
Intro of Key Features of SoftCAAT Pro software
Intro of Key Features of SoftCAAT Pro softwareIntro of Key Features of SoftCAAT Pro software
Intro of Key Features of SoftCAAT Pro software
rafeq
 
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Puppet
 
SDM Presentation V1.0
SDM Presentation V1.0SDM Presentation V1.0
SDM Presentation V1.0
KirSinc
 
Making a Quantum Leap with Continuous Analytics-Based QA
Making a Quantum Leap with Continuous Analytics-Based QAMaking a Quantum Leap with Continuous Analytics-Based QA
Making a Quantum Leap with Continuous Analytics-Based QA
Cognizant
 
Take your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena SoftwareTake your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena Software
Serena Software
 
Importance of software quality metrics
Importance of software quality metricsImportance of software quality metrics
Importance of software quality metrics
Piyush Sohaney
 
How to improve your system monitoring
How to improve your system monitoringHow to improve your system monitoring
How to improve your system monitoring
Andrew White
 
OberservePoint - The Digital Data Quality Playbook
OberservePoint - The Digital Data Quality  PlaybookOberservePoint - The Digital Data Quality  Playbook
OberservePoint - The Digital Data Quality Playbook
ObservePoint
 
Scale
ScaleScale
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
RTTS
 
Automated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI Reports
RTTS
 
How Data.com Drives Data Quality
How Data.com Drives Data QualityHow Data.com Drives Data Quality
How Data.com Drives Data Quality
Salesforce Partners
 
Big Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data QualityBig Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data Quality
RTTS
 
DataOps , cbuswaw April '23
DataOps , cbuswaw April '23DataOps , cbuswaw April '23
DataOps , cbuswaw April '23
Jason Packer
 
Kiran_CV
Kiran_CVKiran_CV
Kiran_CV
kiran kumar
 
Data Quality_ the holy grail for a Data Fluent Organization.pptx
Data Quality_ the holy grail for a Data Fluent Organization.pptxData Quality_ the holy grail for a Data Fluent Organization.pptx
Data Quality_ the holy grail for a Data Fluent Organization.pptx
Balvinder Hira
 
Best Practices for Rating and Policy Administration System Replacement
Best Practices for Rating and Policy Administration System ReplacementBest Practices for Rating and Policy Administration System Replacement
Best Practices for Rating and Policy Administration System Replacement
Edgewater
 

Similar to State of the Market - Data Quality in 2023 (20)

Implementing AI for improved performance testing – Cuneiform.pdf
Implementing AI for improved performance testing – Cuneiform.pdfImplementing AI for improved performance testing – Cuneiform.pdf
Implementing AI for improved performance testing – Cuneiform.pdf
 
How to Build High-Performing IT Teams - Including New Data on IT Performance ...
How to Build High-Performing IT Teams - Including New Data on IT Performance ...How to Build High-Performing IT Teams - Including New Data on IT Performance ...
How to Build High-Performing IT Teams - Including New Data on IT Performance ...
 
Puppet Labs EMC DevOps Day NYC Aug-2015
Puppet Labs  EMC DevOps Day NYC Aug-2015Puppet Labs  EMC DevOps Day NYC Aug-2015
Puppet Labs EMC DevOps Day NYC Aug-2015
 
Intro of Key Features of SoftCAAT Pro software
Intro of Key Features of SoftCAAT Pro softwareIntro of Key Features of SoftCAAT Pro software
Intro of Key Features of SoftCAAT Pro software
 
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
 
SDM Presentation V1.0
SDM Presentation V1.0SDM Presentation V1.0
SDM Presentation V1.0
 
Making a Quantum Leap with Continuous Analytics-Based QA
Making a Quantum Leap with Continuous Analytics-Based QAMaking a Quantum Leap with Continuous Analytics-Based QA
Making a Quantum Leap with Continuous Analytics-Based QA
 
Take your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena SoftwareTake your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena Software
 
Importance of software quality metrics
Importance of software quality metricsImportance of software quality metrics
Importance of software quality metrics
 
How to improve your system monitoring
How to improve your system monitoringHow to improve your system monitoring
How to improve your system monitoring
 
OberservePoint - The Digital Data Quality Playbook
OberservePoint - The Digital Data Quality  PlaybookOberservePoint - The Digital Data Quality  Playbook
OberservePoint - The Digital Data Quality Playbook
 
Scale
ScaleScale
Scale
 
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
Big Data Testing : Automate theTesting of Hadoop, NoSQL & DWH without Writing...
 
Automated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI Reports
 
How Data.com Drives Data Quality
How Data.com Drives Data QualityHow Data.com Drives Data Quality
How Data.com Drives Data Quality
 
Big Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data QualityBig Data Testing: Ensuring MongoDB Data Quality
Big Data Testing: Ensuring MongoDB Data Quality
 
DataOps , cbuswaw April '23
DataOps , cbuswaw April '23DataOps , cbuswaw April '23
DataOps , cbuswaw April '23
 
Kiran_CV
Kiran_CVKiran_CV
Kiran_CV
 
Data Quality_ the holy grail for a Data Fluent Organization.pptx
Data Quality_ the holy grail for a Data Fluent Organization.pptxData Quality_ the holy grail for a Data Fluent Organization.pptx
Data Quality_ the holy grail for a Data Fluent Organization.pptx
 
Best Practices for Rating and Policy Administration System Replacement
Best Practices for Rating and Policy Administration System ReplacementBest Practices for Rating and Policy Administration System Replacement
Best Practices for Rating and Policy Administration System Replacement
 

More from RTTS

JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
QuerySurge AI webinar
QuerySurge AI webinarQuerySurge AI webinar
QuerySurge AI webinar
RTTS
 
TestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data TestingTestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data Testing
RTTS
 
Creating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing AssignmentCreating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing Assignment
RTTS
 
RTTS Postman and API Testing Webinar Slides.pdf
RTTS Postman and API Testing Webinar  Slides.pdfRTTS Postman and API Testing Webinar  Slides.pdf
RTTS Postman and API Testing Webinar Slides.pdf
RTTS
 
How to Automate your Enterprise Application / ERP Testing
How to Automate your  Enterprise Application / ERP TestingHow to Automate your  Enterprise Application / ERP Testing
How to Automate your Enterprise Application / ERP Testing
RTTS
 
QuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing WebinarQuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing Webinar
RTTS
 
Webinar - QuerySurge and Azure DevOps in the Azure Cloud
 Webinar - QuerySurge and Azure DevOps in the Azure Cloud Webinar - QuerySurge and Azure DevOps in the Azure Cloud
Webinar - QuerySurge and Azure DevOps in the Azure Cloud
RTTS
 
Creating a Data validation and Testing Strategy
Creating a Data validation and Testing StrategyCreating a Data validation and Testing Strategy
Creating a Data validation and Testing Strategy
RTTS
 
Implementing Azure DevOps with your Testing Project
Implementing Azure DevOps with your Testing ProjectImplementing Azure DevOps with your Testing Project
Implementing Azure DevOps with your Testing Project
RTTS
 
An introduction to QuerySurge webinar
An introduction to QuerySurge webinarAn introduction to QuerySurge webinar
An introduction to QuerySurge webinar
RTTS
 
Data Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical IndustryData Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical Industry
RTTS
 
Completing the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessCompleting the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = Success
RTTS
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World Distilled
RTTS
 
QuerySurge for DevOps
QuerySurge for DevOpsQuerySurge for DevOps
QuerySurge for DevOps
RTTS
 
Leveraging HPE ALM & QuerySurge to test HPE Vertica
Leveraging HPE ALM & QuerySurge to test HPE VerticaLeveraging HPE ALM & QuerySurge to test HPE Vertica
Leveraging HPE ALM & QuerySurge to test HPE Vertica
RTTS
 
Whitepaper: Volume Testing Thick Clients and Databases
Whitepaper:  Volume Testing Thick Clients and DatabasesWhitepaper:  Volume Testing Thick Clients and Databases
Whitepaper: Volume Testing Thick Clients and Databases
RTTS
 
Query Wizards - data testing made easy - no programming
Query Wizards - data testing made easy - no programmingQuery Wizards - data testing made easy - no programming
Query Wizards - data testing made easy - no programming
RTTS
 
Case study: Open Source Automation Framework using Selenium WebDriver
Case study: Open Source Automation Framework using Selenium WebDriverCase study: Open Source Automation Framework using Selenium WebDriver
Case study: Open Source Automation Framework using Selenium WebDriver
RTTS
 
Enterprise Business Intelligence & Data Warehousing: The Data Quality Conundrum
Enterprise Business Intelligence & Data Warehousing: The Data Quality ConundrumEnterprise Business Intelligence & Data Warehousing: The Data Quality Conundrum
Enterprise Business Intelligence & Data Warehousing: The Data Quality Conundrum
RTTS
 

More from RTTS (20)

JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
QuerySurge AI webinar
QuerySurge AI webinarQuerySurge AI webinar
QuerySurge AI webinar
 
TestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data TestingTestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data Testing
 
Creating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing AssignmentCreating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing Assignment
 
RTTS Postman and API Testing Webinar Slides.pdf
RTTS Postman and API Testing Webinar  Slides.pdfRTTS Postman and API Testing Webinar  Slides.pdf
RTTS Postman and API Testing Webinar Slides.pdf
 
How to Automate your Enterprise Application / ERP Testing
How to Automate your  Enterprise Application / ERP TestingHow to Automate your  Enterprise Application / ERP Testing
How to Automate your Enterprise Application / ERP Testing
 
QuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing WebinarQuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing Webinar
 
Webinar - QuerySurge and Azure DevOps in the Azure Cloud
 Webinar - QuerySurge and Azure DevOps in the Azure Cloud Webinar - QuerySurge and Azure DevOps in the Azure Cloud
Webinar - QuerySurge and Azure DevOps in the Azure Cloud
 
Creating a Data validation and Testing Strategy
Creating a Data validation and Testing StrategyCreating a Data validation and Testing Strategy
Creating a Data validation and Testing Strategy
 
Implementing Azure DevOps with your Testing Project
Implementing Azure DevOps with your Testing ProjectImplementing Azure DevOps with your Testing Project
Implementing Azure DevOps with your Testing Project
 
An introduction to QuerySurge webinar
An introduction to QuerySurge webinarAn introduction to QuerySurge webinar
An introduction to QuerySurge webinar
 
Data Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical IndustryData Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical Industry
 
Completing the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessCompleting the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = Success
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World Distilled
 
QuerySurge for DevOps
QuerySurge for DevOpsQuerySurge for DevOps
QuerySurge for DevOps
 
Leveraging HPE ALM & QuerySurge to test HPE Vertica
Leveraging HPE ALM & QuerySurge to test HPE VerticaLeveraging HPE ALM & QuerySurge to test HPE Vertica
Leveraging HPE ALM & QuerySurge to test HPE Vertica
 
Whitepaper: Volume Testing Thick Clients and Databases
Whitepaper:  Volume Testing Thick Clients and DatabasesWhitepaper:  Volume Testing Thick Clients and Databases
Whitepaper: Volume Testing Thick Clients and Databases
 
Query Wizards - data testing made easy - no programming
Query Wizards - data testing made easy - no programmingQuery Wizards - data testing made easy - no programming
Query Wizards - data testing made easy - no programming
 
Case study: Open Source Automation Framework using Selenium WebDriver
Case study: Open Source Automation Framework using Selenium WebDriverCase study: Open Source Automation Framework using Selenium WebDriver
Case study: Open Source Automation Framework using Selenium WebDriver
 
Enterprise Business Intelligence & Data Warehousing: The Data Quality Conundrum
Enterprise Business Intelligence & Data Warehousing: The Data Quality ConundrumEnterprise Business Intelligence & Data Warehousing: The Data Quality Conundrum
Enterprise Business Intelligence & Data Warehousing: The Data Quality Conundrum
 

Recently uploaded

GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Ukraine
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
UiPathCommunity
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
Sease
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
ScyllaDB
 
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdfLee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
leebarnesutopia
 
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
FilipTomaszewski5
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)
HarpalGohil4
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's TipsGetting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
ScyllaDB
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!
Ortus Solutions, Corp
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
DianaGray10
 

Recently uploaded (20)

GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
 
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdfLee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
 
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's TipsGetting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
 

State of the Market - Data Quality in 2023

  • 1. Bill Hayduk CEO State of the Market: Data Quality in 2023
  • 2. a software division of Statistics & Expert Opinions State of the Market: Data Quality
  • 3. a software division of Study Driving Business Value from Data in the Face of Fragmentation and Complexity Organizations Face Data Fragmentation and Complexity More volume in more sources, spread across more clouds makes it difficult to discover, manage, and control data. State of the Market: Data Quality
  • 4. a software division of 79% of organizations have >100 data sources and 30% have >1,000” survey of global Chief Data Officers (Dec. 2021) Study State of the Market: Data Quality
  • 5. a software division of Study of organizations have been able to standardize across all data management functions State of the Market: Data Quality
  • 6. a software division of Report: How to Improve Your Data Quality Poor data quality costs organizations an average $12.9 million, increases the complexity of data ecosystems and leads to poor decision-making. Good quality data provides better leads, better understanding of customers and better customer relationships. Data quality is a competitive advantage that Data & Analytics leaders need to improve upon continuously.” By the end of 2022, 70% of organizations will rigorously track data quality levels via metrics, improving it by 60% to significantly reduce operational risks and costs.” State of the Market: Data Quality
  • 7. a software division of State of the Market: Data Quality World Quality Report 2022 – 2023 • from • WQR’s the industry’s largest research study • Provides the most comprehensive assessment of the current state of quality engineering practices o from around the world o across different industries • Survey population includes: o > 1,750 senior executives globally o across multiple sectors o from 32 countries
  • 8. a software division of 89% agree that a robust data validation capability will not only improve efficiencies in terms of time and resources but, more importantly, will help to improve business decision-making.” 88% agree that a robust data validation capability has a direct impact on customer satisfaction and accuracy of insights and that it will help to boost business profitability” Executive Survey State of the Market: Data Quality
  • 9. a software division of World Quality Report Survey State of the Market: Data Quality
  • 10. a software division of World Quality Report Survey State of the Market: Data Quality
  • 11. a software division of Key Observations • An increasing number of organizations see data validation as an important task to be carried out for better decision-making. • Most organizations agree that effective data validation has a direct impact on customer experience and business outcomes. Key Recommendation • Invest in skills and the right level of thought leadership to get the best results and outcomes from data analytics. Key Observations & Recommendation State of the Market: Data Quality
  • 12. a software division of QuerySurge Prospect Survey Survey population includes: • 1,090 managers & practitioners • across 13 industries • from 45 countries • looking for an automated Data Validation solution State of the Market: Data Quality
  • 13. Are you currently performing data validation & ETL testing? How are you currently performing it? Which Use Case are you interested in? a software division of Manual Data Sampling: Excel / Utility: Minus Queries: ETL Testing solution: DevOps for Data: State of the Market: Data Quality
  • 14. a software division of State of the Market: Data Quality
  • 15. The Key Focus of Data Quality in 2022 - 2023: DEV OPS DevOps for Data & Continuous Testing State of the Market: Data Quality
  • 16. Some Descriptions: State of the Market: Data Quality
  • 17. William Blair: Equity Research The most comprehensive report on DevOps • Update on DevOps Market Consolidation • Where Infrastructure and DevOps Meet • Revisiting the CD Bottleneck: Balancing Speed and Risk With Progressive Delivery • Software Testing Coming Into Its Own • Preventing the Next Sunburst Attack: Shifting Security Further Left • Dawn of a New Category: DevOps Management • You Can Be a Developer Too: The Low Code/No Code Boom5 • Key Players of Scale in the DevOps Market State of the Market: Data Quality
  • 18. Source: Markets And Markets, William Blair What % of your apps are developed & deployed using DevOps? Most enterprises have deployed < 50% of their apps using DevOps, as companies struggle to scale their DevOps models. State of the Market: Data Quality
  • 19. Sources: IDC U.S. DevOps Survey and William Blair Equity Research What are the core challenges to building DevOps capabilities at your organization? State of the Market: Data Quality
  • 20. Source: Markets And Markets, William Blair Optimizing Test Automation should = DevOps There are 6 primary goals of DevOps: 1) Increase speed of development and release processes 2) Make builds more reliable 3) Reduce turnaround for new features and bug fixes 4) Provide greater scalability of applications and infrastructure 5) Increase security by automating compliance practices 6) Improve collaboration throughout the development lifecycle State of the Market: Data Quality
  • 21. State of the Market: Data Quality “The DevOps Market Remains Highly Fragmented and Relatively Immature” • $8.5 billion DevOps market (IDC) • Immense 28% growth rate • IDC estimates a $28 billion market in 2024 • Continuous testing & DevOps will remove testing bottlenecks & clear path for Continuous Delivery (CD) - Source: Agility Now 2.0 - Key Trends Shaping the DevOps Landscape, William Blair Equity Research QuerySurge seamlessly integrates with all other solutions in the DevOps marketplace Key Players of Scale now in DevOps Market
  • 22. a software division of ETL Test Tool Vendors and DevOps Tools in the market that are fully DevOps-enabled with a robust API State of the Market: Data Quality
  • 23. a software division of Patterned after CMMI Source: QuerySurge Data Maturity Model for Validation & Testing Data Testing & Commercial Solution Level 4 Fully repeatable test automation, centralized reporting, training, knowledge base, professional support, partner ecosystem. DevOps-for-Data Testing Solution Level 5 CI/CD pipeline test automation, RESTful API, minimum human interaction, predictive data issues, auditable history & automated reports. Business value is fully supported by management. 1% of companies 9% of companies Data Sampling & Excel Level 1 Using Excel spreadsheets to compare a small % of data sets visually. Ad-hoc reporting. Doesn’t scale & isn’t repeatable. Data Compares & Minus Queries Level 2 Utilizing SQL editor & minus queries to test data. Ad hoc reporting. Data Testing & Home-Grown Framework Level 3 Built internally or by a system integrator. Expensive upfront costs to build and maintain. No documentation, limited or custom reporting. 90% of companies on the Data Maturity Level
  • 24. In most instances, QuerySurge is the only data validation & ETL testing tool mentioned… State of the Market: Data Quality
  • 25. QuerySurge’s DevOps for Data features provides testers with the capability to dynamically generate, execute, and update tests and data stores utilizing API calls • It’s the first full DevOps for Data solution • improve their data quality at speed QuerySurge DevOps for Data integrates with: • Continuous integration/ETL solutions • Automated build/release/deployment solutions • Operations and DevOps monitoring solutions • Test management/issue tracking solutions • Scheduling and workload automation solutions 60+ API calls with almost 100 different properties that users can utilize to retrieve, edit, update, or delete information. QuerySurge DevOps for Data
  • 26. a software division of QuerySurge provides two different ways to trigger API calls: Command Line RESTful – Documented with Swagger QuerySurge DevOps for Data
  • 27. a software division of How to implement… The QuerySurge team has expanded our REST API. Almost all UI commands can now be executed through API request: QuerySurge Focus for 2023
  • 28. Test Suite Results Individual Test Results Source and Target Data Failed Record Data Test Suite Execution Status Retrieve QueryPairs Create / Modify / Delete Datastore Connections Test Suites Staging Tables Query Snippets Staging Queries Design, execute & analyze QuerySurge operations externally from the user interface QuerySurge Server Run Test Scenario Kill Test Scenario Execution Run QueryPair QuerySurge DevOps for Data
  • 29. Unlimited Use Cases…. Tests are automatically initiated after ETL execution completes and conditional logic is applied based on specific results When new environments are deployed, new connections are automatically created, and tests are duplicated & executed Tests automatically analyze and detect changes to the data store schemas and those changes are reflected in the tests. QuerySurge DevOps for Data
  • 30. a software division of The QuerySurge team will be releasing new training courses and new certifications that have a specific focus on DevOps How we are going to help you get there… Use discount code TESTGUILD2023 Learn. Earn. Inform. • Learn DevOps for Data Testing • Earn your digital badge • Inform your social network of your new skills • Receive access to free self-paced training • take free certification exams • receive Labor Market insights on jobs, locations, salaries FREE Training & Certs QuerySurge Focus for 2023
  • 31. We’re seeing a huge culture shift…in business, people want their code to be delivered immediately, and good-to-go. This is why we’re seeing a move toward continuous testing, where we’re aiming to prevent defects, rather than find them” - The World Quality Report, 2021-2022 State of the Market: Data Quality
  • 32. 32 a software division of State of the Market: Data Quality www.QuerySurge.com