Learn statistics and expert opinions on the state of the market regarding data quality in 2023. Learn about: - statistics and expert opinions - the key focus of data quality in 2023 - the Data Maturity Model - DevOps for data and CI/CD pipelines - data validation and ETL testing - test automation

1. Bill Hayduk
CEO
State of the Market: Data Quality in 2023

2. a software division of
Statistics & Expert Opinions
State of the Market: Data Quality

3. a software division of
Study Driving Business Value from Data in the Face of
Fragmentation and Complexity
Organizations Face Data Fragmentation and Complexity
More volume in more sources, spread across more clouds makes it difficult to discover, manage, and control data.
State of the Market: Data Quality

4. a software division of
79% of organizations have
>100 data sources and 30%
have >1,000”
survey of global Chief Data
Officers (Dec. 2021)
Study
State of the Market: Data Quality

5. a software division of
Study
of organizations
have been able to
standardize across all
data management
functions
State of the Market: Data Quality

6. a software division of
Report:
How to Improve Your Data Quality
Poor data quality
costs organizations an
average $12.9 million,
increases the complexity of
data ecosystems and leads
to poor decision-making.
Good quality data
provides better leads,
better understanding of
customers and better
customer relationships.
Data quality is a
competitive advantage that
Data & Analytics leaders
need to improve upon
continuously.”
By the end of 2022, 70% of organizations will rigorously track data quality levels
via metrics, improving it by 60% to significantly reduce operational risks and costs.”
State of the Market: Data Quality

7. a software division of
State of the Market: Data Quality
World Quality Report 2022 – 2023
• from
• WQR’s the industry’s largest research study
• Provides the most comprehensive
assessment of the current state of quality
engineering practices
o from around the world
o across different industries
• Survey population includes:
o > 1,750 senior executives globally
o across multiple sectors
o from 32 countries

8. a software division of
89% agree that a robust data validation capability will not
only improve efficiencies in terms of time and resources but, more
importantly, will help to improve business decision-making.”
88% agree that a robust data validation capability has a
direct impact on customer satisfaction and accuracy of insights and
that it will help to boost business profitability”
Executive Survey
State of the Market: Data Quality

9. a software division of
World Quality Report Survey
State of the Market: Data Quality

10. a software division of
World Quality Report Survey
State of the Market: Data Quality

11. a software division of
Key Observations
• An increasing number of organizations see data validation as an
important task to be carried out for better decision-making.
• Most organizations agree that effective data validation has a direct
impact on customer experience and business outcomes.
Key Recommendation
• Invest in skills and the right level of thought leadership to get the
best results and outcomes from data analytics.
Key Observations & Recommendation
State of the Market: Data Quality

12. a software division of
QuerySurge Prospect Survey
Survey population includes:
• 1,090 managers & practitioners
• across 13 industries
• from 45 countries
• looking for an automated
Data Validation solution
State of the Market: Data Quality

13. Are you currently performing
data validation & ETL testing?
How are you currently performing it?
Which Use Case are you interested in?
a software division of
Manual Data Sampling:
Excel /
Utility:
Minus Queries:
ETL Testing solution:
DevOps for Data:
State of the Market: Data Quality

14. a software division of
State of the Market: Data Quality

15. The Key Focus of Data Quality in 2022 - 2023:
DEV OPS
DevOps for Data
& Continuous Testing
State of the Market: Data Quality

16. Some Descriptions:
State of the Market: Data Quality

17. William Blair: Equity Research
The most comprehensive report on DevOps
• Update on DevOps Market Consolidation
• Where Infrastructure and DevOps Meet
• Revisiting the CD Bottleneck: Balancing Speed and Risk With
Progressive Delivery
• Software Testing Coming Into Its Own
• Preventing the Next Sunburst Attack: Shifting Security Further
Left
• Dawn of a New Category: DevOps Management
• You Can Be a Developer Too: The Low Code/No Code Boom5
• Key Players of Scale in the DevOps Market
State of the Market: Data Quality

18. Source: Markets And Markets, William Blair
What % of your apps are developed & deployed using DevOps?
Most enterprises have
deployed < 50% of
their apps using
DevOps, as companies
struggle to scale their
DevOps models.
State of the Market: Data Quality

19. Sources: IDC U.S. DevOps Survey and William Blair Equity Research
What are the core challenges to building DevOps capabilities at
your organization?
State of the Market: Data Quality

20. Source: Markets And Markets, William Blair
Optimizing Test Automation should = DevOps
There are 6 primary goals of DevOps:
1) Increase speed of development and release processes
2) Make builds more reliable
3) Reduce turnaround for new features and bug fixes
4) Provide greater scalability of applications and infrastructure
5) Increase security by automating compliance practices
6) Improve collaboration throughout the development lifecycle
State of the Market: Data Quality

21. State of the Market: Data Quality
“The DevOps Market Remains Highly Fragmented and Relatively Immature”
• $8.5 billion DevOps market (IDC)
• Immense 28% growth rate
• IDC estimates a $28 billion market in 2024
• Continuous testing & DevOps will remove testing bottlenecks & clear path for Continuous Delivery (CD)
- Source: Agility Now 2.0 - Key Trends Shaping the DevOps Landscape, William Blair Equity Research
QuerySurge
seamlessly
integrates with all
other solutions in
the DevOps
marketplace
Key Players of Scale
now in DevOps
Market

22. a software division of
ETL Test Tool Vendors and DevOps
Tools in the market that are fully DevOps-enabled with a robust API
State of the Market: Data Quality

23. a software division of
Patterned after CMMI
Source: QuerySurge
Data Maturity Model for Validation & Testing
Data Testing & Commercial Solution
Level
4
Fully repeatable test automation, centralized reporting, training,
knowledge base, professional support, partner ecosystem.
DevOps-for-Data Testing Solution
Level
5
CI/CD pipeline test automation, RESTful API, minimum human
interaction, predictive data issues, auditable history & automated
reports. Business value is fully supported by management.
1%
of companies
9%
of companies
Data Sampling & Excel
Level
1
Using Excel spreadsheets to compare a small % of data sets
visually. Ad-hoc reporting. Doesn’t scale & isn’t repeatable.
Data Compares & Minus Queries
Level
2
Utilizing SQL editor & minus queries to test data. Ad hoc reporting.
Data Testing & Home-Grown Framework
Level
3
Built internally or by a system integrator. Expensive upfront costs to
build and maintain. No documentation, limited or custom reporting.
90%
of companies
on the
Data Maturity
Level

24. In most instances, QuerySurge is the only data validation & ETL testing tool mentioned…
State of the Market: Data Quality

25. QuerySurge’s DevOps for Data features provides
testers with the capability to dynamically
generate, execute, and update tests and data
stores utilizing API calls
• It’s the first full DevOps for Data solution
• improve their data quality at speed
QuerySurge DevOps for Data integrates with:
• Continuous integration/ETL solutions
• Automated build/release/deployment solutions
• Operations and DevOps monitoring solutions
• Test management/issue tracking solutions
• Scheduling and workload automation solutions
60+ API calls with almost 100 different properties
that users can utilize to retrieve, edit, update, or delete
information.
QuerySurge DevOps for Data

26. a software division of
QuerySurge provides two different ways to trigger API calls:
Command Line
RESTful – Documented with Swagger
QuerySurge DevOps for Data

27. a software division of
How to implement…
The QuerySurge team has expanded our REST API. Almost all UI
commands can now be executed through API request:
QuerySurge Focus for 2023

28. Test Suite Results
Individual Test Results
Source and Target Data
Failed Record Data
Test Suite Execution Status
Retrieve
QueryPairs
Create / Modify / Delete
Datastore Connections
Test Suites
Staging Tables
Query Snippets
Staging Queries
Design, execute & analyze
QuerySurge operations externally from the user interface
QuerySurge Server
Run Test Scenario
Kill Test Scenario
Execution
Run QueryPair
QuerySurge DevOps for Data

29. Unlimited Use Cases….
Tests are automatically initiated after ETL execution completes and
conditional logic is applied based on specific results
When new environments are deployed, new connections are automatically
created, and tests are duplicated & executed
Tests automatically analyze and detect changes to the data store
schemas and those changes are reflected in the tests.
QuerySurge DevOps for Data

30. a software division of
The QuerySurge team will be releasing new training courses
and new certifications that have a specific focus on DevOps
How we are going to help you get there…
Use discount code TESTGUILD2023
Learn. Earn. Inform.
• Learn DevOps for Data Testing
• Earn your digital badge
• Inform your social network of your new skills
• Receive access to free self-paced training
• take free certification exams
• receive Labor Market insights on jobs, locations, salaries
FREE Training & Certs
QuerySurge Focus for 2023

31. We’re seeing a huge culture shift…in business, people want
their code to be delivered immediately, and good-to-go.
This is why we’re seeing a move toward continuous testing,
where we’re aiming to prevent defects, rather than find them”
- The World Quality Report, 2021-2022
State of the Market: Data Quality

32. 32
a software division of
State of the Market: Data Quality
www.QuerySurge.com