Data Quality for AI or AI for Data quality: advances in Data Quality Management for the success and sustainability of emerging technologies, business and society

DATA QUALITY FOR AI OR AI FOR DATA QUALITY:
ADVANCES IN DATA QUALITY MANAGEMENT
FOR THE SUCCESS AND SUSTAINABILITY OF
EMERGING TECHNOLOGIES, BUSINESS AND SOCIETY
ANASTASIJA NIKIFOROVA
University of Tartu, Institute of Computer Science, Estonia
European Open Science Cloud, Task Force «FAIR metrics and data quality»
Expert of the Latvian Council of Sciences, Associate member of the Latvian Open Technology Association
https://anastasijanikiforova.com/
Guest Lecture for the University of South-Eastern Norway (USN), October 2023

“It is among the top 1% of the world's universities, making it
one of Northern Europe's leading universities and the best-
ranked university in the Baltics”
University of Tartu : Rankings, Fees & Courses Details | Top Universities, University of Tartu | World
University Rankings | THE (timeshighereducation.com)

PHD IN COMPUTER SCIENCE – DATA PROCESSING SYSTEMS AND DATA NETWORKING
RESEARCH INTERESTS: DATA MANAGEMENT WITH A FOCUS ON DATA QUALITY, PUBLIC
ADMINISTRATION, OPEN DATA- AND OPEN GOVERNMENT DATA (ECOSYSTEMS)- RELATED TOPICS,
COVERING BOTH TECHNOLOGICAL AND SOCIETAL ASPECTS OF THE ABOVE, SOCIETY 5.0, SDG,
SMART CITY, SUSTAINABLE DEVELOPMENT, IOT, HCI AND DIGITIZATION.
✔ASSISTANT PROFESSOR AT THE UNIVERSITY OF TARTU, FACULTY OF SCIENCE AND TECHNOLOGY, INSTITUTE OF COMPUTER SCIENCE,
CHAIR OF SOFTWARE ENGINEERING
✔EUROPEAN OPEN SCIENCE CLOUD TASK FORCE “FAIR METRICS AND DATA QUALITY”
✔EDSC AMBASSADOR (EUROPEAN DIGITAL SKILLS CERTIFICATE, AS PART OF ACTION 9 OF THE DIGITAL EDUCATION ACTION PLAN (2021-
2027) – JRC/SVQ/2022/OP/0013)
✔IFIP WG8.5 ON ICT AND PUBLIC ADMINISTRATION MEMBER
✔ASSOCIATE MEMBER OF THE LATVIAN OPEN TECHNOLOGY ASSOCIATION
✔EXPERT OF THE LATVIAN COUNCIL OF SCIENCES IN (1) NATURAL SCIENCES – COMPUTER SCIENCE & INFORMATICS, (2) ENGINEERING &
TECHNOLOGY-ELECTRICAL ENGINEERING, ELECTRONICS, ICT, (3) SOCIAL SCIENCES – ECONOMICS & BUSINESS
✔EXPERT OF THE COST – EUROPEAN COOPERATION IN SCIENCE & TECHNOLOGY
✔EDITORIAL BOARD MEMBER FOR SEVERAL JOURNALS, PROGRAM COMMITTEE MEMBER FOR SEVERAL INTERNATIONAL
CONFERENCES (20+), PART OF AN ORGANIZING COMMITTEE (5+), INVITED REVIEWER FOR 15+ HIGH-QUALITY JOURNALS
✔ASSISTANT PROFESSOR AT THE UNIVERSITY OF TARTU, FACULTY OF SCIENCE AND TECHNOLOGY, INSTITUTE OF COMPUTER SCIENCE,
CHAIR OF SOFTWARE ENGINEERING
✔EUROPEAN OPEN SCIENCE CLOUD TASK FORCE “FAIR METRICS AND DATA QUALITY”
✔EDSC AMBASSADOR (EUROPEAN DIGITAL SKILLS CERTIFICATE, AS PART OF ACTION 9 OF THE DIGITAL EDUCATION ACTION PLAN (2021-
2027) – JRC/SVQ/2022/OP/0013)
✔IFIP WG8.5 ON ICT AND PUBLIC ADMINISTRATION MEMBER
✔ASSOCIATE MEMBER OF THE LATVIAN OPEN TECHNOLOGY ASSOCIATION
✔EXPERT OF THE LATVIAN COUNCIL OF SCIENCES IN (1) NATURAL SCIENCES – COMPUTER SCIENCE & INFORMATICS, (2) ENGINEERING &
TECHNOLOGY-ELECTRICAL ENGINEERING, ELECTRONICS, ICT, (3) SOCIAL SCIENCES – ECONOMICS & BUSINESS
✔EXPERT OF THE COST – EUROPEAN COOPERATION IN SCIENCE & TECHNOLOGY
✔EDITORIAL BOARD MEMBER FOR SEVERAL JOURNALS, PROGRAM COMMITTEE MEMBER FOR SEVERAL INTERNATIONAL
CONFERENCES (20+), PART OF AN ORGANIZING COMMITTEE (5+), INVITED REVIEWER FOR 15+ HIGH-QUALITY JOURNALS
✔VISITING RESEARCHER AT THE DELFT UNIVERSITY OF TEHNOLOGY, FACULTY TECHNOLOGY POLICY AND MANAGEMENT (TPM)
✔ASSISTANT PROFESSOR AT THE FACULTY OF COMPUTING, UNIVERSITY OF LATVIA
✔RESEARCHER IN THE INNOVATION LABORATORY, FACULTY OF COMPUTING, UNIVERSITY OF LATVIA
✔IT-EXPERT AT THE LATVIAN BIOMEDICAL RESEARCH AND STUDY CENTRE, BBMRI-ERIC LV NATIONAL NODE
✔ADVISOR FOR THE INSTITUTE FOR SOCIAL AND POLITICAL STUDIES, UNIVERSITY OF LATVIA
✔DATA SECURITY SOLUTIONS, LATVIA
✔VISITING RESEARCHER AT THE DELFT UNIVERSITY OF TEHNOLOGY, FACULTY TECHNOLOGY POLICY AND MANAGEMENT (TPM)
✔ASSISTANT PROFESSOR AT THE FACULTY OF COMPUTING, UNIVERSITY OF LATVIA
✔RESEARCHER IN THE INNOVATION LABORATORY, FACULTY OF COMPUTING, UNIVERSITY OF LATVIA
✔IT-EXPERT AT THE LATVIAN BIOMEDICAL RESEARCH AND STUDY CENTRE, BBMRI-ERIC LV NATIONAL NODE
✔ADVISOR FOR THE INSTITUTE FOR SOCIAL AND POLITICAL STUDIES, UNIVERSITY OF LATVIA
✔DATA SECURITY SOLUTIONS, LATVIA
MOST RECENT EXPERIENCE
PAST EXPERIENCE
BRIEFLY
ABOUT ME…

https://www.linkedin.com/posts/georgefirican_data-dataquality-datamanagement-activity-7001229524768108544-v-ne/?originalSubdomain=mv

+
=
https://starwars.fandom.com/wiki/Destruction_of_Despayre, https://www.linkedin.com/posts/georgefirican_data-dataquality-datamanagement-activity-7001229524768108544-v-ne/?originalSubdomain=mv, History in Objects: Death Star Plans Datacard • Lucasfilm, Video Analysis of an Exploding Death Star | WIRED

DATA QUALITY - WHAT, WHY, HOW, 10 BEST PRACTICES & MORE - Enterprise Master Data Management • Profisee

DATA … DATA ARE EVERYWHERE
M-Files on Twitter: "Data is the New Oil – Especially in Oil and Gas! https://t.co/zFlrvQqlMs https://t.co/qE3Q4aLNQy" / Twitter

DATA … DATA ARE EVERYWHERE
Sources: Premium Vector | Artificial intelligence logo, icon. vector symbol ai, deep learning blockchain neural network concept. machine learning, artificial intelligence, ai. (freepik.com), Top 10 Successful Data Science Companies in 2023 - Learn | Hevo (hevodata.com),
How to Use Business Intelligence (BI) to Improve Organizational Alignment | Wyn Enterprise (grapecity.com), Machine learning logo - Wi6Labs, Business Intelligence Icon Gráfico por aimagenarium · Creative Fabrica, Open Data – GEOAFRICA,
https://www.gartner.com/en/articles/4-emerging-technologies-you-need-to-know-about?utm_medium=social&utm_source=linkedin&utm_campaign=SM_GB_YOY_GTR_SOC_SF1_SM-SWG&utm_content=&sf267111387=1

https://dataladder.com/the-impact-of-poor-data-quality-risks-challenges-and-solutions/

https://twitter.com/bright_data/status/1346443370718240768

🤨 "Data is the new oil." | LinkedIn

Data is the New Oil - HubMeta
NOT REALLY

“DATA IS THE NEW OIL” WHY IT IS NOT?
BUT!
✓
Source: Here's Why Data Is Not The New Oil (forbes.com), Image sources: Oil well – Wikipedia, How do we get oil and gas out of the ground? (world-petroleum.org), Customized Silos For Effective Storage of Food | Nextech Solutions (nextechagrisolutions.com)
DATA, LIKE OIL is a source of power,
and those, who control them,
are establishing themselves as «masters of the universe»,
just as oil barons did 100 years ago

effectively infinitely durable and reusable
treating like oil –storing in siloes, has little benefit & reduces its usefulness
a finite resource
can be replicated indefinitely & moved around the world at
the speed of light, at low cost, through fiber optic networks
OIL
requires huge amounts of resources to be
transported to where it is needed
when used, its energy being lost as heat or light, or
permanently converted into another form (e.g., plastic)
becomes more useful the more it is used - once
processed, data often reveals further applications
as the world’s oil reserves dwindle, extracting
it becomes increasingly difficult and expensive
becoming increasingly available as computer
technology advances
data mining doesn’t intrinsically involve damage to the
environment & exploitation of finite natural resources
*apart from the electricity used to run the system
oil drilling involve causing damage to the natural
environment and exploitation of finite natural resources
“DATA IS THE NEW OIL” WHY IT IS NOT?
✘
Source: Here's Why Data Is Not The New Oil (forbes.com), Image sources: Oil well – Wikipedia, How do we get oil and gas out of the ground? (world-petroleum.org), Customized Silos For Effective Storage of Food | Nextech Solutions (nextechagrisolutions.com)
DATA
✘
✘
✘
✘

IF WE THINK ABOUT DATA AS A POWER SOURCE OR FUEL,
IT WOULD MAKE MORE SENSE TO COMPARE THEM WITH
RENEWABLE SOURCES LIKE THE
SUN, WIND AND TIDES”
-B. Marr, Forbes
Here's Why Data Is Not The New Oil (forbes.com)
Letter from the Editor: Here comes the sun (medicalnewstoday.com), A healthy wind | MIT News | Massachusetts Institute of Technology, Tidal phenomenon: high and low tides | Ponant Magazine

AMONG OTHER “NUANCES”,
DATA QUALITY IS USE-CASE DEPENDENT AND DYNAMIC IN NATURE
“ABSOLUTE DATA QUALITY”
DATA QUALITY LEVEL AT WHICH THE DATA WOULD SATISFY
ALL POSSIBLE USE CASES - IS IMPOSSIBLE TO ACHIEVE,
BUT IT IS A GOAL TO BE PURSUED

Def. 1: FITNESS-FOR-USE
Def. 2: FITNESS-FOR-PURPOSE
Def. 3: FREE OF ERRORS

Def. 1: FITNESS-FOR-USE
Def. 2: FITNESS-FOR-PURPOSE
Def. 3: FREE OF ERRORS
UTILITY*
WARRANTY*
=
=
According to ITIL® 4: the framework for the management of IT-enabled service

ISO def.: THE DEGREE TO WHICH
DATA SATISFIES THE REQUIREMENTS
OF ITS INTENDED PURPOSE
ISO/IEC 25012

DATA QUALITY IS
NOT ONLY ABOUT WHAT,
BUT
ALSO ABOUT HOW

NOT ONLY ABOUT WHAT, BUT
ALSO ABOUT HOW?
IT IS A PROCESS

NOT ONLY ABOUT WHAT, BUT
ALSO ABOUT HOW?
IT IS A PROCESS –
DATA QUALITY MANAGEMENT PROCESS

DEFINE
MEASURE
ANALYSE
IMPROVE TDQM
DATA QUALITY MANAGEMENT PROCESS
TOTAL DATA QUALITY MANAGEMENT LIFCYCLE (BY MIT)
DEFINE: IDENTIFY RELEVANT DQ DIMENSIONS
MEASURE: PRODUCE DQ METRICS
ANALYSE: IDENTIFY ROOT CAUSES FOR DQ PROBLEMS AND
DETERMINE THE IMPACT OF POOR DQ
IMPROVE: IDENTIFY AND EMPLOY TECHNIQUES FOR
IMPROVING DQ

•Lacagnina, C., David, R., Nikiforova, A., Kuusniemi, M. E., Cappiello, C., Biehlmaier, O., Wright, L.,
Schubert, C., Bertino, A., Thiemann, H., & Dennis, R. (2023). Towards a data quality framework
for EOSC. Zenodo. https://doi.org/10.5281/zenodo.7515816

Source: https://healthinstitute.illinois.edu/connect/news/berd-tips-dimensions-of-data-quality
AVAILABILITY
INTERNAL CONSISTENCY
EXTERNAL CONSISTENCY
ACCESSIBILITY
COMPREHENSIVENESS
INTEGRITY
SEMANTIC ACCURACY
SYNTACTIC ACCURACY
RELEVANCE
BELIEVABILITY
TRUSTWORTHINESS
UNAMBIGUITY
DQ DIMENSIONS
CURRENCY
VOLATILITY
EASE OF UNDERSTANDING
CREDIBILITY
PORTABILITY
RESPONSIVENESS
OBJECTIVITY
REPUTATION
RELIABILITY
AND MANY MORE…

Relevance
Availability
Internal consistency
External consistency
Accessibility
Comprehensiveness
Believability
Integrity
Trustworthiness
Semantic accuracy
Unambiguity
Syntactic accuracy
Source: https://healthinstitute.illinois.edu/connect/news/berd-tips-dimensions-of-data-quality
THERE ARE MORE THAN 100 DATA QUALITY DIMENSIONS

IS THERE ANY COMMONLY ACCEPTED DQ DIMENSION
CLASSIFICATION?
https://iso25000.com/index.php/en/iso-25000-standards/iso-25012/136-iso-iec-2012
ISO 25012
SOFTWARE ENGINEERING — SOFTWARE
PRODUCT QUALITY REQUIREMENTS
AND EVALUATION (SQUARE) — DATA
QUALITY MODEL

DIMENSIONS VARY IN DEFINITION AND SCOPE
ONE AND THE SAME NOTION CAN REFER TO DIFFERENT DIMENSIONS
ONE AND THE SAME DIMENSION CAN HAVE
DIFFERENT NOTIONS [IN DIFFERENT SOURCES]
DATA QUALITY RULES ARE THEN DEFINED
FOR EACH DIMENSION
METRICS ARE THEN SELECTED FOR THEM

SIMPLER
USER-ORIENTED
APPROACH
BASED ON USER DEFINED DATA
QUALITY REQUIREMENTS
Nikiforova, A. (2020). Definition and Evaluation of Data Quality: User-Oriented Data Object-Driven Approach to Data Quality
Assessment. Baltic Journal of Modern Computing, 8(3).

BUT WHAT ABOUT SCALING UP?
IS THERE
AUTOMATED DATA QUALITY
MANAGEMENT?

BUT WHAT ABOUT SCALING UP?
IS THERE AUTOMATED DATA QUALITY
MANAGEMENT?
OR
«Manual Data Quality Doesn’t Cut It in 2023»
-Attacama

ARTIFICIAL INTELLIGENCE FOR DATA QUALITY
OR
DATA QUALITY FOR ARTIFICAL INTELLIGENCE?

ARTIFICIAL INTELLIGENCE FOR DATA QUALITY
AND
DATA QUALITY FOR ARTIFICAL INTELLIGENCE?

✓ STANDARDIZATION, NORMALIZATION AND PARSING
✓ MATCHING / DEDUPLICATION AND MERGING
✓ DATA CLEANSING
✓ VALIDATION
✓ DATA PROFILING / AUDITING
✓ SOME A FEW OF THEM SUPPORT (SEMI-)AUTOMATED DQ RULE RECOGNITION
DQ TOOLS FOR (SEMI-)AUTOMATED DQM

Systematic Search of DQ Tools
Research papers:
Searched from Scopus using
keywords
Technology reviewers:
❏ 16 technology reviewers -
128 DQ tools
Suggestions by DQ
professionals
Martinsaari H. (2023). Toward an Automated Data Quality Rule Detection in Data Warehouses. Master Thesis (supervisor: Nikiforova Anastasija)

47
DQ management is closely related to other information management functionalities like
metadata management and master data management.
Tool Environment and Connectivity

10DQ tools out of 151 are able to detect DQ rules in DW
DQ rules were mainly discovered using
metadata, built-in rules and machine learning

SO FAR…
DEFINITION USER TIME
DIMENSION
PROCESS PURPOSE

SO FAR…
DEFINITION USER TIME
DIMENSION
PROCESS PURPOSE
WHAT ELSE?

DATA OBJECT
DATASET
DATABASE DATA REPOSITORY INFORMATION SYSTEM
SOFTWARE
NO ONE-SIZE-FITS-ALL

DATA OBJECT
DATASET
SOFTWARE
DATA OWNER
KNOWN
THIRD-PARTY

DATA OBJECT
DATASET
SOFTWARE
DATA OWNER
KNOWN
THIRD-PARTY
Nikiforova, A. (2020). Definition and Evaluation of Data Quality: User-Oriented Data Object-Driven Approach to Data Quality Assessment. Baltic Journal of Modern Computing, 8(3).
Nikiforova, A. (2020). Definition and Evaluation of Data Quality: User-Oriented Data Object-Driven Approach to Data Quality Assessment
Nikiforova, A. (2018). Open Data Quality Evaluation: A Comparative Analysis of Open Data in Latvia
Nikiforova, A. (2019). Analysis of open health data quality using data object-driven approach to data quality evaluation: insights from a Latvian context
Nikiforova, A. (2020, October). Timeliness of open data in open government data portals through pandemic-related data: a long data way from the
publisher to the user
The most frequently occurred data quality issues (for OGD) are: (a) contextual data
quality issues, (b) empty values even for primary data; (c) multiple denotation for the
same object within one data object and even a parameter; (d) issues on interrelated
parameters

DATA OBJECT
DATASET
SOFTWARE
DATA STRUCTURE
STRUCTURED DATA UNSTRUCTURED DATA
SEMI-STRUCTURED DATA
Image sources: https://monkeylearn.com/blog/semi-structured-data/, https://www.pngitem.com/middle/ioJTTbR_organization-structure-icon-png-download-structures-icon-png/

DATA OBJECT
DATASET
SOFTWARE
Running Analytics on the Data Lake - The Databricks Blog

DATA QUALITY-AWARE SOFTWARE
DEVELOPMENT
&
DATA QUALITY MODEL-BASED TESTING

THINK DATA QUALITY FIRST!!! OR TOWARDS DATA
QUALITY BY DESIGN
Guerra-García, C., Nikiforova, A., Jiménez, S., Perez-Gonzalez, H. G., Ramírez-Torres, M., & Ontañon-
García, L. (2023). ISO/IEC 25012-based methodology for managing data quality requirements in the
development of information systems: Towards Data Quality by Design. Data & Knowledge
Engineering, 145,
DAQUAVORD - A METHODOLOGY FOR PROJECT MANAGEMENT OF DATA QUALITY REQUIREMENTS
SPECIFICATION - AIMED AT ELICITING DQ REQUIREMENTS ARISING FROM DIFFERENT USERS’ VIEWPOINTS
THESE DQ REQUIREMENTS SERVE AS DATA QUALITY SOFTWARE REQUIREMENT AT THE TIME
OF THE DEVELOPMENT OF SOFTWARE THAT TAKES DATA QUALITY INTO ACCOUNT BY
DEFAULT.
IS BASED ON THE VIEWPOINT-ORIENTED REQUIREMENTS DEFINITION (VORD) METHOD, AND
THE LATEST AND MOST GENERALLY ACCEPTED ISO/IEC 25012 STANDARD.

DATA OBJECT
DATASET
SOFTWARE
DATA WAREHOUSE DATA LAKE
Maybe even something else?

DATA OBJECT
DATASET
SOFTWARE

Image source: https://www.grazitti.com/blog/data-lake-vs-data-warehouse-which-one-should-you-go-for/, https://www.qubole.com/data-lakes-vs-data-warehouses-the-co-existence-argument/
SCHEMA ON READ
SCHEMA ON WRITE
“SINGLE SOURCE
OF TRUTH”

Implementing a Data Lake or Data Warehouse Architecture for Business Intelligence? | by Lan Chu | Towards Data Science
NB: EXTRACT-TRANSFORM-LOAD
IS NOT DQM!!!

https://www.slideteam.net/data-lake-it-avoid-data-swamp-in-a-data-lake.html
HOW TO AVOID DATA SWAMP?

Image source: The abstracted future of data engineering | by Justin Gage | Datalogue | Medium
OR HOW TO AVOID GIGO*?
*“GARBAGE IN, GARBAGE OUT”

DATA LAKE FOR BI
BUSINESS DATA LAKE
https://www.capgemini.com/wp-content/uploads/2017/07/pivotal_data_lake_vs_traditional_bi_20140805.pdf

DATA LAKE
+
DATA WRANGLING
[an asset, not a silver bullet]
✔
Source: https://monkeylearn.com/blog/data-wrangling/, https://www.altair.com/what-is-data-wrangling/ , https://pediaa.com/what-is-the-difference-between-data-wrangling-and-data-cleaning

Image source: https://www.google.com/url?sa=i&url=https%3A%2F%2Ftwitter.com%2Frokar9%2Fstatus%2F1452339921629302784&psig=AOvVaw2IUSKtgUWxeaplk56f7CoK&ust=1668004535620000&source=images&cd=vfe&ved=0CA4QjhxqFwoTCJDHwbjnnvsCFQAAAAAdAAAAABAM

THE DATA WRANGLING PROCESS TO PREPARE DATA AND INTEGRATE IT INTO IS
DEPENDING ON THE IS AND THE DESIRED OR REQUIRED TARGET QUALITY*, INDIVIDUAL STEPS
SHOULD BE CARRIED OUT SEVERAL TIMES ➔ !!! DATA WRANGLING IS A CONTINUOUS PROCESS
!!! THAT REPEATS ITSELF REPEATEDLY AT REGULAR INTERVALS.
Information
System
Azeroual, O., Schöpfel, J., Ivanovic, D., & Nikiforova, A. (2022). Combining data lake and
data wrangling for ensuring data quality in CRIS. Procedia Computer Science, 211, 3-16.

DATA LAKE VS DATA WAREHOUSE
HOW TO TAKE
THE ADVANTAGES OF BOTH?

DATA LAKE VS DATA WAREHOUSE
HOW TO TAKE
THE ADVANTAGES OF BOTH?
DATA LAKEHOUSE

DATA LAKEHOUSE IS SEEN AS A COMBINATION OF DATA WAREHOUSING WORKLOADS & DATA LAKE ECONOMICS

Running Analytics on the Data Lake - The Databricks Blog, Build a Lake House Architecture on AWS | AWS Big Data Blog (amazon.com), The Data Lakehouse, the Data Warehouse and a Modern Data platform architecture - Microsoft Community Hub

DATA ARTIFACT
WHAT DQM APPROACH DEPENDS ON?
DEFINITION USER
TIME
DIMENSION
PROCESS PURPOSE

MUSK’S TOP PRIORITY: TO IMPROVE THE
PRODUCT…
Q: HOW DOES ONE ENSURE THE RELIABILITY OF DATA
AND DECISIONS MADE BASED ON SAID DATA?
THE ANSWER LIES NOT IN MANAGING THE DATA ALONE,
BUT ALSO THE INFORMATION AROUND AND ABOUT DATA
ACQUISITION, TRANSFORMATIONS AND VISUALIZATION
TO PROVIDE A BETTER UNDERSTANDING AND SUPPORT
DECISION MAKERS
https://www.gqindia.com/get-smart/content/5-things-elon-musk-did-to-become-one-of-the-richest-men-in-the-world

MUSK’S TOP PRIORITY: TO IMPROVE THE
PRODUCT…
Q: HOW DOES ONE ENSURE THE RELIABILITY OF DATA
AND DECISIONS MADE BASED ON SAID DATA?
THE ANSWER LIES NOT IN MANAGING THE DATA ALONE,
BUT ALSO THE INFORMATION AROUND AND ABOUT DATA
ACQUISITION, TRANSFORMATIONS AND VISUALIZATION
TO PROVIDE A BETTER UNDERSTANDING AND SUPPORT
DECISION MAKERS
BY FOCUSING ON SUSTAINABLE DATA, CLEAR
DATA GOVERNANCE
AND STRONG DATA MANAGEMENT

https://www.softcrylic.com/blogs/data-catalogs-in-data-governance/

DATA GOVERNANCE IS THE ANSWER
https://www.edq.com/blog/data-quality-vs-data-governance/
Azeroual O., Nikiforova A., Sha K. (2023) Overlooked Aspects of Data Governance:
Workflow Framework For Enterprise Data Deduplication

DATA GOVERNANCE IS THE ANSWER

BUT IS DATA GOVERNANCE THE
SILVER BULLET?

DATA MESH IS A NEW TREND!?
https://www.edq.com/blog/data-quality-vs-data-governance/, What is a data mesh? | IBM
A DATA MESH IS A DECENTRALIZED DATA ARCHITECTURE*
THAT ORGANIZES DATA BY A SPECIFIC BUSINESS DOMAIN,
E.G., MARKETING, SALES, CUSTOMER SERVICE —
PROVIDING MORE OWNERSHIP TO THE PRODUCERS OF A
GIVEN DATA(SET) ➔ DEMOCRATIZE DATA ACROSS A LARGE
ORGANIZATION
*FOCUSES ON ORGANIZATIONAL CHANGE
“A data mesh involves a cultural shift in the way that companies think about their data”

DATA MESH IS A NEW TREND!?
Data Lakehouse, Data Mesh, and Data Fabric (r2) | PPT (slideshare.net)

TREND#1: (SEMI-)AUTOMATED (THIRD-PARTY) DATA QUALITY
MANAGEMENT / AUGUMENTED DATA QUALITY MANAGEMENT
TREND#2: DATA QUALITY-BY-DESIGN
TREND#3: DATA MESH
TREND OR HYPE?

TREND#3: DATA MESH

TREND#3: DATA MESH
ALL THAT GLITTERS IS NOT GOLD

DATA QUALITY MANAGEMENT IS A CONTINUOUS PROCESS

THINK DATA QUALITY FIRST!
“1-10-100” RULE
1$ SPENT ON PREVENTION
SAVES 10$ ON APPRAISAL AND
100$ ON FAILURE COSTS
https://twitter.com/bright_data/status/1346443370718240768

DEVELOP DATA QUALITY MANAGEMENT AND
GOVERNANCE STRATEGIES
MANTAIN DQM & DQG STRATEGIES
DEFINE
MEASURE
ANALYSE
IMPROVE

FOR FURTHER READING IN CASE OF INTEREST…
✓ Nikiforova, A. (2020). Definition and Evaluation of Data Quality: User-Oriented Data Object-Driven Approach to Data Quality Assessment. Baltic Journal of
Modern Computing, 8(3).
✓ Guerra-García, C., Nikiforova, A., Jiménez, S., Perez-Gonzalez, H. G., Ramírez-Torres, M., & Ontañon-García, L. (2023). ISO/IEC 25012-based methodology for
managing data quality requirements in the development of information systems: Towards Data Quality by Design. Data & Knowledge Engineering, 145,
102152.
✓ Lacagnina, C., David, R., Nikiforova, A., Kuusniemi, M. E., Cappiello, C., Biehlmaier, O., ... & Dennis, R. (2022). TOWARDS A DATA QUALITY FRAMEWORK
FOR EOSC Authorship Community (Doctoral dissertation, EOSC Association).
✓ Nikiforova, A. (2020, October). Timeliness of open data in open government data portals through pandemic-related data: a long data way from the publisher
to the user. In 2020 Fourth International Conference on Multimedia Computing, Networking and Applications (MCNA) (pp. 131-138). IEEE.
✓ Azeroual, O., Jha, M., Nikiforova, A., Sha, K., Alsmirat, M., & Jha, S. (2022). A record linkage-based data deduplication framework with datacleaner
extension. Multimodal Technologies and Interaction, 6(4), 27.
✓ Azeroual, O., Nikiforova, A., & Sha, K. (2023, June). Overlooked Aspects of Data Governance: Workflow Framework For Enterprise Data Deduplication. In
2023 International Conference on Intelligent Computing, Communication, Networking and Services (ICCNS) (pp. 65-73). IEEE.
✓ Azeroual, O., Schöpfel, J., Ivanovic, D., & Nikiforova, A. (2022). Combining data lake and data wrangling for ensuring data quality in CRIS. Procedia
Computer Science, 211, 3-16.
✓ Nikiforova, A., Bicevskis, J., Bicevska, Z., & Oditis, I. (2020, December). Data quality model-based testing of information systems: the use-case of E-
scooters. In 2020 7th International Conference on Internet of Things: Systems, Management and Security (IOTSMS) (pp. 1-8). IEEE.
✓ Nikiforova, A., & Kozmina, N. (2021, November). Stakeholder-centred Identification of Data Quality Issues: Knowledge that Can Save Your Business. In 2021
Second International Conference on Intelligent Data Science Technologies and Applications (IDSTA) (pp. 66-73). IEEE.

Contact information:
https://anastasijanikiforova.com/
nikiforova.anastasija@gmail.com
https://www.linkedin.com/in/anastasija-nikiforova-466b99b3/
THANK YOU FOR ATTENTION!
https://cdn-haenh.nitrocdn.com/xgTmfzpSonftdbidYtTkgHxuTjkANBFu/assets/images/optimized/rev-47ba389/wp-content/uploads/2021/12/improve-data-quality.webp

Data Quality for AI or AI for Data quality: advances in Data Quality Management for the success and sustainability of emerging technologies, business and society

Recommended

Recommended

More Related Content

Similar to Data Quality for AI or AI for Data quality: advances in Data Quality Management for the success and sustainability of emerging technologies, business and society

Similar to Data Quality for AI or AI for Data quality: advances in Data Quality Management for the success and sustainability of emerging technologies, business and society (20)

More from Anastasija Nikiforova

More from Anastasija Nikiforova (20)

Recently uploaded

Recently uploaded (20)

Data Quality for AI or AI for Data quality: advances in Data Quality Management for the success and sustainability of emerging technologies, business and society