DQM Certification Program in Korea Data Quality Workshop 52 nd  ISO TC184/SC4 Meeting, Ibusuki, Japan July 4, 2007 Sunho K...
Agenda 1.1 Introduction to Korea Database Promotion Center 1.2 Survey Results for DQM 1.3 DQM Certification Processes 1.4 ...
1. DQM Certification Program 1.1 Introduction to Korea Database Promotion Center (KDPC) <ul><li>KDPC, established in 1993,...
1.   DQM Certification Program Strategic use of data Data types and volume Data interchange among enterprises Request for ...
1.   DQM Certification Program Application / Requisition 1.3 DQM Certification Processes Enterprise Data Quality Appraiser...
1.   DQM Certification Program 1.4 Korean Industries Under Preparation for Certification KDB POSCO KTF FOA Kyobo … KT KIST...
Agenda 2.1 Characteristics of the Model 2.2 Architecture   of   the Model 2.3 Data Quality Criteria 2.4 DQM Processes 2.5 ...
2.   DQM Maturity Model 2.1 Characteristics of the  Model DQM Target <ul><li>Data quality secur ed by the measure of the m...
2.   DQM Maturity Model 2.1 Characteristics of the  Model <ul><li>The model evaluates maturity levels of not data quality ...
2.   DQM Maturity Model - Three Components of the Model Data Quality Criteria DQM Processes DQM Maturity Levels 2.2 Archit...
2.   DQM Maturity Model Data Validity Maturity Level Data Utility Maturity Level Data Management Processes DQM Process Dat...
2.   DQM Maturity Model Different Awareness about Quality Imprecise Target Quality Management Problems Occurrence Consiste...
2.   DQM Maturity Model Validity Utility Data Quality Criteria <ul><li>Consistency implies that there are no variations be...
2.   DQM Maturity Model Requirements Management Data Architecture Management Data Flow Management Database Management 2.4 ...
2.   DQM Maturity Model Data Application Management Data Standardization Management Data Ownership Management User View Ma...
2.   DQM Maturity Model Processes Application (Monitoring) Requirements (Functional) User View Data Standardization Data A...
2.   DQM Maturity Model Data Quality Criteria Validity Utility Security Accuracy Consistency Timeliness Accessibility Usab...
2.   DQM Maturity Model Maturity Quantification  :  DQM is performed by quantitative measurement methods. Integration  :  ...
2.   DQM Maturity Model Accuracy The required operational rules are implemented as DQM problems are realized. In addition,...
2.   DQM Maturity Model 2.6 DQM Maturity Levels <ul><li>[Standard Data Management] </li></ul><ul><li>Terminology, codes, d...
2.   DQM Maturity Model 2.6 DQM Maturity Levels - DQM Maturity Levels for Accuracy <ul><li>Define biz rules clearly and ke...
2.   DQM Maturity Model Consistency 1 2 3 4 5 Process Maturity Level Data duplication is defined; documents show the relat...
2.   DQM Maturity Model Usability 1 2 3 4 5 Process Maturity Level Requirements are defined in this stage. User requiremen...
2.   DQM Maturity Model Accessibility 1 2 3 4 5 Process Maturity Level Basic user views are provided including menu struct...
2.   DQM Maturity Model Timeliness 1 2 3 4 5 Process Maturity Level Nonfunctional user requirements are defined in this st...
2.   DQM Maturity Model Security 1 2 3 4 5 Process Maturity Level In this stage, subjects for security control and the sec...
2.   DQM Maturity Model - An Example of the Maturity Level Certification  Accuracy Level 5 Consistency Level 5 Usability L...
<ul><li>This model has been designed to evaluate DQM processes. We need to extend the model to cover the evaluation for da...
Thank You
Upcoming SlideShare
Loading in …5
×

11.DQM Certification Program in Kor...

473 views
390 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
473
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

11.DQM Certification Program in Kor...

  1. 1. DQM Certification Program in Korea Data Quality Workshop 52 nd ISO TC184/SC4 Meeting, Ibusuki, Japan July 4, 2007 Sunho Kim Myongji University, Korea [email_address]
  2. 2. Agenda 1.1 Introduction to Korea Database Promotion Center 1.2 Survey Results for DQM 1.3 DQM Certification Processes 1.4 Certified Industries in Korea 1 DQM Certification Program DQM Maturity Model 2 Conculsion 3
  3. 3. 1. DQM Certification Program 1.1 Introduction to Korea Database Promotion Center (KDPC) <ul><li>KDPC, established in 1993, is a government organization under the Ministry of Information and Communication. It aims to expedite the realization of ubiquitous knowledge & information society by way of promoting utilization of information in public and supporting advancement of database industry in Korea. </li></ul><ul><li>Database Industry Promotion and Advancement </li></ul><ul><li>toward u-Korea Realization </li></ul><ul><li>Knowledge Innovation Center for Database </li></ul><ul><li>Excellencies in 21 st Century </li></ul><ul><li>The DQM (Data Quality Management) Certification Program was developed in 2006 </li></ul><ul><li>by KDPC </li></ul>
  4. 4. 1. DQM Certification Program Strategic use of data Data types and volume Data interchange among enterprises Request for quick & correct data In the fast environmental change <ul><li>The Survey of DQM needs in Korea [ Sep.,2005, KDPC ] </li></ul>Implementation of DQM No 70% Yes 30% Needs for DQM No 17.4% Yes 82.6% Rising 1.2 Survey Results for DQM Trend
  5. 5. 1. DQM Certification Program Application / Requisition 1.3 DQM Certification Processes Enterprise Data Quality Appraiser Certification Center Evaluation Verification Certification Consultation Plan establishment Requisition Approval Contract conclusion Evaluation Result Result Consideration Verification Certificate Issuance Certificate acquisition
  6. 6. 1. DQM Certification Program 1.4 Korean Industries Under Preparation for Certification KDB POSCO KTF FOA Kyobo … KT KISTI IITA SAMSUNG SDI IBK … DQM Certification Financial Industry Public Institution IT Industry Manufacturing industry
  7. 7. Agenda 2.1 Characteristics of the Model 2.2 Architecture of the Model 2.3 Data Quality Criteria 2.4 DQM Processes 2.5 Data Quality Criteria & DQM Processes 2.6 DQM Maturity Levels 1 DQM Certification Program 2 DQM Maturity Model Conculsion 3
  8. 8. 2. DQM Maturity Model 2.1 Characteristics of the Model DQM Target <ul><li>Data quality secur ed by the measure of the moment doesn’t last long </li></ul><ul><li>Data quality can be preserved or improved when DQM is performed continuously </li></ul><ul><li>Need to take both data and process into account </li></ul>Issues of DQM Strength Weakness Action <ul><li>It can reduce fundamental causes of data errors </li></ul><ul><li>It is possible to improve data quality continuously and step by step by process improvement </li></ul><ul><li>It is not possible to respond quickly to data errors </li></ul><ul><li>Data quality is measured qualitatively </li></ul><ul><li>It’s possible to respond quickly against data errors by measurement criteria </li></ul><ul><li>Data quality is measured quantitatively </li></ul><ul><li>It is not easy to measure quality levels. </li></ul><ul><li>After a certain level of quality, the quality declines as time goes by </li></ul>Improve and preserve DQM processes Measure data quality and improve data of bad quality Management Process Data
  9. 9. 2. DQM Maturity Model 2.1 Characteristics of the Model <ul><li>The model evaluates maturity levels of not data quality itself but DQM processes </li></ul><ul><li>The model primarily focuses on the certification of general master data and application data. </li></ul><ul><li>The model was developed based on CMMI (Capability Maturity Model Integration ) and EAMM (Enterprise Architecture Maturity Model) </li></ul>
  10. 10. 2. DQM Maturity Model - Three Components of the Model Data Quality Criteria DQM Processes DQM Maturity Levels 2.2 Architecture of the Model
  11. 11. 2. DQM Maturity Model Data Validity Maturity Level Data Utility Maturity Level Data Management Processes DQM Process Data Validity Data Utility Organization Maturity Level Extract DQM Processes Measure the maturity level of a whole organization Measure the maturity level by criterion 2.2 Architecture of the Model Integration Quantification Match DQM Processes To data quality criteria
  12. 12. 2. DQM Maturity Model Different Awareness about Quality Imprecise Target Quality Management Problems Occurrence Consistent Definition of Quality Problems in Case of No Criteria Criteria Establishment Validity Utility Data Quality Criteria 2.3 Data Quality Criteria Accuracy Consistency Usability Accessibility Timeliness Security
  13. 13. 2. DQM Maturity Model Validity Utility Data Quality Criteria <ul><li>Consistency implies that there are no variations between identical data content within an information system. </li></ul><ul><li>Data usability implies whether to satisfies the scope of data and its detailed level that users require. </li></ul><ul><li>Accessibility implies how easily users can access data that they need. </li></ul><ul><li>Timeliness implies response time, up-to-datedness of data, etc. </li></ul><ul><li>Accurate data should essentially reflect values of an object in the real world without errors. </li></ul><ul><li>It implies whether or not data is being aptly protected from internal or external dangerous factors . </li></ul>Consistency Usability Accessibility Timeliness Security Authenticity, Compatibility, Necessity, Relevancy Compatibility, Unanimity, Referential Integrity Sufficiency, Flexibility, Usability, Traceability Protection, Liability, Safety Accessibility Timeliness Accuracy Consistency Usability Accessibility Timeliness Security Accuracy 2.3 Data Quality Criteria
  14. 14. 2. DQM Maturity Model Requirements Management Data Architecture Management Data Flow Management Database Management 2.4 DQM Processes - DQM includes eight different processes: This process is intended for solving and managing users’ demands for data . User requirement is further separated into functional requirements required in business processes and nonfunctional requirements such as response time, data supply interval, etc. This is the process that manages the data structures of data modeling and database planning, and provides users with their data requirements. The processes include data flexibility, data redundancy, data integrity, and data conformity management. The various processes that create, extract, convert, and store the source data to meet the target. The processes include flow cycle management and circulation flow management. This process administers and oversees databases that actually store the value of data. Performance management and security management are the sub-processes.
  15. 15. 2. DQM Maturity Model Data Application Management Data Standardization Management Data Ownership Management User View Management 2.4 DQM Processes This process evaluates the present condition of data provided to users and the environment in which data maximization can be achieved. Sub-processes are b usiness rules verification and usage monitoring To maintain a standardized view on data, this process manages definitions of data standardization rules, standard modifications, and standard applications management. Examples of standard data; terminology, codes, domain, conversion rules for data exchange, etc. Data ownership management defines the various procedures, processes, and activities encompassed in DQM on the basis of owner responsibility and rights. The owner has authority over the creation, change, application, and deletion of data. The process implies the management of environments where end users can apply data. The purpose of this process is to minimize barriers of user access and maximize data utilization. Hence, the user view should be designed and implemented from the users’ point of view.
  16. 16. 2. DQM Maturity Model Processes Application (Monitoring) Requirements (Functional) User View Data Standardization Data Architec-ture Data Database (Performance) Database (Security) Application (Biz Rule) Flow Ownership Requirements (Nonfunctional) Requirements Management Data Architecture Management Data Flow Management Database Management Data Application Management Data Standardization Management Data Ownership Management User View Management Process Relationship Diagram 2.4 DQM Processes
  17. 17. 2. DQM Maturity Model Data Quality Criteria Validity Utility Security Accuracy Consistency Timeliness Accessibility Usability Requirements Management Data Architecture Management Data Flow Management Database Management Data Application Management Data Standardization Management Data Ownership Management User View Management 2.5 Data Quality Criteria & DQM Processes DQM Processes
  18. 18. 2. DQM Maturity Model Maturity Quantification : DQM is performed by quantitative measurement methods. Integration : DQM is performed consistently from the viewpoint of enterprise integration. Formalization : Overall processes for DQM are formalized. Introduction : The organization recognizes problems and necessity of data quality, and performs partial DQM activities. Optimization : Factors to improve DQM are derived and applied continuously. 2.6 DQM Maturity Levels 5 4 3 2 1 Quantification Integration Formalization Introduction Optimization - Definition of DQM Maturity Levels
  19. 19. 2. DQM Maturity Model Accuracy The required operational rules are implemented as DQM problems are realized. In addition, the administrator personally manages various activities in parts. Even if the problem is recognized and inspected, there are still many errors since there is no standardization or systemization. The administrator systematically inspects and enhances support tool applications, and produces standardized documents that clearly reflect operational rules needed at hand. Also, standard domain and code are recorded, and the database is periodically checked and improved to assure that the standards are acceptable. In regard to the operational rules, causes of the error are tracked and analyzed after inspection, and the present conditions of the error enhancement are summarized in a record log and database. Data quality inspection, follow-up, and enhancement are safely administered and managed qualitatively. The success of inspection and data error enhancement requires an index of quantitative goals to be measured periodically. Additionally, the responsibility and role of the process administrator adjusts controls according to the recognized data error. In the transferring dimension, quantitative management switches to qualitative management, and the level clearly measures and acknowledges the outcomes of quality. The operational rules and error patterns are analyzed so that error does not occur, and the root cause of error is removed by continuous process improvement. This means that the current quality management can be optimized and tackle any situation that may arise in the environment. 1 2 3 4 5 Process Maturity Level 2.6 DQM Maturity Levels Data Quality Criteria
  20. 20. 2. DQM Maturity Model 2.6 DQM Maturity Levels <ul><li>[Standard Data Management] </li></ul><ul><li>Terminology, codes, domain, etc. </li></ul><ul><li>Define standards for accuracy </li></ul><ul><li>[Data Application Management ( Biz rules verification ) ] </li></ul><ul><li>Define data objects for management </li></ul><ul><li>Confirm accuracy </li></ul><ul><li>Define data objects for management systematically </li></ul><ul><li>Confirm accuracy under the plan </li></ul><ul><li>Keep track of data errors systematically </li></ul><ul><li>Keep management indices </li></ul><ul><li>Accumulate knowledge of error resolution </li></ul><ul><li>Remove fundamental causes of data errors </li></ul><ul><li>[Data Ownership Management] </li></ul><ul><li>Define roles and responsibilities clearly </li></ul><ul><li>Evaluate performance </li></ul>- DQM Processes for Accuracy
  21. 21. 2. DQM Maturity Model 2.6 DQM Maturity Levels - DQM Maturity Levels for Accuracy <ul><li>Define biz rules clearly and keep them up-to-dated </li></ul><ul><li>Perform verification by software and record results </li></ul><ul><li>Data accuracy highly relies on personal ability </li></ul><ul><li>Identify clearly biz rules for management </li></ul><ul><li>Keep them in the form of formalized documents or diagrams </li></ul><ul><li>Improve accuracy errors by tools or systematic ways </li></ul><ul><li>Verify biz rules enterprise-wide, grasp causes of errors, and improve them </li></ul><ul><li>Perform consistently verification, tracking, and status monitoring from the viewpoint of enterprise </li></ul><ul><li>Perform normal DQM activities </li></ul><ul><li>Determine performance measures for standards, applications, and ownership </li></ul><ul><li>Perform evaluation regularly, and store knowledge of errors </li></ul><ul><li>Adjust roles and responsibilities when necessary </li></ul><ul><li>Analyze knowledge-based standards, applications, ownership and various error patterns </li></ul><ul><li>Remove fundamental causes of errors, and improve corresponding processes </li></ul>1 2 3 4 5 5 4 3 2 1 Quantification Integration Formalization Introduction Optimization
  22. 22. 2. DQM Maturity Model Consistency 1 2 3 4 5 Process Maturity Level Data duplication is defined; documents show the relationship between tables and columns; relationships of data defined within duplicated columns are inspected periodically. A list of successful and failed attempts is managed by inspecting and using separate mapping logic programs since data flow is documented in the programs and structure maps of databases. There are standardized methods such as the ERD that help to clearly understand the relations between tables and columns, and compliance procedures are used periodically to follow up on the use of the standards. Data flow tools and formulated methods, for example, are developed based on batch movement. Some features that are checked in mapping logic are the layout modification and the actual progress of work flow, and a procedure is arranged to control errors that show up during flow monitoring. It is feasible to check the tables and programs that define the standards, analyze the effects of quality modifications, and check the nonstandard functions. In the dimension of the enterprise, the cause of various kinds of errors is inspected according to the administrator in charge of the particular standard in question. The administrator organizes and prepares documents that describe the present state of error enhancement and database utilization. The state of the error must be tracked and managed qualitatively in the viewpoint of data inspection. Compliance of standards is measured periodically within the realm of conformity and application to fulfill quantitative goals, and the role and responsibility are adjusted to learned errors. In respect to the enterprise, qualitative management can involve quantitative controls to develop data quality and clearly assess the outcomes. Even if errors are not found, learned error patterns should be analyzed to remove the root cause of potential errors, and process improvement activities should be continuously recorded and managed. This practice enables the present data quality to be optimized and any changes in the environment to be managed effectively. 2.6 DQM Maturity Levels Data Quality Criteria
  23. 23. 2. DQM Maturity Model Usability 1 2 3 4 5 Process Maturity Level Requirements are defined in this stage. User requirements are collected, documented, and then confirmed by the users. As the user grasps the response situation of the requirements, the processing status of the requirements is recorded. Also, the user confers about the problems and boundaries of the response to the requirements. In this stage, the response process to the requirements fixes any problems and user responses are reflected in this stage. Compilation and confirmation of the requirements and the response process are established. Modifications can be made upon the agreement of the person in charge. Also, user instructions are offered, the status of data applications is checked, and user dissatisfactions or requirements for improvement are addressed. Data architecture is designed based on the requirements and modifications to the data architecture resulting from new requirements or changes to existing requirements are evaluated. There is a systematic organization in charge of managing the data architecture of the enterprise. Organizations and processes for conferring on the requirements are defined, and priorities of resource distribution for the response of the requirements are defined and managed. Also, two-way tracking system is maintained between requirements and project plans/output. Quantitative goals are set about the quality of individual data, and progress is evaluated regularly through a measurement index. Based on the result of the evaluation, response knowledge about functional user requirements is systematically classified and documented. When necessary, roles and responsibilities of the person in charge are changed. From the viewpoint of the enterprise, qualitative control is transferred to quantitative control, and product quality is clearly measured and documented in this stage. The best practices of the industry are reflected in the control for its maintenance and development. In addition, data utility is maximized through application of utility technologies and optimization of limited resources. 2.6 DQM Maturity Levels Data Quality Criteria
  24. 24. 2. DQM Maturity Model Accessibility 1 2 3 4 5 Process Maturity Level Basic user views are provided including menu structure, site maps and references. In addition, users can search dispersed data. There are rules on integrating or converting dispersed data. Users are provided with standardized user views and get regular education to improve their ability to utilize data. User manuals are consistently updated. As dispersed data are provided through subject-oriented, integrated database, users can have easy access to and utilize data. Data dispersed to various systems are provided through integrated user views and channels that help access to data that are necessary for a business become more diverse. Dispersed data can be analyzed from the viewpoint of the enterprise. In addition to the basic data provided by enterprises, an environment is provided to enable users to carry out a more advanced analysis on data. Also, the data is profound enough to crease added value. Finally, meta information is provided and shared among users though the basic data architecture. Quantitative goals are set about the quality of individual data, and progress is evaluated regularly through a measurement index. Based on the result of the evaluation, accessibility is improved and user utility patterns are analyzed. When necessary, roles and responsibilities of the person in charge are changed. From the viewpoint of the enterprise, qualitative control is transferred to quantitative control, and product quality is clearly measured and documented. In this stage, new technologies and utility situations are applied to the efforts to optimize user views. The optimization is not confined to the current environment, but also applies to future environments. 2.6 DQM Maturity Levels Data Quality Criteria
  25. 25. 2. DQM Maturity Model Timeliness 1 2 3 4 5 Process Maturity Level Nonfunctional user requirements are defined in this stage. User requirements are collected, documented, and then confirmed by the users. As the users grasp the response situation of the requirements, the processing status of the requirements is recorded. Also, the users confer about the problems and boundaries of the response to the requirements. Schedule management is carried out on basic data flow and the result is evaluated on a regular basis. The response process fixes and adjusts to the requirements, data flow cycle is coordinated, and roles and responsibilities for managing timeliness are defined. Compilation and confirmation of the requirements are established. The organization sets up schedules, responds to the requirements, and tunes the operating system regularly. Data flow cycle is clearly defined, and data analysis cycle, deadline and summary are documented. In addition, operations before and after data flow are coordinated. Resources for the response to the requirements are distributed and distribution plans are set up and implemented in order. In this stage, data flow is managed from the viewpoint of the enterprise and roles and responsibilities are clearly defined. Also, professional response to the nonfunctional user requirements is established. Quantitative goals are set about the quality of individual data, and progress is evaluated regularly through a measurement index. Based on the result of the evaluation, response knowledge about nonfunctional user requirements is systematically classified and documented. When necessary, roles and responsibilities are changed. From the viewpoint of the enter-prise, qualitative control is transferred to quantitative control, and product quality is clearly measured and documented. The organization consistently looks for room for improvement and optimizes performance by tuning the systems and redistributing resources. In terms of data flow, it eliminates the cause for unnecessary flow, and finds out and applies technologies that enable improvement of performance. 2.6 DQM Maturity Levels Data Quality Criteria
  26. 26. 2. DQM Maturity Model Security 1 2 3 4 5 Process Maturity Level In this stage, subjects for security control and the security level are defined. The subjects are divides and evaluated. Based on controls on access of systems, basic identification and confirmation are conducted. Also, back-up plans are set up to ensure continuity. In this stage, organizations are constituted, businesses are divided, and guidelines and processes are established to ensure security. The organization in charge conducts standardized management of security. All processes including controls on access, back-ups and restorations are established and roles and responsibilities of people are defined. Besides, all processes are defined such as the response process in case of an emergency. From the perspective of the enterprise, security management of data is carried out consistently, and loggings and analyses are conducted on all uses. Loggings and audits analyze and decide threats, which are reflected in the security policy. Vulnerabilities are assessed and threats are removed on a regular basis. Loggings and audits apply to all uses. Data security management is stepped up in this stage. People in charge of different businesses can keep each other in check, and when permitted, proper controls are implemented on one. Complex identification and confirmation systems including a certificate system are implemented and tools are established to prevent an insider from revealing secret information. Authority division and important data are encoded. Data security management is completed in this stage. The optimization is not confined to the current environment, but also applies to future environments. The organization assesses the current security management, examines whether it is effective in protecting the data, and complement weaknesses. Besides, the organization sets up threat control system that consists of several levels and completes a defense system that can actively handle any unexpected threat. 2.6 DQM Maturity Levels Data Quality Criteria
  27. 27. 2. DQM Maturity Model - An Example of the Maturity Level Certification Accuracy Level 5 Consistency Level 5 Usability Level 5 Accessibility Level 5 Timeliness Level 5 Security Level 5 Validity Utility Data Quality Criteria Accuracy Level 4 Consistency Level 4 Usability Level 4 Accessibility Level 4 Timeliness Level 4 Security Level 4 Accuracy Level 3 Consistency Level 3 Usability Level 3 Accessibility Level 3 Timeliness Level 3 Security Level 3 Accuracy Level 2 Consistency Level 2 Usability Level 2 Accessibility Level 2 Timeliness Level 2 Security Level 2 Accuracy Level 1 Consistency Level 1 Usability Level 1 Accessibility Level 1 Timeliness Level 1 Security Level 1 DQM Maturity In Validity Level 3 DQM Maturity in Utility Level 2 DQM Maturity of Organization Level 2 2.6 DQM Maturity Levels
  28. 28. <ul><li>This model has been designed to evaluate DQM processes. We need to extend the model to cover the evaluation for data quality </li></ul><ul><li>DQM Maturity Model can be provided as a reference framework for ISO 8000 data quality. </li></ul><ul><li>For further Information, please contact: Sunho Kim, Myongji University (www.myongji.ac.kr) Inchang Kim, KDPC (ickim@dpc.or.kr) </li></ul>3. Conclusion
  29. 29. Thank You

×