An Empirical Study on Criteria for Assessing Information Quality in Corporate Wikis Therese Friberg Wolfgang Reinhardt University of Paderborn 14th International Conference on Information Quality 2009 (ICIQ 2009) 08.11.2009 Potsdam, Germany

Outline Motivation Basics Frameworks of information quality Development of the Wiki-Set Definition of the criteria Presentation of the study Results with technical approaches Conclusion

Motivation

Basics

Frameworks of information quality

Development of the Wiki-Set

Definition of the criteria

Presentation of the study

Results with technical approaches

Conclusion

Motivation Growing amount of information on the Internet Efficient knowledge management becomes more and more important Organizations want to use knowledge management successfully No reliable statement about the information quality of information 68% criticize the quality of information in organizations (study of 2007 with 610 managers, Information Builders GmbH)

Growing amount of information on the Internet

Efficient knowledge management becomes more and more important

Organizations want to use knowledge management successfully

No reliable statement about the information quality of information

68% criticize the quality of information in organizations (study of 2007 with 610 managers, Information Builders GmbH)

Basics – items What is information quality? Properties of a data object that describes the value of its content Wikis offer many advantages: Every user can create or modify content Changes can be tracked Easy to use Cross-linking between content ïƒ Sharing knowledge and building up communities ïƒ But does there really exist high information quality in wikis?

What is information quality?

Properties of a data object that describes the value of its content

Wikis offer many advantages:

Every user can create or modify content

Changes can be tracked

Easy to use

Cross-linking between content ïƒ Sharing knowledge and building up communities

ïƒ But does there really exist high information quality in wikis?

Basics – information quality Studies about Wikipedia show high information quality E.g.: Wiegand (2007) shows Wikipedia on the same level as the big German encyclopedias (as Encarta, Brockhaus und Bertelsmann) Wikis have great potential for a high information quality Organizations already use the opportunity Assessment? ïƒ manual and automatic But: According to which criteria?

Studies about Wikipedia show high information quality

E.g.: Wiegand (2007) shows Wikipedia on the same level as the big German encyclopedias (as Encarta, Brockhaus und Bertelsmann)

Wikis have great potential for a high information quality

Organizations already use the opportunity

Assessment? ïƒ manual and automatic

But: According to which criteria?

Gräfe (2005) Wang & Strong (1996) Brändle (2005) Alexander & Tate (1999) English (1999) Eppler (2003) Wikipedia Frameworks of information quality

Gräfe (2005) Wang & Strong (1996) Brändle (2005) Alexander & Tate (1999) English (1999) Eppler (2003) Wikipedia Wang & Strong (1996) Accuracy Objectivity Believability Reputation Accessibility Security Relevancy Value-added Timeliness Completeness Amount of information Interpretability Ease of understanding Concise representation Consistent representation Frameworks of information quality

Accuracy

Objectivity

Believability

Reputation

Accessibility

Security

Relevancy

Value-added

Timeliness

Completeness

Amount of information

Interpretability

Ease of understanding

Concise representation

Consistent representation

Modelle der Informationsqualität Gräfe (2005) Wang & Strong (1996) Brändle (2005) Alexander & Tate (1999) English (1999) Eppler (2003) Wikipedia Wikipedia Well-written Comprehensive Factually accurate Neutral Stable Lead Appropriate structure Consistent citations Images Length Frameworks of information quality

Well-written

Comprehensive

Factually accurate

Neutral

Stable

Lead

Appropriate structure

Consistent citations

Images

Length

Initial point: 50 criteria Development of the Wiki-Set Accuracy Amount of Information Derivation integrity Conciseness Stable Objectivity Interpretability Concurrency of red. or distributed data Clarity Speed Believability Ease of understanding Contextual clarity Applicability Transparenz Reputation Concise Representation Equivalence of red. or distr. data Comprehensive-ness Ausgewogenheit und Vielfalt Accessibility Consistent Representation Usability Consistency Gestaltung Security Definition conformance Rightness (or fact completeness) Correctness Multimedialität Relevancy Validity business rule conformance Interaction and Transaction design Convenience Bereitstellung Value-added Accuracy to surrogate source Currency Traceability Neuigkeit Timeliness Precision Coverage and Intended Audience Interactivity Informations-inhalt Completeness Nonduplication Authority Maintainability well-written

1. phase ïƒ 19 criteria Development of the Wiki-Set Accuracy Amount of Information Derivation integrity Conciseness Stable Objectivity Interpretability Concurrency of red. or distributed data Clarity Speed Believability Ease of understanding Contextual clarity Applicability Transparenz Reputation Concise Representation Equivalence of red. or distr. data Comprehensive-ness Ausgewogenheit und Vielfalt Accessibility Consistent Representation Usability Consistency Gestaltung Security Definition conformance Rightness (or fact completeness) Correctness Multimedialität Relevancy Validity business rule conformance Interaction and Transaction design Convenience Bereitstellung Value-added Accuracy to surrogate source Currency Traceability Neuigkeit Timeliness Precision Coverage and Intended Audience Interactivity Informations-inhalt Completeness Nonduplication Authority Maintainability well-written

1. phase ïƒ 19 criteria Development of the Wiki-Set Accuracy Amount of Information Objectivity Interpretability Clarity Believability Reputation Comprehensive-ness Accessibility Usability Gestaltung + Multimedialität Security Correctness Relevancy Value-added Currency Timeliness Precision Completeness

2. phase ïƒ 5 criteria Development of the Wiki-Set Accuracy Amount of Information Objectivity Interpretability Clarity Believability Reputation Comprehensive-ness Accessibility Usability Gestaltung + Multimedialität Security Correctness Relevancy Value-added Currency Timeliness Precision Completeness

2. phase ïƒ 5 criteria Development of the Wiki-Set Accuracy Amount of Information Objectivity Interpretability Clarity Believability Reputation Comprehensive-ness Accessibility Usability Gestaltung + Multimedialität Security Correctness Relevancy Value-added Currency Timeliness Precision Completeness

I N D I C A T O R S Criteria of the Wiki-Set Believability Trust Connection to and recognition of the author Existence of references Access rates of an article

Believability

Trust

Connection to and recognition of the author

Existence of references

Access rates of an article

I N D I C A T O R S Criteria of the Wiki-Set Believability Trust Connection to and recognition of the author Existence of references Access rates of an article Relevance Subjective Impact Title of an article Abstract of an article Access rates of an article Length of an article

Believability

Trust

Connection to and recognition of the author

Existence of references

Access rates of an article

Relevance

Subjective

Impact

Title of an article

Abstract of an article

Access rates of an article

Length of an article

I N D I C A T O R S Criteria of the Wiki-Set Believability Trust Connection to and recognition of the author Existence of references Access rates of an article Relevance Subjective Impact Title of an article Abstract of an article Access rates of an article Length of an article Timeliness Age „ up-to-date“ Date of last modification Creation date Outdated or unreachable links

Believability

Trust

Connection to and recognition of the author

Existence of references

Access rates of an article

Relevance

Subjective

Impact

Title of an article

Abstract of an article

Access rates of an article

Length of an article

Timeliness

Age

„ up-to-date“

Date of last modification

Creation date

Outdated or unreachable links

I N D I C A T O R S Criteria of the Wiki-Set Believability Trust Connection to and recognition of the author Existence of references Access rates of an article Relevance Subjective Impact Title of an article Abstract of an article Access rates of an article Length of an article Timeliness Age „ up-to-date“ Date of last modification Creation date Outdated or unreachable links Completeness Amount References Length of an article

Believability

Trust

Connection to and recognition of the author

Existence of references

Access rates of an article

Relevance

Subjective

Impact

Title of an article

Abstract of an article

Access rates of an article

Length of an article

Timeliness

Age

„ up-to-date“

Date of last modification

Creation date

Outdated or unreachable links

Completeness

Amount

References

Length of an article

I N D I C A T O R S Criteria of the Wiki-Set Believability Trust Connection to and recognition of the author Existence of references Access rates of an article Relevance Subjective Impact Title of an article Abstract of an article Access rates of an article Length of an article Timeliness Age „ up-to-date“ Date of last modification Creation date Outdated or unreachable links Completeness Amount References Length of an article Comprehensibility Linguistic style Interpretation Mistakes in spelling, grammar, or punctuation Structure of an article Existence of additional links Existence of pictures, tables, and other media Declaration of required prior knowledge

Believability

Trust

Connection to and recognition of the author

Existence of references

Access rates of an article

Relevance

Subjective

Impact

Title of an article

Abstract of an article

Access rates of an article

Length of an article

Timeliness

Age

„ up-to-date“

Date of last modification

Creation date

Outdated or unreachable links

Completeness

Amount

References

Length of an article

Comprehensibility

Linguistic style

Interpretation

Mistakes in spelling, grammar, or punctuation

Structure of an article

Existence of additional links

Existence of pictures, tables, and other media

Declaration of required prior knowledge

Description of the survey Online survey with 46 questions in 6 sections 4-stepped Likert scale Runtime: September – November 2008 Participants: Employees from 17 German companies (Wincor-Nixdorf, sd&m AG, Siemens AG, Fraport GmbH …) 206 completed responses

Online survey with 46 questions in 6 sections

4-stepped Likert scale

Runtime: September – November 2008

Participants: Employees from 17 German companies (Wincor-Nixdorf, sd&m AG, Siemens AG, Fraport GmbH …)

206 completed responses

Results of the survey Believability Indicator Mean Connection to the and recognition of the author Existence of references and additional links Number of published articles of the author Access rates of an article 2.85 2.38 2.16 2.12

Believability

Results of the survey Believability Technical approaches for realization Connecting the article with the author profile Automatic author profiles with areas of activity Facilities to directly contact authors of articles Number of accesses of an article Link checker Indicator Mean Connection to the and recognition of the author Existence of references and additional links Number of published articles of the author Access rates of an article 2.85 2.38 2.16 2.12

Believability

Technical approaches for realization

Connecting the article with the author profile

Automatic author profiles with areas of activity

Facilities to directly contact authors of articles

Number of accesses of an article

Link checker

Results of the survey Relevance Indicator Mean Title of an article Abstract of an article Access rates of an article Length of an article 3.66 2.93 2.27 1.66

Relevance

Results of the survey Relevance Technical approaches for realization Tag cloud Rating the relevance of articles to searches Access rates of the article considering time intervals (days, weeks, months) Manual recommendations Indicator Mean Title of an article Abstract of an article Access rates of an article Length of an article 3.66 2.93 2.27 1.66

Relevance

Technical approaches for realization

Tag cloud

Rating the relevance of articles to searches

Access rates of the article considering time intervals (days, weeks, months)

Manual recommendations

Results of the survey Timeliness Indicator Mean Date of last modification Outdated or unreachable links Creation date - 3.18 2.40

Timeliness

Results of the survey Timeliness Technical approaches for realization Creation and modification dates Link checker Definition of timeframes for updating Indicator Mean Date of last modification Outdated or unreachable links Creation date - 3.18 2.40

Timeliness

Technical approaches for realization

Creation and modification dates

Link checker

Definition of timeframes for updating

Results of the survey Completeness Indicator Mean Existence of references and additional links Length of an article 2.33 1.62

Completeness

Results of the survey Completeness Technical approaches for realization Existence of references and links Existence of pictures, tables and other media Length of an article Indicator Mean Existence of references and additional links Length of an article 2.33 1.62

Completeness

Technical approaches for realization

Existence of references and links

Existence of pictures, tables and other media

Length of an article

Results of the survey Comprehensibility Indicator Mean Structure of an article Existence of pictures, tables and other media Existence of additional links Mistakes in spelling, grammar or punctuation Declaration of required prior knowledge 3.61 3.12 3.05 2.96 2.54

Comprehensibility

Results of the survey Comprehensibility Technical approaches for realization Existence of headings Error check (grammar, punctuation, …) Existence of multi media files Existence and reachability of links Flesh reading ease Marking of knowledge levels Indicator Mean Structure of an article Existence of pictures, tables and other media Existence of additional links Mistakes in spelling, grammar or punctuation Declaration of required prior knowledge 3.61 3.12 3.05 2.96 2.54

Comprehensibility

Technical approaches for realization

Existence of headings

Error check (grammar, punctuation, …)

Existence of multi media files

Existence and reachability of links

Flesh reading ease

Marking of knowledge levels

Technical Realization IQ profile for each article in the wiki Presentation of prior ratings of the article Facilitation of simple rating of articles More detailed ratings available upon request In the future consideration of social navigation and search relevancies Technical realization currently under development User tests with a company with heavy wiki usage

IQ profile for each article in the wiki

Presentation of prior ratings of the article

Facilitation of simple rating of articles

More detailed ratings available upon request

In the future consideration of social navigation and search relevancies

Technical realization currently under development

User tests with a company with heavy wiki usage

Conclusion Identified 5 important criteria Believability Relevance Timeliness Completeness Comprehensibility Possibilities for (semi-)automatic and manual realization Participation of wiki users from various companies Outlook Importance of IQ has to be communicated to companies Trying to identify further indicators Investigating the weighting of indicators International study ( looking for partners!!! )

Identified 5 important criteria

Believability

Relevance

Timeliness

Completeness

Comprehensibility

Possibilities for (semi-)automatic and manual realization

Participation of wiki users from various companies

Outlook

Importance of IQ has to be communicated to companies

Trying to identify further indicators

Investigating the weighting of indicators

International study ( looking for partners!!! )

Therese Friberg [email_address] Thank you Therese Friberg University of Paderborn, C.I.K. [email_address] http://cik.upb.de

Therese Friberg University of Paderborn, C.I.K. [email_address] http://cik.upb.de