An Empirical Study on Criteria for Assessing Information Quality in Corporate Wikis

1,295 views
1,223 views

Published on

Slides from ICIQ 2009, final draft of the paper is online at: http://bit.ly/vyusG

Sorry for the template. Had to use that...

Published in: Technology, Design
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,295
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
29
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

An Empirical Study on Criteria for Assessing Information Quality in Corporate Wikis

  1. 1. An Empirical Study on Criteria for Assessing Information Quality in Corporate Wikis Therese Friberg Wolfgang Reinhardt University of Paderborn 14th International Conference on Information Quality 2009 (ICIQ 2009) 08.11.2009 Potsdam, Germany
  2. 2. Outline <ul><li>Motivation </li></ul><ul><li>Basics </li></ul><ul><li>Frameworks of information quality </li></ul><ul><li>Development of the Wiki-Set </li></ul><ul><li>Definition of the criteria </li></ul><ul><li>Presentation of the study </li></ul><ul><li>Results with technical approaches </li></ul><ul><li>Conclusion </li></ul>
  3. 3. Motivation <ul><li>Growing amount of information on the Internet </li></ul><ul><li>Efficient knowledge management becomes more and more important </li></ul><ul><li>Organizations want to use knowledge management successfully </li></ul><ul><li>No reliable statement about the information quality of information </li></ul><ul><li>68% criticize the quality of information in organizations (study of 2007 with 610 managers, Information Builders GmbH) </li></ul>
  4. 4. Basics – items <ul><li>What is information quality? </li></ul><ul><ul><li>Properties of a data object that describes the value of its content </li></ul></ul><ul><li>Wikis offer many advantages: </li></ul><ul><ul><li>Every user can create or modify content </li></ul></ul><ul><ul><li>Changes can be tracked </li></ul></ul><ul><ul><li>Easy to use </li></ul></ul><ul><ul><li>Cross-linking between content  Sharing knowledge and building up communities </li></ul></ul><ul><li> But does there really exist high information quality in wikis? </li></ul>
  5. 5. Basics – information quality <ul><li>Studies about Wikipedia show high information quality </li></ul><ul><ul><li>E.g.: Wiegand (2007) shows Wikipedia on the same level as the big German encyclopedias (as Encarta, Brockhaus und Bertelsmann) </li></ul></ul><ul><li>Wikis have great potential for a high information quality </li></ul><ul><li>Organizations already use the opportunity </li></ul><ul><li>Assessment?  manual and automatic </li></ul><ul><li>But: According to which criteria? </li></ul>
  6. 6. Gräfe (2005) Wang & Strong (1996) Brändle (2005) Alexander & Tate (1999) English (1999) Eppler (2003) Wikipedia Frameworks of information quality
  7. 7. Gräfe (2005) Wang & Strong (1996) Brändle (2005) Alexander & Tate (1999) English (1999) Eppler (2003) Wikipedia Wang & Strong (1996) <ul><li>Accuracy </li></ul><ul><li>Objectivity </li></ul><ul><li>Believability </li></ul><ul><li>Reputation </li></ul><ul><li>Accessibility </li></ul><ul><li>Security </li></ul><ul><li>Relevancy </li></ul><ul><li>Value-added </li></ul><ul><li>Timeliness </li></ul><ul><li>Completeness </li></ul><ul><li>Amount of information </li></ul><ul><li>Interpretability </li></ul><ul><li>Ease of understanding </li></ul><ul><li>Concise representation </li></ul><ul><li>Consistent representation </li></ul>Frameworks of information quality
  8. 8. Modelle der Informationsqualität Gräfe (2005) Wang & Strong (1996) Brändle (2005) Alexander & Tate (1999) English (1999) Eppler (2003) Wikipedia Wikipedia <ul><li>Well-written </li></ul><ul><li>Comprehensive </li></ul><ul><li>Factually accurate </li></ul><ul><li>Neutral </li></ul><ul><li>Stable </li></ul><ul><li>Lead </li></ul><ul><li>Appropriate structure </li></ul><ul><li>Consistent citations </li></ul><ul><li>Images </li></ul><ul><li>Length </li></ul>Frameworks of information quality
  9. 9. Initial point: 50 criteria Development of the Wiki-Set Accuracy Amount of Information Derivation integrity Conciseness Stable Objectivity Interpretability Concurrency of red. or distributed data Clarity Speed Believability Ease of understanding Contextual clarity Applicability Transparenz Reputation Concise Representation Equivalence of red. or distr. data Comprehensive-ness Ausgewogenheit und Vielfalt Accessibility Consistent Representation Usability Consistency Gestaltung Security Definition conformance Rightness (or fact completeness) Correctness Multimedialität Relevancy Validity business rule conformance Interaction and Transaction design Convenience Bereitstellung Value-added Accuracy to surrogate source Currency Traceability Neuigkeit Timeliness Precision Coverage and Intended Audience Interactivity Informations-inhalt Completeness Nonduplication Authority Maintainability well-written
  10. 10. 1. phase  19 criteria Development of the Wiki-Set Accuracy Amount of Information Derivation integrity Conciseness Stable Objectivity Interpretability Concurrency of red. or distributed data Clarity Speed Believability Ease of understanding Contextual clarity Applicability Transparenz Reputation Concise Representation Equivalence of red. or distr. data Comprehensive-ness Ausgewogenheit und Vielfalt Accessibility Consistent Representation Usability Consistency Gestaltung Security Definition conformance Rightness (or fact completeness) Correctness Multimedialität Relevancy Validity business rule conformance Interaction and Transaction design Convenience Bereitstellung Value-added Accuracy to surrogate source Currency Traceability Neuigkeit Timeliness Precision Coverage and Intended Audience Interactivity Informations-inhalt Completeness Nonduplication Authority Maintainability well-written
  11. 11. 1. phase  19 criteria Development of the Wiki-Set Accuracy Amount of Information Objectivity Interpretability Clarity Believability Reputation Comprehensive-ness Accessibility Usability Gestaltung + Multimedialität Security Correctness Relevancy Value-added Currency Timeliness Precision Completeness
  12. 12. 2. phase  5 criteria Development of the Wiki-Set Accuracy Amount of Information Objectivity Interpretability Clarity Believability Reputation Comprehensive-ness Accessibility Usability Gestaltung + Multimedialität Security Correctness Relevancy Value-added Currency Timeliness Precision Completeness
  13. 13. 2. phase  5 criteria Development of the Wiki-Set Accuracy Amount of Information Objectivity Interpretability Clarity Believability Reputation Comprehensive-ness Accessibility Usability Gestaltung + Multimedialität Security Correctness Relevancy Value-added Currency Timeliness Precision Completeness
  14. 14. I N D I C A T O R S Criteria of the Wiki-Set <ul><li>Believability </li></ul><ul><li>Trust </li></ul><ul><li>Connection to and recognition of the author </li></ul><ul><li>Existence of references </li></ul><ul><li>Access rates of an article </li></ul>
  15. 15. I N D I C A T O R S Criteria of the Wiki-Set <ul><li>Believability </li></ul><ul><li>Trust </li></ul><ul><li>Connection to and recognition of the author </li></ul><ul><li>Existence of references </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Relevance </li></ul><ul><li>Subjective </li></ul><ul><li>Impact </li></ul><ul><li>Title of an article </li></ul><ul><li>Abstract of an article </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Length of an article </li></ul>
  16. 16. I N D I C A T O R S Criteria of the Wiki-Set <ul><li>Believability </li></ul><ul><li>Trust </li></ul><ul><li>Connection to and recognition of the author </li></ul><ul><li>Existence of references </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Relevance </li></ul><ul><li>Subjective </li></ul><ul><li>Impact </li></ul><ul><li>Title of an article </li></ul><ul><li>Abstract of an article </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Length of an article </li></ul><ul><li>Timeliness </li></ul><ul><li>Age </li></ul><ul><li>„ up-to-date“ </li></ul><ul><li>Date of last modification </li></ul><ul><li>Creation date </li></ul><ul><li>Outdated or unreachable links </li></ul>
  17. 17. I N D I C A T O R S Criteria of the Wiki-Set <ul><li>Believability </li></ul><ul><li>Trust </li></ul><ul><li>Connection to and recognition of the author </li></ul><ul><li>Existence of references </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Relevance </li></ul><ul><li>Subjective </li></ul><ul><li>Impact </li></ul><ul><li>Title of an article </li></ul><ul><li>Abstract of an article </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Length of an article </li></ul><ul><li>Timeliness </li></ul><ul><li>Age </li></ul><ul><li>„ up-to-date“ </li></ul><ul><li>Date of last modification </li></ul><ul><li>Creation date </li></ul><ul><li>Outdated or unreachable links </li></ul><ul><li>Completeness </li></ul><ul><li>Amount </li></ul><ul><li>References </li></ul><ul><li>Length of an article </li></ul>
  18. 18. I N D I C A T O R S Criteria of the Wiki-Set <ul><li>Believability </li></ul><ul><li>Trust </li></ul><ul><li>Connection to and recognition of the author </li></ul><ul><li>Existence of references </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Relevance </li></ul><ul><li>Subjective </li></ul><ul><li>Impact </li></ul><ul><li>Title of an article </li></ul><ul><li>Abstract of an article </li></ul><ul><li>Access rates of an article </li></ul><ul><li>Length of an article </li></ul><ul><li>Timeliness </li></ul><ul><li>Age </li></ul><ul><li>„ up-to-date“ </li></ul><ul><li>Date of last modification </li></ul><ul><li>Creation date </li></ul><ul><li>Outdated or unreachable links </li></ul><ul><li>Completeness </li></ul><ul><li>Amount </li></ul><ul><li>References </li></ul><ul><li>Length of an article </li></ul><ul><li>Comprehensibility </li></ul><ul><li>Linguistic style </li></ul><ul><li>Interpretation </li></ul><ul><li>Mistakes in spelling, grammar, or punctuation </li></ul><ul><li>Structure of an article </li></ul><ul><li>Existence of additional links </li></ul><ul><li>Existence of pictures, tables, and other media </li></ul><ul><li>Declaration of required prior knowledge </li></ul>
  19. 19. Description of the survey <ul><li>Online survey with 46 questions in 6 sections </li></ul><ul><li>4-stepped Likert scale </li></ul><ul><li>Runtime: September – November 2008 </li></ul><ul><li>Participants: Employees from 17 German companies (Wincor-Nixdorf, sd&m AG, Siemens AG, Fraport GmbH …) </li></ul><ul><li>206 completed responses </li></ul>
  20. 20. Results of the survey <ul><li>Believability </li></ul>Indicator Mean Connection to the and recognition of the author Existence of references and additional links Number of published articles of the author Access rates of an article 2.85 2.38 2.16 2.12
  21. 21. Results of the survey <ul><li>Believability </li></ul><ul><li>Technical approaches for realization </li></ul><ul><ul><li>Connecting the article with the author profile </li></ul></ul><ul><ul><li>Automatic author profiles with areas of activity </li></ul></ul><ul><ul><li>Facilities to directly contact authors of articles </li></ul></ul><ul><ul><li>Number of accesses of an article </li></ul></ul><ul><ul><li>Link checker </li></ul></ul>Indicator Mean Connection to the and recognition of the author Existence of references and additional links Number of published articles of the author Access rates of an article 2.85 2.38 2.16 2.12
  22. 22. Results of the survey <ul><li>Relevance </li></ul>Indicator Mean Title of an article Abstract of an article Access rates of an article Length of an article 3.66 2.93 2.27 1.66
  23. 23. Results of the survey <ul><li>Relevance </li></ul><ul><li>Technical approaches for realization </li></ul><ul><ul><li>Tag cloud </li></ul></ul><ul><ul><li>Rating the relevance of articles to searches </li></ul></ul><ul><ul><li>Access rates of the article considering time intervals (days, weeks, months) </li></ul></ul><ul><ul><li>Manual recommendations </li></ul></ul>Indicator Mean Title of an article Abstract of an article Access rates of an article Length of an article 3.66 2.93 2.27 1.66
  24. 24. Results of the survey <ul><li>Timeliness </li></ul>Indicator Mean Date of last modification Outdated or unreachable links Creation date - 3.18 2.40
  25. 25. Results of the survey <ul><li>Timeliness </li></ul><ul><li>Technical approaches for realization </li></ul><ul><ul><li>Creation and modification dates </li></ul></ul><ul><ul><li>Link checker </li></ul></ul><ul><ul><li>Definition of timeframes for updating </li></ul></ul>Indicator Mean Date of last modification Outdated or unreachable links Creation date - 3.18 2.40
  26. 26. Results of the survey <ul><li>Completeness </li></ul>Indicator Mean Existence of references and additional links Length of an article 2.33 1.62
  27. 27. Results of the survey <ul><li>Completeness </li></ul><ul><li>Technical approaches for realization </li></ul><ul><ul><li>Existence of references and links </li></ul></ul><ul><ul><li>Existence of pictures, tables and other media </li></ul></ul><ul><ul><li>Length of an article </li></ul></ul>Indicator Mean Existence of references and additional links Length of an article 2.33 1.62
  28. 28. Results of the survey <ul><li>Comprehensibility </li></ul>Indicator Mean Structure of an article Existence of pictures, tables and other media Existence of additional links Mistakes in spelling, grammar or punctuation Declaration of required prior knowledge 3.61 3.12 3.05 2.96 2.54
  29. 29. Results of the survey <ul><li>Comprehensibility </li></ul><ul><li>Technical approaches for realization </li></ul><ul><ul><li>Existence of headings </li></ul></ul><ul><ul><li>Error check (grammar, punctuation, …) </li></ul></ul><ul><ul><li>Existence of multi media files </li></ul></ul><ul><ul><li>Existence and reachability of links </li></ul></ul><ul><ul><li>Flesh reading ease </li></ul></ul><ul><ul><li>Marking of knowledge levels </li></ul></ul>Indicator Mean Structure of an article Existence of pictures, tables and other media Existence of additional links Mistakes in spelling, grammar or punctuation Declaration of required prior knowledge 3.61 3.12 3.05 2.96 2.54
  30. 30. Technical Realization <ul><li>IQ profile for each article in the wiki </li></ul><ul><li>Presentation of prior ratings of the article </li></ul><ul><li>Facilitation of simple rating of articles </li></ul><ul><ul><li>More detailed ratings available upon request </li></ul></ul><ul><li>In the future consideration of social navigation and search relevancies </li></ul><ul><li>Technical realization currently under development </li></ul><ul><ul><li>User tests with a company with heavy wiki usage </li></ul></ul>
  31. 31. Conclusion <ul><li>Identified 5 important criteria </li></ul><ul><ul><li>Believability </li></ul></ul><ul><ul><li>Relevance </li></ul></ul><ul><ul><li>Timeliness </li></ul></ul><ul><ul><li>Completeness </li></ul></ul><ul><ul><li>Comprehensibility </li></ul></ul><ul><li>Possibilities for (semi-)automatic and manual realization </li></ul><ul><li>Participation of wiki users from various companies </li></ul><ul><li>Outlook </li></ul><ul><ul><li>Importance of IQ has to be communicated to companies </li></ul></ul><ul><ul><li>Trying to identify further indicators </li></ul></ul><ul><ul><li>Investigating the weighting of indicators </li></ul></ul><ul><ul><li>International study ( looking for partners!!! ) </li></ul></ul>
  32. 32. Therese Friberg [email_address] Thank you <ul><ul><ul><li>Therese Friberg University of Paderborn, C.I.K. [email_address] http://cik.upb.de </li></ul></ul></ul>

×