The document discusses enhancements in the peer review process through automation and new web tools, emphasizing citation linking, plagiarism detection, and the importance of recognizing reviewer contributions. It outlines challenges in knowledge discovery and certification in both retrospective and prospective contexts, highlighting the need for new metrics for evaluating scientific work. Additionally, it addresses the issues surrounding access and rights to full texts, suggesting that open access can mitigate these barriers.

1. Web Tools for Peer Reviewers … and Everyone Richard Akerman ICSTI Public Conference 2007 June 21, 2007

2. The Motivation for Science If Mr. Cavor made [the gravity-blocking substance], it would go down to posterity as Cavorite or Cavorine, and he would be made an F.R.S., and his portrait given away as a scientific worthy with Nature , and things like that. And that was all he saw! - H.G. Wells, The First Men in the Moon (1901)

3. Peer Review also remains basically unchanged And has enduring value Some enhancement through automation of review process citation linking plagiarism detection Perhaps increased recognition or openness Publish list of reviewers annually Publish reviewer comments after some embargo period

5. Overview Exploring the problem space New metrics Certification (and reward) challenges Some example web tools

6. Two Problem Spaces and Two Directions Find existing Articles, experts, clusters, objects (article discovery) Discover new Ideas, concepts, relationships (knowledge discovery) Retrospective The existing body of scientific knowledge, the citation web, known author relationships Prospective (real-time) New and unconnected or weakly-connected work and authors ( Ramanujan )

7. Retrospective Finding Start with some nucleus idea, keywords, article, author, (collection), your history Find a network of related objects How? Recommender services. Use metrics/features and find closeness/similarity in some feature space

8. Features/Quality Metrics Networks of Citations Authors (reputation) Groups and Projects Certification (journal publication) Position in these networks

9. http://www. flickr .com/photos/ darkmatterpaintball /251552464/

10. Forensics & Major Miner Problems Examining the corpse / dead trees Fossilized trees -> coal -> mining! We need to recognize this knowledge metaphor We’re mining material that is already refined

11. http://www. flickr .com/photos/ mekin /399220499/

12. Thinking about mining Currently most demonstrations are only on abstracts due to limited full-text availability So we are mining a refinement of the refinement Rights are a major problem Lack of mining rights is one factor that is driving open access Also knowledge discovery leads into difficult areas of machine reasoning What does it mean to move beyond mining?

13. Talking a walk in the the forest How do we find the new Mr. Cavor? Exploring the World Wide Web (including repositories) as opposed to the World Wide Literature Blogs, wikis, videos, datasets And pre-prints The retrospective metrics we could use are either weaker or non-existent

14. http://www. flickr .com/photos/ jzakariya /191481917/

15. What New Metrics Can We Use? Derived reputation (e.g. Nature blogs) Page rank and similar citation-like connections Bookmarks Intentional rank Comments Page hits / usage / viewing time Derived quality (important person or group) favours this object (many many people) favour this object Lots of privacy issues

16. Challenges: Download/Views The versions challenge – many copies of the article in many places http://www.lse.ac.uk/library/versions/ eprintweb is linking pre-prints to published versions Search engines (spidering)

17. Challenges for Certification What should be certified? Articles in repositories Blog entries Versioned Wiki entries Data sets Videos Annotated mashups How is certification asserted? Some sort of digital signature What sort of scientific rewards can we provide for the new Mr. Cavor?

18. Example: Eigenfactor

19. Example: eprintweb.org astro-ph/0607051 (July 2006) Bulk viscosity of Mixed nucleon-hyperon-quark Matter in Neutron stars Na-Na Pan, Xiao-Ping Zheng and Jia-Rong Li Received. 04 July 2006 Last updated. 04 July 2006 Abstract. We calculate the coefficient of bulk viscosity … Journal-ref. Mon.Not.Roy.Astron.Soc. 371 (2006) 1359 Published Article doi: 10.1111/j.1365-2966.2006.10759.x

20. Example: ScienceBlogs

21. Example: Postgenomic

22. Example: ChemRank

23. Thank You Richard dot Akerman at NRC dot CA Supplementary bookmarks at http://www.connotea.org/user/scilib/tag/icsti2007akerman © 2007 Government of Canada and licensed in the Creative Commons http://creativecommons.org/licenses/by-nc-sa/2.5/