Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Web Tools for Peer Reviewers … and Everyone Richard Akerman ICSTI Public Conference 2007 June 21, 2007
The Motivation for Science <ul><li>If Mr. Cavor made [the gravity-blocking substance], it would go down to posterity as Ca...
Peer Review also remains basically unchanged <ul><li>And has enduring value </li></ul><ul><li>Some enhancement through </l...
 
Overview <ul><li>Exploring the problem space </li></ul><ul><li>New metrics </li></ul><ul><li>Certification (and reward) ch...
Two Problem Spaces and Two Directions <ul><li>Find existing </li></ul><ul><ul><li>Articles, experts, clusters, objects (ar...
Retrospective Finding <ul><li>Start with some nucleus </li></ul><ul><ul><li>idea, keywords, article, author, (collection),...
Features/Quality Metrics <ul><li>Networks of </li></ul><ul><ul><li>Citations </li></ul></ul><ul><ul><li>Authors (reputatio...
http://www. flickr .com/photos/ darkmatterpaintball /251552464/
Forensics & Major Miner Problems <ul><li>Examining the corpse / dead trees </li></ul><ul><li>Fossilized trees -> coal -> m...
http://www. flickr .com/photos/ mekin /399220499/
Thinking about mining <ul><li>Currently most demonstrations are only on abstracts due to limited full-text availability </...
Talking a walk in the the forest <ul><li>How do we find the new Mr. Cavor? </li></ul><ul><li>Exploring the World Wide Web ...
http://www. flickr .com/photos/ jzakariya /191481917/
What New Metrics Can We Use? <ul><li>Derived reputation (e.g.  Nature  blogs) </li></ul><ul><li>Page rank and similar cita...
Challenges: Download/Views <ul><li>The versions challenge – many copies of the article in many places </li></ul><ul><ul><l...
Challenges for Certification <ul><li>What should be certified? </li></ul><ul><ul><li>Articles in repositories </li></ul></...
Example: Eigenfactor
Example: eprintweb.org astro-ph/0607051  (July 2006) Bulk viscosity of Mixed nucleon-hyperon-quark Matter in Neutron stars...
Example: ScienceBlogs
Example: Postgenomic
Example: ChemRank
Thank You <ul><li>Richard dot Akerman at NRC dot CA </li></ul><ul><li>Supplementary bookmarks at http://www.connotea.org/u...
Upcoming SlideShare
Loading in …5
×

Web Tools For Peer Reviewers... and Everyone

1,512 views

Published on

What are the challenges in exploring the existing journal literature and new forms of scientific communication? Ultimately in the online, digital environment, everyone is a potential consumer and reviewer of scientific content. What tools currently exist, and what future types of content certification and metrics can we imagine?

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Web Tools For Peer Reviewers... and Everyone

  1. 1. Web Tools for Peer Reviewers … and Everyone Richard Akerman ICSTI Public Conference 2007 June 21, 2007
  2. 2. The Motivation for Science <ul><li>If Mr. Cavor made [the gravity-blocking substance], it would go down to posterity as Cavorite or Cavorine, and he would be made an F.R.S., and his portrait given away as a scientific worthy with Nature , and things like that. And that was all he saw! - H.G. Wells, The First Men in the Moon (1901) </li></ul>
  3. 3. Peer Review also remains basically unchanged <ul><li>And has enduring value </li></ul><ul><li>Some enhancement through </li></ul><ul><ul><li>automation of review process </li></ul></ul><ul><ul><li>citation linking </li></ul></ul><ul><ul><li>plagiarism detection </li></ul></ul><ul><li>Perhaps increased recognition or openness </li></ul><ul><ul><li>Publish list of reviewers annually </li></ul></ul><ul><ul><li>Publish reviewer comments after some embargo period </li></ul></ul>
  4. 5. Overview <ul><li>Exploring the problem space </li></ul><ul><li>New metrics </li></ul><ul><li>Certification (and reward) challenges </li></ul><ul><li>Some example web tools </li></ul>
  5. 6. Two Problem Spaces and Two Directions <ul><li>Find existing </li></ul><ul><ul><li>Articles, experts, clusters, objects (article discovery) </li></ul></ul><ul><li>Discover new </li></ul><ul><ul><li>Ideas, concepts, relationships (knowledge discovery) </li></ul></ul><ul><li>Retrospective </li></ul><ul><ul><li>The existing body of scientific knowledge, the citation web, known author relationships </li></ul></ul><ul><li>Prospective (real-time) </li></ul><ul><ul><li>New and unconnected or weakly-connected work and authors ( Ramanujan ) </li></ul></ul>
  6. 7. Retrospective Finding <ul><li>Start with some nucleus </li></ul><ul><ul><li>idea, keywords, article, author, (collection), your history </li></ul></ul><ul><li>Find a network of related objects </li></ul><ul><li>How? Recommender services. </li></ul><ul><ul><li>Use metrics/features and find closeness/similarity in some feature space </li></ul></ul>
  7. 8. Features/Quality Metrics <ul><li>Networks of </li></ul><ul><ul><li>Citations </li></ul></ul><ul><ul><li>Authors (reputation) </li></ul></ul><ul><ul><ul><li>Groups and Projects </li></ul></ul></ul><ul><ul><li>Certification (journal publication) </li></ul></ul><ul><li>Position in these networks </li></ul>
  8. 9. http://www. flickr .com/photos/ darkmatterpaintball /251552464/
  9. 10. Forensics & Major Miner Problems <ul><li>Examining the corpse / dead trees </li></ul><ul><li>Fossilized trees -> coal -> mining! </li></ul><ul><li>We need to recognize this knowledge metaphor </li></ul><ul><li>We’re mining material that is already refined </li></ul>
  10. 11. http://www. flickr .com/photos/ mekin /399220499/
  11. 12. Thinking about mining <ul><li>Currently most demonstrations are only on abstracts due to limited full-text availability </li></ul><ul><ul><li>So we are mining a refinement of the refinement </li></ul></ul><ul><ul><li>Rights are a major problem </li></ul></ul><ul><ul><li>Lack of mining rights is one factor that is driving open access </li></ul></ul><ul><li>Also knowledge discovery leads into difficult areas of machine reasoning </li></ul><ul><li>What does it mean to move beyond mining? </li></ul>
  12. 13. Talking a walk in the the forest <ul><li>How do we find the new Mr. Cavor? </li></ul><ul><li>Exploring the World Wide Web (including repositories) as opposed to the World Wide Literature </li></ul><ul><li>Blogs, wikis, videos, datasets </li></ul><ul><ul><li>And pre-prints </li></ul></ul><ul><li>The retrospective metrics we could use are either weaker or non-existent </li></ul>
  13. 14. http://www. flickr .com/photos/ jzakariya /191481917/
  14. 15. What New Metrics Can We Use? <ul><li>Derived reputation (e.g. Nature blogs) </li></ul><ul><li>Page rank and similar citation-like connections </li></ul><ul><ul><li>Bookmarks </li></ul></ul><ul><li>Intentional rank </li></ul><ul><li>Comments </li></ul><ul><li>Page hits / usage / viewing time </li></ul><ul><li>Derived quality </li></ul><ul><ul><li>(important person or group) favours this object </li></ul></ul><ul><ul><li>(many many people) favour this object </li></ul></ul><ul><li>Lots of privacy issues </li></ul>
  15. 16. Challenges: Download/Views <ul><li>The versions challenge – many copies of the article in many places </li></ul><ul><ul><li>http://www.lse.ac.uk/library/versions/ </li></ul></ul><ul><ul><li>eprintweb is linking pre-prints to published versions </li></ul></ul><ul><li>Search engines (spidering) </li></ul>
  16. 17. Challenges for Certification <ul><li>What should be certified? </li></ul><ul><ul><li>Articles in repositories </li></ul></ul><ul><ul><li>Blog entries </li></ul></ul><ul><ul><li>Versioned Wiki entries </li></ul></ul><ul><ul><li>Data sets </li></ul></ul><ul><ul><li>Videos </li></ul></ul><ul><ul><li>Annotated mashups </li></ul></ul><ul><li>How is certification asserted? </li></ul><ul><ul><li>Some sort of digital signature </li></ul></ul><ul><li>What sort of scientific rewards can we provide for the new Mr. Cavor? </li></ul>
  17. 18. Example: Eigenfactor
  18. 19. Example: eprintweb.org astro-ph/0607051 (July 2006) Bulk viscosity of Mixed nucleon-hyperon-quark Matter in Neutron stars Na-Na Pan, Xiao-Ping Zheng and Jia-Rong Li Received. 04 July 2006 Last updated. 04 July 2006 Abstract. We calculate the coefficient of bulk viscosity … Journal-ref. Mon.Not.Roy.Astron.Soc. 371 (2006) 1359 Published Article doi: 10.1111/j.1365-2966.2006.10759.x
  19. 20. Example: ScienceBlogs
  20. 21. Example: Postgenomic
  21. 22. Example: ChemRank
  22. 23. Thank You <ul><li>Richard dot Akerman at NRC dot CA </li></ul><ul><li>Supplementary bookmarks at http://www.connotea.org/user/scilib/tag/icsti2007akerman </li></ul><ul><li>© 2007 Government of Canada and licensed in the Creative Commons http://creativecommons.org/licenses/by-nc-sa/2.5/ </li></ul>

×