3 Js Devdoc (Peer Review)


Published on

OLIT 593 008 Collaborative Knowledge Creation

  • Be the first to comment

  • Be the first to like this

3 Js Devdoc (Peer Review)

  1. 1. John  Jason  Jen <ul><li>Collaborative Knowledge Creation </li></ul><ul><li>Development Presentation </li></ul><ul><li>for </li></ul><ul><li>University Review </li></ul><ul><li>April 10, 2008 </li></ul>
  2. 2. Background: Peer-Review <ul><li>Peer-review is a collaborative knowledge creation process. The intended product is an accurate and significant scientific contribution in the form of a paper. The authors and reviewers engage in an iterative editing process mediated by a journal editor to develop the paper to publishable quality. </li></ul>
  3. 3. Background: Peer-Review <ul><li>A journal exists so as to streamline and centralize the repetitive tasks involved in publishing a work. </li></ul><ul><ul><li>To readers, the journal must offer credible and significant work in the research area that is executed, organized, and displayed in a pleasing manner. </li></ul></ul><ul><ul><li>To the authors, the journal must offer peer-review services and credence to the work. </li></ul></ul><ul><ul><li>To the reviewers, the journal must provide an environment that encourages their voluntary and unpaid labor. </li></ul></ul>
  4. 4. Problem <ul><li>The current implementation of peer-review at University Review does not take advantage of collaborative knowledge creation techniques. </li></ul>
  5. 5. Task <ul><li>We will restructure the peer-review process of University Review by developing a comprehensive workflow plan to more effectively and efficiently produce academic knowledge. </li></ul>
  6. 6. Knowledge Enablers <ul><li>Instill A Vision </li></ul><ul><li>Manage Conversations </li></ul><ul><li>Mobilize Knowledge Artifacts </li></ul><ul><li>Create Right Content </li></ul><ul><li>Globalize Local Knowledge </li></ul>
  7. 7. Enabler 1 Instill a Vision <ul><li>The new vision of University Review is: </li></ul><ul><ul><ul><li>&quot;honoring knowledge and academic integrity through iterative review, grammatical standards, and layout design.&quot; </li></ul></ul></ul>
  8. 8. Criteria of a Good Knowledge Vision <ul><li>Commitment to a Direction </li></ul><ul><li>Generativity </li></ul><ul><li>Specific Style </li></ul><ul><li>Restructuring the Knowledge System </li></ul><ul><li>Restructuring the Task System </li></ul><ul><li>Communication of Values </li></ul><ul><li>Shaping Competitiveness </li></ul>
  9. 9. Criterion I Commitment to a Direction <ul><li>Our knowledge vision places the journal as a steward of knowledge </li></ul><ul><li>“ Honoring Knowledge” </li></ul>
  10. 10. Criterion II Generativity <ul><li>The phrase &quot;honoring knowledge&quot; has the power to stimulate new organizational imagination. </li></ul>
  11. 11. Criterion III Specific Style <ul><li>The phrasing of the vision encompasses a role for every person involved in work for the journal. </li></ul><ul><ul><li>Iterative review </li></ul></ul><ul><ul><li>Grammatical standards </li></ul></ul><ul><ul><li>Layout design </li></ul></ul>
  12. 12. Criterion IV Restructuring the Knowledge System <ul><li>The vision emphasizes team spirit, iteration, and speed, which are three criteria for increasing the effectiveness of the knowledge creation process. </li></ul>
  13. 13. Criterion V Restructuring the Task System <ul><li>The vision is a guide for the new workflow </li></ul>
  14. 14. Criterion VI Communication of Values <ul><li>The phrase &quot;honoring knowledge&quot; is a clear delineator of University Review from the rest of the publishing world. </li></ul><ul><li>The inclusion of “academic integrity” in the parsimonious vision statement places its realization as a priority. </li></ul>
  15. 15. Criterion VII Shaping Competitiveness <ul><li>Authors want to publish in a journal that has a fast turnaround. </li></ul><ul><li>Reviewers want to feel a part of the process. </li></ul><ul><li>Readers want a high quality product. </li></ul><ul><li>This vision incorporates a commitment to all three desires. </li></ul>
  16. 16. <ul><li>Conversations at University Review will have a number of unique attributes: </li></ul><ul><ul><ul><ul><li>they will be asynchronous </li></ul></ul></ul></ul><ul><ul><ul><ul><li>they will be text-based </li></ul></ul></ul></ul><ul><ul><ul><ul><li>they will be (often) held between people who do not know each other </li></ul></ul></ul></ul><ul><ul><ul><ul><li>the editor will be the only person who knows who all the other discussants are </li></ul></ul></ul></ul><ul><ul><ul><ul><li>they will have a clear purpose: to improve the existing paper </li></ul></ul></ul></ul><ul><ul><ul><ul><li>they will be on a deadline </li></ul></ul></ul></ul>Enabler 2 Manage Conversations
  17. 17. Principles of Good Conversations <ul><li>Actively Encourage Participation </li></ul><ul><li>Establish Conversational Etiquette </li></ul><ul><li>Edit Conversations </li></ul><ul><li>Foster Innovative Language </li></ul>
  18. 18. Principle I Actively Encourage Participation <ul><li>Communication from the editor will be courteous. </li></ul><ul><li>Expectations for participation will be stated on the journal website. </li></ul>
  19. 19. Principle II Establish Conversational Etiquette <ul><li>Sample documents will demonstrate proper tone and commenting procedure. </li></ul><ul><li>Expectations for participations will be stated on the journal website. </li></ul>
  20. 20. Principle III Edit Conversations <ul><li>It will be the responsibility of the editor to decide what comments to pass along to the author and to make the final decision as to the paper's fitness for publication. </li></ul>
  21. 21. Principle IV Foster Innovative Language <ul><li>Innovative language is not desired. </li></ul><ul><li>Innovative language must be clearly defined and placed in reference to existing language with appropriate reference to the literature. </li></ul>
  22. 22. Enabler 3 Mobilize Knowledge Activists <ul><li>Empowerment for authors and contributors </li></ul><ul><li>Tacit to Explicit Knowledge: shared on the web for the authors and contributors </li></ul>
  23. 23. Enabler 4 Create the Right Context <ul><li>Virtual and mental spaces created to help authors, editors, reviewers, and readers honor knowledge </li></ul><ul><ul><li>Virtual space: SharePoint site </li></ul></ul><ul><ul><li>Mental space: focused with job aids </li></ul></ul>
  24. 24. Enabler 4 Create the Right Context DOCUMENTING Interactions with the website amongst author, editor, and reviewers INTERNALIZING readers who will be able to become authors once the knowledge is internalized. CONVERSING Exchange of comments amongst author, editor, and reviewers ORIGINATING Paper submission Collective Interaction Individual Interaction
  25. 25. Enabler 5 Globalize Local Knowledge <ul><li>Submission </li></ul><ul><li>Review Process </li></ul><ul><li>Reader Internalization </li></ul><ul><li>Triggering </li></ul><ul><li>Re-packaging </li></ul><ul><li>Re-creating </li></ul>
  26. 26. Knowledge Creation Steps <ul><li>Share Tacit Knowledge </li></ul><ul><li>Create a Concept </li></ul><ul><li>Justify a Concept </li></ul><ul><li>Build a Prototype </li></ul><ul><li>Cross-Leveling Knowledge </li></ul>
  27. 27. Step 1 Share Tacit Knowledge <ul><li>There will be two sources to embed explicit knowledge: </li></ul><ul><ul><li>The job aids included in the journal website compiled by the editor. </li></ul></ul><ul><ul><li>The conversation taking place between the reviewers, editor, and author. </li></ul></ul>
  28. 28. Step 1 Share Tacit Knowledge <ul><li>The role defines the incentives to share tacit knowledge: </li></ul><ul><ul><li>Authors : create a submission that quickly and smoothly passes the peer-review process and is published </li></ul></ul><ul><ul><li>Editors : build a more efficient system, help authors better prepare submissions, and help reviewers better prepare their comments </li></ul></ul><ul><ul><li>Reviewers : be given the privilege of reviewing next time, which includes access to developing works before publication and a professional association with the journal </li></ul></ul>
  29. 29. Step 2 Create a Concept <ul><li>As peer-review is an iterative process, the concept is created while the tacit knowledge is shared. The externalized knowledge will include: </li></ul><ul><ul><li>past publications of the journal to give everyone a feel for the journal </li></ul></ul><ul><ul><li>checklists to serve as job aids at point of entry for particular stages in the process </li></ul></ul><ul><ul><li>the paper itself and associated comments </li></ul></ul>
  30. 30. Step 3 Justify a Concept <ul><li>The entire peer-review process is a process of justifying a concept. It includes: </li></ul><ul><ul><li>Authors : review submission job aid and submits work </li></ul></ul><ul><ul><li>Editors : check for appropriateness of paper and assigns reviewers </li></ul></ul><ul><ul><li>Reviewers : check details of paper and provide feedback </li></ul></ul><ul><ul><li>Authors : rewrite or justify each feedback item </li></ul></ul><ul><ul><li>Editors : approve paper for publication </li></ul></ul>
  31. 31. Step 4 Build a Prototype <ul><li>Throughout the entire process, the author is in control of the prototype construction </li></ul><ul><li>The prototype begins as a submission and may require a resubmission to address reviewers’ comments </li></ul><ul><li>Proofs of the final paper must be accepted by the author before publication </li></ul>
  32. 32. Step 5 Cross-Leveling Knowledge <ul><li>It is the goal of every journal to cross-level knowledge. The journal itself is the documented knowledge and its sale circulates the knowledge. However, this new system will: </li></ul><ul><ul><li>increase efficiency and thereby shorten the time from submission to publication </li></ul></ul><ul><ul><li>document every step of the process by requiring that communication be electronic and then organizing and storing it </li></ul></ul>
  33. 33. Collaboration Process Elements <ul><li>Major steps </li></ul><ul><li>Knowledge product </li></ul><ul><li>Performance objective </li></ul><ul><li>Knowledge required </li></ul>
  34. 34. The Major Step <ul><li>There is only one major step in the peer-review process and that is to produce a publishable paper </li></ul>
  35. 35. The Knowledge Product <ul><li>There is only one knowledge product that will result from peer-review and that is a publishable paper . </li></ul>
  36. 36. The Performance Objectives <ul><li>The objectives are as follows: </li></ul><ul><ul><li>The subject matter of the paper aligns with that of University Review </li></ul></ul><ul><ul><li>The paper is formatted to reflect the standards of University Review </li></ul></ul><ul><ul><li>The paper contains enough information to replicate the results reported </li></ul></ul><ul><ul><li>The paper represents a significant advance in the field </li></ul></ul>
  37. 37. The Knowledge Required <ul><li>The knowledge required to produce the knowledge product can be analyzed by role: </li></ul><ul><ul><li>Authors : must have the specific knowledge to complete the work as well as to write an academic paper </li></ul></ul><ul><ul><li>Editor: must have command of the workflow </li></ul></ul><ul><ul><li>Reviewers : must have domain knowledge of the paper’s subject matter as well as of how to review an academic paper </li></ul></ul>
  38. 38. The Knowledge Required <ul><li>Peer-review at University Review is a Wikinomy: </li></ul><ul><li>“ a way of producing goods and services that relies … on self-organizing, egalitarian communities of individuals who come together voluntarily to produce a shared outcome” (Tapscott & Williams, 2006, p. 67) </li></ul>
  39. 39. Workflow Model <ul><li>Introduction </li></ul><ul><li>Routing </li></ul><ul><li>Triggers </li></ul><ul><li>Petri-net </li></ul><ul><li>Embedding learning into the workflow model </li></ul><ul><ul><li>Documents Embody Factual Knowledge </li></ul></ul><ul><ul><li>Instruction Embodies Conceptual Knowledge </li></ul></ul><ul><ul><li>Examples Embody Procedural Knowledge </li></ul></ul><ul><ul><li>Expert Advice Embodies Metacognitive Knowledge </li></ul></ul>
  40. 40. Workflow Process <ul><li>Submission </li></ul><ul><ul><li>An author submits a paper (email), the paper is received by the journal, the journal sends an acknowledgment of receipt (email), the editor reviews the paper for appropriateness, if not appropriate, the editor emails the author, if appropriate, the peer review process starts. </li></ul></ul>
  41. 41. Workflow Process <ul><li>Peer-Review </li></ul><ul><ul><li>Three reviewers are identified, the paper is sent to them (email), each reviewer agrees (or not) to review it (email), each sends it back in a timely manner or a reminder email is sent by the journal, once all three reviews get back to the editor, the editor emails his/her decision to the author. </li></ul></ul>
  42. 42. Workflow Process <ul><li>Author Submission </li></ul><ul><li>Visit Journal </li></ul><ul><li>website </li></ul><ul><li>Review criteria </li></ul><ul><li>Submit paper </li></ul><ul><li>Editor </li></ul><ul><li>Initial Review </li></ul><ul><li>Confirm receipt </li></ul><ul><li>Decide if appropriate </li></ul><ul><li>If yes - review </li></ul><ul><li>If no - notify author </li></ul><ul><li>Peer </li></ul><ul><li>Review </li></ul><ul><li>Notify author </li></ul><ul><li>Invite reviewers </li></ul><ul><li>Review feedback </li></ul><ul><li>Editor decision </li></ul><ul><li>Editor </li></ul><ul><li>Final Review </li></ul><ul><li>Positive review - publish </li></ul><ul><li>Negative review – do not publish </li></ul><ul><li>Notify author </li></ul>
  43. 43. Workflow Process <ul><li>Routing Required </li></ul><ul><ul><li>Sequential </li></ul></ul><ul><ul><li>Selective - for example, the reviewer accepts or declines to review </li></ul></ul><ul><ul><li>Iterative - for example, the author resubmits the paper with suggested changes </li></ul></ul><ul><li>Routing Not Required </li></ul><ul><ul><li>Parallel </li></ul></ul>
  44. 44. Workflow Process <ul><li>Triggers Required </li></ul><ul><ul><li>Time Signal - for example, if the reviewer is taking too long to reply, they will receive a reminder email </li></ul></ul><ul><ul><li>Resource Initiative - for example, all actions of the author, editor, and reviewers in the system will be triggers </li></ul></ul><ul><ul><li>External Event - for example, receiving feedback from a reviewer or a resubmitted paper </li></ul></ul>
  45. 45. Petri Nets <ul><li>Background: </li></ul><ul><li>Petri nets are visual presentations showing workflow management. </li></ul><ul><li>The process has one entry point and one end point with a series of conditions and tasks. (p 49) </li></ul>
  46. 46. Petri Nets <ul><li>Symbols and Functions: </li></ul><ul><li>Conditions (or “cases”) are shown as places and represented by the circle symbol. </li></ul><ul><li>Tasks are shown as transitions by the square symbol. </li></ul><ul><li>Tokens correspond with particular cases and are shown by a dot. </li></ul><ul><li>The decision process involves joining and/or splitting at transition points. </li></ul><ul><li>Transitions are eager to fire as soon as they are enabled through triggering. </li></ul><ul><li>Arrows demonstrate the directional flow. </li></ul>
  47. 47. Petri Nets Work Items & Activities <ul><li>Work Items involve cases and tasks </li></ul><ul><li>Activities include cases, tasks and resources. (p 84) </li></ul>select_notification Case 3 reviewer_decision Case 2 review_criteria Case 1 Task Case WORK ITEMS Case 3 Case 2 Case 1 Case John select_notification Jason reviewer_decision Jen review_criteria Resources Task ACTIVITIES
  48. 48. Scholarly Journal Review Process <ul><li>Author: </li></ul><ul><li>Submission </li></ul><ul><li>Editor: </li></ul><ul><li>Initial Review </li></ul><ul><li>Peer: Review </li></ul><ul><li>Editor: </li></ul><ul><li>Final Review </li></ul>Start visit website review criteria submit paper confirm receipt editor review invite reviewers accept decline conduct reviews receive feedback publish? no receive submission accept decline editor review yes notify author select notification select reviewers publish evaluation end reviewer decision
  49. 49. SharePoint site <ul><li>Please visit the SharePoint site for University Review at: </li></ul><ul><li> </li></ul><ul><li>Also available as a link on the homepage of University Review found on the class SharePoint site </li></ul><ul><li>Comments Welcome! </li></ul>