1. The document introduces Eli, a peer review tool that allows teachers to assign writing activities, collect student drafts and reviews, and analyze review data. 2. Research shows that routinely engaging in revision is associated with better writing performance, yet students do not often revise without explicit instruction. Eli aims to provide more opportunities for review and revision cycles. 3. The document demonstrates how Eli can be used by assigning a sample writing and review activity, and discusses the types of responses - criteria matching, scaled items, and comments - that Eli allows teachers to incorporate into reviews.

1. One Writer. Many Reviewers. Better Writing.

2. Peer Response & Evidence-Based Teaching in the Humanities Bill Hart-Davidson @billhd Mike McLeod @mcleodm3 Michigan State University WIDE Research Center hartdav2@msu [dotty] edu Note: these slides accompany a hands-on activity conducted with elireview.com @elireview

3. Agenda A brief introduction to and tour of Eli Breakout Sessions: 1. Doing a Review in Eli (Students POV) 2. Building a Review in Eli (Teacher’s POV) 3. Discussion: How Do You Use Peer Response? Wrap Up, Q&A, lunch!

4. Origins of Eli Invented by writing teachers for writing teachers Design grounded in theory & research (peer reviewed & published) Created, tested, and used in our own classes for two years Problem: How can we foster (and teach) more review and revision cycles than current technologies and time allow?

5. Research, Review, Better Writing The 2007 National Assessment of Educational Progress (NAEP) writing assessment:

6. Research, Review, Better Writing The 2007 National Assessment of Educational Progress (NAEP) writing assessment: Routinely engaging in revision was associated with better performance Students who stated that they were required to routinely revise scored highest on the assessment

7. Research, Review, Better Writing It is likely that giving students greater opportunity to revise will lead to improved writing.

8. Research, Review, Better Writing Yet: Students usually do not spontaneously revise at an optimal level and require explicit instruction Experts revise more often than novices, who tend not to spend much time revising

9. Research, Review, Better Writing And so: Instruction focused on a revision strategies was most helpful for writers with an undeveloped writing strategy High quality revisions must be shaped by high quality review processes—this must (and can) be taught

10. What is Eli? Coordination tool to facilitate peer review Allows teachers to focus on teaching and not: Collecting papers Compiling drafts, comments, revisions Multi-day peer review activities

11. What is Eli? Drafts, comments, revisions automatically gathered in one place Collects data about student writers across courses and across academic careers Instant, real-time feedback on writing and reviews Powerful analytic data about student writing and review work

12. What Does Eli Do? Review Analytics: All review information is aggregated in a single dashboard view Trends that correspond to your learning goals are visible Drill down to see individual students ’ work in two important roles: as a writer AND as a reviewer

13. Working with Eli Today With Mike in Room 305 Gain access to today’s workshop via Secret Code Short writing & review activity With Bill in Room 306 Demo: analyzing review results Build a review activity

14. Create an Eli Account Go to http://elireview.com and click ‘Sign Up’ Create an account If you provide an email address, you’ll be asked to verify it (if you want to be a teacher in Eli you must provide an email address) When complete, wait at the Eli dashboard for our secret code

15. The Writing Activity Write a fancy pants conference title of the sort you’d expect to see included in the program of an academic conference in your discipline. Learning goals: Students will understand the structural characteristics of the genre Students will be able to write a compelling title of their own Students will be able to correctly deploy the medial colon

16. The Mini Review Please read the draft titles submitted by your peer group and offer feedback. Use the check boxes to indicate whether their drafts meet the listed criteria. Answer the star questions, and offer a revision suggestion comment. As you will see…the magic happens when we align response types and prompts with our learning goals in ways that will generate helpful data…

17. The Mini Review, II Criteria Matching The draft title contains a medial colon. The draft contains a phrase meant to catch attendees attention. The draft contains a phrase meant to communicate the presentation’s purpose. Scaled Items I would attend this presentation. I am confident the medial colon was used appropriately. Comments Describe one change the writer could make to really improve the draft.

18. Response Types: Criteria Matching In a criteria matching response, the prompt names a trait that you wish reviewers to identify in their peers' work. Criteria matching items offer writers formative feedback the helps them prioritize next steps for revision. Criteria matching elements are also aggregated for the whole class, which helps a teacher to see what kinds of issues to prioritize in discussion with the larger group.

19. Response Types: Scaled Items These are questions that the teacher asks the reviewer to respond to on a scale. They can be used to provide feedback ranging from formal to informal at just about any stage in the process. Like criteria matching, scaled items are aggregated for the whole group so the teacher can see class averages and compare the results of individual students with those of the whole group. Eli also identifies "peer exemplars" by showing the teacher those students whose work scores highest on scaled items.

20. Response Types: Comments Comments are open-ended text responses from reviewers. Comments can be attached to specific passages in a text. Instructors can prompt reviewers to offer specific types of feedback with a comment response if they like. All comments from all reviewers are collected in one place, making it easy for a writer to strategize a revision plan. Instructors can view comments from reviewers as they come in during a review. Instructors can also endorse a comment by clicking a button.

21. One Writer. Many Reviewers. Better Writing. Thank You! @billhd @mcleodm3 @elireview elireview.com