Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evaluating Digital Scholarship, Alison Byerly


Published on

While a number of professional organizations have produced valuable guidelines for evaluation of digital work, many colleges and universities have yet to establish clear protocols and practices for applying them. Alison Byerly, College Professor and former Provost and Executive Vice President at Middlebury College, who has co-led workshops on evaluating digital scholarship at the MLA convention, will review major issues to be considered in the evaluation of digital work, such as: presentation of medium-specific materials, documentation of multiple roles in collaborative work, changing forms of peer review, and identification of appropriate reviewers. She will then talk briefly about how these issues can best be approached from the perspective of the candidate who wishes to present his or her work effectively to review committees, as well as from the perspective of colleagues who wish to provide a well-informed evaluation of such work.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Evaluating Digital Scholarship, Alison Byerly

  1. 1. Evaluating Digital ScholarshipAlison Byerly, College Professor, Middlebury College NITLE Seminar, October 10, 2012
  2. 2. Evaluating Digital ScholarshipNITLE Seminar, Oct. 10, 2012Alison Byerly , Middlebury College
  3. 3. Challenges of evaluating digital scholarshipMay differ from traditional scholarship in:(1) analytic approach or content and/or(2) format, medium, method of publication or disseminationDigital humanities: not really a field or discipline - an approach, an array ofpractices. These practices are often more typical of sciences and socialsciences than of traditional humanities• data-focused or object-focused• visualization and spatial analysis• collaborative; -open-access and non-proprietary• process-oriented rather than product-oriented
  4. 4. Evaluation• Good evaluation depends on shared context: clear expectations, defined roles, recognized measures of success• Both parties – candidate and evaluator(s) –have a role to play in establishing the appropriate context• Ideally, this work begins the moment a new colleague is hired – if not sooner
  5. 5. Essential tasks for digital scholarsEducate your colleagues• Define the audience of peers addressed by your work, and actively engage that audience• Document your specific role and effort, particularly if it is collaborative• Explain the significance of specific achievements, recognition, or prestige markers• Look for opportunities to create narratives describing your work
  6. 6. Essential tasks for evaluatorsEducate yourselves• Review and assess any project in the medium in which it was created.• Recognize the intrinsically collaborative nature of digital projects.• Consult specialists in relevant disciplines regarding the various components of a candidate’s work• Assess candidate’s work in relation to overall institutional expectations of quality, recognizing that specific metrics of success will differ in new areas
  7. 7. Guidelines from professional organizations• MLA (Modern Language Association)– Guidelines for Evaluation of Work in Digital Humanities and Digital Media• AHA (American Historical Association) – Suggested Guidelines for Evaluating Digital Media Activities in Tenure, Review, and Promotion• 2011 volume of MLAs Profession contains some excellent essays (,• -NINES ‘ Evaluating Digital Scholarship site, "Guidelines for Promotion and Tenure Committees in Judging Digital Work" ( ).
  8. 8. New publication and presentation formats websites, online manuscripts, databases, apps…..(1) Website NINES: Nineteenth-Century Scholarship Online 992,065 peer-reviewed digital objects from 119 federated sites:The Old Bailey Online is a fully searchable edition of the largest body of texts detailing the lives of non-elite people ever published, containing 197,745 criminal trials held at Londons central criminal court.The Yellow Nineties Online publishes digitized facsimile editions of a select collection of periodicals…a rich historical archive of paratextual materials related to the production and reception of these periodicals…also publish peer-reviewed scholarship: biographies of the writers, authors, publishers, and others associated with the period; scholarly introductions and commentary; and essays on our process of building the site and encoding its digital objects. All documents are marked-up and fully searchable.
  9. 9. (2) In-process online manuscript Complex TV: The Poetics of Contemporary Television Storytelling by Jason Mittell. ‘This site hosts the peer-to-peer review of the in-progress manuscript Complex TV: The Poetics of Contemporary Television Storytelling by Jason Mittell (copyright 2012). The book proposal was posted in March 2011, with individual chapters released in serialized installments starting in Spring 2012. The project is currently under review contract with NYU Press, who has allowed me to post the pieces here for pre-publication and open-review. The draft manuscript with comments will continue to live online here, even after the book has been published.’See also JM’s blog post, “Thoughts on Blogging for Tenure”
  10. 10. (3) Database or programDocuscope: a text analysis environment with a suite of interactivevisualization tools for corpus-based rhetorical analysis.
  11. 11. Peer review – identifying the relevant scholarlyaudience or community• HASTAC: Humanities, Arts, Sciences, and Technology Advanced Collaboratory “HASTAC ("haystack") -- a network of individuals and institutions inspired by the possibilities that new technologies offer for shaping how we learn, teach, communicate, create… the digital era provides rich opportunities for informal and formal learning and for collaborative, networked research that extends across traditional disciplines, across the boundaries of academe and community, across the "two cultures" of humanism and technology, across the divide of thinking versus making….”)
  12. 12. Digital Humanities – major outlets • JDH: Journal of Digital Humanities “A comprehensive, peer-reviewed, open access journal that features the best scholarship, tools, and conversations produced by the digital humanities community in the previous quarter. JDH offers expanded coverage of the digital humanities publishing scholarly work beyond the traditional research article…by selecting content from open and public discussions in the field…[and] by encouraging continued discussion through peer-to-peer review.” • DHQ: Digital Humanities Quarterly • Anvil Academic – A new all-digital, open-access publisher for the digital humanities. analog-star-an-interview-with-fred-moody-of-anvil-academic/42936
  13. 13. --there are some printed books, too—•A Companion to Digital Literary Studies, ed. Susan Schreibman andRay Siemans. Blackwell, 2008.•Debates in the Digital Humanities. Ed. Matthew Gold. U of Minnesota, 2012.
  14. 14. Other resources• MLA 2012 Workshop on Digital Scholarship “The Challenges of Digital Scholarship,” Chronicle of Higher Ed, January 25, 2012 scholarship/38103• See 10/5/12 #anvil twitterchat re: evaluation of digital scholarship: digital-wo#publicize