Harley AAUP 5.18.10

Assessing the Future Landscape of Scholarly Communication: Faculty Values and Needs in Twelve Disciplines Diane Harley, Ph.D., Principal Investigator Center for Studies in Higher Education University of California, Berkeley Project Website and Associated Document Links: http:// cshe.berkeley.edu/research/scholarlycommunication http://escholarship.org/uc/cshe_fsc AAUP, June 18, 2010, Salt Lake City, UT Copyright 2010, Center for Studies in Higher Education

Why Study Users?
Funded by the A.W. Mellon Foundation
Team: D. Harley, C.J. King, S. Kryzs Acord,
S. Earl-Novell, S. Lawrence
Academic Values and Scholarly Communication Practices—Talk to scholars.
Distinguish between Archival publishing and what we call In-Progress publishing (2005)
Anthropologist, Not a publisher or librarian
Empirical approach

Motivations and Approach
Is “The lack of willingness of the faculty to change” key barrier to moving to more cost-effective (OA?) publishing models? Or something else?
Assess the criteria by which faculty decide when and in what venues to communicate the results of scholarly research—at all stages.
Analyze what roles universities and faculty play in resolution of the perceived “crisis in scholarly communication.”
Premise: Disciplinary traditions and culture matter.

Distinguish: In-progress communication VS fully peer-reviewed archival publication
We see a dichotomous situation:
Electronic forms of “print” (and other) publications consumed heavily.
Perceptions and realities of the reward system = strong adherence to conventional, high-stature “print” publications as record for reporting research and having it evaluated institutionally.
Promotion depends predominantly upon final, fully peer-reviewed archival publication (arts, professional schools different).
Making a reputation also dependent on where one publishes plus “in-progress” communication activities (e.g., conferences, service, etc.)

Methods
Research Universities
45 mostly elite institutions
160+ scholars
12 disciplinary case studies
Literature reviews
Daily environmental scans (news, listservs, blogs, twitter, announcements, etc.)
Planning project . 2005-2006
N=5 Anthropology, Biostatistics, Chemical Engineering, Law and Economics, English-language Literature.
Current project . 2007-Present
N=7 Astrophysics, Archaeology, Biology, Economics, History, Music, Political Science
NO perfect set of disciplines. We focused on diversity, attributes, affordances.

Informants
Relevant stakeholders
Faculty primarily / R1s.
Range of seniority (tenured, pretenure, postdocs, grad students).
PLUS high level administrators, editors, publishers and librarians.
A variety of “Grand Challenge” initiatives.
Many more formal and informal conversations; attendance at meetings, literature and web review, tracking online interchanges.

Interview Protocols/Topics
(1) Tenure and promotion, making a name
(2) Criteria for disseminating research at various stages (publication practices, new publication outlets, new genres)
(3) Sharing (w/ whom, when, why or why not?)
(4) Collaboration (w/ whom, when, why or why not?)
(5) Resources created and consumed; Needs
(6) Public engagement
(7) The Future

Identified Faculty Needs
(1) Development of more nuanced tenure and promotion practices that do not rely exclusively on the imprimatur of the publication or easily gamed citation metrics,
(2) A reexamination of the locus, mechanisms, timing, and meaning of peer review,
(3) Competitive high quality and affordable journals and monograph publishing platforms (with strong editorial boards, peer review, and sustainable business models),
(4) New models of publication that can accommodate arguments of varied length, rich media, and embedded links to data; plus institutional assistance to manage permissions of copyrighted material, and
(5) Support for managing and preserving new research methods and products including components of natural language processing, visualization, complex distributed databases, and GIS, among many others.

The Current Publishing Landscape
Scholars use a range of mechanisms for disseminating scholarship at various stages.
Within any given discipline there may be a variety of publishing strategies available to authors. But…
Scholars in a field can name top 3-5 journals and/or presses w/o missing a beat.
In high paradigm fields like physics, astrophysics, and mathematics , discipline-specific repositories, such as the arXiv, are essential outlets that exist alongside formal commercial- and society-owned journals. Massive data bases, shared, very expensive telescopes, satellites, etc. Computation. Visualization. Multiple authorship running in dozens.
Economists and quantitative political scientists use working paper repositories, such as SSRN and personal websites, for disseminating research but continue to rely heavily upon society and commercial journals for final archival publication. Long lag times to publication. Varied data sources, some proprietary. Sole or dual authorship common. Publishers often require open data.

The current publishing landscape (cont’d)
In computer science , peer-reviewed conference proceedings are the most prestigious archival outlet, but distribution of scholarship using more open methods, such as posting on personal websites, is not uncommon. Software as product. Multiple authorship. Scan = much is locked down.
History and archaeology, and other fields of the humanities, such as languages and media studies , rely heavily on the book for long arguments; journals are still very important as a means for disseminating short arguments, book reviews, and other communications. Rich media and graphics, maps, GIS, NLP. Sole authorship the norm. Sharing data unusual.
Musicology includes “art” historical studies of western music, computer composition, ethnomusicology, and music theory. As expected there are multiple outlets used ranging from books and critical editions to highly competitive and selective society journals to encyclopedias to MP3s, CD’s, new instruments, software, and networked performance. Sole authorship the norm.

The current publishing landscape (cont’d)
In Molecular and cell biology , and perhaps other sciences such as chemistry , that are fast-moving, well-funded, highly competitive, and have commercial potential , there is a MUCH more limited range of outlets (although numerically many more journals in some subfields).
The journal article reigns in these fields and the more prestigious the journal, the better from the perspective of faculty at competitive institutions. Nature, Cell, Science. (Former 2 the enemy of reasonable costs)
Some PloS journals rising in prestige (not PloS One). Society Journals (and scholar editors) the most trusted.
Preprint servers are unheard of. Publication lags are exceptionally short. Large databases linked to publication are common. Impt. of speed, visualization, replication, fraud detection. Multiple authorship. Open data in genomics, etc.

T&P as flexible system?
Stellar Publication record impt for T&P. Service, Teaching, Public Engagement. Impt but age and institution dependent.
“ Groundbreaking, moves field forward, judged of high quality by internal and external reviewers, original”
Quality over quantity. Metrics such as impact factor viewed w/ suspicion.
“ bibliometrics is a deeply corrupt field, which encourages individuals to work to the impact factor algorithms; it should be thrown out entirely.”
Exceptions to the “rules” made; lots of filtering before someone hired (“good fit” within the department).
Databases, cell lines, critical editions, software: credited in T&P decisions; rarely sole criteria in most fields.
Assessing multiple authorship a growing challenge. Interdisciplinary work can present challenges as well.
New Journals (OA), New Genres are acceptable as long as peer reviewed.

Peer Review and Advancement Process
Heavy reliance on peer reviewed publications to aid institution/T&P committees and external reviewers in evaluation of scholarly work.
Impt. of external reviews in T&P.
Damning: “No one has heard of you.” “High impact” publications are a way of making a name.
Many worry that lack of peer review is associated with newer, untested forms of publication. Incl. OA.
Advancement process can and should be supportive (and unprejudiced) of non-traditional publishing models, provided that peer review is strongly embedded.
Written policy: New electronic genres should NOT be undervalued in consideration of advancement. Does actual practice vary? Committees not seeing many examples that deviate from the norm.

Peer Review in Promotion and Publishing are Separate but Deeply Intertwined
Concern that T&P committees rely too heavily on proxy of imprimatur and citation metrics.
Publication as a proxy for assessing scholarship follows from the exponential growth and resulting compartmentalization of knowledge across the Academy, which has meant that individuals within a faculty can no longer effectively review the work of their peers.
Expectations trickle-down from R-1 to aspirant institutions.
Leads to explosion of low quality publications (“arms race,” “inflationary currency”)
Proliferation of (esp. commercial) publishing entities/journals
Peer review and reviewers are being swamped
= Economically unsustainable system

Peer review is the coin of the realm
Q’s = Where faculty publish and why. We did NOT set out to study peer review. We heard:
THE value system supporting assessment of and perceived quality of research.
The primary mechanism through which research quality is nurtured.
The primary mechanism through which research is made both effective and efficient.
Excellent quality filter for the proliferating mass of scholarly information available on the web.
More difficult for time-pressed scholars to sift through it all.
This will get worse. Need more formal gate-keeping embodied in current publishing system? Other systems?

Peer Review: Problems and New Models.
Recent meeting. Courant, Schekman, Yamamoto, Wolpert, Wheatley, Schulenburger, Dougherty, Faran, Lynch, Withey, Hilton, et al.
Working Papers: PEER REVIEW IN ACADEMIC PROMOTION AND PUBLISHING: Its Meaning, Locus, and Future http://cshe.berkeley.edu/publications/publications.php?id=357
Many perceived problems w/ peer review (long history of complaints)
Lack of speed, conservatism, bias, low quality reviews, non-scholar editors.
Finding good reviewers can be a challenge. System overwhelmed.
Exceptionally costly to the institution to subsidize peer review via faculty salaries.
Citation metrics? Can be gamed. Are not always accurate.
“ Open” web-based peer review? No real success so far. Who has time?
Fraud and plagiarism detection.
Ensuring “Trust” will become more pressing.
Creates Echo chambers, “Generational paradigms?”

Abbott’s "generational paradigms" (AAUP 2008)
“ There appeared in many fields in the 1960s and 1970s a new structure supplementing specialization as a strategy for dealing with the sheer mass of scholarship. I have … called this structure "generational paradigms" … specialty-groupings, within disciplines and subdisciplines, that take a particular view of the substantive, methodological, and philosophical debates in their field, and then pursue that view to the exclusion of other approaches.”
“ generational paradigms...are not really a legitimate or valid way to deal with the problem of abundance. They are just ad hoc agreements to ignore some huge body of material.”
Q-- New Media will be the savior to problems with overabundance, peer review, and the publishing system? Or just create new “good old boys” clubs and echo chambers?

Steve Pinker vs Nick Carr Crowdsourced “Digital Humanities Now”
Pinker: Op-Ed Contributor – Mind Over Mass Media – NYTimes.com
Posted by these editors: (6)
Musebrarian : RT @dancohen: Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J 11.06.2010 08.17
cliotropic : Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J (via @dancohen) 11.06.2010 07.50
derekbruff : RT @dancohen: Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J 11.06.2010 08.13
dancohen : Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J 11.06.2010 07.38
CatinStack : RT @dancohen: Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J 11.06.2010 07.42
mkimarnold : Interesting. What do you think? (she asks of twittersphere) NYTimes: Mind Over Mass Media http://nyti.ms/dgu3Qg 11.06.2010 03.29
Posted by others: (11)
DUKEpress : RT @dancohen: Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J 11.06.2010 08.01
kevinmarks : RT @cshirky: Pinker on Carr: "It’s not as if habits of deep reflection and rigorous reasoning ever came naturally to people." http://nyti.ms/bnLGfO 11.06.2010 07.59
profsyn : RT @cliotropic: Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J 11.06.2010 07.57
tanya_roth : RT @dancohen: "Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J" 11.06.2010 08.09
Glinner : RT @jayrosen_nyu: "Far from making us stupid, these technologies are the only things that will keep us smart." http://jr.ly/zgas But keep at it, contrarians! 11.06.2010 07.57
alexismadrigal : Pinker's latest op-ed is good. He doesn't elevate 20th C norms to faux-natural laws. http://instapaper.com/zjhpt7fd 11.06.2010 07.06
NiemanLab : This is your brain on web: Steven Pinker weighs in (via @palafo) http://j.mp/cvVf0j 11.06.2010 07.30
mathewi : some much-needed sanity from Steven Pinker on whether the web and new media are making us stupid: http://nyti.ms/d3LP6f /via @nickbilton 11.06.2010 06.41
palafo : Steven Pinker on whether the web and new media are making us stupid: http://nyti.ms/d3LP6f /via @nickbilton @mathewi 11.06.2010 06.49
cshirky : Pinker on Carr: "It’s not as if habits of deep reflection and rigorous reasoning ever came naturally to people." http://nyti.ms/bnLGfO 11.06.2010 07.32
mikhailg : RT @dancohen: Reality check from Steven Pinker about the Google/Twitter/etc is making us stupid vogue: http://nyti.ms/8YJu2J 11.06.2010 07.41
retweet
Rough Type: Nicholas Carr’s Blog : Steven Pinker and the Internet
Posted by these editors: (3)
jafurtado : Steven Pinker and the Internet, by Nicholas Carr http://tinyurl.com/3x6sgzt 12.06.2010 17.33
alogemann : Steven Pinker and the Internet: http://www.roughtype.com/archives/2010/06/steven_pinker_a.php 12.06.2010 19.32
Openreflections : RT @tcarmody ~ Nick Carr's thoughtful reply to Steven Pinker: http://is.gd/cNafY (via @textpatterns) 13.06.2010 01.27
Posted by others: (1)
berniedodge : Carr rebuts Pinker rebutting Carr. It's like pingpong. http://tinyurl.com/3x6sgzt 12.06.2010 22.33

Faculty Expectations re SC
Assume access to others’ scholarship will be provided and costs covered by the institution.
Want relative speed to publication and the highest prestige outlet as possible given the nature of what is being published and the targeted audience . OA not the issue for many.
Don’t want to slog through dreck/Need peer review as a filtering mechanism.
Want work protected from rip-off/Must receive credit.
Want high quality pubs/editorial/production values/visibility and persistence for their best work
Back-end data support.

Perceptions: Enhanced Capabilities of Electronic Communication (Electronic not always = to OA)
Ability to reach a larger audience
Ease of access by readers
More rapid publication even when peer reviewed
Ability to search within and across texts /surface old literature
Opportunity to make use of hyperlinks, embed media
Enable innovation in scholarly work
Ability to have enough information (e.g., software code, back-end data, text archives, etc.)
OA = Possible democratizing effect on scholars outside of North America /EU

Giving it Away? Sharing Publications
Scholars share but What? With whom? When?
Web 2.0? Wisdom of crowds? Crowd-sourcing?
Personality and disciplinary tradition very impt.
Early “half baked” work? Universally not shared publicly.
Young scholars especially conservative re sharing work.

Giving it Away? Sharing Publications
First w/ trusted circle of colleagues. Modicum of privacy needed.
Conferences/seminars. Impt –smaller the better for early work.
Public Posting of Working papers ( different from posting preprints/post prints)
Not posted until it has been vetted by inner circle.
“ Penultimate” drafts.
Astrophys—arxiv ;
Econ, Quant Poli Sci --IRs, personal websites
Bio? Not much in MCB before publication
History/Arch/Musicology? Not much

Barriers to Sharing Data
“ When I’m finished w/ it.”
Not all data are easily shared.
Not all data can just be uploaded. Much time and work ($$) required to make it readable and clean.
Costs to be born by whom? Publishers? Institution? Individual?
Social sciences — proprietary data, human subjects, heterogeneity of data
Health sciences – anonymity, human subjects, commercial interests.
Archaeology — GSP coordinates can lead pot hunters to protected sites. Heterogeneity of data.
Engineering — Commercial and other impediments?

Collaboration
Collaboration can take place w/o f2f but…
Importance of eye contact and handshake universal.
Grand challenge Qs. Bridging disciplinary silos and cultures
Administrative nodes for distribution of funds
Multi author tensions in TP system.
Disciplinary cultures, shared language (e.g., in bio, what are data?)
What about social sciences and humanities in Grand challenge Q’s?
Rankings of virtual networks vs Universities?

Conclusions
Scholars will read a piece of work once. Time is limited and coveted.
Distinguish—open access journals, postprints of published material, well-developed preprints and working papers published on web site, vs blogs and other casual non-peer reviewed postings.
Disciplinary culture and tradition, PLUS individual’s imperative to advance his/her career and field will always be important in competitive environments.
Sharing is complex. Building systems that assume early sharing of ideas can be a waste of time if ignorant of culture in a field.
Young scholars? Adopting norms of mentors. They are very conservative in archival publication choices.
Don’t confuse predictions about “social scholarship” enabled by Web 2.0 with hard realities of T+P requirements in highly competitive professional environs (and the peer reviewed publishing demands in those fields).

Mythology
Generations are internally homogeneous? Old = Conservative ? Young = more innovative? Socialization and personality impt.
Non-Peer reviewed work can count for tenure? Blogs? Technical work?
New genres are rejected in T&P? Not showing up in dossiers?
Scientists = More Innovation in Publishing?
Open Peer Review Will Work? Who has time?
All Societies are bad. Who are the real bad guys in publishing?
New media won’t create new “old boys” clubs?

Academic Values and Scholarly Communication Practices: An Analysis of Faculty Choice and Opinion in Twelve Disciplines Diane Harley, Principal Investigator Center for Studies in Higher Education University of California, Berkeley Project Website and Associated Document Links: http:// cshe.berkeley.edu/research/scholarlycommunication http:// escholarship.org/uc/cshe_fsc [email_address] Copyright 2010, Center for Studies in Higher Education

Harley AAUP 5.18.10

by Association of American University Presses on Jun 23, 2010

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Harley AAUP 5.18.10 — Presentation Transcript