In this contribution, we enter somewhat murky waters….the posing of questions with ambiguous or uncertain answers, raised in the hope that they foster debate. Again, these are not the official views of SEPM.
Previous JSR Paper Clips posts (available here and here and here) have examined aspects of open access publishing. Careful readers may have noted several fundamental philosophical changes that have accompanied the shift towards open access, leading to new and continuing evolution of relationships among readers, authors, administrators, and even Congress.
As subscribers, readers traditionally have served the role of the customers and arbiters of quality—journals that published important papers would have a greater number of individual and library subscribers, and be well read. Journals with no readers would not be viable. With open access, if “all” papers are free and universally available, the publication system no longer reflects customer-based assessment of the quality of journals and the papers that they contain. Such an outcome would appear to require that pre-publication peer review fill this role of evaluating manuscripts, and in maintaining and rewarding quality papers and publications.
Yet, consider what some open access publishers tell us about peer review. Some open access publishers explicitly state that significance is not considered as a criterion for acceptance. Such a peer review model suggests “The peer review process does not judge the importance of the work, rather focuses on whether the work is done to high scientific and ethical standards and is appropriately described, and that the data support the conclusions.” An alternative open access model of “peer review” includes only “…a ‘binary’ accept or reject decision” that is based solely “on the basis of the validity of the research being reported, including the soundness of the research methods employed and the analysis undertaken to reach conclusions.”
Questions regarding the quality of the communication aside, leaving the evaluation of significance to the post-publication reader represents a fundamental shift from the approach of many journals. For example, a primary criterion for publishing a manuscript in JSR is that the manuscript clearly describes overarching question(s) of global relevance that the manuscript addresses. It is essential that papers in JSR “…..be agenda-setting, generic, and relevant to an international audience. The wider relevance must be explicit, in the title, abstract, introduction, and conclusions” (editorial May 2004). With the new “open access” review model, presumably an evaluation of significance is left to post-publication assessment.
Should post-publication commenting or “review” of some sort take the place of pre-publication peer review as a metric of scientific excellence and significance? What should be the form of “public” commentary—comments by whom? What are their credentials? Or, because chairs and deans will not take the time to read those comments (“The dean can’t read, but he can count”), will this lead to a situation in which the article-level or journal-level impact factors provide the sole metric of “quality” used for career advancement?
Traditionally, it has been regarded that “Publications are the currency of academia.” This statement is predicated on the notion that (at least with a traditional journal employing time-honored external pre-publication peer review) a publication reflects a well-written report on a quality study—the fact that it was published alone meant something. As is evident, however, this situation may not (or may) be the case today.
Beyond these changes, however, lies another basic question: Does the author “pays-for-publication” model create a fundamental conflict of interest? How does this model preclude the situation wherein “more money = more publications,” or, even more simply “money = publications,” especially given the lax standards of some OA publishers? Or, to quote Lord Cutler Beckett from Pirates of the Caribbean, are we now entering the situation wherein “Currency is the currency of the realm”? Does this advance science?
Perhaps it is more complex than that. For if this outcome is to be, consider how authors get some of their funding. Recent legislation working its way around Congress proposes new criteria for evaluation of National Science Foundation proposals that appear, to some, to “replace peer review.” (Having said that, the draft of the bill is somewhat ambiguous - do these replace, or extend beyond, the “intellectual merit” and “broader impacts” criteria already in place?) A draft of the proposed “High Quality Research Act” advocates that Congress should have some sort of oversight on “quality” science:
(a) “CERTIFICATION.—Prior to making an award of any contract or grant funding for a scientific research project, the Director of the National Science Foundation shall publish a statement on the public website of the Foundation that certifies that the research project—
(1) is in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science;
(2) is the finest quality, is ground breaking, and answers questions or solves problems that are of utmost importance to society at large; and
(3) is not duplicative of other research projects being funded by the Foundation or other Federal science agencies.”
Along these lines, to flex their perceived authority, members of Congress actually requested access to “scientific/technical reviews and the Program Officer’s Review Analysis” of specific, individual NSF grants. Does Congress now advocate that it provide oversight beyond (or replace?) the peer-review process? Is this level of oversight appropriate? Should Congress proactively “call the piper’s tune?”
Since at least the 17th century, the scientific community, through the work of scholarly societies, has served an important role in the development (e.g., conferences where ideas are discussed in person), organization (into peer groups, facilitating specialized, topical journals), and validation (through peer review) of quality science communication—perhaps playing the most important role in science, other than that of the scientists themselves. The first few posts of this series (available here and here) explored some of the issues that may arise for scholarly societies as a result of mandated open access. In this new world, with a) no subscriptions, b) limited (no?) pre-publication peer assessment of significance (but with congressional oversight?), and/or c) authors who can pay to publish almost anything, what is the role of scholarly societies in this publishing evolution/revolution? Are the days (indeed … the centuries!) where their journals stand as hallmarks of quality publications soon to be (or already) over? Will there be publications and “publications”? What is the appropriate relationship among readers, authors, and societies? Is there only one relationship? If not, what are the appropriate metrics for excellent scholarly papers and publications, and how do societies best serve to ensure quality?
Or, at an even larger scale, what is the appropriate place for scholarly societies in this politically mandated changing landscape?
One goal of societies is keeping the community abreast of changes in the science world. We at JSR Paper Clips hope that our series has helped inform you about some of the dynamic publication world (as readers, authors, and interested parties), and motivate you to dig deeper, for these changes will impact all of us. Now, however, until further developments, we continue with our regular programming….
This post represents one of a series in JSR Paper Clips. These posts are available here:
We welcome feedback and comments!
No comments:
Post a Comment