What’s new in peer review? Initiatives by the academic community
The increasing number of publications and journals is widening the chasm between the number of peer reviewers required and the number of reviewers willing to review. Many researchers reject reviewing opportunities because they are pressed for time and need to meet their own publication and funding requirements. The burden to perform competent peer reviews, therefore, often rests on a small number of academics, leading to agonizing waits for authors hoping to have their papers published.
Moreover, expectations from peer reviewers themselves can vary by journal, discipline, and region, and with other changes in the academic and publishing landscape. All of this places pressure on the current peer review system and may call into question the effectiveness of the process in maintaining the quality and integrity of published research.
These challenges notwithstanding, the integral role of peer review in academia cannot be denied. Quality assurance in some form or the other will always be a crucial cog in the machine of scholarship, and the academic community is actively trying out new models, signaling hope and change in the way peer review is conducted.
Novel peer review models
The formats of scholarly dissemination keep evolving. From print-only, subscription-based traditional journals, we have now moved to an era of digital publication and open access (OA). There is a thrust toward open science and open data, particularly in the face of global crises such as COVID-19 and climate change.
In response, peer review models have not remained static. For example, during the current pandemic, academic journals followed fast-track systems for COVID-19–related content to hasten review and publication1.Academic publishing and communications organizations such as PLOS and eLife created a Rapid Reviewer Initiative to maximize the efficiency of peer review of COVID-19 research. This was an unprecedented and “uncommon moment of scholarly publishers collaborating to gather useful insights on the performance of the scholarly communication system.”2 Meanwhile, preprints provide unique opportunities for open and effective peer review. The last few years have witnessed a spurt in new forms and ideas for peer review to meet different evolving needs and expectations. Let’s look at some notable developments in the last few years.
Open peer review
Open peer review (OPR) aims to make reviews and publishing decisions more transparent. There is no clear definition for OPR, but it broadly refers to a peer review model wherein elements of the peer review process are made publicly available before or after publication. This system hinges on combinations of the following:
Open identities: Reviewer identity is known.
Open reports: Peer review reports are published alongside the relevant article.
Open participation: The paper is open to scrutiny and feedback by the community.
OPR in various flavors has been implemented by several journals and scholarly platforms. The EMBO Journal allows pre-publication interaction among reviewers, who comment on each other’s reports before the editor makes a decision. Frontiers journals have an interactive collaboration stage involving the authors, reviewers, and Associate Editor, which enables quicker arrival at a consensus.3
Making review reports open means that the research community can evaluate the review process. Open review reports are also amenable to reviewer recognition (see later in this article).
Pros
- OPR can address problems like lack of accountability and unethical review practices, which can otherwise occur under a veil of anonymity.
- Open identities encourage reviewers to be more meticulous and constructive in their evaluations.
Cons
- A single, clear definition of open peer review is not available.
- Some reviewers (especially early-career researchers) might be guarded or hesitant to give critical feedback when reviewing manuscripts by established researchers.
Post-publication peer review
When an article is published before peer reviewers are sought, it is called post-publication peer review (PPPR). PPPR can also bring in much needed transparency. By allowing anyone in the scientific community to evaluate a paper after it has been made public, PPPR extends the window of scrutiny beyond the publication date.
PPPR can take a number of forms, such as Letters to the Editor, blog posts, and social media posts. Some publication platforms like eLife have pioneered this model. According to Michael Eisen, Editor-in-Chief of eLife, and his colleagues, eLife tries to “replace the traditional ‘review, then publish’ model developed in the age of the printing press with a ‘publish, then review’ model optimized for the age of the internet.”4
Twitter peer review may be considered a type of PPPR. Many scientists today spend at least some time daily on social media platforms like Twitter. On Twitter, numerous researchers deconstruct papers and their limitations, strengths, and conclusions—often in lay terms, sometimes even infusing humor and memes! The Twitter “thread” is fast becoming a way to explain complex concepts and phenomena. In a Twitter thread, one is not restricted by a 280-character limit.
That being said, Twitter cannot be a formal system of peer review but can serve as a channel to draw attention to grave issues and amplify important messages.
PPPR in various forms might continue to be a part of the academic landscape, diversifying into newer and important models.
Pros
- PPPR allows rapid publication.
- It can spark meaningful discussions between researchers and a wide range of contributors.
- Critiquing on Twitter can draw attention to aspects of a published paper when it is difficult to do so via traditional channels.
Cons
- Quick publication without prior peer review could lead to poor-quality research being widely disseminated.
- Twitter reviews can be misinterpreted by persons outside the scholarly community.
- The tone of a Twitter review can make it feel like an attack.
Results-masked review
An interesting new form of peer review is “results-masked review” or “results-blind review,” wherein a manuscript is judged based on the research question and methodology rather than the results. In this form of peer review, manuscripts without the results, discussion, or conclusion are first sent to peer reviewers for scrutiny, and only when they are accepted after this stage do they move to the next, where the full manuscripts are reviewed.5
Pros
- It is a great step to avoid publication bias favoring positive findings only.
- It places a greater focus on methodological rigor.
Cons
- It has not been widely adopted.
- It may not be applicable to articles that do not have an experimental section, e.g., review articles.
Other recent trends in peer review
Incentives for reviewers
Stakeholders in academia are realizing the need to recognize peer review reports as scholarly outputs. For example, on the ScienceOpen platform, all reviews are openly available and have DOIs to make them fully citable to allow recognition in the review process. Review histories can be cited in CVs6.
Initiatives like Reviewer Credits and Publons (now part of Web of Science) confer benefits and reviewer “credits” (e.g., access to non-OA journal content) to reward researchers for their inputs as peer reviewers.
Diversity and inclusion in peer review
Peer review by individuals with diverse backgrounds is essential to prevent biases, promote development, and increase scientific rigor. Journals are becoming increasingly cognizant of the importance of diversity, equity, and inclusion (DEI) in their editorial boards and reviewer pools and are actively trying to include academics of different genders, geographical backgrounds, and career stages. Academic publishers are now committed to developing and implementing ways to improve DEI in peer review, which will be crucial in diversifying and improving science.7
Peer review assisted by artificial intelligence
Artificial intelligence (AI)-powered tools for language checks, plagiarism detection, and journal compliance checks are in use by authors and publishers alike. There might even be exciting possibilities for AI in peer review. AI technologies may be used to select reviewers and summarize the conclusions of a manuscript to ease the load on a reviewer. The use of AI in peer review can simplify and semi-automate pre–peer review screening and some aspects of the peer review process.8
Looking ahead
Peer review has come a long way since the inception of science publishing with Philosophical Transactions in 1665. The flexibility of peer review was recently demonstrated in the way academic content was processed during the pandemic. This is testament to how versatile and amenable to change the process can be. Every innovation in peer review has the potential to dispel the limitations and criticisms of the traditional system. Suffice it to say, we can look forward to speedier and more transparent ways of assessing scholarly content, which might become the norm rather than exception.
References
1. Horbach, S. How the pandemic changed editorial peer review – and why we should wonder whether that’s desirable. LSE Impact Blog. (2021). https://blogs.lse.ac.uk/impactofsocialsciences/2021/02/10/how-the-pandemic-changed-editorial-peer-review-and-why-we-should-wonder-whether-thats-desirable/
2. PLOS Blog. Study finds COVID-19 research freely accessible, but research data sharing and preprinting are low. (2021). https://theplosblog.plos.org/2021/12/rori-covid-19-research/
3. Ross-Hellauer, T. What is open peer review? A systematic review What is open peer review? A systematic review [version 2; peer review: 4 approved]. F1000Research 6, 588 (2017). https://doi.org/10.12688/f1000research.11369.2
4. Eisen, M.E., Akhmanova, A., Behrens, T.E., Harper, D.M., Weigel, D., Zaidi, M. Peer Review: Implementing a “publish, then review” model of publishing eLife 9, e64910 (2020). https://doi.org/10.7554/eLife.64910
5. Elsevier (2017). Results masked review: peer review without publication bias. Elsevier Connect. https://www.elsevier.com/connect/reviewers-update/results-masked-review-peer-review-without-publication-bias
6. Review on ScienceOpen. About ScienceOpen https://about.scienceopen.com/peer-review-guidelines/ (2020).
7. Dewidar, O., Elmestekawy, N., Welch, V. Improving equity, diversity, and inclusion in academia. Res Integr Peer Rev 7, 4 (2022). https://doi.org/10.1186/s41073-022-00123-z
8. Checco, A., Bracciale, L., Loreti, P. et al. AI-assisted peer review. Humanit Soc Sci Commun 8, 25 (2021). https://doi.org/10.1057/s41599-020-00703-8
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
