What is post-publication peer review?

Over the past few years there has been a rise in the use of post-publication peer review (PPPR) to complement pre-publication review and improve existing and future research published in the scientific literature. PPPR is not a new concept; post-publication evaluation and discussion of research has always happened organically through written or spoken dialogue. It is a cornerstone of the practice of science and it is how the extensive knowledge base we have today has been built up over time. However, with a greater volume of research now being undertaken and scientific dissemination becoming more digitized, the discussion and evaluation of science has started to migrate from private forums to the internet, a universal platform where scientists can quickly make their thoughts on specific papers more widely available to a much broader audience.

Different types of post-publication peer review

The term PPPR itself is slightly nuanced and can have different meanings depending on the context. It can signify both an add-on to, and a replacement for, traditional pre-publication review. These different types can be best explained as ‘primary’ PPPR and ‘secondary’ PPPR1. Primary PPPR is the initial evaluation of an academic paper once it is has been published (but not reviewed), and is used as an alternative to the more commonplace pre-publication peer review publishing model. Secondary PPPR is an evaluation of an article independent of the peer review it has already received.

Primary PPPR

With scientific articles now being published primarily online, there has been an increase in innovation with respect to scholarly publishing, with a particular emphasis on exploring novel ways to disseminate research that fits the digital age in terms of speed, transparency and cost effectiveness. This has led to the rise of the publishing platform, and has enabled publishers to challenge some of the established processes of the article and devise a publication model that utilizes the web technologies we have at our disposal. F1000Research2 is an author-led publishing platform that uses a PPPR model. Articles go through a rapid initial check by the in-house editorial team before being published immediately with the status ‘awaiting peer review’. Once published, formally invited peer review commences and when referee reports are received, the reviewer’s name, affiliation and report are published alongside the article. This PPPR model enables the readers to see and access all of the reviews, and by making them publicly available enables the peer-review process to become an intrinsic part of a paper’s ecosystem.

Secondary PPPR

Typically in the traditional peer review model, an article is published after it has been given approval by a few chosen referees. In some cases this might not be enough to identify flaws, provide constructive criticism or highlight the significance of a particular body of work, and so a published article may benefit from further evaluation to provide a fuller picture. This discussion can be conducted on the articles themselves using the publisher’s commenting functionality, or the discussion can happen externally on independent blogs and through social media channels. Recently, new platforms have been developed specifically for the discussion of scientific papers after publication, such as Open Review3, PubPeer4 and PubMed Commons5, which facilitate interchange on their individual platforms and store them in their centralized online databases. The F1000Prime6 recommendation service is also a form of positive PPPR, through its qualitative assessment of published research articles based on the selections of over 6,000 F1000 Faculty Members, who are world-renowned experts in biology and medicine.

PPPR – the challenges it faces

The biggest challenge for PPPR today is the ‘publish or perish’ culture7. There is considerable pressure to publish research papers to guarantee career security and advancement. These pressures reinforce the misconception that research is but a means to an end. Publishing has become a product of a hypercompetitive system8 driven by the need for academic survival, and publications are seen as the magic bullet to garner respect, prestige, recognition and professional opportunities. In short, researchers are working to the mantra of completing a study and then quickly moving on to the next one. This status quo means that once an article is published, then that work is essentially completed – it becomes indexed and ultimately a permanent record in the published literature. Due to the large workloads scientists have, there is little motivation for them to participate in PPPR, particularly if their contributions to a paper are not acknowledged by the authors and acted upon. In part, this could explain why the PPPR of the current literature consists of mainly anonymous and negative comments9; researchers are more likely to contribute to PPPR that may lead to a retraction or major correction as currently that is the only way their contributions are noticed and accepted by their peers.

For PPPR to be more widely adopted, publishers will need to look beyond the notion of publication as the final step of quality assurance, and introduce a more dynamic way of publishing papers with version control where any PPPR contributions can be processed and applied. The F1000Research PPPR model has version control at its core, and all the papers published are never completed; PPPR can happen at any stage and the authors can update their paper whenever they would like in response to any peer review and comments it may receive.

Another challenge is the lack of tangible incentives available to increase PPPR participation. Peer review in all forms is the main arbiter of regulating scientific quality, a routine procedure in academia that provides no perceivable cachet. Peer review often improves a paper, but it is the author who takes all the credit and the reviewers are often left bereft of any noticeable kudos for their contributions. With all the importance heavily weighted towards publication of articles, which in itself requires the community to provide extensive reviewing duties, there is no obligation to contribute to what is seen as a further reviewing burden. Writing papers will always take precedence in the current reward system, so to encourage a PPPR culture, there need to be meaningful incentives for the community to get involved and these need to correlate in some way with academic assessment.

Of course, open PPPR enables much greater credit to be obtained from an individual’s contribution. This could cause some trepidation amongst younger researchers as openly criticizing a more senior scientist could result in retaliatory measures such as having future grants blocked, or getting a less favourable tenure evaluation. Again, this fear manifests from the publish or perish culture and young researchers take a guarded approach to protect their livelihoods as an academic. So for PPPR to become the norm, better appreciation and understanding of the value of scientific criticism will be required to ensure that those who may point out errors in publications are providing a beneficial contribution to the research.

Another challenge is the fragmentation of PPPR discussion that currently exists on the web10. PPPR can happen in a multitude of places and is not intuitively linked to a ‘central hub’. At present, researchers read and access articles through workflows and products that suit them, so some PPPR contributions may not be receiving full exposure, which leads to unnecessary information shortfalls.

Is there a future for PPPR?

One thing that is clear is that PPPR provides a useful additional layer of information that extends the context of a published piece of research. Publishing is now an online process and PPPR can enhance our understanding of the ever growing scientific literature. There are hurdles that will have to be jumped, in particular the shift from the publish or perish culture that governs how researchers are currently assessed. There is also a need for real incentives to be put in place to propagate community contributions, encourage researchers to speak objectively and constructively without fear of retribution. If these hurdles can be overcome, then PPPR will play an important role in how science is communicated and discussed; the technology is certainly there to embrace it. PPPR is faster than traditional forms of evaluation and, implemented correctly, it has the potential to improve the quality of research and reduce waste in science11. Scientists have been slow to adapt to information-sharing and collaboration made possible by network technologies. However, there are signs of transition, and PPPR could well be at the forefront of this paradigm shift: PPPR, in all its guises, is here to stay.

Competing interests

The author is an employee of Faculty of 1000 Ltd, where F1000Research is an open science publishing platform with secondary PPR, and F1000Prime is a recommendation service based upon primary PPPR.