Scholarly peer review or academic peer review (also known as refereeing) is the process of having a researcher's methods and findings reviewed (usually anonymously) by experts (or "peers") in the same field. Peer review is widely used for helping the academic publisher (i.e., the editor-in-chief, the editorial board, or the program committee) decide whether the work should be accepted, considered acceptable with revisions, or rejected for official publication in an academic journal, a monograph, or in the proceedings of an academic conference. If the identities of authors are not revealed to each other, the procedure is called dual-anonymous peer review.
Academic peer review requires a community of experts in a given (and often narrowly defined) academic field, who are qualified and able to perform reasonably impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields, may be difficult to accomplish, and the significance (good or bad) of an idea may never be widely appreciated among its contemporaries. Peer review is widely used to increase academic quality and is used in most major scholarly journals. However, criticism of peer review include failure to prevent publication of invalid research or scientific fraud, low correlation with future impact, slowness, and lack of accountability.
History The first record of an editorial pre-publication peer-review is from 1665 by Henry Oldenburg, the founding editor of Philosophical Transactions of the Royal Society at the Royal Society of London.Committee on Science, Engineering, and Public Policy, National Academy of Sciences, National Academy of Engineering, and Institute of Medicine On Being a Scientist: A Guide to Responsible Conduct in Research National Academies Press, Washington, D.C., 1995, 82 pages,
The first peer-reviewed publication might have been the Medical Essays and Observations published by the Royal Society of Edinburgh in 1731. The present-day peer-review system evolved from this 18th-century process, began to involve external reviewers in the mid-19th-century, and did not become commonplace until the mid-20th-century.
Peer review became a touchstone of the scientific method, but until the end of the 19th century was often performed directly by an editor-in-chief or editorial committee. Editors of scientific journals at that time made publication decisions without seeking outside input, i.e. an external panel of reviewers, giving established authors latitude in their journalistic discretion. For example, Albert Einstein's four revolutionary Annus Mirabilis papers in the 1905 issue of Annalen der Physik were evaluated by the journal's editor-in-chief, Max Planck, and its co-editor, Wilhelm Wien, both future Nobel prize winners and together experts on the topics of these papers. On a much later occasion, Einstein was severely critical of the external review process, saying that he had not authorized John Torrence Tate Sr., the editor in chief of Physical Review, to show his manuscript "to specialists before it is printed", and informing him that he would "publish the paper elsewhere"which he did, with substantial modifications.
While some medical journals started to systematically appoint external reviewers, it is only since the middle of the 20th century that this practice has spread widely and that external reviewers have been given some visibility within academic journals, including being thanked by authors and editors. A 2003 editorial in Nature stated that, in the early 20th century, "the burden of proof was generally on the opponents rather than the proponents of new ideas." Nature itself instituted formal peer review only in 1967. Journals such as Science and the American Journal of Medicine increasingly relied on external reviewers in the 1950s and 1960s, in part to reduce the editorial workload. In the 20th century, peer review also became common for science funding allocations. This process appears to have developed independently from that of editorial peer review.
Pragmatically, peer review refers to the work done during the screening of submitted manuscripts. This process encourages authors to meet the accepted standards of their discipline and reduces the dissemination of irrelevant findings, unwarranted claims, unacceptable interpretations, and personal views. Publications that have not undergone peer review are likely to be regarded with suspicion by academic scholars and professionals. Non-peer-reviewed work does not contribute, or contributes less, to the academic credit of a scholar (such as the h-index), although this heavily depends on the field.
Justification
It is difficult and time-consuming for authors and researchers, whether individually or in a team, to spot and provide feedback on every mistake or flaw in a complicated piece of work. This is not necessarily a reflection on those concerned, but because with a new and perhaps eclectic subject, an opportunity for improvement may be more obvious to someone with special expertise or who simply looks at it with a fresh eye. Therefore, showing work to others increases the probability that weaknesses will be identified and improved. For both grant-funding and publication in a scholarly journal, it is also normally a requirement that the subject is both novel and substantial.
The decision whether or not to publish a scholarly article, or what should be modified before publication, ultimately lies with the publisher (editor-in-chief or the editorial board) to which the manuscript has been submitted. Similarly, the decision whether or not to fund a proposed project rests with an official of the funding agency. These individuals usually refer to the opinion of one or more reviewers in making their decision. This is primarily for three reasons: Workload. A small group of editors/assessors cannot devote sufficient time to each of the many articles submitted to many journals. Miscellany of ideas. Were the editor/assessor to judge all submitted material themselves, approved material would solely reflect their opinion. Limited expertise. An editor/assessor cannot be expected to be sufficiently expert in all areas covered by a single journal or funding agency to adequately judge all submitted material.
Reviewers are often anonymous and independent. However, some reviewers may choose to waive their anonymity, and in other limited circumstances, such as the examination of a formal complaint against the referee, or a court order, the reviewer's identity may have to be disclosed. Anonymity may be unilateral or reciprocal (single- or double-blinded reviewing).
Since reviewers are normally selected from experts in the fields discussed in the article, the process of peer review helps to keep some invalid or unsubstantiated claims out of the body of published research and knowledge. Scholars will read published articles outside their limited area of detailed expertise, and then rely, to some degree, on the peer-review process to have provided reliable and credible research that they can build upon for subsequent or related research. Significant scandal ensues when an author is found to have falsified the research included in an article, as other scholars, and the field of study itself, may have relied upon the invalid research.
For US universities, peer reviewing of books before publication is a requirement for full membership of the Association of American University Presses.
As experimentally controlled studies of this process are difficult to arrange, direct evidence that peer review improves the quality of published papers is scarce. One recent analysis of randomized controlled trial abstracts found that editorial and peer review processes led to substantive improvements between submission and publication.
Procedure In the case of proposed publications, the publisher (editor-in-chief or the editorial board, often with assistance of corresponding or associate editors) sends advance copies of an author's work or ideas to researchers or scholars who are experts in the field (known as "referees" or "reviewers"). Communication is nowadays normally by e-mail or through a web-based manuscript processing system such as ScholarOne, Scholastica, or Open Journal Systems. Depending on the field of study and on the specific journal, there are usually one to three referees for a given article. For example, Springer states that there are two or three reviewers per article.
The peer-review process involves three steps:
Step 1: Desk evaluation An editor evaluates the manuscript to judge whether the paper will be passed on to journal referees. At this phase many articles receive a "desk reject", that is, the editor chooses not to pass along the article. The authors may or may not receive a letter of explanation.
Desk rejection is intended to be a streamlined process so that editors may move past nonviable manuscripts quickly and provide authors with the opportunity to pursue a more suitable journal. For example, the European Accounting Review editors subject each manuscript to three questions to decide whether a manuscript moves forward to referees: 1) Is the article a fit for the journal's aims and scope, 2) is the paper content (e.g. literature review, methods, conclusions) sufficient and does the paper make a worthwhile contribution to the larger body of literature, and 3) does it follow format and technical specifications? If "no" to any of these, the manuscript receives a desk rejection.
Desk rejection rates vary by journal. For example, in 2017 researchers at the World Bank compiled rejection rates of several global economics journals; the desk rejection rate ranged from 21% (Economic Lacea) to 66% (Journal of Development Economics). The American Psychological Association publishes rejection rates for several major publications in the field, and although they do not specify whether the rejection is pre- or post- desk evaluation, their figures in 2016 ranged from a low of 49% to a high of 90%.
Step 2: External review If the paper is not desk rejected, the editors send the manuscript to the referees, who are chosen for their expertise and distance from the authors. At this point, referees may reject, accept without changes (rare) or instruct the authors to revise and resubmit.
Reasons vary for acceptance of an article by editors, but Elsevier published an article where three editors weigh in on factors that drive article acceptance. These factors include whether the manuscript: delivers "new insight into an important issue", will be useful to practitioners, advances or proposes a new theory, raises new questions, has appropriate methods and conclusion, presents a good argument based on the literature, and tells a good story. One editor notes that he likes papers that he "wished he'd done" himself.
These referees each return an evaluation of the work to the editor, noting weaknesses or problems along with suggestions for improvement. Typically, most of the referees' comments are eventually seen by the author, though a referee can also send 'for your eyes only' comments to the publisher; scientific journals observe this convention almost universally. The editor then evaluates the referees' comments, her or his own opinion of the manuscript before passing a decision back to the author(s), usually with the referees' comments.
Referees' evaluations usually include an explicit recommendation of what to do with the manuscript or proposal, often chosen from options provided by the journal or funding agency. For example, Nature recommends four courses of action: to unconditionally accept the manuscript or the proposal, to accept it in the event that its authors improve it in certain ways to reject it, but encourage revision and invite re-submission to reject it outright.