Peer Review and Research
The article "Peer Review in Scientific Publications" discusses the importance of peer review: "Peer review ... is now used not only to ensure that a scientific manuscript is experimentally and ethically sound, but also to determine which papers sufficiently meet the journal’s standards of quality and originality before publication. Peer review is now standard practice by most credible scientific journals, and is an essential part of determining the credibility and quality of work submitted" (Kelly, Sadeghieh, & Adeli; 2014).
Peer review offers a general expectation that the research presented is as follows:
- Conducted ethically (approved by an IRB if necessary)
- Under FDA regulations, an Institutional Review Board (IRB) is group that has been formally designated to review and monitor biomedical research involving human subjects... This group review serves an important role in the protection of the rights and welfare of human research subjects (fda.gov)
- Conducted with repeatable and accurate results
- Focused on the subject of journal/issue where it was published
- Of academic quality
- Original research
History
Before the modern peer review process was established, collaboratively validating information was still practiced, even earlier than the printing press. Once printing made publication a faster process,
"Philosophical Transactions of the Royal Society is thought to be the first journal to formalize the peer review process in 1665, however, it is important to note that peer review was initially introduced to help editors decide which manuscripts to publish in their journals, and at that time it did not serve to ensure the validity of the research. It did not take long for the peer review process to evolve, and shortly thereafter papers were distributed to reviewers with the intent of authenticating the integrity of the research study before publication" (Kelly, Sadeghieh, & Adeli; 2014). [Emphasis added]
Structure of an Article
The parts of a peer reviewed paper include the abstract, introduction, the method and material, results, the conclusions, and the references.
Abstract: The author of a paper will need to also write an abstract to explain in short what the research will address. An abstract will not be more than a paragraph and is not part of the actual research presentation. It should not be cited as a source as it functions more like the blurb on the back of a book.
Introduction: The introduction will present the research question, some context for why that question is important, and probably some information about how they will conduct their research. This is a good place to scan when you are determining how useful an article will be.
Methodology / Materials: Any original experimental research paper will include a section talking about how their experiment was conducted and what material they used. For a humanities article, this could be a survey with conversational answers (qualitative results) or a numerical dataset for how many individuals from certain demographics utilized a social program and reported measurable effects like reduced debt, increased terminal education degrees, fewer teen pregnancies or drug overdoses (quantitative results). For STEM articles, this could include chemical components, strings of coding, building material, or soil samples and explicate descriptions of those materials, quantities, and actions.
Results: With any experiment or research project, the results are very important even if they don't give you the result you were expecting. Even a negative result will tell you something. These results could be qualitative (interview questions, personal experiences, emotional or subjective perceptions), quantitative (numerical, statistics, measurable, tangible) or mixed methods (both qualitative and quantitative results). A peer reviewed article will present their results and explain their analysis of those results.
Conclusion: This section will often offer advice, future research opportunities, possible conflicts of interest from the authors or their funders, restate the answer they arrived at or why they didn't find an answer at all during their research. This is another good place to scan for a quick overview.
References: This is the list of citations for the resources that were used in the research process. Often there is a Literature Review section before the description of the experiment/survey/original work that puts this paper into context. Any material cited will be included in the list of references.
Process of Peer Review
"When a scholarly work is submitted to a scientific journal, it first undergoes a preliminary check known as a desk review. The editor decides if the manuscript should be sent for peer review or be immediately rejected. The next step is to select experts from the same field who are qualified and able to review the work impartially. Ideally the work is evaluated by multiple experts...
"Peer reviewers normally provide their assessment in the form of a questionnaire which they return to the editor. This forms the basis for deciding whether the work should be accepted, considered acceptable with revisions, or rejected. Submissions with serious failings will be rejected, though they can be re-submitted once they have been thoroughly revised....
"If a work is rejected, this does not necessarily mean it is of poor quality. A paper may also be rejected because it doesn't fall within the journal's area of specialization or because it doesn't meet the high standards of novelty and originality required by the journal in question... It is therefore common for authors to submit their paper to a different journal after receiving a rejection" (Publisso, 2017).
Types of Peer Review
- single-blind peer review: the name of the reviewer is hidden from the author;
- double-blind peer review: both the reviewer and the author remain anonymous to each other. (Publisso, 2017)
- Open peer review—reviewers are aware of the authors’ identities and authors are aware of the reviewers’ identities. A disadvantage of this method (and single-blinded peer review) is that reviewers may be biased by knowing the identities of the authors.
- Post-publication peer review—this is a newer form of peer review, whereby peer review takes place after publication. Manuscripts or “preprints,” that have not yet been peer reviewed can be published on “preprint servers.” Important scientific organizations such as the Medical Research Council, now actively encourage preprint servers, as well as citation of preprints and their use in grant applications3. This means that traditional forms of publication bias are reduced. This includes the “file drawer problem” whereby studies with positive findings are more likely to be published, while those with negative/nonconfirmatory results are ignored. As content can be evaluated by the scientific community as a whole, the biases of individual reviewers are minimized.
- Preprint publication also bypasses the bottleneck of peer reviewing, so manuscripts can be published more quickly. However, in the interim between publication and peer review, this can mean mistakes and poor quality research is open to the scientific community2,4.
- Collaborative peer review—during the peer review process, peer reviewers can view each other’s comments, interact, and produce a final peer review report. This has been investigated by Elsevier, in the journals Molecular Cell, Neuron, and Cell5. Compared with the traditional process, results were largely positive from reviewers, editors and authors in terms of the final product. However, the process usually took longer than traditional peer review.
- Interactive peer review—in this process, work is first screened by an editor for ethical issues and unacceptable writing. Then peer reviewers work directly with authors—often on an online forum. Reviewers provide real-time feedback to authors and can collaborate with other reviewers in this process and hold discussion with authors about recommendations. Once all authors and reviewers unanimously agree on the final product, the manuscript can be published. Peer reviewers are also listed on the paper and are invited to provide a supplementary commentary6. This process has been used by the frontiers publishing group. Although seen to be a more collaborative process and potentially faster due to real-time discussion, this method has not been adopted by many mainstream journals (Koshy, Fowler, Gundogan, & Agha, 2018).
Why Peer Review?
Arguments Against:
- Bias—reviewers also have inherent bias due to their own research interests and a myriad of other reasons. As they can act as gatekeepers to publication, this can skew the published literature and disillusion authors.
- Inconsistency—rarely do 2 peer reviewers form the same impression and provide similar recommendations. This can provide conflicting messages to authors.
- Delay in time to publication—as peer review forms the main bottleneck to publication, this can be frustrating for authors and lead to publications that are no longer relevant or contemporaneous at the time of publication (Koshy, Fowler, Gundogan, & Agha, 2018).
Arguments For:
-
Importance to the scientific community and readers - Peer review is important because it serves to uphold the quality of the literature as well as advance the scientific knowledge base.
-
Importance to governments, responsible authorities, and industry - Scientific research has far reaching applications beyond academia. It is instructive in governmental policies, regional schemes, and in industry. All of these areas rely on high quality research, of which peer review in instrumental.
-
Importance to authors - Providing comments and suggesting revisions to authors ... with the intent of improving the ultimately finished product... is often an invaluable tool for authors since it allows them to produce a more polished and rigorous piece of work.
-
Importance to editors - A well peer-reviewed manuscript is essential for journal editors in deciding whether a manuscript is suitable for publication.
-
Prestige - Being invited to peer review a paper suggests the reviewer’s knowledge of the field and/or critical appraisal skills are respected enough to be entrusted with gauging the quality of scientific research.
-
Awareness of the field - Peer reviewing allows access to a larger breadth of the scientific literature. It also enables reviewers to read the most up-to-date research that others do not yet have access to and before it is put into the public domain (Koshy, Fowler, Gundogan, & Agha, 2018).
Peer Review and You
For most of you, any research you will do at UNCP will require you to use peer reviewed articles. Basically, peer-reviewed articles is a fairly certain method of finding academic level information that has already been evaluated for correctness. Information found online will need more of your time and energy to decide if it is accurate and academic. By using peer reviewed articles, you can engage with information that both you and your professor will consider appropriate for academic research.
Resources:
Kelly, J., Sadeghieh, T., & Adeli, K. (2014). Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide. EJIFCC, 25(3), 227–243.
Koshy, K., Fowler, A., Gundogan, B., & Agha, R. (2018) Peer review in scholarly publishing part A: why do it? International Journal of Surgery Oncology, 3(2), pp 56. doi: 10.1097/IJ9.0000000000000056
Peer Review: Why is it important? (2017, November). Publisso. https://www.publisso.de/en/advice/publishing-advice-faqs/peer-review/
