Peer Review Guidelines
Objective
Counter-Insider Threat Research and Practice (CITRAP) is an online, unclassified, peer-reviewed journal aimed at professionals, and published by The Threat Lab in partnership with National Insider Threat Task Force (NITTF) and the Department of Defense’s Counter-Insider Threat Program. As a peer reviewer, you play an integral role in maintaining the integrity of the journal by assessing the validity, quality, contribution to the field, and originality of articles for publication. CITRAP editors and authors greatly appreciate your time and valuable input.
Reviewer Assignment and Acceptance
When receiving a request to provide a peer review, please consider the following in deciding whether to accept:
- Does the article match your area of expertise?
* Only accept if you feel you can provide a high-quality review. - Do you have a potential conflict of interest?
* Disclose this to the editor when you respond. - Do you have time?
* Reviewing can be a lot of work – before you commit, make sure you can meet the deadline. - Consult the COPE guidelines for more information.
Please respond to the invitation and all CITRAP correspondence as soon as possible to avoid delays in the review process and unnecessary wait times for authors.
Material Handling
Upon acceptance, reviewers must treat the materials as confidential documents. If a reviewer wishes to consult a colleague or trainee for assistance with the review, please seek prior authorization from the editor and ensure the person is free of any competing interests. Reviewers may not retain copies of submitted manuscripts and may not use the knowledge of manuscript content for any purpose unrelated to the peer review process. The review process is conducted anonymously for all submissions.
Note: In some rare cases, and for some specific submission formats (e.g., “Perspectives”), it may be necessary to reveal the identity of the author(s) or sponsoring organization to provide context for the submission. In these rare instances, reviewers should take care to evaluate the paper on its merits and not let specific affiliations bias their judgments.
Conducting the Review
Reviewers should familiarize themselves with the Submission Guidelines for the type of article they are reviewing—Research, Brief Report, or Notes from the Field.
To help peer reviewers assess submissions and to ensure consistency, CITRAP editors customized a Peer Reviewer Form on Scholastica. This form lists specific questions on a 5-point rating scale along with text boxes for specific written feedback as outlined in the fourth step below. Reviewers will receive access to the form upon accepting the invitation to review.
The review will comprise the following steps:
1. Read the article carefully. First, reviewers should read the article all the way through. On the second read, they should assess the paper according to the Peer Review Template.
2. Provide Recommendation. Three recommendation options are available:
a. Accept
b. Revise and Resubmit
c. Reject
3. Provide Ratings on Review Criteria. Numeric ratings are requested for criteria such as originality, contribution to the field, organization and completeness of the paper; additional criteria apply to original research papers concerning more specific elements of the paper (e.g., method, results, discussion, etc.).
4. Provide Written Feedback/Commentary. The following written feedback is requested in a separate text box:
a. Summary. Provide a brief summary of the study and findings to communicate your understanding of the paper.
b. Major Issues. Give the author(s) your assessment and detailed description of any major conceptual, methodological or other issues.
c. Minor Issues. Provide the author(s) a description of minor issues, such as where meaning is ambiguous, issues with citations, minor errors in reporting, etc.
d. Confidential Comments to the Editor. You may include additional comments or considerations for the editor to take into account—these comments will not be seen by the author(s).
Making a Revise and Resubmit Recommendation
When recommending Revise and Resubmit recommendation, reviewers should provide specific guidance. The review should ultimately help the author improve the article. Be objective and constructive, not subjective and destructive. The following questions are designed to help structure the reviewer’s comments:
- Major Revisions
- Are there any major flaws? State what they are and what the severity of their impact is on the paper.
- Has similar work already been published without the authors acknowledging this?
- If the paper presents findings that challenge current thinking, is the evidence strong enough to prove the author’s case?
- Have they cited all the relevant work that would contradict their thinking and addressed it appropriately?
- If major revisions are required, please indicate clearly what they are.
- Are there any major presentational problems?
- Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work?
- Are there any ethical issues?
- Consider whether to disclose these in the confidential comments section to the editor.
- Minor Revisions
- Are there places where meaning is ambiguous?
- How can this be corrected?
- Are the correct references cited?
- If not, which should be cited instead/also? Are citations excessive, limited, or biased?
- Are there any factual, numerical or unit errors?
- If so, what are they?
- Are all tables and figures appropriate, sufficient, and correctly labelled?
- If not, say which are not.
- Are there places where meaning is ambiguous?
How to Write a Constructive Review
- Give your main impressions of the article. For example, is it novel and interesting, does it have a sufficient impact, does it add to the knowledge base?
- Give positive feedback. Authors are more likely to be open to constructive criticism if you start and end with positive comments. But don’t overdo it if you will be recommending rejection—doing so could trigger an appeal.
- Keep the focus on the research and not the author.
- Try to put the findings of the paper into the context of the existing literature and current knowledge.
- State any major flaws or weaknesses. Also note any special considerations—for example, if previously held theories are being overlooked.
- Be clear in your feedback. Give specific comments and suggestions in short, clear paragraphs and make it easy for the author(s) to see what section you’re referring to so they can reply to each point.
- Explain how the article conforms to the Submission Guidelines.
- Keep your comments strictly factual. Don’t speculate on the motives of the author(s).
- Carefully review the methodology, statistical errors, results, conclusion/discussion, and references.
- Be aware of the possibility for bias in your review. Unconscious bias can lead us all to make questionable decisions which impact negatively on the academic publishing process.
- Treat the author’s work the way you would like your own to be treated.
Pitfalls to Avoid
- Don’t focus unduly on spelling/grammar concerns: Try not to be distracted by spelling or grammar errors. If the research is sound but difficult to comprehend due to poor grammar, recommend to the editor that the author(s) have their paper edited.
- Don’t rely on cursory remarks: Avoid terse remarks such as “great work,” “this is interesting,” “poor structure,” “did not execute” without explaining why or how.
- Don’t make ad hominem comments: Avoid any personally attributable comments towards the author(s). Rather, comments should focus on content and the position of the author(s).
- Don’t allow possible biases to influence your assessment: Avoid unjustified dismissal of alternative viewpoints or theories that might conflict with your personal opinions on a topic.
Submitting a Review
When you are ready to submit your report, follow the instructions in the Scholastica online Peer Reviewer system. If you have any questions, contact CITRAP. For technical issues, contact the Scholastica Help Desk.