Automated Review Guide

A skill for leveraging AI-assisted tools in the peer review process, covering both author-side self-review and editor-side manuscript screening. Addresses tool selection, prompt engineering for review tasks, limitations and biases of LLM-generated reviews, quality assurance workflows, and ethical guidelines for AI use in peer review.

Overview of AI in Peer Review

Current Landscape

AI-assisted peer review tools operate at multiple stages of the publication pipeline. Understanding where automation adds genuine value and where it introduces risk is essential for responsible adoption.

Where AI assists in peer review:

Author-side (pre-submission):
  - Grammar and style checking (Grammarly, Writefull)
  - Statistical result verification (statcheck, GRIM/SPRITE)
  - Reference completeness checking
  - Plagiarism detection (iThenticate, Turnitin)
  - Readability scoring
  - Structural completeness (IMRAD compliance)

Editor-side (triage and assignment):
  - Desk rejection screening (scope, quality threshold)
  - Reviewer matching (expertise alignment)
  - Conflict of interest detection
  - Duplicate submission detection
  - Plagiarism and image manipulation screening

Reviewer-side (review assistance):
  - Paper summarization for rapid assessment
  - Statistical claim verification
  - Reference checking (do cited papers support claims)
  - Comparison with related work
  - Structured review template generation

Post-review:
  - Decision consistency analysis
  - Review quality assessment
  - Revision compliance checking

Self-Review with AI Before Submission

Structured Self-Review Prompts

Pre-submission AI review checklist:

1. Abstract completeness check:
   Prompt: "Analyze this abstract. Does it contain:
   (a) background/motivation, (b) research gap,
   (c) methodology summary, (d) key results with
   numbers, (e) conclusion/implication? Identify
   any missing elements."

2. Claim-evidence alignment:
   Prompt: "For each claim in the Discussion section,
   identify the specific result (table, figure, or
   statistical test) that supports it. Flag any claims
   without corresponding evidence in the Results."

3. Methods reproducibility:
   Prompt: "Read the Methods section and list every
   piece of information that another researcher would
   need to replicate this study. Identify any gaps:
   missing sample sizes, unspecified parameters,
   ambiguous procedures, unnamed software versions."

4. Statistical reporting:
   Prompt: "Check all statistical results in this paper
   for completeness. Each test should report: test name,
   test statistic, degrees of freedom, p-value, and
   effect size. List any incomplete reports."

5. Reference audit:
   Prompt: "For each citation in the Introduction, verify
   that the cited claim matches the in-text description.
   Flag any cases where the citation might not support
   the specific claim being made."