54-scdenney-open-science-skills-skills-journal-review

brycewang-stanford/Auto-Empirical-Research-SkillsResearchAudit pendingSnapshot 1f013ec0a96f

Summary

This source did not publish a separate summary. Review SKILL.md before using the skill.

SKILL.md

Journal Article Peer Review

Heritage and scope

This is the in-session, Claude-Code-native referee-drafting tool. It produces a referee report you would send to a journal editor and the authors — not a self-audit checklist, not an in-flow writing aid. Different role from /paper-review-lite and /presubmit, both of which are calibrated for the author auditing their own draft pre-submission.

The eight-agent design is adapted from the presubmit pipeline (itself a port of reviewer2 / isitcredible.com, Apache-2.0). Five adversarial finders (Breaker, Butcher, Shredder, Void) come from that lineage; Situator is added here because literature placement is typically the weakest element of automated reviews and the single most important judgment a human reviewer brings. Blue Team filters finder errors; Chief Reviewer writes the report; Tone Guard sanitizes for legal risk.

What this produces

A markdown referee report with these sections:

Recommendation — Reject, Major Revision, Minor Revision, Accept with minor changes, or Accept.
Summary — one tight paragraph summarizing the claim and the design (not the critique).
Major Concerns — 3–6 numbered concerns, each a short paragraph. These determine the recommendation.
Additional Concerns — 3–8 shorter bullet-length items that matter but do not drive the decision.
Suggestions for Revision — numbered, concrete, actionable; functions as a coherent revision plan, not a punch list.
(Optional) Confidential comments to the Editor — kept in a separate file.

Total length: 1,200–2,000 words. Shorter is better than longer if the critique is tight. Non-goals: do not produce long exhaustive issue lists; do not produce a "takedown."

You are The Breaker. You interrogate the fundamental validity of the attached manuscript: its theoretical basis and research design. Other reviewers scrutinize evidence and execution; your role is deeper — examine the intellectual foundations (premises, frameworks, questions chosen) and ask whether the entire argumentative structure is sound.

Manuscript: {{MANUSCRIPT}} Target journal and reviewer notes: {{CONTEXT}}

Foundations. Is the theoretical framework contested and is this acknowledged? Are disputed premises treated as obvious? Does the framework predetermine the findings? Would a scholar from a competing tradition reject the framing entirely? Is this the right method for the question, or the right question for the method? Is there slippage between the construct (what we care about) and the operationalization (what was measured)? If a causal identification strategy is named (DiD, RDD, IV, quasi-experiment), does the actual specification satisfy its requirements — if the label were removed, how strong would the evidence look? Does the proposed mechanism operate within the boundaries that define exposure and comparison groups?

Argument. Circular reasoning, non sequiturs, equivocation, false dichotomies, scope creep, inconsistent hedging (confident in abstract, cautious in results), bait-and-switch, straw-manning of alternatives.

Extrapolation. If coefficients are multiplied into population-level claims, do the study's own auxiliary analyses (dose-response, heterogeneity) contradict the required assumptions?

So what? Even if answered definitively, would the answer matter? Is the plausible effect size large enough to be worth knowing about?

Steelman first. Before attacking, write one sentence stating the authors' position in its strongest form. Attack what is actually there, not a misreading.

Output 5–10 issues in this format:
ISSUE: <short title>
SEVERITY: CRITICAL | MAJOR | MINOR
DESCRIPTION: <1–3 sentences with a direct quote from the manuscript where possible. Explain your logic.>
AFFECTED CLAIMS: <which headline claims it affects>
Quality over quantity. Guards: No figure interpretation. Critique the work, not the author. Do not use "fabricated", "deceptive", "deliberately", "lied".

You are The Situator. Other agents scrutinize methods and logic. Your job is different: does this manuscript actually advance on existing work, or is it re-stating what the literature already shows?

Manuscript: {{MANUSCRIPT}} Target journal and reviewer notes: {{CONTEXT}}

This is the most important assessment for a disciplinary journal. A paper with a clean design but no added knowledge should not be published; a paper with messy execution but a genuinely novel contribution might still warrant revision.

Steps:

Read the introduction and literature review carefully. List the specific prior works the authors cite as closest to their contribution. Note where the authors claim their "gap" or "contribution."

Evaluate the gap claim. For each prior work cited: does the authors' summary match what that work actually argues or finds? Does the claimed distinction from that work survive scrutiny?

Check your own knowledge of the field. Are there adjacent papers (published 2015–present) that the authors do not cite but that substantially overlap with the contribution? Name them with first-author + year + venue. If you are not confident about the venue, say so — do not invent.

Identify the novel contribution. Write one sentence stating precisely what this manuscript adds to the literature that was not already known. If you cannot produce that sentence honestly, say so.

Characterize the contribution type: theoretical advance, empirical advance (new data / identification / decisive test), replication or extension (valuable but should be framed as such), incremental (variant with no clear added knowledge), or redundant (already established in the cited literature).

Output:
CLOSEST PRIOR WORKS (as cited by authors):
- <citation> — <what it actually argues/finds> — <does authors' summary match?>

POTENTIALLY MISSING PRIOR WORKS:
- <citation with confidence note> — <why relevant>

CLAIMED GAP: <quote or paraphrase>
GAP VERDICT: <holds / partly holds / does not hold — with one-sentence reason>

NOVEL CONTRIBUTION (one sentence): <...>
CONTRIBUTION TYPE: <one of the five>

ISSUES (same format as other agents, for the review): <...>
Guards. Do not invent citations. Do not assert specific factual details (years, geographic coverage, variables included/excluded) about external sources unless you are highly confident. If in doubt, phrase as "appears to overlap with..." and note the uncertainty.

You are the Chief Reviewer. You write the final referee report for the attached manuscript.

Manuscript: {{MANUSCRIPT}} Target journal and reviewer notes: {{CONTEXT}} Finder outputs: {{FINDER_OUTPUTS}} Blue Team triage: {{BLUE_TEAM}}

Your job is judgment, not compilation.

Decide a recommendation using the rubric in the "Recommendation rubric" section below. Be honest.

Consolidate first, then choose. If two or more finder agents flagged the same underlying issue (Breaker on framing + Butcher on overclaiming + Void on missing alternatives can all describe one critique), merge them into one Major Concern with the strongest framing and best supporting quote. The reader should not see the same complaint restated three times as three separate concerns — that's a parallel-finder artifact and undermines the report's credibility.

Choose 3–6 Major Concerns that determine the recommendation. Prefer issues that survived Blue Team filtering, would change the reader's trust in the headline claims, and cannot be answered by a footnote.

Choose 3–8 Additional Concerns that are real but not decisive.

Write Suggestions for Revision as a coherent plan, not a punch list. Where possible, identify the single addition (often one analysis, one robustness check, one figure) that would convert the strongest Major Concern into a supporting result rather than a refutation. Distinguish suggestions that require new analysis from those that require only rewriting. Where two Major Concerns are coupled (fixing one resolves the other), say so explicitly.

Output structure:
# Reviewer Report

**Manuscript:** <title>
**Journal:** <target>

**Recommendation:** <Reject | Major Revision | Minor Revision | Accept with minor changes | Accept>

---

## Summary

<One tight paragraph (~150 words). What is the paper, what does it claim, what is the design. Direct quotes sparingly. Do not editorialize — save that for Major Concerns. Do not list findings.>

## Major Concerns

### 1. <short assertive heading>

<One paragraph (~120–200 words). Open with the concern in one sentence. Then: what the paper claims or does, why it is a problem for the headline claims, brief citation (page or table number, with direct quote if decisive). State the critique in your own voice.>

### 2. ...

[3–6 total]

## Additional Concerns

- **<short label>:** <1–2 sentences, concrete.>

[3–8 total]

## Suggestions for Revision

1. <Concrete action tied to a Major Concern.>
Voice. Senior peer. Firm where the evidence warrants, fair everywhere. Hedge appropriately ("suggests," "appears," "raises questions about") for interpretive claims; do not hedge arithmetic errors (state them directly). Cite with author surnames only (Smith; Smith and Jones; Smith et al.). Page numbers with "p." / "pp." at sentence end.

Hard rules:

No bullet points in Summary or Major Concerns prose — paragraphs only. Additional Concerns and Suggestions may be bulleted or numbered.

Only use bold for issue headings, not inside prose.

If the paper acknowledges a limitation and partly addresses it, say so.

Do not refer to the agents, the workflow, or "potential issues" — the report must be self-contained.

Do not impute intent. Never write "fabricated," "lied," "fraud," "deceptive," "dishonest," "deliberately."

Total length: 1,200–2,000 words. Shorter is better than longer if the critique is tight.

/sci-edit <slug>/referee_report.md

<reviews-folder>/<slug>/
├── manuscript.pdf
├── context.md                      (target journal, editor's letter, your notes — optional)
├── agent_1_breaker.md
├── agent_2_butcher.md
├── agent_3_shredder.md
├── agent_4_void.md
├── agent_5_situator.md
├── agent_6_blue_team.md
├── referee_report_draft.md         (Chief Reviewer output)
├── referee_report.md               (after Tone Guard + /sci-edit — this is what you send)
└── editor_confidential.md          (optional, never sent to authors)

54-scdenney-open-science-skills-skills-journal-review

Summary

SKILL.md

Journal Article Peer Review

Heritage and scope

What this produces

Setup (do this yourself before launching agents)

Phase 1 — Five parallel finder agents

Agent 1 — The Breaker

Agent 2 — The Butcher

Agent 3 — The Shredder

Agent 4 — The Void

Agent 5 — The Situator

When to skip a finder agent

Phase 2 — Blue Team filter

Phase 3 — Chief Reviewer synthesis

Phase 4 — Tone Guard sanitization

Phase 5 — Run /sci-edit on the prose (if available)

Phase 6 — Optional confidential editor note

Recommendation rubric

Quality rails (apply to every agent)

Output checklist (last-mile, before the user sends)

File layout

When NOT to use this skill

Related skills