Use when auditing a finished or near-finished AER, AER:Insights, or AEJ manuscript for internal consistency: headline numbers across abstract, introduction, results, and tables; sample sizes; log-point and percentage-point conversions; cross-references; and in-text-citation/bibliography matching. Apply after the body and exhibits exist, before aer-referee-sim and aer-submission.
SKILL.md
AER Consistency
Overview
Referees and editors run cheap integrity checks before engaging with ideas:
does the abstract's number appear in the tables? Do the Ns add up? Does
"Table 4" exist? Does every citation resolve? A single mismatch reframes the
entire report from "is this right?" to "what else is wrong?" — and for
AI-assisted manuscripts these mismatches are the modal failure, because
text and tables are often generated in separate passes.
This skill is the full-manuscript integrity audit. It is mechanical by
design: every check below has a yes/no answer obtained by comparing two
artifacts, not by judgment. Run it after every revision round, not only
before first submission.
When to Use
The body sections and exhibits exist and the manuscript is being assembled
After an R&R revision, when numbers and exhibit ordering changed
Before aer-referee-sim (so the simulated referees attack substance, not
typos) and before aer-submission
Any time results were re-run — even "tiny" re-runs desynchronize text
Audit 1 — The Headline-Number Register
Build a register of every number that appears more than once in the
manuscript, then verify each row against its single source of truth (the
table or the replication output):
NUMBER SOURCE ABSTRACT INTRO RESULTS CONCL MATCH
4.2 log points Tab 3 col 4 yes yes yes yes OK
s.e. 1.1 Tab 3 col 4 yes no yes no OK
$84 billion cited source no yes no yes OK
N = 37,824 Tab 1 no no yes no OK
Rules:
Every quoted estimate matches its table to the digit, including the
standard error. No re-rounding in prose: if the table says 0.042 (0.011),
the text says 4.2, not 4 or 4.20.
Installs
0
The abstract, introduction, and conclusion quote the same headline
specification. Quoting column 3 in the abstract and column 4 in the intro
is a real and common failure.
Externally sourced numbers (the hook's "$84 billion") carry a citation at
first use, and the same value everywhere.
Audit 2 — Sample-Size Integrity
The Data section's sample funnel arithmetic is exact: raw N minus each
documented drop equals the analysis N.
The analysis N in Table 1 equals the N in the main results table for the
matching specification; every deviation (balanced panel, IV subsample) is
explained in the table notes and the text.
Observation counts are consistent with the unit of analysis (12,400
county-years from 620 counties × 20 years — check the multiplication).
Heterogeneity subsample Ns sum to the full-sample N (minus documented
exclusions).
Audit 3 — Units and Conversions
The conversion table for prose claims about coefficients:
Outcome form
Coefficient β means
Exact percent effect
log(Y), binary D
100·β log points
100·(e^β − 1)
log(Y), log(X)
elasticity
β% per 1% of X
Y in levels, binary D
β units of Y
100·β / mean(Y)
Y is a rate (share)
β·100 percentage points
100·β / baseline rate percent
Checks:
Every "percent" vs. "percentage point" usage verified against the
outcome's units. A coefficient of 0.02 on an employment rate is 2
percentage points; on log employment it is 2.0 percent (exactly 2.02).
The exact exponential conversion used whenever |β| > 0.10; 0.31 log points
is 36 percent, not 31.
Standardized effects state which SD (cross-sectional? within? whose
sample?) and the SD's value.
Currency years and deflators consistent across all sections; one base
year, named.
Signs: a negative coefficient on an inverted scale narrated correctly —
the most embarrassing class of referee catch.
Audit 4 — Stars, Standard Errors, and Stated Significance
For each starred coefficient, |β/se| is consistent with the stars under
the declared convention (1.65 / 1.96 / 2.58 thresholds approximately).
Every prose claim of "significant at the X percent level" matches the
table's stars and the CI.
Claims of "no effect" are backed by a CI the text reports, not by absence
of stars (see aer-robustness on null-result discipline).
One star convention across all tables (aer-tables-figures sets it; this
audit verifies it held).
Audit 5 — Cross-Reference Integrity
Every \ref / \autoref resolves; no "Table ??" anywhere in the PDF.
Every table and figure is referenced in the text at least once, in order
of first reference; exhibits nobody cites get cut or moved to the
appendix.
Section references survive renumbering ("see Section V" after V became
IV is the classic R&R injury).
Appendix references point to existing appendix objects, including the
supplemental file if separate.
Equation references match the renumbered equations.
Audit 6 — Citation Two-Way Match
Every in-text citation has a bibliography entry; every bibliography entry
is cited at least once. LaTeX builds with zero unresolved citation
warnings.
Author spellings and years in text match the entries (Cattaneo 2020 in
text, 2019 in the bib — fail).
The deeper verification — that entries are real and claims accurate — is
aer-literature's integrity protocol; this audit confirms it was run and
the ledger has no open rows.
Audit 7 — Claim-Evidence Map
For each empirical claim in the abstract, introduction, and conclusion,
record where the evidence lives:
CLAIM EVIDENCE STATUS
"raises 90/10 ratio by 4.2 log points" Tab 3 c4 OK
"driven by gains at the top" Fig 3 / Tab 4 OK
"absent in retail and construction" Tab 5 c2-c3 OK
"consistent with skill-biased adoption" Sec V battery OK (consistency claim)
Causal claims trace to design-based exhibits; mechanism claims are
worded as consistency ("consistent with"), matching aer-paper-body
rules.
Any claim with no exhibit or citation is rewritten or deleted — "we find"
with nothing to point at is how overclaiming enters a manuscript.
When a replication package skeleton is available, maintain
docs/claim-evidence-ledger.csv: use label:<tex-label> for manuscript
exhibits, cite:<bib-key> for externally sourced claims, and
file:<relative-output> for generated output files. Rows must be OK or
PASS before handoff.
Mechanical Procedure
Run the bundled script for the deterministic LaTeX checks (citations
two-way, ref/label two-way, duplicate labels, abstract word count):
Extract every number from the abstract and introduction (grep for
digits); locate each in a table or a cited source; build the register.
Recompute the sample funnel and the unit conversions by hand — these are
arithmetic, not judgment.
Diff the table files in the manuscript against the replication package's
output/tables/ — they must be the same files, not lookalikes
(aer-replication requires this anyway).
Produce the consistency report (below) and fix every FAIL before
handing off.
The Consistency Report
AUDIT RESULT DETAIL
1 headline numbers PASS 12 numbers, 12 matched
2 sample sizes FAIL Tab 4 N=37,824 vs Tab 1 N=37,284
3 units and conversions PASS 2 exact conversions applied
4 stars vs SEs PASS
5 cross-references PASS 31 refs, 0 dangling
6 citations two-way FAIL 2 bib entries uncited
7 claim-evidence map PASS 9 claims mapped
Fix-and-rerun until all PASS. The report travels with the handoff so
aer-referee-sim and aer-submission know the floor is solid.
Common Failure Modes
Tables regenerated after the prose was written, desynchronizing every
quoted estimate
Abstract edited for word count, changing "4.2" to "about 4" while the
tables stayed exact
Percentage points and percent swapped exactly once, in the abstract
Two tables both numbered 3 after an R&R reshuffle
The conclusion claiming a mechanism the Results section only called
"suggestive"
Repository Resources
Bundled with the installed skill, no repository checkout needed --- read it
before the repo resources below:
references/audit-checklist.md --- manual audit passes plus what the bundled script does and does not catch
When working from the AER-skills repository or plugin bundle, load only the
relevant resource:
Exhibit-to-script mapping that fixes each number's source of truth:
examples/replication-package-skeleton/docs/exhibit-register.md
Claim-to-evidence template checked by the bundled script:
examples/replication-package-skeleton/docs/claim-evidence-ledger.csv
Narration rules the claim-evidence map enforces:
skills/aer-paper-body/SKILL.md
