TOP_PRIORITY_PAPERS.md - Curated priority list organized by tier
evaluated-papers.json - Rich structured data for programmatic access
For small searches (<50 papers), stick to core files only. For large searches (>100 papers), auxiliary files add significant organizational value.
Phase 3: Search Literature
Use searching-literature skill:
Construct PubMed query from keywords
Execute search (start with 100 results)
Save results to initial-search-results.json
Report: "🔎 Found N papers matching query"
Phase 4: Evaluate Papers
Use evaluating-paper-relevance skill:
For each paper:
Check papers-reviewed.json (skip if already processed)
Stage 1: Score abstract (0-10)
If score ≥ 7: Stage 2 deep dive
Extract findings to SUMMARY.md
Download PDF and supplementary if available
Update papers-reviewed.json (for ALL papers, even low-scoring ones)
If score ≥ 7: proceed to Phase 5 for this paper
CRITICAL: Add every paper to papers-reviewed.json regardless of score. This prevents re-review and tracks complete search history.
Report progress for EVERY paper:
📄 [15/100] Screening: "Paper Title"
Abstract score: 8 → Fetching full text...
✓ Found IC50 data for 8 compounds
→ Added to SUMMARY.md
📄 [16/100] Screening: "Another Paper"
Abstract score: 3 → Skipping (not relevant)
📄 [17/100] Screening: "Third Paper"
Abstract score: 7 → Relevant, adding to queue...
Every 10 papers, give summary update
Phase 5: Traverse Citations
Use traversing-citations skill:
For papers scoring ≥ 7:
Get references (backward)
Get citations (forward)
Filter for relevance (score ≥ 5)
Add to processing queue
Evaluate queued papers (return to Phase 4)
Report progress:
🔗 Following citations from highly relevant paper
→ Found 12 relevant references
→ Found 8 relevant citing papers
→ Adding 20 papers to queue
Phase 6: Checkpoint
Check after:
Every 50 papers reviewed
Every 5 minutes of processing
Queue exhausted
Ask user:
⏸️ Checkpoint: Reviewed 50 papers, found 12 relevant
Papers with data: 7
Continue searching? (y/n/summary)
Options:
y - Continue processing
n - Stop and finalize
summary - Show current findings, then decide
Phase 7: Synthesize Findings
When stopping (user says no or queue empty):
Option A: Manual synthesis (small research sessions)
Review SUMMARY.md - Organize by relevance and topic
Extract key findings - Group by data type
Add synthesis section:
## Key Findings Summary
### IC50 Values for BTK Inhibitors
- Compound A: 12 nM (Smith et al., 2023)
- Compound B: 45 nM (Doe et al., 2024)
- [More compounds...]
### Selectivity Data
- Compound A shows >80-fold selectivity vs other kinases
- Tested against panel of 50 kinases (Jones et al., 2023)
### Synthesis Methods
- Lead compounds synthesized via [method]
- Yields: 30-45%
- Full protocols in [papers]
### Gaps Identified
- No data on selectivity vs [specific kinase]
- Limited in vivo data
- Few papers on resistance mechanisms
Update search progress stats
List all files downloaded
Option B: Script-based synthesis (large research sessions >50 papers)
For large research sessions, consider creating a synthesis script:
Filter papers-reviewed.json to extract only relevant papers (score ≥ 7):
# Read papers-reviewed.json
with open('papers-reviewed.json') as f:
all_papers = json.load(f)
# Filter for relevant papers (score >= 7)
relevant_papers = {
doi: data for doi, data in all_papers.items()
if data.get('score', 0) >= 7
}
# Save to relevant-papers.json
with open('relevant-papers.json', 'w') as f:
json.dump(relevant_papers, f, indent=2)
Small searches (<50 papers): Core files only (papers-reviewed.json, SUMMARY.md, citation-graph.json)
All searches: Create relevant-papers.json at end; enhance SUMMARY.md with methodology
Large searches (>100 papers): May add auxiliary files (README.md, TOP_PRIORITY_PAPERS.md, evaluated-papers.json) for better organization
Error Handling
No results found:
Try broader keywords
Remove constraints
Check spelling
Try different synonyms
API rate limiting:
Report to user: "⏸️ Rate limited, waiting..."
Wait required time
Resume automatically
Full text unavailable:
Note in SUMMARY.md
Continue with abstract-only evaluation
Flag for manual retrieval if highly relevant
Too many results (>500):
Suggest narrowing query
Process first 100, ask if continue
Focus on most recent or most cited
Quick Reference
Phase
Skill
Output
Parse
(built-in)
Keywords, data types, constraints
Initialize
(built-in)
Folder, SUMMARY.md, tracking files
Search
searching-literature
List of papers with metadata
Evaluate
evaluating-paper-relevance
Scored papers, extracted findings
Traverse
traversing-citations
Additional papers from citations
Synthesize
(built-in)
Enhanced SUMMARY.md with methodology + findings
Consolidate
(built-in)
relevant-papers.json (filtered to score ≥7)
Common Mistakes
Not tracking all papers: Only adding relevant papers to papers-reviewed.json → Add EVERY paper to prevent re-review, track complete history
Creating unnecessary auxiliary files for small searches: For <50 papers, stick to core files (papers-reviewed.json, SUMMARY.md, citation-graph.json). For large searches (>100 papers), auxiliary files like README.md and TOP_PRIORITY_PAPERS.md add value.
Silent work: User can't see progress → Report EVERY paper, give updates every 10
Non-clickable identifiers: Plain text DOIs/PMIDs → Always use markdown links
Jumping to evaluation without good search: Too narrow results → Optimize search first
Not tracking papers: Re-reviewing same papers → Always use papers-reviewed.json
Following all citations: Exponential explosion → Filter before traversing
No checkpoints: User loses context → Report and ask every 50 papers
Poor synthesis: Just list papers → Group by data type, extract key findings
Batch reporting: Reporting 20 papers at once → Report each one as you go
User Communication (CRITICAL)
NEVER work silently! User needs continuous feedback.
Report frequency:
Every paper: Brief status as you screen (📄 [N/Total] Title... Score: X)
Every 5-10 papers: Progress summary with counts
Every finding: Immediately report what data you found
Every decision point: Ask before changing direction
Be specific in progress reports:
✅ "Found IC50 = 12 nM for compound 7 (Table 2)"
❌ "Found data"
✅ "Screening paper 25/127: Not relevant (score 3)"
❌ Silently skip papers
Ask for clarification when needed:
✅ "Are you looking for in vitro or in vivo IC50 values?"
❌ Assume and potentially waste time
Report blockers immediately:
✅ "⚠️ Paper behind paywall - evaluating from abstract only"
