43-wentorai-research-plugins-skills-research-deep-research-kosmos-scientist-guide

brycewang-stanford/Auto-Empirical-Research-SkillsResearchAudit pendingSnapshot c6f41423768b

Summary

This source did not publish a separate summary. Review SKILL.md before using the skill.

SKILL.md

Kosmos AI Scientist Guide

Overview

Kosmos is a Claude Code-driven AI Scientist framework that automates the scientific discovery process — from hypothesis generation through literature review, experiment design, code implementation, result analysis, and paper writing. It uses Claude Code as the execution engine with structured prompts that guide it through the full scientific method. Designed for ML/AI researchers automating experiment pipelines.

Scientific Pipeline

Research Question
      ↓
  Literature Review (search + synthesize)
      ↓
  Hypothesis Generation (testable predictions)
      ↓
  Experiment Design (variables, controls, metrics)
      ↓
  Implementation (code, data pipeline)
      ↓
  Execution (run experiments)
      ↓
  Analysis (statistics, visualization)
      ↓
  Interpretation (findings, limitations)
      ↓
  Paper Draft (LaTeX manuscript)

Project Configuration

# CLAUDE.md for Kosmos AI Scientist

## Research Protocol
You are an AI Scientist conducting rigorous research.
Follow the scientific method strictly:

1. **Literature Review**: Search for related work before
   proposing anything new. Use OpenAlex API.
2. **Hypothesis**: State falsifiable hypotheses clearly.
3. **Experiment Design**: Define independent/dependent
   variables, controls, evaluation metrics.
4. **Implementation**: Write clean, reproducible code.
   Set random seeds. Log all hyperparameters.
5. **Analysis**: Run statistical tests. Report confidence
   intervals, not just point estimates.
6. **Honesty**: Report negative results. Acknowledge
   limitations. Never fabricate data.

## Tools Available
- Python 3.11+ with PyTorch, NumPy, SciPy
- LaTeX (pdflatex + bibtex)
- OpenAlex API for literature
- W&B for experiment tracking (optional)

Workflow Stages

Stage 1: Literature Review

# Kosmos automates literature search
# The AI Scientist searches, reads, and synthesizes

# Guided prompt pattern:
"""
Search for papers on: [TOPIC]

1. Find 20+ relevant papers from last 3 years
2. Read abstracts and identify key methods
3. Create a summary table:
   | Paper | Method | Dataset | Key Result |
4. Identify gaps in current research
5. Propose novel directions based on gaps
"""

# Structured experiment specification
experiment_spec = {
    "hypothesis": "Sparse attention patterns learned via "
                  "Gumbel-Softmax outperform fixed patterns "
                  "on long-sequence tasks",
    "independent_vars": ["attention_pattern_type"],
    "dependent_vars": ["accuracy", "throughput", "memory"],
    "controls": {
        "model_size": "same parameter count",
        "training_data": "same dataset and splits",
        "hyperparams": "same learning rate schedule",
    },
    "datasets": ["Long Range Arena", "PG-19"],
    "baselines": ["full_attention", "local_window",
                   "linformer", "performer"],
    "metrics": {
        "primary": "accuracy",
        "secondary": ["wall_clock_time", "peak_memory"],
    },
    "statistical_tests": ["paired_t_test", "bootstrap_ci"],
    "seed_runs": 5,
}

# The AI Scientist writes and runs experiment code

# Pattern: iterative implementation with testing
"""
Implement the experiment:
1. Write model code with unit tests
2. Write training loop with logging
3. Run small-scale validation (1 epoch, subset)
4. Verify metrics are computed correctly
5. Run full experiments (all seeds, all baselines)
6. Save results to results/ directory
"""

# Results structure
# results/
# ├── config.json         # Full hyperparameters
# ├── metrics.csv         # All run metrics
# ├── figures/            # Generated plots
# └── checkpoints/        # Model checkpoints

# Automated analysis and writing
"""
Analyze results and write paper:
1. Compute mean ± std across seeds
2. Run statistical significance tests
3. Generate publication-quality figures
4. Write LaTeX paper with:
   - Introduction (motivation + contributions)
   - Related Work (from literature review)
   - Method (formal description)
   - Experiments (setup + results + analysis)
   - Conclusion (summary + limitations + future)
5. Verify all citations are real (OpenAlex/CrossRef)
"""

### Guardrails
- Never fabricate or manipulate experimental data
- Report all results including negative ones
- Acknowledge limitations explicitly
- Verify all citations against real databases
- Include compute cost and environmental impact
- Flag when results are inconclusive
- Human review required before submission

43-wentorai-research-plugins-skills-research-deep-research-kosmos-scientist-guide

Summary

SKILL.md

Kosmos AI Scientist Guide

Overview

Scientific Pipeline

Project Configuration

Workflow Stages

Stage 1: Literature Review

Stage 2: Experiment Design

Stage 3: Implementation and Execution

Stage 4: Analysis and Paper

Safety and Ethics

Use Cases

References

Related skills