integration-testing-discipline

microsoft/amplifier-bundle-superpowersDeveloper ToolsAudit passedSnapshot 0c4a0865603c

Installs: 1

Summary

4 principles for E2E testing discipline — observe first, fix in batches, expect long durations, check container state directly. NO FIXES DURING OBSERVATION RUNS.

SKILL.md

Integration Testing Discipline

The Core Principle

OBSERVE FIRST. FIX IN BATCHES. NEVER FIX DURING ACTIVE E2E RUNS.

When running integration tests or E2E validation, your job is to CAPTURE ALL failures first, then fix them systematically. Making code changes during a running test invalidates that test run.

The Four Principles

Principle 1: Don't Fix During Observation Runs

DO: Let the E2E run complete (or fail with a real error), capture ALL failure points, then fix everything as a coordinated batch.

DON'T: See one failure, fix it immediately, and continue the same E2E run.

Why: Code changes during E2E runs invalidate the running test. You lose the ability to trust that run's results.

Example:

❌ WRONG:
1. E2E run finds validation error
2. Fix validation immediately  
3. Continue same E2E run
4. Find auth error
5. Fix auth immediately
6. Continue same E2E run
→ Result: Can't trust this run's success

✅ RIGHT:
1. E2E run finds validation error — RECORD IT
2. E2E run finds auth error — RECORD IT  
3. E2E run completes — STOP OBSERVING
4. Fix validation AND auth as a batch
5. Start fresh E2E run to validate fixes
→ Result: Clean validation of coordinated fixes

Principle 2: Long-Running Processes Are Normal

DO: Check for actual error signals: non-zero exit codes, error messages in logs, process death, hung processes.

DON'T: Declare "stuck" or "failed" based on wall clock time alone.

Expected Durations:

Container setup: 60-90 seconds
Simple spec (1 endpoint): ~13 minutes
Medium spec (4 CRUD endpoints): ~25 minutes
Complex spec (8+ endpoints): ~40 minutes
Each convergence iteration: 5-8 minutes

Example:

❌ WRONG:
"Process has been running 20 minutes with no output — it's stuck"

✅ RIGHT:
"Process has been running 20 minutes. Checking:
- Exit code: still running (0)
- Error logs: none
- New files: tracker.json updated 30s ago
- Container: process alive, making progress
→ Status: WORKING, not stuck"

❌ WRONG:
"API shows no progress for 10 minutes — declaring failure"

✅ RIGHT:
"API shows no progress. Checking container directly:
$ docker exec container-name ls -la /workspace/
$ docker exec container-name cat tracker.json  
$ docker exec container-name ps aux
→ Found: New files created 2 min ago, process active
→ Status: API lag, not failure"

❌ WRONG:
Run 1: Find validation error → fix → new run
Run 2: Find auth error → fix → new run  
Run 3: Find storage error → fix → new run
→ Result: 3 full E2E cycles for related issues

✅ RIGHT:
Run 1: Find validation + auth + storage errors → RECORD ALL
Fix: Address validation first (dependency), then auth, then storage
Run 2: Validate all fixes together
→ Result: 1 observation run + 1 validation run

integration-testing-discipline

Summary

SKILL.md

Integration Testing Discipline

The Core Principle

The Four Principles

Principle 1: Don't Fix During Observation Runs

Principle 2: Long-Running Processes Are Normal

Principle 3: Check Container State Directly

Principle 4: One Run, All Errors

When to Apply

Red Flags — Return to Observation Mode

Impact

Related skills