This source did not publish a separate summary. Review SKILL.md before using the skill.
SKILL.md
Load Testing Plan Skill
Produce a complete load and performance testing plan for a service — covering test objectives, scenario definitions, tooling configuration, success thresholds, and CI integration. A good load testing plan eliminates ambiguity about what "performance is acceptable" means, so engineers can run tests and get a pass/fail answer without having to interpret raw numbers themselves.
Required Inputs
Ask for these if not already provided:
Service name and key endpoints — which endpoints are under test (path, method, typical request/response shape)
Current traffic baseline — current requests/sec, p50/p99 latency, error rate under normal load
Peak traffic expectations — expected peak RPS (e.g. 10× baseline for flash sales, or seasonality peak)
Preferred testing tool — k6, Locust, JMeter, Gatling, or no preference
Test environment availability — dedicated load test environment, staging, or production (with traffic shaping)
Output Format
Load Testing Plan: [Service Name]
Author: [Name] | Team: [Team name]
Date: [Date] | Review cycle: Before each major release and quarterly
Testing tool: [k6 / Locust / JMeter / Gatling]
Test environment: [Environment name and URL]
1. Objectives and Scope
What we are testing: [Service name] handles [describe function — e.g. "user authentication requests from the mobile and web clients"]. This plan validates that the service meets its SLOs under expected and elevated traffic conditions.
In scope:
[Endpoint 1: METHOD /path — description]
[Endpoint 2: METHOD /path — description]
[Endpoint 3: METHOD /path — description]
Out of scope:
[Any endpoints explicitly excluded and why — e.g. "admin APIs — low traffic, excluded from load test"]
[Third-party integrations that cannot be load-tested — mock them instead]
2. Performance Targets (Success Criteria)
Every scenario has explicit pass/fail thresholds. A test run FAILS if any threshold is breached.
Metric
Baseline scenario
Stress scenario
Spike scenario
Soak scenario
p50 latency
< [X] ms
< [X × 1.5] ms
< [X × 2] ms
< [X] ms
p95 latency
< [Y] ms
< [Y × 1.5] ms
< [Y × 2] ms
< [Y] ms
p99 latency
< [Z] ms
< [Z × 2] ms
< [Z × 3] ms
< [Z] ms
Error rate
< [0.1]%
< [1]%
< [2]%
< [0.1]%
Throughput
≥ [N] RPS
≥ [N × 3] RPS
N/A
≥ [N] RPS
Failed requests
0 (5xx)
< [threshold]
< [threshold]
0 (5xx)
SLO reference: These thresholds are derived from the service SLOs — p99 < [Z ms], error rate < [0.1]%, availability [99.9]%.
3. Traffic Model
Baseline traffic (current production):
Average RPS: [N] req/sec
Peak RPS (observed): [N] req/sec
Request distribution by endpoint:
[Endpoint 1]: [X]% of traffic
[Endpoint 2]: [Y]% of traffic
[Endpoint 3]: [Z]% of traffic
Simulated user behaviour:
Think time between requests: [X–Y] seconds (randomised)
Purpose: Confirm the service performs acceptably under normal production load.
Duration: 10 minutes
Load profile: Ramp to [N] RPS over 2 minutes, hold for 8 minutes.
Concurrency: [N] virtual users
Pass criteria: All thresholds in the Baseline column of the targets table above.
Scenario 2: Stress Test
Purpose: Find the breaking point — how much load can the service handle before SLOs are breached?
Duration: 20–30 minutes
Load profile: Ramp from [N] RPS (baseline) to [N × 5] RPS in 5-minute steps. Hold each step for 5 minutes. Stop at first SLO breach.
Concurrency: Scales with RPS target
What to record:
RPS at which p99 latency first exceeds SLO
RPS at which error rate first exceeds SLO
Whether the service recovers when load drops back to baseline
Scenario 3: Spike Test
Purpose: Simulate a sudden traffic surge (flash sale, viral event, bot attack).
Duration: 15 minutes
Load profile: Hold at [N] RPS (baseline) for 3 minutes, spike to [N × 10] RPS instantly, hold for 5 minutes, drop back to baseline for 7 minutes.
What to record:
Latency during spike and recovery
Whether the service sheds load gracefully (rate limiting, queue depth)
Time to recover to baseline latency after spike ends
Scenario 4: Soak / Endurance Test
Purpose: Detect memory leaks, connection pool exhaustion, and slow degradation over time.
Duration: 4–8 hours (run overnight)
Load profile: Steady [N × 1.5] RPS (50% above baseline) for entire duration.
What to watch:
Memory usage trend over time (should not grow unboundedly)
Error rate trend (should be flat, not creeping up)
GC pause frequency (JVM/Go services)
Database connection pool utilisation
p99 latency trend (should not creep up over hours)
5. Test Environment Requirements
Infrastructure
Component
Requirement
Notes
Service under test
Isolated from production
[N] replicas, matching prod resource limits
Database
Separate instance with production-scale data
Seed script in section 7
Cache (Redis/Memcached)
Empty at test start
Ensures cold-start conditions are tested
Load generator
Separate from service under test
[N] vCPUs, [N] GB RAM minimum
Network
Low-latency path to service
Do not run generator on same host
Data Seeding
Before every test run, ensure the environment has:
# Seed test users (needed for authenticated endpoint tests)
[seed command or script path — e.g. python scripts/seed_load_test_users.py --count 10000]
# Seed test data for read endpoints
[seed command — e.g. ./scripts/seed_products.sh --count 50000]
# Verify seed completed
[verification command — e.g. psql $DB_URL -c "SELECT COUNT(*) FROM users WHERE load_test=true"]
Test data rules:
Never use real production user data in load tests
Tag all test-generated records with load_test=true for easy cleanup
from locust import HttpUser, task, between
import random
class [ServiceName]User(HttpUser):
wait_time = between([THINK_TIME_MIN], [THINK_TIME_MAX])
token = None
def on_start(self):
"""Called once per simulated user — authenticate."""
user_id = random.randint(1, 10000)
response = self.client.post("/auth/login", json={
"username": f"load_test_user_{user_id}@example.com",
"password": "[LOAD_TEST_PASSWORD]",
})
self.token = response.json()["access_token"]
self.headers = {"Authorization": f"Bearer {self.token}"}
@task([WEIGHT_1]) # Weight = relative frequency
def [endpoint_1_task](self):
"""[Endpoint 1 description]"""
with self.client.get(
"/[endpoint-1]",
headers=self.headers,
catch_response=True
) as response:
if response.elapsed.total_seconds() > [LATENCY_THRESHOLD]:
response.failure(f"Too slow: {response.elapsed.total_seconds()}s")
@task([WEIGHT_2])
def [endpoint_2_task](self):
"""[Endpoint 2 description]"""
self.client.post(
"/[endpoint-2]",
json={"[key]": "[value]"},
headers=self.headers,
)
Running Tests
# k6 — run baseline scenario
k6 run --env BASE_URL=https://[test-env-url] scripts/load_test.js
# k6 — run stress scenario with output to InfluxDB
k6 run --out influxdb=http://[influxdb-host]:8086/k6 \
--env SCENARIO=stress \
scripts/load_test.js
# Locust — headless run
locust -f locustfile.py \
--headless \
--users [N] \
--spawn-rate [N] \
--run-time 10m \
--host https://[test-env-url] \
--csv=results/[run-id]
# Locust — web UI (interactive)
locust -f locustfile.py --host https://[test-env-url]
7. Metrics to Capture
Capture all of the following during every test run. Missing any of these makes result comparison unreliable.
Metric
Source
Why it matters
p50, p95, p99, p999 latency per endpoint
Load tool
SLO validation
Error rate (4xx, 5xx) per endpoint
Load tool
SLO validation
Requests/sec (throughput)
Load tool
Capacity baseline
CPU utilisation (%)
Infra monitoring
Saturation signal
Memory utilisation (%)
Infra monitoring
Leak detection
GC pause time / frequency
JVM/Go metrics
Latency spike root cause
DB connection pool: active/idle/waiting
DB metrics
Pool exhaustion detection
DB query latency (p99)
DB metrics
Downstream bottleneck
Cache hit rate
Cache metrics
Miss storm detection
Pod/instance count (if autoscaling)
Infra
Scaling behaviour
Network in/out bytes
Infra
Bandwidth saturation
8. Result Analysis Framework
After each test run, work through this analysis in order:
Step 1 — Pass/fail check
Compare all captured metrics against the thresholds in Section 2. Record pass/fail per scenario.
Step 2 — Latency distribution
Plot the full latency histogram, not just percentiles. A bimodal distribution (two humps) indicates two distinct code paths — investigate the slow hump.
Step 3 — Error correlation
If errors occurred, correlate them with:
Time of occurrence (was it during ramp-up, steady state, or spike?)
Specific endpoint (is it one endpoint or all?)
Infrastructure events (CPU spike, OOM, DB connection exhaustion?)
Step 4 — Saturation analysis
Graph CPU, memory, and connection pool over time. If any resource reached 80%+ of capacity, it is a candidate bottleneck — even if SLOs passed this run.
Step 5 — Compare to baseline run
Every run should be compared to the previous run. A 10% regression in p99 latency warrants investigation even if it is still within SLO.
Regression classification:
Change
Classification
Action
p99 within 5% of previous run
Green — no regression
No action
p99 5–15% worse than previous
Yellow — watch
Investigate before next release
p99 >15% worse than previous
Red — regression
Block release, file ticket
Error rate increased vs previous
Red — regression
Block release
SLO threshold breached
Critical
Block release, page on-call
9. CI Integration
Add load tests as a gated step in the release pipeline. Run the baseline scenario on every release candidate; run all scenarios weekly.
# Example: GitHub Actions step (adapt for your CI platform)
load-test:
runs-on: ubuntu-latest
needs: [deploy-staging]
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
- name: Install k6
run: |
curl -s https://dl.k6.io/key.gpg | sudo apt-key add -
echo "deb https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update && sudo apt-get install k6
- name: Seed test data
run: [seed command]
- name: Run baseline load test
run: |
k6 run \
--env BASE_URL=${{ secrets.LOAD_TEST_ENV_URL }} \
--out json=results.json \
scripts/load_test.js
env:
LOAD_TEST_ENV_URL: ${{ secrets.LOAD_TEST_ENV_URL }}
- name: Check thresholds
run: |
# k6 exits with non-zero if any threshold fails — this step fails the build
echo "k6 threshold check complete"
- name: Upload results
uses: actions/upload-artifact@v3
if: always()
with:
name: load-test-results-${{ github.run_id }}
path: results.json
- name: Cleanup test data
if: always()
run: [cleanup command]
CI gates summary:
Baseline scenario runs on every release to staging
Full scenario suite (stress, spike, soak) runs weekly on a schedule
Any threshold failure blocks promotion to production
Results are archived for trend analysis
Quality Checks
All key endpoints are covered by at least one test scenario — no production endpoint is untested
Thresholds are derived from actual SLO targets, not guesses
Test data seeding is scripted and reproducible — tests do not rely on pre-existing environment state
The load generator runs on separate infrastructure from the service under test
CI integration blocks promotion on threshold failure — not just records results
Soak test has been run at least once to establish a memory and connection pool baseline
Results comparison to previous run is part of the analysis — not just absolute pass/fail
Anti-Patterns
Do not set thresholds without grounding them in actual SLO targets or production baselines — arbitrary numbers produce meaningless pass/fail results
Do not run the load generator on the same host as the service under test — this contaminates both the test results and the service metrics
Do not use production user data in load test seeding — all test data must be synthetic, tagged, and cleaned up after each run
Do not skip the soak test on first deployment — only a soak test reveals slow memory leaks and connection pool exhaustion that short tests miss
Do not treat a passing baseline test as evidence the service handles spikes — baseline, stress, spike, and soak scenarios test fundamentally different failure modes
