CodeRabbit Production Checklist

Overview

Complete checklist for deploying CodeRabbit as a production merge gate. Covers the transition from "optional AI review" to "required status check that blocks merges." Ensures configuration is tuned, team is onboarded, and fallback procedures are documented.

Prerequisites

• CodeRabbit installed and running in non-blocking mode for 1-2 weeks
• Team has seen CodeRabbit reviews and provided feedback
• .coderabbit.yaml tuned based on pilot feedback
• GitHub organization admin access

Instructions

Step 1: Pre-Launch Configuration Audit

# Run through each item before going live:

## App Installation
- [ ] CodeRabbit GitHub App installed on all target repos
- [ ] Repository access scoped correctly (all repos vs select)
- [ ] Seat assignment policy set (active committers vs manual)
- [ ] Bot accounts excluded from seats (dependabot, renovate)

## Configuration
- [ ] `.coderabbit.yaml` committed to main branch
- [ ] YAML syntax validated (no parse errors)
- [ ] `auto_review.enabled: true`
- [ ] `auto_review.drafts: false` (skip draft PRs)
- [ ] `base_branches` includes all target branches (main, develop)
- [ ] `path_filters` excludes generated/lock/vendor files
- [ ] `path_instructions` configured for key directories
- [ ] `chat.auto_reply: true`

## Review Behavior
- [ ] Profile set to "assertive" (or team's preferred level)
- [ ] `request_changes_workflow` decision documented:
  - `true`: CodeRabbit blocks merge when issues found
  - `false`: CodeRabbit only comments (non-blocking)
- [ ] Org-level defaults in `.github/.coderabbit.yaml` if multi-repo

Step 2: Validate Configuration

set -euo pipefail
echo "=== CodeRabbit Production Readiness Check ==="

# 1. YAML syntax
echo "--- Config Validation ---"
if [ -f .coderabbit.yaml ]; then
  python3 -c "
import yaml, sys
try:
    config = yaml.safe_load(open('.coderabbit.yaml'))
    print('YAML syntax: PASS')

    reviews = config.get('reviews', {})
    auto = reviews.get('auto_review', {})

    checks = {
        'auto_review.enabled': auto.get('enabled', False) == True,
        'auto_review.drafts': auto.get('drafts', True) == False,
        'path_filters configured': len(reviews.get('path_filters', [])) > 0,
        'path_instructions configured': len(reviews.get('path_instructions', [])) > 0,
        'chat.auto_reply': config.get('chat', {}).get('auto_reply', False) == True,
    }

    for name, passed in checks.items():
        status = 'PASS' if passed else 'WARN'
        print(f'  {name}: {status}')

    if all(checks.values()):
        print('Overall: READY FOR PRODUCTION')
    else:
        print('Overall: Review warnings above before going live')

except yaml.YAMLError as e:
    print(f'YAML syntax: FAIL ({e})')
    sys.exit(1)
" 2>&1
else
  echo "FAIL: .coderabbit.yaml not found"
fi