A queued, staged pipeline that generates structured, line-accurate review comments using Codex in read-only mode with global RPM/TPM-aware throttling.
Handling 5,000 concurrent PR reviews means up to 5,000 review jobs outstanding at any time. The challenge is controlling active concurrency to match API rate limits and CI capacity without blowing rate limits or spamming engineers with stale or duplicate reviews.
This gives the cleanest permissions and SCM integration (CI), plus one place to enforce org-wide rate limits, caching, routing, and quality controls (service). Model recommendation: gpt-5.2-codex
Trigger on PR events, create idempotent jobs keyed by (repo, pr_number, head_sha). Cancel superseded runs to avoid reviewing stale diffs.
Input = diff + minimal surrounding context (50 lines around hunks), plus repo signals: language, touched modules, test failures, lint output.
Force strict JSON schema for findings: file path, line range, severity/priority, title/body, confidence. Run in read-only sandbox.
For high-risk PRs, fan out specialist agents (security/perf/correctness/maintainability), then fan in via meta-agent for dedup and ranking.
Global limiter for RPM and TPM, exponential backoff/retry on 429s. Batch within single request only where safe.
Triage route: small/fast model to decide review depth. Flex processing for non-urgent PRs, Batch API for offline tasks.
End-to-end flow from webhook to published review
If your actual TPM < that, you must: increase W, reduce T (context packing/triage), reduce calls/PR, or accept backlog.
Pilot on 1-2 repos: implement the Codex structured-review workflow end-to-end, then add central throttling + evals before scaling to org-wide 5000-PR concurrency.