I want one runtime that can give Codex and Claude the same task, isolate their work, record every decision and tool result, select the best candidate with tests, and learn when each model is worth its cost. “Ask two agents” is not an architecture; a typed protocol, bounded authority, and reproducible evaluation are.
The unit of work is a contract
Each task begins as an immutable envelope: goal, repository
revision, scoped paths, constraints, acceptance commands, resource
budget, and a definition of done. Repo conventions belong in
AGENTS.md or the provider's equivalent; the task
envelope carries only what changes for this job. That keeps prompts
small enough to inspect and makes a run replayable.
{
"task_id": "auth-refresh-017",
"base_sha": "8b14…",
"goal": "rotate refresh tokens after use",
"scope": ["src/auth/**", "tests/auth/**"],
"acceptance": ["npm test -- auth", "npm run typecheck"],
"limits": {"wall_minutes": 20, "usd": 6, "writes": "worktree"},
"artifacts": ["patch", "test-results", "decision-log"]
}A model response never advances the state; only typed outputs and gate results do.
Use disagreement deliberately
The simplest strong pattern is asymmetric. One agent inspects and plans; another implements without seeing persuasive prose from the first, only the task contract and accepted plan. A fresh reviewer receives the patch, tests, and original acceptance criteria. The reviewer cannot edit. This reduces shared-context anchoring and makes “looks plausible” compete with executable evidence.
Provider assignment is a policy decision learned from results, not a permanent hierarchy.
The six components I would build
| Component | Responsibility | Evidence it must emit |
|---|---|---|
| Task compiler | Turns an issue into the immutable contract and rejects ambiguous acceptance criteria. | Validated JSON, base revision, dependency snapshot. |
| Workspace manager | Creates a disposable worktree/container with path and network policy. | Environment digest and complete filesystem diff. |
| Provider adapters | Normalize Codex and Claude events into one stream without erasing provider-native detail. | Raw event log plus normalized trace. |
| Gate runner | Runs formatting, types, unit/integration tests, policy scans, and task-specific checks. | Commands, exit codes, stdout/stderr hashes, durations. |
| Router | Selects role/model using task features, prior pass rate, latency, and marginal cost. | Routing rationale and counterfactual baseline. |
| Replay UI | Shows prompt, tools, edits, retries, token/cost estimates, and final artifact as a timeline. | Shareable, redacted run bundle. |
How Codex and Claude plug in
Codex already supports parallel subagent workflows, repository
guidance through AGENTS.md, sandbox and approval
controls, and review/test-oriented workflows. Parallelism is most
valuable for independent, read-heavy work; parallel writers need
non-overlapping scope or separate worktrees.
Claude Code exposes a useful non-interactive adapter surface:
claude -p, JSON or streaming JSON output, explicit
allowed/disallowed tools, model selection, permission mode, and a
maximum-turn limit. MCP can provide the same external tool
contracts to either provider. The harness should preserve these
native controls rather than hiding everything behind a lowest-common
denominator.
Evaluation before clever routing
Start with 30–50 versioned tasks drawn from the target repositories: bug fixes, feature work, refactors, test generation, migrations, and review. Each task needs a frozen base commit and hidden behavioral checks. Report first-pass success, human-accepted success, regression rate, wall time, tokens, estimated cost, review precision, and unnecessary diff size. Run both providers on matched tasks before fitting a router.
The critical anti-metric is self-reported completion. The agent's prose is an artifact, never the label. A task succeeds only when gates pass and a human accepts the diff. Traces must scrub secrets, tool outputs need size limits, network access must be declared, and destructive or externally visible actions stay approval-gated.
Milestones that turn this page into a case study
- Replayable single-agent runner: one contract, one worktree, one provider, one complete trace.
- Cross-provider adapter: Codex and Claude produce the same normalized event schema.
- Independent review: blind patch review with findings mapped to test evidence.
- Matched benchmark: at least 30 frozen tasks with cost, latency, and accepted-pass results.
- Learned routing: only after the always-Codex, always-Claude, and cheapest-model baselines exist.
Primary documentation
- OpenAI Codex use cases: verified workflows, evaluation, and engineering patterns.
- OpenAI: repository guidance with AGENTS.md.
- OpenAI: Codex subagents.
- Anthropic: Claude Code CLI reference.
- Anthropic: Model Context Protocol.