TL;DREvery trap in backfilling tests is an ordering trap: make Codex read and describe current behavior first, make it test what the code does rather than what it should do, add tests one batch at a time, and gate each batch on green. Plus the key rule for failures: fix the code or fix the test?
Sources SOURCES
Distilled from public community discussions. This page only breaks them down and verifies — it never reposts the original; opinions belong to their authors.
Facts FACTS
Check these claims against the listed sources. For allowance mechanics, first-party account displays outrank community summaries.
- Codex can run the project's test command, read the failure output, and iterate on fixes — the precondition that makes test backfilling semi-automatable.
- The AGENTS.md optimization thread on the community forum lists 'tests first' among the most effective instructions for agents: spell out the test command and the acceptance bar, and agent output quality rises noticeably.
Concepts CONCEPTS
Not everything said in a community thread is a fact — each concept carries its nature label. Read the label first.
The classic wipeout in legacy testing: you ask Codex to write tests for 'the behavior the code should have' and end up testing its own imagination. The correct framing is the reverse — have it describe how the code actually behaves today, then freeze that current behavior into tests.
The rule for failures: behavior matches business expectations, so fix the test (the assertion was wrong); behavior is a clear bug, so fix the code and log it. The rule itself is simple. The discipline is not.
Step-by-step SOP STEP BY STEP
- 01Make it read first
Ask: 'Read this module and list every exported function's inputs, outputs, and side effects — write no code.' Check that understanding by hand for obvious errors.
- 02Confirm test infrastructure
Ask: 'What test framework does this project use? If none exists, propose the minimal setup.' Get one empty test passing before anything else.
- 03Batch one: core paths
Have it write 'test what is' cases — known input, current actual output — for only the 2–3 most-used functions.
- 04Green one batch, then the next
Once everything passes, request one batch of edge cases. One batch at a time; handle failures immediately; never stockpile.
- 05Dispose of failures by rule
Apply the fix-code-or-fix-test rule to every failure, and record the bugs you squashed in the commit message.
- 06Persist into AGENTS.md
Write the test command and the 'test what is' policy into the repo's AGENTS.md, so every future agent follows them automatically.
Keep reading NEXT
Want today's verdict? Back to the live radar.