Working with AI coding agents: what actually helps

Coding agents can now read a repository, edit files, run commands and open a pull request. Getting code out of one is easy. Getting code you would merge takes what a new teammate needs: a clear ticket, a way to check their own work, and a careful review. None of that depends on which tool you use.

Where things stand this year

The first half of 2025 put an agent into most of the places developers already work.

• Claude Code, Anthropic's agent for the terminal, arrived as a limited research preview on 24 February alongside Claude 3.7 Sonnet. It became generally available on 22 May with the Claude 4 models.
• Codex CLI, OpenAI's open-source agent that runs locally in your terminal, came out on 16 April. On 16 May OpenAI followed with a research preview of Codex in ChatGPT, which runs each task in its own cloud sandbox with your repository loaded.
• Cursor shipped an early agent inside Composer in version 0.43 last November, and made Agent the default mode in 0.46 in February.
• GitHub announced the Copilot coding agent on 19 May. You assign it an issue, it works in an environment powered by GitHub Actions, and it pushes commits to a draft pull request for you to review.

They differ in where they run and how much they do before checking with you. Underneath, the loop is the same: read some code, change some files, run a command, look at the result, go again. The habits below are about feeding that loop and checking what comes out, so they carry across tools.

Write the task like a ticket

An agent fills every gap in a request with a guess. A wrong guess looks exactly like a right one until you read the diff, and by then more code may sit on top of it.

The fix is old and dull: write the task as you would a ticket for a colleague new to the codebase. Four parts cover most of it.

• Goal. What should be true when this is done, in a sentence or two.
• Constraints. What must not change: signatures, dependencies, behaviour for existing inputs, files that are out of scope.
• Where to look. The files and tests involved, and an existing example of the pattern if there is one.
• Definition of done. Checks the agent can run itself. "Make it work" is not one.

Written out in full, a brief like that is still short. I'd put it in the issue, or paste it as the first message:

## Goal
`parse_duration` in `durations.py` should accept compound durations
such as `1h30m` and `2m15s`, as well as the single units it takes today.
 
## Context
- Code: `durations.py`. Tests: `tests/test_durations.py`.
- Callers pass user input, so bad input must raise `ValueError`,
  never return 0.
 
## Constraints
- Units are h, m and s, in that order. No days, no decimals.
- No new dependencies. Keep the function's signature.
- Change only the two files above. If something else looks wrong,
  note it in your summary instead of fixing it.
 
## Definition of done
1. Add test cases first: `1h30m` -> 5400, `2m15s` -> 135, and
   `""`, `"1h1h"`, `"30m1h"` raising `ValueError`. Run them and
   show that the new ones fail.
2. Make them pass without editing existing tests.
3. `./scripts/check.sh` exits 0.
4. Summarise what changed and anything you were unsure about.

The last line matters more than it looks. Asking for doubts in writing gives the agent somewhere to put "I wasn't sure whether..." instead of quietly picking an answer. Anthropic's guidance for Claude Code makes the wider point: the more specific the instructions, the better it does, most of all on a first attempt.

Give the repository an instruction file

Each of these tools reads standing instructions from a file in the repository. The names differ:

Write it as onboarding notes for someone fast, literal and new. Three things earn their place:

• The exact commands to set up, lint and test, written so they can be pasted. They're what let the agent check its own work.
• The rules you would otherwise repeat in every review: don't touch the tests to make them pass, no new dependencies without asking, where new files go.
• The traps: the command that only works with a particular flag, the service that has to be running first, the generated folder nobody should edit.

Leave out style rules a linter already enforces, the project's history, and anything long. The file is read on every task, so every line competes for attention. Anthropic suggests treating it as a prompt you keep tuning, not a file you only add to.

# Notes for coding agents
 
## Commands
- Set up: `python -m venv .venv && .venv/bin/pip install pytest ruff`
- Check everything (lint, format, tests): `./scripts/check.sh`
- One test file: `pytest -q tests/test_durations.py`
 
## Rules
- Run `./scripts/check.sh` before you say a task is done.
- Never pipe test output through `head` or `tail`. The exit code is the result.
- Don't edit, skip or delete existing tests to make a change pass.
  If a test looks wrong, stop and explain why.
- No new dependencies without asking.
- Stay inside the files the task names.
 
## Layout
- `durations.py`: parsing helpers. Pure functions, no I/O.
- `tests/`: pytest, one file per module. Tests import from the
  repo root (`pythonpath = ["."]` in `pyproject.toml`).

Name it for the tool you use. The same content works as an AGENTS.md.

One command that means "done"

The most useful line in that file points at a single script that runs exactly what CI runs. People, agents and CI then share one definition of green.

#!/usr/bin/env bash
# One command for people, agents and CI: lint, format, tests.
set -euo pipefail
 
ruff check .
ruff format --check .
pytest -q

On the small example project from the brief, with the change in place (Python 3.11, pytest 8.4.0, ruff 0.11.13):

All checks passed!
2 files already formatted
..........                                                               [100%]
10 passed in 0.04s

set -e stops at the first failing step, so the script's exit code is the answer. Run it from the repository root, as CI does: a check that passes on src/ alone can still fail on a file outside it.

Make tests the contract

A test is the clearest brief you can give an agent: an input, an expected output, and a command that says pass or fail. It turns "I think it works" into something both of you can check.

Order matters. Anthropic describes a test-first workflow for Claude Code that suits any agent:

1. Have it write tests from pairs of inputs and expected outputs, and say plainly that this is test-driven development, so it doesn't write stand-in implementations for code that doesn't exist yet.
2. Have it run the tests and confirm they fail, without writing implementation code.
3. Commit the tests once you're happy with them.
4. Have it write code until they pass, without changing the tests.

For the duration example, step one is a test file:

import pytest
 
from durations import parse_duration
 
 
@pytest.mark.parametrize(
    ("text", "seconds"),
    [("90s", 90), ("5m", 300), ("2h", 7200), ("1h30m", 5400), ("2m15s", 135)],
)
def test_parses(text, seconds):
    assert parse_duration(text) == seconds
 
 
@pytest.mark.parametrize("text", ["", "30", "5x", "1h1h", "30m1h"])
def test_rejects(text):
    with pytest.raises(ValueError):
        parse_duration(text)

Step two, pytest -q --tb=no against the old parser, which only knows single units:

...FF.....                                                                                   [100%]
===================================== short test summary info ======================================
FAILED tests/test_durations.py::test_parses[1h30m-5400] - ValueError: not a duration: '1h30m'
FAILED tests/test_durations.py::test_parses[2m15s-135] - ValueError: not a duration: '2m15s'
2 failed, 8 passed in 0.03s

That is the output you want before any code changes. The new cases fail for the reason you expect, and the old ones still pass. A test that passes before the change isn't testing the change. The change itself is a few lines in one file, and afterwards check.sh prints the green run shown earlier.

Two more things make the contract hold.

The agent runs the full check itself and shows you the result, not "the tests should pass". Codex builds this in: its summaries cite the terminal logs and test output behind each step. Whatever the tool, ask for that evidence.

Existing tests are off limits. If the agent may edit tests, the shortest route from red to green is to change what a test expects. Say so in the brief and the instruction file, then check in review.

Here it is against the failing suite above:

pytest -q --tb=no | tail -n 1
echo "exit code: $?"
 
set -o pipefail
pytest -q --tb=no | tail -n 1
echo "exit code: $?"

2 failed, 8 passed in 0.03s
exit code: 0
2 failed, 8 passed in 0.04s
exit code: 1

An agent trimming output to save context can do this as easily as a person. If anything reads that exit code to decide what to do next, two failing tests count as a pass. Run test commands bare, or with pipefail set, as check.sh does.

Keep diffs small

A diff you can't read in one sitting is a diff you will skim, and an agent can produce a long one in minutes. Size is the one thing about the output you control before it exists.

• One concern per task. If the goal has an "and" in it, it's probably two tasks.
• Plan before code for anything past a file or two. Ask for a plan, say no code yet, and read it. Anthropic notes that, skipping this step, Claude often starts writing code straight away. A plan is cheap to correct; a long diff isn't.
• Keep mechanical and meaningful apart. Formatting, renames and moves go in their own commit, so the one that changes behaviour stays short.
• Commit at every green point, so there's always a passing state to return to.
• Stop early. If the first steps go the wrong way, interrupt (Escape, in Claude Code), fix the brief and restart rather than stacking corrections.

Review it like a teammate's pull request

An agent's diff deserves the review you would give a capable new colleague: careful, specific, and assuming nothing. OpenAI's launch post for Codex says the same: even with its logs and test results to check, a person still has to read and validate the code before it's merged or run.

Four things are worth looking for every time.

Invented APIs

Methods missing from your version of a library, config keys nothing reads, and packages that don't exist at all. A study of 16 code-generating models (Spracklen et al., first posted June 2024, revised March 2025) found that, on average, at least 5.2% of packages in code from commercial models were hallucinated, against 21.7% for open-source models. A made-up package name is also a supply-chain risk, because someone else can publish a real package under it. Check every new import against your lockfile, and every new call against the docs for the version you actually run.

Skipped error handling

Look for except Exception: pass, errors logged and then ignored, a default returned where the brief said raise (the return 0 the duration brief ruled out), removed retries or timeouts, and TODO stubs standing in for the real thing. Code that never fails loudly still passes a happy-path test.

Silenced tests

New skip or xfail markers. Assertions loosened from == 5400 to "is truthy". Expected values edited to match whatever the code now returns. Tests deleted. Mocks so broad that the code under test never runs. Any change to an existing test needs a reason you agree with.

Over-broad changes

Files outside the brief, reformatting mixed in with logic, new dependencies, edits to CI config, lint rules switched off, or # noqa and # type: ignore added to quiet a warning instead of fixing its cause. Lockfile changes deserve a look too.

Finally, run check.sh yourself. The agent's summary of its test run is a claim. The exit code on your machine is evidence.

Permissions and safety

An agent runs commands with your permissions. Treat it like a script you didn't write.

Keep production credentials away from it. Anything the agent can read can end up in its context, in a log or in a commit. Give it development keys and a local or disposable database, and keep live .env files out of the working copy it uses.

Read commands before approving them. By default, Claude Code asks before file writes and many shell commands, and Codex CLI's default Suggest mode asks before every file write and every command other than reading files. Allow the commands that are safe or easy to undo, like the test runner or git diff, so the prompts that remain are worth reading. In Claude Code that list can live in .claude/settings.json, checked in so the team shares it.

Run unattended work in a sandbox. Here is what each tool offers or recommends:

• Codex CLI's Full Auto mode runs commands with the network disabled and writes confined to your working directory, using Apple's Seatbelt on macOS. On Linux, its README recommends running it inside Docker.
• Codex in ChatGPT runs each task in an isolated cloud container. At launch, internet access was off while a task ran. Since 3 June you can choose to give it access.
• Copilot's coding agent can only reach internet destinations on an allow-list you control, and its pull requests need a human's approval before any CI workflows run. The person who asked for the pull request can't be the one who approves it.
• For Claude Code, Anthropic suggests using the flag that skips permission checks only inside a container without internet access, and points to a Docker dev container setup as a reference.

Expect prompt injection. An agent reads issues, web pages, dependency source and test fixtures, and any of them can contain text written to look like instructions. Anthropic's guidance lists data exfiltration through prompt injection among the risks of letting an agent run arbitrary commands. No secrets and no network keep the damage small if it happens.

Make git your undo button. Start each task from a clean working tree on a fresh branch. Codex CLI warns you if you start its automatic modes in a folder git isn't tracking, so that there is always a way back.

Where agents still struggle

The misses fall into patterns. GitHub describes its coding agent as best at "low-to-medium complexity tasks in well-tested codebases", which is a fair description of where any of these tools do well. The list below is mostly the opposite cases.

• Vague tasks. Without a definition of done, the agent decides when it's done.
• Large refactors without tests. If nothing pins down today's behaviour, neither of you can tell whether it survived. Write characterisation tests first, then refactor in steps that each pass.
• Private APIs it has never seen. Your internal SDK almost certainly isn't in any model's training data, so its methods get guessed from their names. Put the interface, a working example call or the docs in the repo, and point the brief at them.
• UI it can't see. CSS that compiles can still look wrong, and a test suite won't notice. Give it a mock to aim at and a way to see its result, then look at the screen yourself. Anthropic suggests giving Claude Code a screenshot tool, such as a Puppeteer MCP server, and iterating against a visual mock.
• A test setup it can't start. If the suite needs a database or service that isn't running, the agent can't close the loop. Expect untested code, or a test changed until it passes. Say in the instruction file how to start what the tests need.

What I'd take away

The agent is quick in the middle of the loop: reading, editing, running and trying again. The two ends are still yours. Deciding what "done" means comes before it starts, and deciding whether it is done comes after.

If I were setting up a repository for agents today, I'd start with three things: a check.sh that runs exactly what CI runs, a short instruction file that points at it, and a failing test written into every brief. None of that is specific to agents. It's what a good team already does for new people. With an agent, the payoff for doing it shows up sooner, and so does the cost of skipping it.