Almost Right. The Hidden Cost of Code That Passes Every Check

Suddenly I am a full-time code reviewer.

That was not really the plan. What happened is that the agents got good enough that features which took me two or three days now lands inside an afternoon, and suddenly my week stopped being about writing software and started being about reading it. I am not complaining, because the speed is real and I would not hand it back. The bit I did not see coming is that code now arrives faster than anybody's understanding of it, mine very much included.

I am not talking about code that is obviously broken. Invalid syntax, imaginary libraries and imports that refuse to resolve are irritating, but they at least announce themselves, and the compiler or the test run will tell you within seconds that something has gone wrong so you fix it and carry on.

The failures worth talking about are the ones that look entirely fine. The implementation compiles, the types are satisfied, the tests may well be green, and every individual line reads sensibly. Somewhere between the requirement and the implementation, though, the problem being solved has shifted very slightly to one side.

What you are left with is code that is almost right.

Almost-right code is not a new invention, engineers have been producing it for as long as we have been writing software, usually at about four on a Friday afternoon. What the agents change is the rate. If I can generate five times as much code, then I am also generating five times as many small assumptions that nobody consciously agreed to, and if those do not get caught during review, the hour I saved generating the code reappears later at a considerably worse moment while I work out why something does not quite behave.

Five versions of this keep turning up often enough that they have changed how I read a diff.

1. Permuting four digits and calling it six

Take a requirement to start with four existing digits, add two more, and build six-digit permutations from the result.

An implementation might look something like this.

from itertools import permutations

for ordering in permutations("1234"):
    prefix = "".join(ordering)

    for a in range(10):
        for b in range(10):
            candidate = prefix + str(a) + str(b)

At first glance it seems reasonable enough. It reaches for itertools.permutations, it generates the two additional digits and it produces a six-digit candidate. What it does not do is permute six digits.

Only the original four are being shuffled. The two new digits are welded to the end and they stay there.

I find this sort of mistake much more interesting than broken syntax, because nothing in the implementation looks ridiculous. Every step maps neatly onto a phrase in the requirement, which is how it survives a quick read, and the error only shows up when you stop reading the lines and start asking what the algorithm as a whole actually does.

That has been one of the bigger adjustments for me: checking that an implementation contains the right operations is not enough, because you also have to check that those operations describe the problem somebody asked about.

Tests can catch it, but only if they were written from the requirement rather than from the implementation. When the same misunderstanding produces both the code and its tests, you get a beautifully tested version of the wrong algorithm.

2. Making TypeScript stop complaining

TypeScript gives us another good one. Imagine an account lookup.

function findAccount(accountId: string) {
    const account = accounts.find(x => x.id === accountId)!;
    return account;
}

That ! is providing a considerable amount of emotional support.

Array.find() can return undefined, so TypeScript is quite correctly asking the code to acknowledge the possibility, and the non-null assertion replies that we know better and that account definitely exists.

Runtime is unaffected, of course. If the account is missing, then it is still missing - the generated code has not handled that case, it has removed the warning that told us the case was there.

So this is one of the things I now look for specifically in generated TypeScript. Did the implementation resolve the uncertainty, or did it persuade the type checker to stop mentioning it?

The same goes for enthusiastic use of as, any, non-null assertions and my personal favourite.

value as unknown as SomeType

Casts are sometimes entirely justified and the presence of one is not automatically a problem. When an agent has been handed a quality gate saying the type checker must pass, though, there is a meaningful difference between making the code type-safe and making the type checker green.

Agents are very good at satisfying explicit constraints, which is usually exactly what we want from them. If the constraint is that npm run typecheck must pass, then making the warning disappear does technically achieve the goal, and the runtime state TypeScript was worried about is still sitting there waiting for someone.

3. Logging an error is not handling an error

Generated error handling has a similar habit.

async function updateProfile(profile: Profile) {
    try {
        await database.save(profile);
    } catch (err) {
        console.error("Failed to save profile", err);
    }
}

This looks responsible. There is a try, there is a catch, and there is an error message with the relevant object attached, so during a fast review it is very easy to tick the error handling box and keep scrolling.

The problem lives outside the function. From the caller's point of view, this line completed successfully.

await updateProfile(profile);

await updateProfile(profile);

The database write failed, the function swallowed the exception, and the promise resolved as though everything had gone to plan. Anything further up the stack now proceeds on the assumption that the profile was saved, so a clear failure has quietly become an inconsistent state, which is a considerably more expensive thing to debug three weeks later.

What the right answer looks like depends on the application. The operation might want a retry, the caller might need an error result, or the request might simply need to fail loudly. There is no universally correct implementation here, but there does need to be a decision, and a console.error is not one.

The same distinction holds well beyond try/catch, because generated code is often excellent at making an immediate error disappear, which is not the same thing as making the system behave sensibly when that error occurs.

4. The empty state nobody mentioned

Some of these are much smaller.

matches = find_matches(query)
best_match = matches[0]

There is nothing unusual about taking the first item in a collection and I have written that line hundreds of times. The interesting part is the assumption sitting inside it, which is that find_matches() always returns at least one result.

That may be perfectly valid. It may also be true only of the examples the model had in front of it while writing the feature, in which case the first real query with no matches becomes an IndexError.

A more explicit version might be this.

matches = find_matches(query)

if not matches:
    return None

return matches[0]

Returning None is not automatically the right answer either. Depending on the domain, an empty result might be ordinary, exceptional, or a sign that something has already gone wrong further upstream, and what matters is that somebody has noticed the state exists and decided what it means.

Generated code has made me considerably more suspicious of implicit assumptions like this one, because the happy path is usually well described in a ticket or a prompt and the absence of something rarely is. An empty result set, a missing configuration value, a relationship that was never created, an API returning nothing at all, these are thoroughly ordinary states in real systems, and generated code can be completely convincing right up until production supplies one of them.

5. Solving the nearest problem - not the actual one

A classic example is a timesheet page showing hours against the wrong day.

Entries submitted late in the evening were landing on the following date, which gets reported as "my Monday hours are on Tuesday now" and sounds like a five minute job. The proposed fix was this.

// Adjust for timezone offset so entries land on the correct day
const day = format(new Date(entry.submittedAt.getTime() - 3600000), "yyyy-MM-dd");

This does solve a problem. The failing test went green, the hours moved back onto the right day, and the person who raised it confirmed it was fixed.

It stays fixed until the clocks go back in October, but it is wrong in the other direction, and it is wrong immediately for anyone submitting from outside the UK. The application was storing a timestamp with no timezone attached and rendering it in whatever zone the browser happened to be in, so the day boundary had never actually been defined anywhere, and subtracting an hour does not define it either. The generated solution had gone straight for the nearest visible obstacle, which was one wrong date in one failing test, and moved the number until that obstacle went away.

I think this is one of the more useful patterns to look out for, because it turns up in so many shapes. A query is slow so an index appears, and nobody asks why the query is running eight hundred times on one page load. A type error appears so a cast appears. An exception occurs so a catch block occurs. A test fails so the test data changes.

Every one of those responses can be technically reasonable while operating a layer below the actual problem. The question I now ask during review is not only whether the change fixes the error, but why the error was there in the first place and whether this does anything at all about that.

Green tests. Wrong code

Tests are meant to protect us from most of this and good ones truly do. It gets more complicated when the same agent is writing both the implementation and the tests.

Suppose the model misreads a requirement. It implements its interpretation, then writes tests against that same interpretation, and now the tests pass, coverage looks excellent and the implementation is still wrong.

The permutation example illustrates it nicely. Tests that check the candidates are six digits long and that the original four appear in varying orders would give that broken implementation a clean bill of health, whereas a test written from the requirement, proving that one of the newly generated digits can appear at the front, fails immediately.

This is why I have become much less interested in coverage as a number when working with agents. High coverage is still worth having and I still want the suite to pass, but the more useful question is where the expectations in those tests came from, because when the implementation and the tests come from the same mistaken reading they will agree with each other perfectly, and that is consensus rather than verification.

Acceptance criteria and worked examples that describe behaviour independently of the implementation get a lot more valuable once the thing writing the implementation can also produce four hundred tests before you have finished your coffee.

Congratulations, you can generate technical debt faster now

None of these are dramatic failures, and that is rather the point.

The permutation code genuinely uses permutations, the non-null assertion is valid TypeScript, the catch block does catch the exception, matches[0] does return the first match when one exists, and the hour subtracted from the timestamp does put Monday's hours back on Monday. Every implementation has done something correct, it just has not done everything the requirement needed.

When you write the code yourself you spend time inside those decisions. You know why a particular branch is there because you put it there, and you remember the awkward edge case because it cost you forty minutes on a Tuesday, so even when the result is not beautiful you have accumulated context as a side effect of producing it.

Generated code breaks that relationship. An agent can hand me six hundred lines in a few minutes carrying dozens of small decisions I was not present for, and if I review those six hundred lines at the speed they arrived then I have not suddenly become better at code review, I have just given each decision less thought.

That is where a productivity gain turns into a debt multiplier, and the unpleasant part is that the bill does not arrive straight away. For the first few weeks everything is faster in a way that feels wonderful, with tickets closing and pull requests ready before I have finished whatever I moved on to, the code compiles and the application works.

Then an edge case turns up three months later and I have to understand an implementation I did not write, resting on an assumption I never consciously made, inside a feature I can barely remember reviewing. If later agents have copied the same shape, I may also discover that the original almost-right implementation has quietly become the house convention.

Congratulations to me, then. I generated the technical debt faster as well.

So where did the expensive part go?

All of this has changed how I review.

The temptation is to review the signals around an implementation rather than the implementation itself. The build passes, the tests pass, the types pass, the diff looks tidy and the agent reports the task complete. Those are useful signals and I would not want to work without them, but not one of them proves that the implementation represents the requirement, and the more code an agent produces the more tempting it becomes to treat them as a shortcut, because spending an hour on a five hundred line change that appeared in four minutes feels like it defeats the whole point.

I think it is the opposite of defeating the point. The value was never that every part of software development should now run at the speed of code generation, the value is that I no longer have to spend that hour typing the implementation, so I can spend more of it on the decisions inside the implementation instead.

In practice that means going to the places where assumptions like to hide, which tends to be boundaries, empty states, error paths, casts, database interactions and anywhere the code stops resembling the wording of the requirement. It also means being willing to tell an agent that something which compiles, passes its tests and looks perfectly sensible is still wrong, which is a slightly strange conversation to have and a necessary one.

Producing an implementation has become dramatically cheaper. Understanding the problem has not, and neither has architecture, knowing which edge cases matter, deciding what failure ought to mean, or checking that what was built is what was asked for. If anything, those get more valuable, precisely because code now arrives faster than I can properly read it.

None of this is a new engineering principle. Agents did not invent edge cases, swallowed exceptions, dubious type assertions or algorithms that solve a slightly different problem, they just changed the rate at which I can accumulate them, and an agent can produce a great deal of almost-right code in the time it would once have taken me to produce a small amount of almost-right code by hand. That makes the ordinary disciplines less optional rather than more.

Which brings me back to moving up the stack, because this is the other half of it. Getting the plan right before the agents start is human work, and so is deciding afterwards whether what came back is actually the thing you asked for. The speed is real and I have no intention of giving it up, it only holds if the code I generate this afternoon does not become the thing slowing me down in November. Writing code has become cheap, and working out whether it is right is still charging full price.