James Duffy

AI is a Superpower That Ships Technical Debt

I use AI tools every day now, and they have made me faster. I can scaffold a new service, write a database migration, or put together a first pass at some Terraform in a fraction of the time it used to take me. I am not going to pretend otherwise.

What I have been noticing, though, is the debt that comes along with it, and it piles up without much warning.

Every model has a knowledge cutoff. It learned from code, docs, and Stack Overflow threads up to some point in time, and when it writes code for you today it is pulling from that snapshot. It has no idea what changed last month or what got deprecated last quarter. Most of the problems I have run into come from that gap.

#Old package versions

The most obvious one is dependency versions. Ask an AI to set up a new project and it will usually grab whatever versions were current when it was trained, which can be months or even years old. Everything runs and the tests pass, so you ship it. Then six months later you are dealing with version conflicts or patching a CVE that was fixed two major releases ago.

I don’t really blame the tool for this. It is working from an old map and has no way of knowing that.

#Patterns that have moved on

This one is harder to catch because nothing breaks right away.

Infrastructure tooling changes fast. The way you should handle identity, secrets, and network policy in Kubernetes today is not the same as it was two years ago, but the AI will happily generate the old way because that is what most of its training data looked like.

So you end up with code that works but is built on assumptions from a few years ago. Things like Azure AD Pod Identity instead of Workload Identity, API versions that are already deprecated, or sidecars held together with init container and lifecycle hook workarounds instead of native sidecar containers. None of it is broken, but it shouldn’t be there anymore and someone is going to have to rip it out later.

#Things it never learned

The third problem is the one that limits you the most. Models only know what was publicly documented before their cutoff, so anything newer, anything that was never documented well, and anything internal to your company is basically invisible to them.

Ask about your internal platform or an SDK your vendor shipped three months ago and you either get a confident answer that looks right but isn’t, or the model tells you it doesn’t know. Both cost you time, but the confident wrong answer costs a lot more.

It is great for things lots of people have already written about, but once I am working on something less common I have learned not to trust how sure it sounds.

#When it all stacks up

Any one of these on its own is manageable, but a project built mostly with AI can end up with all three at the same time: outdated dependencies, deprecated patterns, and spots where newer tooling would have been the better choice. It works and probably looks fine in code review. Then someone tries to extend it or upgrade it, all of that debt shows up at once, and cleaning it up is still manual work.

#What I do about it

I treat AI like a really fast collaborator who has been out of the loop for a while. It is great for a first draft, but I review what it gives me with the cutoff in mind.

In practice that means checking package versions against what is actually current before I accept them, knowing which patterns my ecosystem has moved away from so I can spot them, and doing my own research for anything newer than the model. Agents that can search the web, read your lockfile, or pull current docs help a lot here, but only if you tell them to go check instead of answering from memory.

I still think it is worth it, but the overhead is easy to miss. The people I see getting the most out of these tools are the ones who push back on the first answer.