How I'm Using LLMs For Programming

I love programming. Programming has been a huge part of my life since I was a child. I'm obsessed with how computers work, how people interact with computers and source code and with how other people are programming. I have been for a long time.

Now, with AI coding being the biggest disruption in the software industry, arguably ever, it's such a fascinating time to be alive. Like many developers, I've been living and breathing this stuff for a while now; I've been reading a lot of AI thought-pieces, digesting people's takes and, most importantly, experimenting myself.

I write this, because I want to give you context. I'm a very capable programmer who is interested in using AI agents, not to grant me the ability to do more complex things or to write better code, but to arrive at broadly the same code, just faster. This is different than the goals that other people have, and, if you start with different assumptions and goals than I do, it's likely that you'll come up with different conclusions too.

I also want you to understand that I'm not just writing CRUD apps; I'm working on things that don't have a well-trodden path. For instance, right now, I'm working on a system in typescript for self-adjusting computation. It's quite complex. Again, if your goals are different, you conclusions might be too.

With that said, this is where I've settled, at least for the time being:

I've tried vibe coding (asking agents for programs without deeply understanding the results) and my impression is that, when you vibe code, you're making a bet.

I won't have to ever understand how this works.

You might be right - it depends what it is - but if you are wrong, it's going to take you substantially more time to build an understanding all at once than if you'd tried to build that understanding as you went (where you could change course to make things easier to understand).

And, why might you need to understand how that code works?

Well, perhaps you care about security; it's well documented by now that LLMs often write vulnerable code. I'm sure you can mitigate that risk by also asking LLMs to analyze the code for vulnerabilities, but there really isn't a substitute for a well-trained human, at least, not yet.

Another issue is just that agents get stuck. They are good at making small changes, but they aren't good at refactoring as they go. They aren't good at keeping things tidy and so they end up in situations where they can't make progress.

This is an awful scenario to be in. If it's difficult for an LLM to make sensible changes to the codebase, it's likely not easy for a human to gain an understanding of it either.

So, with that said, I'm happy to vibe code scripts and small utilities that I don't foresee maintaining, but for anything else, I have a different strategy.

My rule of thumb is that, if claude can't more or less one-shot it, then I need to do work to decide on an overarching design that makes sense. Once I have an outline, I'll happy attempt to ask claude to fill in the blanks, working recursively.

The key, I think, is that, if you ask claude to do something, but you don't have a clear understanding of what a solution should look like, reviewing anything that it produces is very difficult work. Actually, it's more difficult and time-consuming than arriving at a good understanding and then writing it yourself. On the other hand, if you know what you expect to see, the calculus changes and using LLMs becomes a net benefit, because, in this case, you can (or at least I can) review the output faster than I can write it myself.

So, claude will always be the junior partner in this relationship, because the bottleneck - the thing that I'm optimizing for - is my ability to understand the output. I make the architectural decisions and decide how everything is structured. If I give up any control, it's only in the leaves of this tree, since any one leaf (a self-contained component, a class or a function) can be rewritten quite easily.

All of this assumes that you are working on green-field projects, but, unfortunately, most people aren't. However, my strategy for using LLMs on established codebases is much less stable, so I'll hold off on writing about that for another day.