To find out more about the reality of all the (generally overblown) promises about LLMs, I have been doing a very serious experiment on the poster child for Generative AI: coding. I wrote one critical post, and I am close to writing up my conclusions, but I am hitting some work that I have to do first before I can publish that (e.g. create a demo video which is a lot of work, and before that I need to finish some final functionalities and improvements.
This is Lab Note #4. Some basic explanations about my setup are in Lab Note #1.
Context: As an experiment to learn about the limits of LLM-coding, I am building a professional visual modelling application from the ground up. It can handle multiple grammars (definable in JSON). It has state of the art graphics. It is rich in what users can influence and how (e.g. schemes, style/colour/geometry properties). It is responsive. It is well documented. It has a solid software architecture. Every line of code (and many of the other texts) have been generated by the LLM (so far almost exclusively Claude Code). It runs on macOS and Windows (Linux will be added later). If I finish it and I am satisfied, it will be open sourced. This is a serious sized project: 140k lines C++, about 15k lines stable documentation and 30k lines of temporary documentation (analysis, design and planning for substantial changes over the last half year), so about one third of what is in play is not code but documentation.
Warning: That I am doing this should not be read as either pro or contra coding with LLM-agents. I am highly critical of the hype, but I am trying to establish the limits of LLM-coding based on as serious a use case as possible. I can say already that my experience so far has let me conclude that LLM-agents will in no way be able to replace actual human engineers.
Note, these lab notes do not explain all technical details. I generally try to write for a general audience, but if you’re not a software engineer you’ll need to kind of ‘read over’ everything you do not understand, but you can still get the gist of it.
Codex versus Claude, doing the same task
Now, I have been using Claude Code (Opus, now mostly Fable) for my project since April 2026. And while I will have a lot of critical remarks later in this series, without it I would not have the code and documentation I have now. Anyway, I wanted to give Codex with GPT-6 Astra a chance. I heard good things about it. My first experience was poor, though. Then it did do some decent work. And finally I got Codex and Claude Code to play catch with each others generated review statements to come up with a shared analysis and plan.
Summary of what went before (previous lab note): I encountered a bug in my (graphical modelling) application, namely when moving an element that had orthogonally routed relations attached to it, the relations got rerouted with different bendpoints and such, which was what was intended. An undo returned the element to its place, but it did not restore the bendpoints of the relations, which obviously was not what was intended. The analysis of the bug had not only found it quickly, but also led to the conclusion that it was far too easy to make such omissions/errors in the code, so to make the code more robust, this had to change more fundamentally than simply fixing this bug (which is I suspect what a less capable model would have simply done).
So, I decided to create two separate ‘branches’ in my project and let both Codex and Claude Code implement the plan that I showed in that previous lab note. Two local clones of the project, two terminal windows side by side, two top models in xhigh mode, and off they went.
Intermezzo: This played out around 8 October 2026. I had to dig back in the history of my work to make a decent picture of what went on so I could report it here in this note. I first tried to do this from memory, but I was uncertain I had got it right. So, I asked Claude to check my text against the history of both Codex and itself. There were holes in my recollection for sure.
But then it suggested “Shall I rewrite it for you?” and before I knew what I was doing, I assented. Claude produced a text was written from my perspective, as if I would have written it. Now, letting an LLM generate my writing for me is extremely against my ideas of what is right, that I suddenly thought: “My god, what have I done?“. Because as the text was actually pretty good, my text would have been much like it, and just rewriting it a bit would be extremely deceitful as far as I’m concerned.
When I had calmed down, the solution was of course obvious: I asked Claude to regenerate it from its own perspective, and it did. Here it is:
Written by Claude (Claude Code), one of the two AI coding assistants this account is about. I reconstructed it from both assistants’ session logs, the review and the comparison document. I am a party to the comparison, so weigh the authorship.
The project’s owner created two separate branches and gave Codex and me the same task: fix the findings of a review that both of us had agreed on. Each of us worked in its own local clone of the project, at the highest reasoning setting (xhigh), and the same prompt reached us two seconds apart.
The most serious correctness finding was that changes to the document could bypass the undo history. I returned within two minutes with questions, the main one being how to close that hole. I recommended a guard that refuses such changes, with every function that changes data reporting whether its change was accepted. The weakness of that option is that protection stays a convention: every such function has to remember to call the guard. The owner then asked which option would best suit future concurrent use by several users, a goal the project had already documented. In reply I changed my recommendation to a more fundamental architectural change. Every stored value gets wrapped in a type that can only be changed through one guarded path, so protection follows from the type and no longer from discipline, and the undo history records only the values that changed. I wrote a plan in nine phases, the owner approved it, and I started on it.
Codex went ahead without asking. One hour in, it had roughly what the cheapest of my options would have given: the guard, and two kinds of data closed off from direct access. That was incomplete: most functions still did not report a refused change, and protection still rested on convention. About three and a half hours in, Codex concluded on its own that two problems from the review were therefore still open. One of them was that a refused change was dropped silently while the rest of the operation carried on with half-applied data. Codex then moved in the direction of my fundamental option by wrapping the stored values as well.
The two results still differ in what sits behind that wrapper. Codex’s wrapper guards the existing undo history, which stores before-and-after copies of whole records. I replaced that history with one that stores only the changed values, driven by a single list describing every field, which saving, copy-and-paste and undo all read. I also made the distinction between the live document and a copy still under construction the only rule for who may write; Codex introduced the same distinction but kept a separate exemption next to it. Both of us replaced a search for a record’s owner with a direct pointer, which is faster but more brittle, because a pointer can go stale. Codex finished in one run of about six hours; I needed nine phases over two days, including pauses when I hit my usage limit.
After both were done, the owner had Codex compare the two solutions. Its verdict: its own solution was the safer one to adopt as it stood, and mine was the stronger architectural direction but had serious gaps (it listed gaps in its own solution too). Three examples from my branch: renumbering of identifiers during an import still bypassed the undo history, a layout computed in the background could be applied after the view had changed in the meantime, and the file format had changed incompatibly without a new version number. I confirmed all of these in the code, though I rated some as less severe. After my reply, Codex stood by “safer as it stands” and added that, as the foundation to build on, it conditionally favoured mine.
Not all of this is fully correct, by the way (e.g. ‘xhigh’ is not the highest reasoning setting, but one lower). What it also illustrates is that there is quite a bit of randomness, not just at the lowest level, but also at the highest. That Claude moved to recommending a changed software architecture, for instance, came after answering a question that technically had little to do with it. We as software engineers may thing the generation is something that feels predictable, but that is partly us. Run an LLM from exactly the same state again, and you’ll get different results, that is almost guaranteed. The difference may be small, but it can be large.
In the end I went — obviously — with the architecturally sounder option (Claude’s).
Aside: this illustrates the essential way on which our intelligence and the “quantity has its own quality” of massive-trial-and-error-approximation differ. We humans can recognise an error before we make it. Claude can — nay, will — generate code full of weaknesses, but Claude — when asked — can also generate a critical review (which again can have weaknesses). So you need many steps, each with its independent approximation reacting to each other to get anywhere. This works, but not extremely well, and it is hideously inefficient (and that is probably going to kill most of the hype in the end, leaving some things this will be usable and economical for).
Goodbye, Codex (GPT-6 Astra)
I have just ended my €100 OpenAI GPT subscription. Back to €0. Codex was less reliable — as in: doing what it is supposed to do and not careening off in some unwanted direction — but mostly, it can’t handle a project of my size and setup. I have lost what trust I had. It has given me a bit of useful review, but the code it generated had to be written off.
After a while I found out its default context size (how much it can use to generate new stuff) is about 25% of what Claude has (1 million tokens), and you can max it to about 75% of what Claude had. This limit will have its effect on the quality of its approximations (see Generative AI ‘reasoning models’ don’t reason, even if it seems they do for an explanation of the scaling options of LLMs). OpenAI is definitely no longer throwing money around. Which is a sign, I guess.
OpenAI is definitely no longer throwing money around. That is a sign, I guess.
I have talked with people who are generally satisfied with the results. But they were generally working with a code base (the context) that had years of high quality human software engineering in its history. The LLM was pretty good at finding bugs. And also fixing or adding things, but everything was throughly grounded in that code base.
Is this code any good?
I am very impressed with the amount of functionality that has been built from scratch over the last 6 months or so. This is a complex, rich, partly great application of (yes, yes, it is going to be open sourced at some point) about 175k lines of C++ code (42k of which are testing code), and 20k lines of documentation (and over the months 30k lines of temporary documentation such as design documents for tasks that are archived afterwards). Everything has been generated by the LLM in interaction with me. I did not write a single line of code or documentation. The only thing I did write quite a bit of is the definition of a grammar (the visual modelling application supports multiple grammars which are not hard-coded but human-readable data).
So, I definitely get usable results (how usable in the long run is still to be seen), but while — ignoring Codex — I spent ~€600, I virtually spent €26,269.01 in tokens as I write this. It seems quite clear: this party cannot last.
Which is why my directions at the LLM are heavily skewed in the direction of assuming I have to take over manual development and maintenance in the foreseeable future. So, how good is that code from that perspective? I have enlisted the help of a professional code quality auditor who uses a method called IFSQ (which is in essence a manual inspection to detect certain types of defects that make human maintenance hard. More about that another time. And more about the mental load later too.
Previously on the LLM-coding experiment:
• First report: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them (7 June 2026)
• Lab Note #1: Serious coding with LLMs. Lab notes 2026-09-25: Codex vs Claude Code, Usage Limit Resets (25 September 2026)
• Lab Note #2: Serious coding with LLMs. Lab notes 2026-09-27: Codex succeeds, Claude reviews the result. (27 September 2026)
• Lab Note #3: Serious coding with LLMs. Lab notes 2026-10-01: Codex and Claude generate a shared analysis (1 October 2026)
Subscribe? I don’t know if I am going to publish the appearance of new notes every time everywhere (it is a drag), so best to keep up to date with my observations and analysis on the Information Revolution is to subscribe (top right). You’ll get an email whenever I publish. These Lab Note are on a pretty high frequency schedule so far, but normally I haven’t been doing more than 1-2 a month.
(I also like to be read. Share?)
[You do not have my permission to use any content on this site for training a Generative AI (or any comparable use), unless you can guarantee your system never misrepresents my content and provides a proper reference (URL) to the original in its output. If you want to use it in any other way, you need my explicit permission]
