Serious coding with LLMs. Lab notes 2026-11-17: Wait…11,000 lines generated for every line of code of my system??

To find out more about the reality of all the (generally overblown) promises about LLMs, I have been doing a very serious experiment on the poster child for Generative AI: coding. I wrote one critical post, and I am close to writing up my conclusions, but I am hitting some work that I have to do first before I can publish that (e.g finish some final functionalities and improvements).

This is Lab Note #9. Some basic explanations about my setup are in Lab Note #1. Some basic technical notes (on tokens, context) are in Lab Note #5.

I am doing a very short note today. In preparation of a presentation/demo tomorrow, I did some back-of-the-envelope calculations based on my own experiment’s data.

Here is the size of my project now:

                                                lines        size
C++ application
  eacmplib (logic)                            108,524     6.85 MB
  qtgui (presentation/GUI)                     69,014     4.30 MB
Project support (tests, tools, migration)
  C++ tests                                    69,743     4.73 MB
  Python tests                                  7,720    366.9 kB
  LUA tests and demos                          10,653    764.4 kB
DATA
  grammars                                     12,695    561.9 kB
  appearance schemes                            1,433     56.3 kB
  JSON schemas                                  3,130    167.2 kB
  SVG                                           2,077    141.7 kB
Stable documentation
  docs/EACMP*.md                               26,358     2.20 MB
Temporary documentation (202*.md)
  docs/                                             0      0.0 kB
  Archive/                                     19,668     1.55 MB
  ../EACMPDev-Archive/                         24,069     1.71 MB

Totals
  C++                                         247,281    15.87 MB
  LUA                                          10,653    764.4 kB
  SVG                                           2,077    141.7 kB
  JSON                                         17,258    785.4 kB
  Stable documentation                         26,358     2.20 MB
  Temporary documentation                      43,737     3.26 MB

GRAND TOTAL                                   355,084    23.39 MB

The reusable library and application together are now about ~180,000 lines of C++ code. The stable documentation is ~26,000 lines of markdown. With those, you have the entire function application and its documentation.

This result has been created — all of it generated by Claude Code. Roughly, I also know how much this has cost in ‘tokens’. And that is a staggering ~47,000,000,000 tokens (for which I haven’t paid the API-cost of ~$32,000, but about ~$700 subscription.

Now assume that we have in our code something like 15 tokens per line on average. ~47 billion tokens translates to ~3 billion lines. With something like ~270k lines of ‘end product’ that means that for every line in the end product, the LLMs had to generate the equivalent of ~11,000 of them. I am roughly taking stuff from the code base above, e.g. a lot of the stuff is stuff that is not so much building the system, but using it (e.g. the LUA code, which then in part useful and comfortable, but not strictly necessary). It is all very, very back-of-the-envelope.

But the observation becomes:

For every single line of code in my product, the LLM had to generate the equivalent of roughly 11,000 of them.

And adding to that. For every next token out of a frontier model, a back-of-the-envelope calculation gives us the following numbers we should multiply:

  • 50% of the 1 million token context = ~500k
  • ~100k size token dictionary
  • ~12k dimensions per token
  • ~80 layers (GPT3 had 96)

This gets you ~300,000.000.000.000 MACs (Multiplication-Addition Combinations) for each next generated token, which thanks to KV-caching goes down to ~100,000.000.000 MACs.

For each token in your code.

1.5 trillion MACs for each line of code.

These numbers do not include the parallelism in the models (multiple generation ‘threads in parallel with pruning and spawning), or the recursiveness of the ‘thinking’ (urgh) models. So, you easily may multiply this by another 100 or so. And yes, there also is optimisation like ‘sparse attention’ etc., but you get the picture.

From a human perspective these numbers are truly unimaginable, they lose all meaning. But they seem to be real nonetheless.

Previously on the LLM-coding experiment:

• First report: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them (7 June 2026)
• Lab Note #1: Serious coding with LLMs. Lab notes 2026-09-25: Codex vs Claude Code, Usage Limit Resets (25 September 2026)
• Lab Note #2: Serious coding with LLMs. Lab notes 2026-09-27: Codex succeeds, Claude reviews the result. (27 September 2026)
• Lab Note #3: Serious coding with LLMs. Lab notes 2026-10-01: Codex and Claude generate a shared analysis (1 October 2026)
• Lab Note #4: Serious coding with LLMs. Lab notes 2026-10-13: Codex vs Claude on the same coding task. (13 October 2026)
• Lab Note #5: Serious coding with LLMs. Lab notes 2026-10-21: The throttling of subscriptions by OpenAI and Anthropic is really happening. (21 Oct 2026)
• Lab Note #6: Serious coding with LLMs. Lab notes 2026-10-27: A code audit of LLM-generated code: improving from ‘sludge’ to ‘vakwerk’. (27 Oct 2026)
• Lab Note #7: Serious coding with LLMs. Lab notes 2026-10-31: Teaser Demo of the LLM-coded EACMP application, and a quiz (31 Oct 2026)
• Lab Note #8: Serious coding with LLMs. Lab notes 2026-11-03: If you offload your thinking, you’re onboarding risks (3 Nov 2026)

Leave a Reply