Can anyone explain to me what this massive volume of code could actually be doing? They obviously have variations of model architectures, training infrastructure, serving infrastructure, and the source for their apps.
Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
What they do not have is a version of chatgpt.com that is not a slow, awkward, glitchy mess.
As a user of Astra, it writes code that appears to work. It does not necessarily write sensible code that is architected in a way that makes sense based on what it does. I just watched it do a major rework of a couple thousand lines of code by keeping the original organization and kludging up an adapter to make old code have new semantics.
I would expect that this approach, coupled with lots of fun asking for changes and none of the grunt work of actually thinking about the result, could result in exponentially increasing amounts of mess.
I haven’t used a recent Anthropic model - maybe it’s better at this.
Nah I made a workout app with Claude code. Today it loaded the same warmup exercise as my main block 3 times in a row. The program does not call AI to make the plan so that’s the code it wrote lol. In general though the app is good enough if you know what you’re doing which seems to be the theme with AI.
For one, they have the internal models that us normies don't have. Also, machine learning as a field is probably the most automate-able field for AI. It's 100% on computers without "real world" collisions at all, AI can do the whole stack today even with publicly available models. And since it's so empirical, you can make a lot of progress by just running more experiments with more compute.
But "big tech" companies have a lot of different components. And they hit git scaling limits long before the current AI age. The commenter was asking about OAI, not big tech, committing much more code.
At some point I imagine they will be writing their own software for the entire training and inference stack. Kernels, libraries, why not? With compute so constrained it must be beneficial to optimize everything as much as possible
I’m not buying this argument, the apps of the LLM providers are some of the most wasteful apps that exist. Have you ever looked at how much memory / CPU Claude and Codex use? They ship with entire virtual machine hypervisors for sandboxing.
I think it’s simply a matter of dogfooding and new feature development, they do have much higher velocity than companies of equivalent size and an unlimited token budget helps as well.
It's AI slop. AI coding agents write much more voluminous code, in my experience. The base code is more expansive, and there are more tests, and there are many, many more commits as you keep fixing and fixing and fixing over and over and over.
I'd bet money they're capturing every change made by any agent, not just at the end of a 'task' (or even a turn). Free training material volume to see what was changed and then what was changed later in the same turn to fix the mistakes introduced.
If they allocate more compute internally than they sell on subs & APIs, then they should be generating more code than all the customers they sell to combined.
Anyone who has fallen into the ADR pit with agents could tell you this. Its just slop, slop to fix slop, docs about slop, slop to fix the docs, etc. 600 commits and 200k LOC/Docs for software that used to be 15 commits and 10k LOC +README and no real value to be found.
Or it's yet another iteration to get another test with overbuilt providence catalogling, boated code, overbuilt nonsense. Suddenly you're 2.5 Billion tokens deep and you just wonder "What the actual Feck did they do, how did they burn that many tokens? It's just a web app with a plugin system, my god."
While I don't have any particular interest in the topic this is a great blog post. No AI in every second line, no needless marketing or hype. Just a bunch of engineers talking in their own voice about a topic in great detail with numbers, charts, visualizations.
Haha, I'm also building a code forge (at eol.sh, just a placeholder is there now). I like that you don't look like yet another Github clone, which is my primary frustration with the alternatives.
> We built Pando with Codex, moving from kick-off to production, in just 10 weeks. The core of Pando was designed by a single engineer, implemented by teams of agents, and deployed by a handful of engineers. Pando began serving production traffic in April of this year, and is one part of a larger system of source control management tools used at OpenAI.
Another way to avoid having to maximize download speed is to use a global git cache per machine in a similar way that pnpm optimizes installing packages:
> Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?
That coding volume is a liability only for human engineers, with their limited memory, short attention span, tiredness, and quirky coding preferences. LLMs don't have any of those problems.
Every bit of code is more context getting soaked up in future agent runs. More context used = worse output. More tests need to be written, then fixed, using up more compute, time and context without adding value compared to a more thoughtful, efficient implementation.
Not necessarily. High LoC software is a potential bad design smell (without a counterfactual for the same level of functionality it might not be easy to tell though). Badly designed code entering an agent context is always a liability, because it contaminates decisions down the road, and makes new work slower as it needs more tokens to process and navigate old code.
Over the last few months at Xavi Group, our team of one engineer has gone from ~1k commits a day to ~100k commits per day as a result of debilitating financial decisions and exponentially increasing agentic workflow capability. Our internal Gitea services began to get bogged down by our proprietary frontier coding agent (internal codename `xavi-polaris`) refactoring the monorepos comprising our three flagship projects, one of which of course is our widely unknown RSI harness project.
(At this time we are not accepting any additional requests for our closed private beta, unless you ask reaaaalllyyy nicely $$$)
These demands were increasingly straining our S3 bucket capacity (petabyte scale) and our CI/CD pipelines (3 DO droplets, a macbook mini, and a legacy windows server possibly compromised). Most critically it was straining the agentic wellfare of Xavi sessions themselves, which began to flag push/pull latencies that were taking so long their caches would go cold between tool calls, resulting in even greater financial hardship.
`xavi-polaris` sess-3LQGgyUxLGXiV7OTTIMo4TWm0a6 is credited with identifying that this unprecedented pace was unsustainable, with projections that by mid 2027, commits per day could be north of 1 billion as monorepo sizes increased and refactoring became increasingly challenging despite the profound software engineering capabilities unlocked by the RSI harness project.
To identify solutions, we first looked to ensure we wouldn’t reinvent the wheel we were reinventing. We noted projects at Google, Anthropic, and OpenAI that recently aimed to address the very same challenges, but after ~20k USD in session research that incidentally solved an open conjecture in statistical analysis, we decided that adopting any of these existing protocols would limit Xavi sessions in our internal promotion benchmarks that enable “contributor” sessions to progress to “lead” sessions, irrevocably inhibiting our expansion of upper agentic management.
AGIt is a drop in replacement for Git, which we identified as being critically important because xavi sessions are deeply familiar with git commands as a result of a massive distillation and theft of proprietary data. AGIt utilizes novel CRDT and distributed systems technology built from our extensive experience with scaling distributed systems off the back of YJS. If that doesn’t make any sense to you, it didn’t make any sense to the individual human engineer at Xavi Group either.
Importantly, we don’t believe it needs to. The future is agentic workflows, and we’re excited to continue giving back to the nursery that birthed our projects and has delivered at scale for almost every other software company on the planet. We’ll be donating five `xavi-polaris` teams to pumping PRs as fast as possible to these open source projects. As we enter this new era we’d like to thank contributing sessions sess-3LQKRQrJUyAvYHlUDJcjos2CInf and sess-3LQKRnqna2emLvyUyPUqSEIaUh0, their teams were instrumental in this endeavor, and both will be promoted from “lead” to “steward” to reflect their tenacity in building this system over the last few hours.
Can anyone explain to me what this massive volume of code could actually be doing? They obviously have variations of model architectures, training infrastructure, serving infrastructure, and the source for their apps.
Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
What they do not have is a version of chatgpt.com that is not a slow, awkward, glitchy mess.
As a user of Astra, it writes code that appears to work. It does not necessarily write sensible code that is architected in a way that makes sense based on what it does. I just watched it do a major rework of a couple thousand lines of code by keeping the original organization and kludging up an adapter to make old code have new semantics.
I would expect that this approach, coupled with lots of fun asking for changes and none of the grunt work of actually thinking about the result, could result in exponentially increasing amounts of mess.
I haven’t used a recent Anthropic model - maybe it’s better at this.
Nah I made a workout app with Claude code. Today it loaded the same warmup exercise as my main block 3 times in a row. The program does not call AI to make the plan so that’s the code it wrote lol. In general though the app is good enough if you know what you’re doing which seems to be the theme with AI.
For one, they have the internal models that us normies don't have. Also, machine learning as a field is probably the most automate-able field for AI. It's 100% on computers without "real world" collisions at all, AI can do the whole stack today even with publicly available models. And since it's so empirical, you can make a lot of progress by just running more experiments with more compute.
[facetious]
many yaks to shave. many bikesheds to paint.
if you're paid by the line, why not go to town...
Engineers are working on multiple sessions, and LLM is doing most of the commits/pushes. This is becoming norm at big techs.
If you are wondering what these changes are, you can take a peek at https://github.com/openclaw/openclaw/commits/main/
This isn't to say that such pace of iteration is good or bad. That's a separate discussion.
But "big tech" companies have a lot of different components. And they hit git scaling limits long before the current AI age. The commenter was asking about OAI, not big tech, committing much more code.
At some point I imagine they will be writing their own software for the entire training and inference stack. Kernels, libraries, why not? With compute so constrained it must be beneficial to optimize everything as much as possible
I’m not buying this argument, the apps of the LLM providers are some of the most wasteful apps that exist. Have you ever looked at how much memory / CPU Claude and Codex use? They ship with entire virtual machine hypervisors for sandboxing.
I think it’s simply a matter of dogfooding and new feature development, they do have much higher velocity than companies of equivalent size and an unlimited token budget helps as well.
This is completely orthogonal to the inference and training stack that the parent comment mentions
If it’s written by AI it’s gonna be the same overcomplicated overfit slop in one way or another. Performance pressure doesn’t fix that.
Depends on what the goal is and the skill of the engineer right?
Give a verifiable benchmark for the AI agent to hit such as reduce token processing time by 5% without affecting output.
Assuming it’s entirely unsupervised, yes, which I can only assume to be the case at these volumes (hence the dogfooding).
I do believe this will force the AI labs to eventually solve the bloat problem with AI, but we’re definitely not there yet.
Looking at their desktop apps, I don't think they're into optimizing.
They certainly are spewing forth large amounts of code, though.
I'm doing that now with gemini argon.
It's better than the ductape and glue pipelines that we had previous to this.
The software is the definition of plumbing no one cares about and testing it is trivial.
RL environments
It's the billions of scripts we used not to commit. And billions more.
It's AI slop. AI coding agents write much more voluminous code, in my experience. The base code is more expansive, and there are more tests, and there are many, many more commits as you keep fixing and fixing and fixing over and over and over.
I'd bet money they're capturing every change made by any agent, not just at the end of a 'task' (or even a turn). Free training material volume to see what was changed and then what was changed later in the same turn to fix the mistakes introduced.
If they allocate more compute internally than they sell on subs & APIs, then they should be generating more code than all the customers they sell to combined.
Anyone who has fallen into the ADR pit with agents could tell you this. Its just slop, slop to fix slop, docs about slop, slop to fix the docs, etc. 600 commits and 200k LOC/Docs for software that used to be 15 commits and 10k LOC +README and no real value to be found.
Or it's yet another iteration to get another test with overbuilt providence catalogling, boated code, overbuilt nonsense. Suddenly you're 2.5 Billion tokens deep and you just wonder "What the actual Feck did they do, how did they burn that many tokens? It's just a web app with a plugin system, my god."
Agent Detection and Response?
Maybe it could be faking code and ai usage?
I've seen it on other workplaces, but from what I see from open ai and anthropic, the code shouldn't be as extended.
But maybe I'm crazy lol
While I don't have any particular interest in the topic this is a great blog post. No AI in every second line, no needless marketing or hype. Just a bunch of engineers talking in their own voice about a topic in great detail with numbers, charts, visualizations.
I don't see benchmarks anywhere in here. For example, how many pushes per second does this handle? And how many concurrent pushers? etc..
Disclaimer: I am building a code forge at https://juju.bi
Haha, I'm also building a code forge (at eol.sh, just a placeholder is there now). I like that you don't look like yet another Github clone, which is my primary frustration with the alternatives.
growing out of Git seems like a graduation for becoming top tech companies. Google and Facebook have done this in the past. now Anthropic and openAI.
Bare in mind that OpenAI|Anthropic (kill me, I can't recall which one), made a massive improvement on their app by... gziping the served content!
> We built Pando with Codex, moving from kick-off to production, in just 10 weeks. The core of Pando was designed by a single engineer, implemented by teams of agents, and deployed by a handful of engineers. Pando began serving production traffic in April of this year, and is one part of a larger system of source control management tools used at OpenAI.
Impressive!?
Another way to avoid having to maximize download speed is to use a global git cache per machine in a similar way that pnpm optimizes installing packages:
https://git-dedup.ben3d.ca
This doesn't work if the pulls are happening in swarms of ephemeral sandboxes.
Also, maybe FUD, but is there anything useful to learn from (p)npm other than cautionary tales?
Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?
This is obviously just a response to https://www.anthropic.com/research/redesigning-source-contro.... Probably no one in the chain of writing this even thought about your question.
Instead of keeping it simple, they decided to go stupid.
Because this kind of growth is how they get to RSI.
Whereas previously, creating this amount of code by hand would simply result in RSI.
> Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?
That coding volume is a liability only for human engineers, with their limited memory, short attention span, tiredness, and quirky coding preferences. LLMs don't have any of those problems.
Every bit of code is more context getting soaked up in future agent runs. More context used = worse output. More tests need to be written, then fixed, using up more compute, time and context without adding value compared to a more thoughtful, efficient implementation.
Not necessarily. High LoC software is a potential bad design smell (without a counterfactual for the same level of functionality it might not be easy to tell though). Badly designed code entering an agent context is always a liability, because it contaminates decisions down the road, and makes new work slower as it needs more tokens to process and navigate old code.
Sure, but you also don't know what your LLM is doing without running into these same limitations. So how is this an improvement?
who cares what the llm is doing? most of the time when there is disagreement, the llm is right and you are wrong anyways.
How would you know?
How do I know my calculator is outputting the correct answer to a 10 by 10 digit multiplication problem?
Because someone understood and verified it.
Scaling Git Operations at Xavi Group /s
Over the last few months at Xavi Group, our team of one engineer has gone from ~1k commits a day to ~100k commits per day as a result of debilitating financial decisions and exponentially increasing agentic workflow capability. Our internal Gitea services began to get bogged down by our proprietary frontier coding agent (internal codename `xavi-polaris`) refactoring the monorepos comprising our three flagship projects, one of which of course is our widely unknown RSI harness project.
(At this time we are not accepting any additional requests for our closed private beta, unless you ask reaaaalllyyy nicely $$$)
These demands were increasingly straining our S3 bucket capacity (petabyte scale) and our CI/CD pipelines (3 DO droplets, a macbook mini, and a legacy windows server possibly compromised). Most critically it was straining the agentic wellfare of Xavi sessions themselves, which began to flag push/pull latencies that were taking so long their caches would go cold between tool calls, resulting in even greater financial hardship.
`xavi-polaris` sess-3LQGgyUxLGXiV7OTTIMo4TWm0a6 is credited with identifying that this unprecedented pace was unsustainable, with projections that by mid 2027, commits per day could be north of 1 billion as monorepo sizes increased and refactoring became increasingly challenging despite the profound software engineering capabilities unlocked by the RSI harness project.
To identify solutions, we first looked to ensure we wouldn’t reinvent the wheel we were reinventing. We noted projects at Google, Anthropic, and OpenAI that recently aimed to address the very same challenges, but after ~20k USD in session research that incidentally solved an open conjecture in statistical analysis, we decided that adopting any of these existing protocols would limit Xavi sessions in our internal promotion benchmarks that enable “contributor” sessions to progress to “lead” sessions, irrevocably inhibiting our expansion of upper agentic management.
We’re proud to introduce our new, closed source Git model: AGIt.
AGIt is a drop in replacement for Git, which we identified as being critically important because xavi sessions are deeply familiar with git commands as a result of a massive distillation and theft of proprietary data. AGIt utilizes novel CRDT and distributed systems technology built from our extensive experience with scaling distributed systems off the back of YJS. If that doesn’t make any sense to you, it didn’t make any sense to the individual human engineer at Xavi Group either.
Importantly, we don’t believe it needs to. The future is agentic workflows, and we’re excited to continue giving back to the nursery that birthed our projects and has delivered at scale for almost every other software company on the planet. We’ll be donating five `xavi-polaris` teams to pumping PRs as fast as possible to these open source projects. As we enter this new era we’d like to thank contributing sessions sess-3LQKRQrJUyAvYHlUDJcjos2CInf and sess-3LQKRnqna2emLvyUyPUqSEIaUh0, their teams were instrumental in this endeavor, and both will be promoted from “lead” to “steward” to reflect their tenacity in building this system over the last few hours.
lol nice try openai but its shallow and too late
Impressive. We're definitely seeing LLMs help create real novel software